Task processing method, device and equipment for database

Abstract

The present invention provides a task processing method, device and equipment for a database, which can be applied to the field of computer technology. The method includes: evenly distributing multiple target tasks determined from multiple tasks to be processed to multiple available threads to obtain allocation methods for multiple target tasks; when it is determined that there are remaining tasks other than the target tasks in the multiple tasks to be processed, based on the number of remaining tasks and the number of available threads, simulating the allocation of the remaining tasks to multiple available threads to obtain multiple simulation allocation results; based on a preset preprocessing time and a preset task processing time, determining the task processing time of each of the multiple simulation allocation results; using the simulation allocation result with the shortest task processing time among the multiple simulation allocation results as the allocation method for the remaining tasks; based on the allocation method for the target task and the allocation method for the remaining tasks, obtaining the allocation method for the multiple tasks to be processed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a task processing method, device, equipment, medium and program product for a database. Background Art

[0002] Since one core of a CPU can execute at least one thread, a multi-core CPU can realize parallel processing of multiple computing tasks. When multiple computing tasks are processed in parallel, there are usually computing tasks that are not enough to be evenly distributed to each thread for processing. For this computing task, it is usually processed by a fixed thread.

[0003] In the process of implementing the concept of the present invention, it was found that there are at least the following problems in the related technology: because a fixed thread is used to process computing tasks that are not enough to be evenly distributed to each thread, the task processing method for different scenarios is not flexible, and in some scenarios, there may be problems of long task processing time and low task processing efficiency. Summary of the invention

[0004] In view of the above problems, the present invention provides a task processing method, apparatus, device, medium and program product for a database.

[0005] According to one aspect of the present invention, a task processing method for a database is provided, comprising: evenly distributing a plurality of target tasks determined from a plurality of pending tasks to a plurality of available threads, and obtaining an allocation method for the plurality of target tasks, wherein the target task is a pending task processed in parallel by a plurality of available threads; in the case of determining that there are remaining tasks other than the target task among the plurality of pending tasks, simulating the allocation of the remaining tasks to the plurality of available threads based on the number of the remaining tasks and the number of available threads, and obtaining a plurality of simulated allocation results; determining the task processing time of each of the plurality of simulated allocation results based on a preset preprocessing time and a preset task processing time; using the simulated allocation result with the shortest task processing time among the plurality of simulated allocation results as the allocation method for the remaining tasks; and obtaining an allocation method for the plurality of pending tasks based on the allocation methods for the plurality of target tasks and the allocation method for the remaining tasks.

[0006] According to an embodiment of the present invention, there is at least one remaining task; multiple simulation allocation results include a first simulation allocation result, a second simulation allocation result and a third simulation allocation result; based on the number of remaining tasks and the number of available threads, the remaining tasks are simulated and allocated to multiple available threads to obtain multiple simulation allocation results, including: at least one remaining task is simulated and allocated to respective target threads to obtain a first simulation allocation result, wherein the remaining tasks and the target threads are in a one-to-one correspondence, and the target threads are determined from multiple available threads; for each remaining task, multiple first remaining slice tasks obtained after splitting the remaining tasks based on the number of available threads are simulated and allocated to respective available threads to obtain a second simulation allocation result; in response to the number of available threads being in a multiple relationship with the number of remaining tasks, the remaining tasks split based on the multiple relationship are simulated and allocated to respective available threads to obtain a third simulation allocation result.

[0007] According to an embodiment of the present invention, at least one remaining task is simulated and assigned to a respective target thread to obtain a first simulation allocation result, including: determining a target thread from multiple available threads based on the number of remaining tasks, wherein the number of target threads is the same as the number of remaining tasks; and simulating and assigning a target thread to each remaining task to obtain a first simulation allocation result.

[0008] According to an embodiment of the present invention, multiple first remaining slice tasks obtained after splitting the remaining tasks based on the number of available threads are respectively simulated and allocated to respective available threads to obtain a second simulated allocation result, including: for each remaining task, the remaining tasks are split based on the number of available threads to obtain multiple first remaining slice tasks, wherein the number of first remaining slice tasks is equal to the number of available threads; and each first remaining slice task is respectively simulated and allocated an available thread to obtain a second simulated allocation result.

[0009] According to an embodiment of the present invention, the remaining tasks after being divided based on the multiple relationship are simulated and assigned to respective threads to obtain a third simulated allocation result, including: dividing each remaining task based on the multiple relationship to obtain a second remaining slice task for at least one remaining task; simulating and assigning an available thread to each second remaining slice task to obtain a third simulated allocation result.

[0010] According to an embodiment of the present invention, based on a preset preprocessing duration and a preset task processing duration, the task processing duration of each of a plurality of simulation allocation results is determined, including: for each simulation allocation result, based on the simulation allocation result, determining the number of parallel times; based on the number of task slices of the remaining tasks in the simulation allocation result and the preset task processing duration, obtaining the target task processing duration; based on the number of parallel times, the target task processing duration and the preset preprocessing duration, obtaining the task processing duration of the simulation allocation result.

[0011] According to an embodiment of the present invention, the method also includes: determining multiple data block information to be generated based on the number of data rows of each of the multiple data to be processed in the database, the preset number of data rows of the data block to be generated, and the data index corresponding to each of the multiple data to be processed, wherein the data index is used to search for the data to be processed in the database; inputting the multiple data block information to be generated into the task generation template to obtain multiple tasks to be processed.

[0012] According to an embodiment of the present invention, the method further includes: based on the allocation mode for the plurality of tasks to be processed, calling available threads corresponding to the plurality of tasks to be processed to process the tasks to be processed.

[0013] Another aspect of the present invention provides a task processing device for a database, comprising: a first mode determination module, used to evenly distribute multiple target tasks determined from multiple tasks to be processed to multiple available threads to obtain an allocation mode for the multiple target tasks, wherein the target task is a task to be processed by multiple available threads in parallel, and the number of target tasks processed by each available thread is the same; a simulation module, used to simulate the allocation of the remaining tasks to multiple available threads based on the number of remaining tasks and the number of available threads when it is determined that there are remaining tasks other than the target task among the multiple tasks to be processed, so as to obtain multiple simulated allocation results; a time determination module, used to determine the task processing time of each of the multiple simulated allocation results based on a preset preprocessing time and a preset task processing time; a second mode determination module, used to use the simulated allocation result with the shortest task processing time among the multiple simulated allocation results as the allocation mode for the remaining tasks; a third mode determination module, used to obtain the allocation mode for the multiple tasks to be processed based on the allocation mode for the multiple target tasks and the allocation mode for the remaining tasks.

[0014] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] Another aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions implement the steps of the above method when executed by a processor.

[0016] Another aspect of the present invention further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0017] According to the task processing method for database of the present invention, by determining the target task from multiple tasks to be processed, and having the same number of target tasks processed by available threads, and determining the remaining tasks except the target tasks from the multiple tasks to be processed, a plurality of simulation allocation methods are used to simulate the allocation of the remaining tasks according to the number of the remaining tasks and the number of available threads, to obtain a plurality of simulation allocation results, and by quantifying the preprocessing time and the task processing time, the task processing time of each simulation allocation result is determined, so as to determine the remaining task allocation method suitable for the current task processing scenario from the plurality of simulation allocation methods based on the task processing time, and then obtain the allocation method of the overall tasks to be processed. By simulating the allocation of remaining tasks to multiple available threads and quantifying the task processing time of each simulated allocation method, it is possible to determine a targeted and time-saving remaining task allocation method for different task processing scenarios, and obtain the shortest overall task allocation method by evenly allocating the target task to each available thread and the target task allocation method and the above-mentioned remaining task allocation method. Therefore, the technical problems of long task processing time and low task processing efficiency caused by inflexible task processing methods existing in related technologies are at least partially solved, and technical effects such as flexible processing of pending tasks, shortening the overall task processing time and improving task processing efficiency are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0019] Figure 1 An application scenario diagram of a task processing method, apparatus, device, medium, and program product for a database according to an embodiment of the present invention is shown.

[0020] Figure 2 A flow chart of a task processing method for a database according to an embodiment of the present invention is shown.

[0021] Figure 3 A flow chart of a task processing method for a database according to another embodiment of the present invention is shown.

[0022] Figure 4A structural block diagram of a task processing device for a database according to an embodiment of the present invention is shown.

[0023] Figure 5 A block diagram of an electronic device suitable for implementing a task processing method for a database according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] It should be noted that the task processing method and device for database of the present invention can be used in the field of computer technology. In the database task processing scenario, it can also be used in any field other than the field of computer technology, such as the field of artificial intelligence technology, etc. The present invention does not limit the application field of the task processing method and device for database.

[0029] During the research, we found that with the rapid development of central processing units (CPUs), the number of CPU cores in mainstream applications has grown from a few cores in a single CPU, 1-20 cores, to 64 cores or even hundreds of cores in a new generation of CPUs, resulting in the number of available cores in a 2-way or 4-way server reaching more than 128 cores. With the addition of hyperthreading technology, a single server can even reach 512 cores. With the increase in the number of CPU cores, for applications such as databases that require computing, a computing task used to be computed by fewer cores, but now there are more computing cores. While the efficiency of computing task processing has increased, how to use threads to process computing tasks has become particularly important.

[0030] In view of this, an embodiment of the present invention provides a task processing method for a database, comprising: evenly distributing multiple target tasks determined from multiple tasks to be processed to multiple available threads to obtain an allocation method for the multiple target tasks, wherein the target task is a task to be processed that is processed in parallel by multiple available threads; in the case of determining that there are remaining tasks other than the target task among the multiple tasks to be processed, based on the number of remaining tasks and the number of available threads, simulating the allocation of the remaining tasks to the multiple available threads to obtain multiple simulated allocation results; based on a preset preprocessing time and a preset task processing time, determining the task processing time of each of the multiple simulated allocation results; using the simulated allocation result with the shortest task processing time among the multiple simulated allocation results as the allocation method for the remaining tasks; and obtaining an allocation method for the multiple tasks to be processed based on the allocation methods for the multiple target tasks and the allocation methods for the remaining tasks.

[0031] Figure 1 An application scenario diagram of a task processing method, apparatus, device, medium, and program product for a database according to an embodiment of the present invention is schematically shown.

[0032] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0033] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0035] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0036] It should be noted that the task processing method for a database provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the task processing device for a database provided in the embodiment of the present invention can generally be set in the server 105. The task processing method for a database provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the task processing device for a database provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The task processing method for a database according to an embodiment of the present invention is described in detail.

[0039] Figure 2 The flowchart of the task processing method for a database according to an embodiment of the present invention is schematically shown.

[0040] like Figure 2 As shown, the method includes operations S210 to S250.

[0041] In operation S210, a plurality of target tasks determined from a plurality of to-be-processed tasks are evenly distributed to a plurality of available threads to obtain a distribution method for the plurality of target tasks, wherein the target tasks are to-be-processed tasks that are processed in parallel by a plurality of available threads.

[0042] In operation S220, when it is determined that there are remaining tasks except the target task among the plurality of tasks to be processed, the remaining tasks are simulated and allocated to the plurality of available threads based on the number of the remaining tasks and the number of available threads to obtain a plurality of simulated allocation results.

[0043] In operation S230 , based on a preset preprocessing duration and a preset task processing duration, the task processing duration of each of the plurality of simulation allocation results is determined.

[0044] In operation S240 , the simulation allocation result with the shortest task processing time among the multiple simulation allocation results is used as the allocation method for the remaining tasks.

[0045] In operation S250 , an allocation method for a plurality of tasks to be processed is obtained based on the allocation methods for a plurality of target tasks and the allocation methods for the remaining tasks.

[0046] According to the embodiment of the present invention, there is no limitation on the task to be processed, and it may be a task generated according to the data to be processed in the database.

[0047] According to an embodiment of the present invention, there is no limitation on the method for determining available threads. It can be determined based on the number of physical cores of the central processing unit, for example: one physical core corresponds to one available thread; it can also be determined based on other methods, such as: one physical core corresponds to multiple available threads.

[0048] According to the embodiment of the present invention, since the amount of data to be processed for each task to be processed is the same, the task processing time and preprocessing time of using available threads to process the task to be processed are the same.

[0049] According to an embodiment of the present invention, multiple pending tasks can be divided into target tasks and remaining tasks. The target tasks are pending tasks that can be processed in parallel by multiple available threads, i.e., all available threads, that is, pending tasks that can be evenly distributed to each available thread, and the number of target tasks processed by each available thread is the same; the remaining tasks are tasks that are not enough to be distributed to each available thread. For example: there are 10 pending tasks and 8 available threads, then one pending task is assigned to each available thread, that is, the target tasks are 8, and 8 pending tasks can be randomly selected from the 10 pending tasks as target tasks, and the remaining 2 pending tasks can be used as remaining tasks.

[0050] According to an embodiment of the present invention, when determining the target task, the number of pending tasks can be first divided by the number of available threads to determine the number of pending tasks processed by each available thread, thereby randomly determining the target task assigned to each available thread from multiple pending tasks.

[0051] According to an embodiment of the present invention, a modulo operation may be performed on the number of tasks to be processed and the number of available threads to determine the number of remaining tasks, and the tasks to be processed other than the remaining tasks may be used as target tasks.

[0052] According to an embodiment of the present invention, one available thread can process one target task.

[0053] According to an embodiment of the present invention, preprocessing may be a preprocessing operation of an available thread when processing a task to be processed, such as an operation of applying for memory.

[0054] According to an embodiment of the present invention, the preset preprocessing time is the time consumed for preprocessing, and the preset task processing time is the time consumed for processing the task to be processed. The preset preprocessing time and the preset processing time can be set by the ratio between the preprocessing time and the task processing time, and the ratio is determined through multiple specific experiments.

[0055] According to an embodiment of the present invention, the remaining tasks may be simulated and allocated to multiple available threads for processing based on the number of remaining tasks, the number of available threads, or the quantitative relationship between the number of remaining tasks and the number of available threads.

[0056] According to an embodiment of the present invention, the task processing time consumed by each simulation allocation result for completing the processing of all remaining tasks can be determined by presetting the preprocessing time and the presetting the task processing time.

[0057] According to an embodiment of the present invention, the allocation method for each task to be processed can be determined by the allocation method for the target task and the allocation method for the remaining tasks, so that the task to be processed can be processed according to the available thread corresponding to each task to be processed.

[0058] According to the task processing method for database of the present invention, by determining the target task from multiple tasks to be processed, and having the same number of target tasks processed by available threads, and determining the remaining tasks except the target tasks from the multiple tasks to be processed, a plurality of simulation allocation methods are used to simulate the allocation of the remaining tasks according to the number of the remaining tasks and the number of available threads, to obtain a plurality of simulation allocation results, and by quantifying the preprocessing time and the task processing time, the task processing time of each simulation allocation result is determined, so as to determine the remaining task allocation method suitable for the current task processing scenario from the plurality of simulation allocation methods based on the task processing time, and then obtain the allocation method of the overall tasks to be processed. By simulating the allocation of remaining tasks to multiple available threads and quantifying the task processing time of each simulated allocation method, it is possible to determine a targeted and time-saving remaining task allocation method for different task processing scenarios, and obtain the shortest overall task allocation method by evenly allocating the target task to each available thread and the target task allocation method and the above-mentioned remaining task allocation method. Therefore, the technical problems of long task processing time and low task processing efficiency caused by inflexible task processing methods existing in related technologies are at least partially solved, and technical effects such as flexible processing of pending tasks, shortening the overall task processing time and improving task processing efficiency are achieved.

[0059] According to an embodiment of the present invention, there is at least one remaining task; the multiple simulation allocation results include a first simulation allocation result, a second simulation allocation result, and a third simulation allocation result; based on the number of remaining tasks and the number of available threads, the remaining tasks are simulated and allocated to multiple available threads to obtain multiple simulation allocation results, which may include the following operations.

[0060] Simulate and allocate at least one remaining task to the respective target thread to obtain a first simulated allocation result, wherein the target thread is determined from multiple available threads; for each remaining task, simulate and allocate multiple first remaining slice tasks obtained by splitting the remaining task based on the number of available threads to the respective available threads to obtain a second simulated allocation result, wherein the first remaining slice tasks and the available threads are in a one-to-one correspondence; in response to the number of available threads being a multiple of the number of remaining tasks, simulate and allocate the remaining tasks after being split based on the multiple relationship to the respective available threads to obtain a third simulated allocation result.

[0061] According to an embodiment of the present invention, the method of simulating allocation is not limited. Target threads having the same number as the remaining tasks can be selected from multiple available threads, and target threads can be simulated and allocated to at least one remaining task respectively, so that one target thread simulates processing one remaining task.

[0062] According to an embodiment of the present invention, each remaining task may be divided. Specifically, the remaining task may be divided according to the number of available threads to obtain a plurality of first remaining slice tasks, and one remaining slice task may be assigned to one available thread for processing.

[0063] According to an embodiment of the present invention, each remaining task may be divided according to a multiple relationship between the number of available threads and the number of remaining tasks, and all the remaining tasks after division may be simulated and assigned to multiple available threads for processing.

[0064] According to the embodiment of the present invention, other simulation allocation methods may also be adopted, and the present invention is not limited here.

[0065] According to the embodiment of the present invention, by using a plurality of simulated allocation modes to simulate the allocation of the remaining tasks, the most appropriate allocation mode of the remaining tasks can be determined.

[0066] According to an embodiment of the present invention, determining the task processing time consumption of each of a plurality of simulation allocation results based on a preset preprocessing time and a preset task processing time may include the following operations.

[0067] For each simulation allocation result, the number of parallel times is determined based on the simulation allocation result; the target task processing time is obtained based on the number of task slices of the remaining tasks in the simulation allocation result and the preset task processing time; the task processing time of the simulation allocation result is obtained based on the number of parallel times, the target task processing time and the preset preprocessing time.

[0068] According to an embodiment of the present invention, the number of parallel processing times is the number of parallel processing times when multiple available threads perform parallel processing on at least one remaining task.

[0069] According to an embodiment of the present invention, since the task volume of each task to be processed is the same, the task processing time of each task to be processed is the same. Therefore, the time taken by multiple available threads to process tasks in parallel is equal to the time taken by one available thread to process one task.

[0070] According to an embodiment of the present invention, the number of task slices is the number of slice tasks obtained after splitting a remaining task.

[0071] According to an embodiment of the present invention, since the preset task processing time is determined by a ratio, it can be considered that after the remaining tasks are divided, the target task processing time allocated to each available thread to process the task is the preset task processing time divided by the number of slices of the remaining tasks. For example: the preset task time is 1, and the remaining tasks are divided into 2 slice tasks, then the target task processing time for one available thread to process one slice task is 0.5.

[0072] According to an embodiment of the present invention, the target task processing time and the preset preprocessing time can be used to determine the time required for one parallel process when multiple available threads process the remaining tasks in parallel. The task processing time required for each simulation allocation result can be obtained by multiplying the time required for one parallel process by the number of parallel processes.

[0073] According to an embodiment of the present invention, after obtaining the task processing time of each simulated allocation result, information such as the number of available threads, the number of tasks to be processed, and the task processing time of each simulated allocation result can be recorded and stored as a table of the relationship between simulated allocation scenarios and task processing time. When the same scenario is encountered later, the task processing time of each simulated allocation result can be obtained directly based on the data in the table of the relationship between simulated allocation scenarios and task processing time, and then a suitable method for allocating the remaining tasks can be quickly selected.

[0074] According to an embodiment of the present invention, since the calculation process of simulating the allocation of each remaining task and determining the task processing time of each simulated allocation result is simple and time-consuming, compared with the longer time caused by inappropriate remaining task processing methods, the above process will not affect the overall task processing time.

[0075] According to an embodiment of the present invention, since the process of splitting the remaining tasks takes a short time, in some embodiments, the time consumed by splitting can be ignored, or a preset splitting duration can be set based on the relationship between the splitting duration and the task processing duration, that is, when there is a splitting step, the time consumed for a parallel processing can be obtained based on the preset preprocessing duration, the target task processing duration and the splitting duration.

[0076] According to an embodiment of the present invention, by quantifying the preprocessing time and the task processing time, and calculating the specific time consumption of each simulated allocation method under different simulated allocation methods, it is possible to select a more reasonable and less time-consuming allocation method to process the remaining tasks in the subsequent actual allocation, thereby shortening the overall task processing time when processing multiple tasks to be processed, and avoiding the problem of long overall task processing time caused by inappropriate remaining task processing methods.

[0077] According to an embodiment of the present invention, simulating and allocating at least one remaining task to respective target threads to obtain a first simulated allocation result may include the following operations.

[0078] Based on the number of remaining tasks, a target thread is determined from a plurality of available threads, wherein the number of the target threads is the same as the number of the remaining tasks; and a target thread is simulated and allocated to each remaining task to obtain a first simulation allocation result.

[0079] According to an embodiment of the present invention, there is a one-to-one correspondence between the remaining tasks and the target threads.

[0080] According to an embodiment of the present invention, when determining the task processing time of the first simulation allocation result, since one target thread processes one remaining task, its parallel number is 1. Since the remaining tasks are not segmented, its task shard number is 1, and its target task processing time is equal to the preset task processing time.

[0081] According to an embodiment of the present invention, the task processing time consumption of the first simulation allocation result is: the sum of a preset task processing time length and a preset preprocessing time length.

[0082] According to an embodiment of the present invention, for example, if the preset preprocessing duration is 0.01, the preset task processing duration is 1, the number of remaining tasks is 4, and the number of available threads is 8, then the task processing duration of the first simulation allocation result is 1.01. For another example, if the preset preprocessing duration and the preset task processing duration remain unchanged, the number of available threads is 128, and the number of remaining tasks is 100, then the task processing duration of the first simulation allocation result is still 1.01.

[0083] According to an embodiment of the present invention, simulating and allocating multiple first remaining slice tasks obtained by dividing the remaining tasks based on the number of available threads to respective available threads to obtain a second simulated allocation result may include the following operations.

[0084] For each remaining task, the remaining task is divided based on the number of available threads to obtain multiple first remaining slice tasks, wherein the number of first remaining slice tasks is equal to the number of available threads; and an available thread is simulated and allocated to each first remaining slice task to obtain a second simulated allocation result.

[0085] According to an embodiment of the present invention, there is a one-to-one correspondence between the first remaining slice tasks and the available threads.

[0086] According to an embodiment of the present invention, when determining the task processing time of the first simulation allocation result, since the remaining tasks are divided based on the number of available threads, the number of task slices is the number of available threads and the number of parallel times is the number of remaining tasks.

[0087] According to an embodiment of the present invention, for example: the preset preprocessing duration is 0.01, the preset task processing duration is 1, the number of remaining tasks is 4, and the number of available threads is 8, then the task processing time of the second simulation allocation result is 4*(1 / 8+0.01)=0.54. For another example: the preset preprocessing duration and the preset task processing duration remain unchanged, the number of available threads is 128, and the number of remaining tasks is 100, then the task processing time of the second simulation allocation result is 100*(1 / 128+0.01)=1.78.

[0088] According to an embodiment of the present invention, simulating and allocating the remaining tasks after being split based on the multiple relationship to respective threads to obtain a third simulation allocation result may include the following operations.

[0089] Each remaining task is divided based on the multiple relationship to obtain a second remaining slice task of at least one remaining task; and an available thread is simulated and allocated to each second remaining slice task to obtain a third simulation allocation result.

[0090] According to an embodiment of the present invention, the multiple relationship may be an integer multiple.

[0091] According to an embodiment of the present invention, splitting each remaining task based on a multiple relationship may be splitting the remaining task into the same number of slice tasks as the multiple relationship. For example, if the multiple relationship is 2, then one remaining task is split into 2 slice tasks.

[0092] According to an embodiment of the present invention, a second remaining slice task may be allocated to each available thread.

[0093] According to an embodiment of the present invention, the process of determining the task processing time of the third simulation allocation result is, for example: the preset preprocessing time is 0.01, the preset task processing time is 1, the number of remaining tasks is 4, and the number of available threads is 8. It can be seen that since the number of available threads is twice the number of remaining tasks, one remaining task is divided into 2 second remaining slice tasks, and the number of parallel tasks is 1. Therefore, the task processing time of the third simulation allocation result is (1 / 2+0.01)=0.51.

[0094] According to an embodiment of the present invention, the relationship between the parallel times and the simulation allocation results can be recorded in advance for the first historical simulation allocation result and the third historical simulation allocation result, so that the parallel times can be obtained through the record.

[0095] According to an embodiment of the present invention, it can be seen from the above examples that in different scenarios, there are obvious differences in the advantages and disadvantages of different simulation allocation results. Therefore, performing simulation allocation in advance and determining the appropriate remaining task allocation method based on the task processing time corresponding to different simulation allocation results can shorten the task processing time to a large extent.

[0096] According to an embodiment of the present invention, the above-mentioned task processing method for a database may further include the following operations.

[0097] Based on the number of data rows of each of the multiple data to be processed in the database, the preset number of data rows of the data block to be generated, and the data index corresponding to each of the multiple data to be processed, determine the multiple data block information to be generated, wherein the data index is used to find the data to be processed in the database; input the multiple data block information to be generated into the task generation template to obtain multiple tasks to be processed.

[0098] According to an embodiment of the present invention, a task to be processed can be generated based on relevant information of multiple data to be processed in a database, wherein the relevant information includes the number of data rows, the number of data columns, the data index, etc. of the multiple data to be processed, and the relevant information can be stored in a format such as a data table.

[0099] According to an embodiment of the present invention, the task generation template may be a task script for generating the task to be processed, and the task script stores relevant program codes for generating the task to be processed and information about the data block to be generated.

[0100] According to an embodiment of the present invention, in response to receiving the relevant information, a task to be processed may be generated.

[0101] According to the embodiments of the present invention, there is no limitation on the tasks to be processed. In some embodiments, it can be a task to store the data to be processed in the database, such as storing it in a target disk, a target memory or a target cloud storage database. In other embodiments, it can be a task to perform other processing on the tasks to be processed, such as calculation, statistical analysis, etc.

[0102] According to an embodiment of the present invention, when the task to be processed is a task to separately store the data to be processed in the database, the data to be processed can be stored in the format of a data block in the storage space for separate storage. Therefore, the storage task of a data block can be regarded as a task to be processed.

[0103] According to an embodiment of the present invention, the data to be processed stored in each data block can be determined by the number of data rows of each of the multiple data to be processed and the preset number of data rows of the data block to be generated, and the data index corresponding to each data block to be generated can be determined by the data index corresponding to each of the multiple data to be processed, and the data index corresponding to each data block to be generated can be stored in the data block information to be generated corresponding to each data block to be generated.

[0104] According to an embodiment of the present invention, the to-be-generated data block information corresponding to each to-be-generated data block can be embedded into the task generation script, thereby generating a to-be-processed task corresponding to each to-be-generated data block information.

[0105] According to an embodiment of the present invention, the data to be processed is divided into multiple data block storage tasks of the same size to be stored, and the data blocks of the same size are determined based on the number of data rows of each of the multiple data to be processed and the preset number of data rows of the data blocks to be generated, so that the task amount of each task to be processed is the same, and each time the task to be processed is processed in parallel, the task processing time is equal to the time of one processing.

[0106] According to an embodiment of the present invention, by dividing the storage tasks of multiple data to be processed into the generation tasks of multiple data blocks, fast parallel processing of multiple data to be processed can be achieved through multiple available threads, and because the data blocks can be stored in columns, the processing speed of the data to be processed is also accelerated to a certain extent.

[0107] According to an embodiment of the present invention, the above-mentioned task processing method for a database may further include the following operations.

[0108] Based on the allocation method for the multiple tasks to be processed, available threads corresponding to the multiple tasks to be processed are called to process the tasks to be processed.

[0109] According to an embodiment of the present invention, based on the processing method for multiple pending tasks, multiple pending tasks or sliced ​​pending tasks can be assigned to the task queues of corresponding available threads, so that each available thread can process multiple pending tasks or sliced ​​tasks of pending tasks according to the order of tasks in its task queue, and realize parallel processing of multiple available threads.

[0110] According to an embodiment of the present invention, for each available thread, in the process of processing the task to be processed, the data to be processed can be obtained from the database based on the data index of the data to be processed included in the task to be processed and processed.

[0111] According to an embodiment of the present invention, by using the data index as key information for processing the data to be processed, the memory overhead caused by the task information of the task to be processed can be reduced.

[0112] Figure 3 A flowchart of a task processing method for a database according to another embodiment of the present invention is schematically shown.

[0113] like Figure 3 As shown, the method includes operations S301 to S308.

[0114] In operation S301 , the number of target tasks and the number of remaining tasks are determined based on the number of tasks to be processed and the number of available threads.

[0115] According to an embodiment of the present invention, the number of tasks to be processed and the number of available threads may be divided, so that a result is the number of target tasks, and the remainder is the number of remaining tasks.

[0116] In operation S302, it is determined whether there is a historical simulation allocation scenario that is the same as the current simulation allocation scenario. If not, operations S303 to S305 are performed. If so, operation S306 is performed.

[0117] According to an embodiment of the present invention, it can be determined from the relationship table between simulation allocation scenarios and task processing time whether there is a historical simulation allocation scenario that is the same as the number of remaining tasks and the number of available threads, that is, the current simulation allocation scenario.

[0118] In operation S303, at least one remaining task is respectively assigned to respective target threads in a simulated manner to obtain a first simulated assignment result.

[0119] In operation S304, for each remaining task, a plurality of first remaining slice tasks obtained by dividing the remaining task based on the number of available threads are respectively simulated and allocated to respective available threads to obtain a second simulated allocation result.

[0120] In operation S305, in response to the number of available threads being in a multiple relationship with the number of remaining tasks, the remaining tasks divided based on the multiple relationship are respectively simulated and allocated to respective available threads to obtain a third simulated allocation result.

[0121] According to an embodiment of the present invention, the target task and the remaining tasks may be determined from a plurality of tasks to be processed according to the number of the target tasks and the number of the remaining tasks.

[0122] In operation S306 , a simulation allocation result with the shortest task processing time is determined from the multiple simulation allocation results as an allocation method for the remaining tasks.

[0123] In operation S307 , the plurality of target tasks are evenly distributed to the plurality of available threads to obtain a distribution method for the target tasks, wherein each available thread processes the same number of target tasks.

[0124] In operation S308 , based on the allocation method for the target task and the allocation method for the remaining tasks, an allocation method for the plurality of tasks to be processed is obtained.

[0125] Based on the above-mentioned task processing method for a database, the present invention also provides a task processing device for a database. Figure 4 The device is described in detail.

[0126] Figure 4The structure block diagram of the task processing device for a database according to an embodiment of the present invention is schematically shown.

[0127] like Figure 4 As shown, the task processing device 400 for a database in this embodiment includes a first mode determination module 410 , a simulation module 420 , a time consumption determination module 430 , a second mode determination module 440 and a third mode determination module 450 .

[0128] The first mode determination module 410 is used to evenly distribute multiple target tasks determined from multiple tasks to be processed to multiple available threads to obtain a distribution mode for the multiple target tasks, wherein the target tasks are tasks to be processed in parallel by multiple available threads.

[0129] The simulation module 420 is used to simulate and allocate the remaining tasks to multiple available threads based on the number of remaining tasks and the number of available threads to obtain multiple simulation allocation results when it is determined that there are remaining tasks other than the target task among the multiple tasks to be processed.

[0130] The time consumption determination module 430 is used to determine the task processing time consumption of each of the multiple simulation allocation results based on a preset preprocessing time and a preset task processing time.

[0131] The second method determination module 440 is used to use the simulation allocation result with the shortest task processing time among the multiple simulation allocation results as the allocation method for the remaining tasks.

[0132] The third mode determination module 450 is used to obtain the allocation mode for the plurality of tasks to be processed based on the allocation modes for the plurality of target tasks and the allocation modes for the remaining tasks.

[0133] According to an embodiment of the present invention, there is at least one remaining task. The plurality of simulation allocation results include a first simulation allocation result, a second simulation allocation result and a third simulation allocation result. The simulation module 420 includes: a first simulation submodule, a second simulation submodule and a third simulation submodule.

[0134] The first simulation submodule is used to simulate and allocate at least one remaining task to respective target threads to obtain a first simulation allocation result, wherein the remaining tasks and the target threads are in a one-to-one correspondence and the target thread is determined from multiple available threads.

[0135] The second simulation submodule is used to simulate and allocate, for each remaining task, a plurality of first remaining slice tasks obtained by dividing the remaining task based on the number of available threads to respective available threads, to obtain a second simulation allocation result.

[0136] The third simulation submodule is used for simulating and allocating the remaining tasks divided based on the multiple relationship to the respective available threads in response to the number of available threads being in multiple relationship with the number of remaining tasks, to obtain a third simulation allocation result.

[0137] According to an embodiment of the present invention, the first simulation submodule includes: a thread determination unit and a first allocation unit.

[0138] The thread determination unit is used to determine a target thread from a plurality of available threads based on the number of remaining tasks, wherein the number of the target threads is the same as the number of remaining tasks.

[0139] The first allocation unit is used to simulate and allocate a target thread to each remaining task to obtain a first simulation allocation result.

[0140] According to an embodiment of the present invention, the second simulation submodule includes: a first segmentation unit and a second allocation unit.

[0141] The first segmentation unit is used to segment each remaining task based on the number of available threads to obtain a plurality of first remaining slice tasks, wherein the number of the first remaining slice tasks is equal to the number of available threads.

[0142] The second allocation unit is used to simulate and allocate an available thread to each of the first remaining slice tasks to obtain a second simulation allocation result.

[0143] According to an embodiment of the present invention, the third simulation submodule includes: a second segmentation unit and a third allocation unit.

[0144] The second splitting unit is used to split each remaining task based on a multiple relationship to obtain a second remaining fragment task for each of the at least one remaining tasks.

[0145] The third allocation unit is used to simulate and allocate an available thread to each of the second remaining slice tasks to obtain a third simulation allocation result.

[0146] According to an embodiment of the present invention, the time consumption determination module includes: a parallel times determination submodule, a duration determination submodule and a time consumption determination submodule.

[0147] The parallel times determination submodule is used to determine the parallel times for each simulation allocation result based on the simulation allocation result.

[0148] The duration determination submodule is used to obtain the target task processing duration based on the number of task slices of the remaining tasks in the simulation allocation result and the preset task processing duration.

[0149] The time-consuming determination submodule is used to obtain the task processing time of the simulation allocation result based on the number of parallel times, the target task processing time and the preset preprocessing time.

[0150] According to an embodiment of the present invention, the task processing device 400 for a database further includes: an information generation module and a task generation module.

[0151] The information generation module is used to determine the information of multiple data blocks to be generated based on the number of data rows of each of the multiple data to be processed in the database, the preset number of data rows of the data block to be generated, and the data index corresponding to each of the multiple data to be processed, wherein the data index is used to find the data to be processed in the database.

[0152] The task generation module is used to input information of multiple data blocks to be generated into a task generation template to obtain multiple tasks to be processed.

[0153] According to an embodiment of the present invention, the task processing device 400 for a database further includes: a task processing module.

[0154] The task processing module is used to call available threads corresponding to the plurality of tasks to be processed to process the tasks to be processed based on the allocation method for the plurality of tasks to be processed.

[0155] According to an embodiment of the present invention, any multiple modules of the first mode determination module 410, the simulation module 420, the time-consuming determination module 430, the second mode determination module 440 and the third mode determination module 450 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first mode determination module 410, the simulation module 420, the time-consuming determination module 430, the second mode determination module 440 and the third mode determination module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the first mode determination module 410, the simulation module 420, the time consumption determination module 430, the second mode determination module 440 and the third mode determination module 450 can be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function can be executed.

[0156] Figure 5A block diagram of an electronic device suitable for implementing a task processing method for a database according to an embodiment of the present invention is schematically shown.

[0157] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0158] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0159] According to an embodiment of the present invention, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that the computer program read therefrom is installed into the storage portion 508 as needed.

[0160] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0161] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0162] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the task processing method for a database provided by the embodiment of the present invention.

[0163] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 501. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0164] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0165] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0166] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0167] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0169] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A task processing method for a database, characterized in that: The method comprises: Allocating a plurality of target tasks determined from a plurality of pending tasks to a plurality of available threads to obtain an allocation method for the plurality of target tasks, wherein the target tasks are the pending tasks processed in parallel by the plurality of available threads; When it is determined that there are remaining tasks other than the target task among the plurality of tasks to be processed, based on the number of the remaining tasks and the number of the available threads, the remaining tasks are simulated and allocated to the plurality of the available threads to obtain a plurality of simulated allocation results; Determining the task processing time of each of the plurality of simulation allocation results based on a preset preprocessing time and a preset task processing time; Using the simulation allocation result with the shortest task processing time among the plurality of simulation allocation results as the allocation method for the remaining tasks; Based on the allocation methods for the plurality of target tasks and the allocation methods for the remaining tasks, an allocation method for the plurality of tasks to be processed is obtained; Wherein, the remaining task is at least one; the plurality of simulation allocation results include a first simulation allocation result, a second simulation allocation result and a third simulation allocation result; The simulating allocation of the remaining tasks to the plurality of available threads based on the number of the remaining tasks and the number of the available threads to obtain a plurality of simulation allocation results includes: Simulate and allocate at least one of the remaining tasks to respective target threads to obtain the first simulation allocation result, wherein the remaining tasks and the target threads are in a one-to-one correspondence, and the target thread is determined from the multiple available threads; For each of the remaining tasks, a plurality of first remaining slice tasks obtained by dividing the remaining tasks based on the number of the available threads are respectively simulated and allocated to respective available threads to obtain the second simulated allocation result; In response to the fact that the number of the available threads is in a multiple relationship with the number of the remaining tasks, the remaining tasks divided based on the multiple relationship are respectively simulated and allocated to the respective available threads to obtain the third simulated allocation result.

2. The method according to claim 1, characterized in that The step of simulating and allocating at least one of the remaining tasks to respective target threads to obtain the first simulation allocation result includes: Based on the number of the remaining tasks, determining the target thread from the plurality of available threads, wherein the number of the target threads is the same as the number of the remaining tasks; A target thread is respectively assigned to each of the remaining tasks in a simulated manner to obtain the first simulated assignment result.

3. The method according to claim 1, characterized in that The plurality of first remaining slice tasks obtained by dividing the remaining tasks based on the number of available threads are respectively simulated and allocated to respective available threads to obtain the second simulation allocation result, including: For each of the remaining tasks, Divide the remaining tasks based on the number of available threads to obtain a plurality of the first remaining slice tasks, wherein the number of the first remaining slice tasks is equal to the number of available threads; Simulate and allocate one of the available threads to each of the first remaining slice tasks respectively to obtain the second simulation allocation result.

4. The method according to claim 1, characterized in that: The remaining tasks divided based on the multiple relationship are respectively simulated and allocated to respective available threads to obtain the third simulation allocation result, including: Splitting each of the remaining tasks based on the multiple relationship to obtain a second remaining fragment task for at least one of the remaining tasks; Simulate and allocate one of the available threads to each of the second remaining slice tasks respectively to obtain the third simulation allocation result.

5. The method according to claim 1, characterized in that The determining of the task processing time of each of the plurality of simulation allocation results based on the preset preprocessing time and the preset task processing time comprises: For each of the simulated allocation results, Based on the simulation allocation result, determining the number of parallel times; Based on the number of task slices of the remaining tasks in the simulation allocation result and the preset task processing duration, a target task processing duration is obtained; The task processing time consumption of the simulation allocation result is obtained based on the parallel number, the target task processing time and the preset preprocessing time.

6. The method according to claim 1, characterized in that The method further comprises: Determine information of multiple data blocks to be generated based on the number of data rows of each of the multiple data to be processed in the database, the preset number of data rows of the data block to be generated, and the data index corresponding to each of the multiple data to be processed, wherein the data index is used to search the data to be processed in the database; Input the information of the plurality of data blocks to be generated into a task generation template to obtain the plurality of tasks to be processed.

7. The method according to claim 1, characterized in that The method further comprises: Based on the allocation method for the plurality of tasks to be processed, available threads corresponding to the plurality of tasks to be processed are called to process the tasks to be processed.

8. A task processing device for a database, characterized in that: The device comprises: A first mode determination module is used to evenly distribute a plurality of target tasks determined from a plurality of pending tasks to a plurality of available threads, and obtain a distribution mode for the plurality of target tasks, wherein the target tasks are the pending tasks processed in parallel by the plurality of available threads, and the number of the target tasks processed by each of the available threads is the same; A simulation module, configured to, when it is determined that there are remaining tasks other than the target task among the plurality of tasks to be processed, simulate allocating the remaining tasks to the plurality of available threads based on the number of the remaining tasks and the number of the available threads, to obtain a plurality of simulation allocation results; A time-consuming determination module, used to determine the task processing time of each of the plurality of simulation allocation results based on a preset preprocessing time and a preset task processing time; A second mode determination module is used to use the simulation allocation result with the shortest task processing time among the plurality of simulation allocation results as the allocation mode for the remaining tasks; A third mode determination module, used to obtain an allocation mode for the plurality of tasks to be processed based on the allocation modes for the plurality of target tasks and the allocation modes for the remaining tasks; There is at least one remaining task; the plurality of simulation allocation results include a first simulation allocation result, a second simulation allocation result and a third simulation allocation result; the simulation module includes a first simulation submodule, a second simulation submodule and a third simulation submodule; The first simulation submodule is used to simulate and allocate at least one of the remaining tasks to respective target threads to obtain the first simulation allocation result, wherein the remaining tasks and the target threads are in a one-to-one correspondence, and the target thread is determined from the multiple available threads; The second simulation submodule is used to simulate and allocate, for each of the remaining tasks, a plurality of first remaining slice tasks obtained by dividing the remaining tasks based on the number of available threads, to respective available threads to obtain the second simulation allocation result; The third simulation submodule is used for simulating and allocating the remaining tasks divided based on the multiple relationship to the respective available threads in response to the number of the available threads being in a multiple relationship to obtain the third simulation allocation result.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

Images