Task execution method and device, electronic equipment and storage medium

Abstract

The invention relates to a task execution method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a to-be-processed target task of a server cluster; according to a preset segmentation rule, segmenting the target task to obtain a to-be-executed target sub-task of each server in the server cluster; predicting an execution score of each server when the server executes the corresponding target sub-task to obtain a prediction score set corresponding to the server cluster; and when the predicted score set satisfies a preset condition, adjusting a segmentation rule according to the predicted score set, so as to re-execute segmentation of the target task based on the adjusted segmentation rule. Under the condition that the performance of the server is abnormal, the task segmentation rule executed by the server cluster is adjusted in time, and compared with current static load configuration, the response speed of the service is remarkably improved; besides, when the performance of the server is reduced, the task is transferred to the server of the same type, so that the task execution speed is prevented from being reduced, and the task execution efficiency is further improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task execution method, device, electronic device and storage medium. Background Art

[0002] When executing tasks, there are usually multiple servers working together to provide services. The performance of each server varies. Existing load balancing strategies often regard these servers as equivalent and implement a unified task allocation plan for the server cluster.

[0003] However, the performance of each server is not the same, and the unified task allocation scheme ignores the differences in server performance. If the performance of a certain type of server gradually declines, the load balancing component cannot transfer the load of the server with the degraded performance to the server with the same performance, and cannot dynamically adjust the task division of the server according to the changes in the server. The execution speed of the tasks undertaken by the server with abnormally degraded performance will slow down, thereby affecting the execution efficiency of the entire target task.

[0004] Therefore, there is an urgent need to develop a task execution method, device, electronic device and storage medium to solve one or more of the above-mentioned problems. Summary of the invention

[0005] In view of this, in order to solve the above technical problems or part of the technical problems, an embodiment of the present invention provides a task execution method, device, electronic device and storage medium.

[0006] In a first aspect, the present application provides a task execution method, the method comprising:

[0007] Get the target tasks to be processed by the server cluster;

[0008] According to a preset segmentation rule, the target task is segmented to obtain a target subtask to be executed by each server in the server cluster;

[0009] Predicting the execution score of each server when executing the corresponding target subtask, and obtaining a set of prediction scores corresponding to the server cluster;

[0010] When the prediction score set meets a preset condition, the segmentation rule is adjusted according to the prediction score set, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

[0011] In a possible implementation, predicting the execution score of each server when executing the corresponding target subtask includes:

[0012] Query the regional information of each server and the execution parameters of each server to execute the corresponding target subtask;

[0013] The regional information and corresponding execution parameters of each server are input into the multidimensional prediction model to obtain the execution score of each server when executing the corresponding target subtask.

[0014] In a possible implementation manner, the region information includes a primary region, a secondary region, and a tertiary region where the server is located;

[0015] The method for constructing the multidimensional prediction model comprises:

[0016] Obtain prediction rules for each level of regions where the server is located, respectively, wherein the prediction rules are weight allocation rules for executing parameters in the process of executing score prediction by the server;

[0017] Using the prediction rules of the regions at each level, respectively construct a first prediction sub-model, a second prediction sub-model, and a third prediction sub-model;

[0018] The first prediction sub-model, the second prediction sub-model, and the third prediction sub-model are combined to obtain a multi-dimensional prediction model.

[0019] In a possible implementation, adjusting the segmentation rule according to the prediction score set includes:

[0020] According to the set of prediction scores, the server cluster is divided into a first server sub-cluster and a second server sub-cluster, wherein the first server sub-cluster includes a first type of servers whose execution scores are lower than a normal execution threshold, and the second server sub-cluster includes a second type of servers whose execution scores are higher than or equal to the normal execution threshold;

[0021] Determine a server to be adjusted corresponding to each of the first-category servers in the first server sub-cluster in the second server sub-cluster;

[0022] Determine the split adjustment strategy for each of the first-class servers according to the execution score of each of the first-class servers corresponding to the server to be adjusted;

[0023] The segmentation rule is adjusted using the segmentation adjustment strategy of each type of server in the first server sub-cluster.

[0024] In a possible implementation, the method for determining the segmentation rule includes:

[0025] Get the server type of each server in the server cluster, as well as the corresponding number of servers of each type;

[0026] According to the server types and the number of servers of each type, a division rule of the target task is determined so that the target task is divided into a set of target subtasks corresponding to the server cluster.

[0027] In a possible implementation manner, determining the server to be adjusted corresponding to each of the first-category servers in the first server sub-cluster in the second server sub-cluster includes:

[0028] Establishing a mapping relationship between the first server and the second server according to the server type of each server in the server cluster;

[0029] By using the mapping relationship, a second-category server corresponding to each first-category server in the first server sub-cluster in the second server sub-cluster is determined as a server to be adjusted corresponding to each first-category server.

[0030] In a possible implementation, the method further includes:

[0031] According to the adjusted segmentation rules, re-predict the execution scores of a type of servers when executing the corresponding target subtasks;

[0032] When the execution score is lower than the normal execution threshold, the step of determining the partitioning adjustment strategy between each of the first-class servers and the corresponding adjustment servers is performed again.

[0033] In a second aspect, the present application provides a data processing device, the device comprising:

[0034] An acquisition module is used to obtain target tasks to be processed by the server cluster;

[0035] A segmentation module, used to segment the target task according to a preset segmentation rule to obtain a target subtask to be executed by each server in the server cluster;

[0036] A prediction module is used to predict the execution score of each server when executing the corresponding target subtask, and obtain a set of prediction scores corresponding to the server cluster;

[0037] An adjustment module is used to adjust the segmentation rule according to the prediction score set when the prediction score set meets a preset condition, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

[0038] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data processing method described in any one of the embodiments of the first aspect when executing the computer program.

[0039] In a fourth aspect, the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data processing method described in any embodiment of the first aspect are implemented.

[0040] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the task execution method provided by the embodiment of the present application introduces a dynamically adjusted load balancing strategy during the task execution process, evaluates the performance of the server by periodically obtaining the execution score of the server, and timely adjusts the task segmentation rules executed by the server cluster in the event of abnormal server performance, which significantly improves the response speed of the service compared to the current static load configuration; in addition, when the server performance declines, the task is transferred to a server of the same type, thereby preventing the task execution speed from slowing down and further improving the efficiency of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0044] Figure 1 A flowchart of a task execution method provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of another task execution method provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a process for executing score prediction provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of a task execution device provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0050] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0051] In order to solve the technical problem that multiple servers work together but each server has different performance in the prior art, the existing task allocation scheme ignores the difference in server performance, and the load balancing component cannot transfer the load of the server with degraded performance to the server with the same performance, and thus cannot dynamically adjust the task division of the server according to the changes in the server, which makes the task execution speed undertaken by the server with abnormally degraded performance slow down, thereby affecting the execution efficiency of the entire target task. The present application provides a task execution method, device, electronic device and storage medium, which introduces a dynamic adjustment load balancing strategy during the task execution process, and timely adjusts the task division rules executed by the server cluster by periodically obtaining the execution score of the server, which significantly improves the response speed of the service compared with the current static load configuration; and transfers tasks to servers of the same type, thereby preventing the task execution speed from slowing down, further improving the efficiency of task execution.

[0052] Figure 1 A flowchart of a task execution method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method specifically includes:

[0053] S101, obtaining target tasks to be processed by the server cluster;

[0054] Server performance affects service response time. Servers with better performance and higher configuration can provide faster and more services. Usually, there are many servers providing target task services to be processed at the same time, and the performance of each server may not be exactly the same.

[0055] In this embodiment, in order to ensure the efficient operation of the system, unfinished pending tasks are periodically extracted from the server cluster. The pending tasks may include data processing, file transfer or other operations that need to be performed. By obtaining the target tasks to be processed, server resources are arranged for the target tasks, and the task processing process is optimized to ensure that the tasks can be completed in a timely manner.

[0056] S102, dividing the target task according to a preset division rule to obtain a target subtask to be executed by each server in the server cluster;

[0057] In this embodiment, due to the different performance of each server, task segmentation rules are specified according to the differences in server performance. According to the pre-set segmentation rules, the described target tasks are divided in detail to obtain specific target subtasks that each server in the server cluster needs to execute.

[0058] S103, predicting the execution score of each server when executing the corresponding target subtask, and obtaining a set of prediction scores corresponding to the server cluster;

[0059] The server can provide a predicted execution score. Based on the collected predicted execution score set of the server cluster, more tasks can be assigned to servers with higher execution scores, so that "the more capable, the more work", thereby improving the efficiency of task execution.

[0060] In this embodiment, the execution efficiency of each server when executing its corresponding target subtask is predicted, and the execution score of each server when completing its specific task is calculated. The specific prediction process involves a detailed analysis and evaluation of the server performance to ensure the accuracy of the prediction results. Through the prediction method, a set of prediction scores for the entire server cluster when executing the task is obtained, thereby providing an important reference basis for task scheduling and resource allocation, optimizing the resource utilization of the server cluster, and improving the operation efficiency of the task.

[0061] Specifically, the prediction score is based on a series of algorithms and models, which can comprehensively consider various factors, such as the server's response time, processing power, resource utilization, etc., to evaluate the server's task execution performance in the future.

[0062] S104: When the prediction score set meets a preset condition, adjust the segmentation rule according to the prediction score set, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

[0063] In this embodiment, when the prediction score set mentioned above reaches or satisfies the pre-set conditions, it is necessary to make corresponding adjustments according to the prediction score set, modify and optimize the existing segmentation rules, so that the execution of the target task can adapt to the performance of the server cluster. This makes the segmentation of the target task more compatible with the current server cluster, thereby improving the execution effect and quality of the overall task, and ensuring that when facing different prediction score sets, the segmentation strategy can be flexibly adjusted to adapt to the changing task requirements and environmental changes.

[0064] It should be noted that the preset conditions in this embodiment can be varied, for example, when there are significant performance differences in the prediction score set, or when the prediction score of a server is lower than a preset threshold, or when the average prediction score of the entire server cluster is lower than a certain standard; the setting of specific preset conditions depends on the actual application scenario and requirements, and is not fixed, so as to ensure that the task execution method can flexibly respond to various situations.

[0065] The task execution method provided in this embodiment first obtains the target task to be processed from the server cluster. In order to improve the efficiency of task processing, the target task is split and processed, and the split subtasks are assigned to different servers for processing. After the splitting process, the execution score of each server when executing the target subtask is predicted, and the execution score is used to reflect the ability and efficiency of each server in processing tasks. Finally, based on the predicted execution score, the segmentation rule of the target task is readjusted to ensure that the segmentation of the target task is more in line with the actual processing capacity of the current server cluster, so that the task allocation is more reasonable and efficient, which significantly improves the execution effect and quality of the overall task and makes the task processing faster and more efficient. Compared with the traditional static load configuration, the task execution method can better adapt to the changes of the server cluster through real-time monitoring and dynamic adjustment, thereby improving the response speed of the service and the efficiency of task processing.

[0066] A unified computing center is needed to collect and calculate the indicator data output by multiple servers, and then process it through multiple dimensions to finally summarize the scoring results of various indicators such as single server, single computer room, and single region. The specific calculation method can adopt a weighted average algorithm, where indicators of different levels have different weights, and the weights of the same indicator of different services can also be different. Finally, the scoring results are mapped into multi-dimensional weight reference data of different servers, different computer rooms, and different regions, which are processed by module three.

[0067] In an optional solution of the embodiment of the present invention, the predicting the execution score of each server when executing the corresponding target subtask includes:

[0068] Query the regional information of each server and the execution parameters of each server to execute the corresponding target subtask;

[0069] In this embodiment, the server should reasonably output some A-level indicators and B-level indicators according to the characteristic requirements of different services. A-level indicators mainly refer to indicators that can directly affect the service response time, such as CPU utilization and the number of requests hitting the cache per unit time; B-level indicators refer to indicators that indirectly affect the service response time, such as memory utilization, the remaining size of the connection pool, etc.

[0070] Specifically, the server's regional information covers three main dimensions, namely, region, computer room, and server's IP address; specifically, region refers to the geographical location of the server, such as a country, province or city; computer room refers to the specific data center where the server is stored, which usually has good physical security measures and stable power supply; the server's IP address is the unique identifier of the server in the network, which is used for data transmission and communication. The three dimensions together constitute the server's regional information, helping users better locate and manage server resources.

[0071] For example, when scoring the server IP dimension in the pending tasks with more reads than writes: Since the CPU has a greater impact on service performance, the CPU utilization accounts for 30%; since memory utilization generally has a lower priority than CPU utilization, the memory utilization accounts for 25%; since the disk mainly affects the response speed, the disk IO accounts for 20%, and other parameters account for 25%. When scoring the computer room dimension, the host restart rate is 30%, the packet loss rate is 25%, the average response time is 20%, and other parameters are 25%.

[0072] For services with more writes than reads, the weights of the service indicators may be greater than the disk IO than the memory utilization, the average response time than the restart rate, etc.

[0073] The weights of indicators in different scenarios are different. In addition, indicators of different levels are generally AB as described above. Level C generally includes restart rate and thread utilization rate, and level D includes disk fragmentation rate, cache hit rate, etc.

[0074] In this embodiment, the regional information of the three dimensions of the query server and the execution parameters of each server in executing the corresponding target task are

[0075] In this embodiment, the regional information of the server in three different dimensions is queried, and the specific execution parameters used by each server when executing its corresponding target task are analyzed to better understand the working status and performance of the server and provide data support for optimizing system performance and resource management.

[0076] Specifically, the execution parameters of the target task can be indicators that directly affect the server response time, such as CPU utilization, the number of requests hitting the cache per unit time, etc.; or indicators that simply affect the service response time, such as memory utilization, the remaining size of the connection pool, etc.

[0077] The regional information and corresponding execution parameters of each server are input into the multidimensional prediction model to obtain the execution score of each server when executing the corresponding target subtask.

[0078] In this embodiment, the regional location information of each server and the execution parameters related thereto are input into a multidimensional prediction model to obtain the execution score of each server when executing the corresponding target subtask. Various execution factors are comprehensively considered to accurately evaluate the performance of each server when executing a specific subtask, thereby providing data support for task scheduling and resource optimization.

[0079] In addition, the scoring calculation results are stored in Redis or other storage components. The load balancing component reads the server scoring results and adjusts the weights of each machine based on the scoring results. A dynamic weighted round-robin algorithm can be used to assign higher weights to servers with higher scores, but this weight is also subject to an upper limit (a possible upper limit is that the QPS of a single server cannot exceed 1000). Other computer rooms and regional dimensions can also refer to this strategy to dynamically adjust weights. However, the computer room-level load adopts a weighted least link algorithm to superimpose the computer room scoring results, and prioritizes assigning high weights to computer rooms with the least connections and the highest scores in the same region.

[0080] The execution score prediction method provided in this embodiment provides data support for task scheduling and resource optimization, which not only improves the accuracy of task allocation, but also can dynamically adjust the task load according to the actual working status and performance of the server, thereby ensuring the efficient operation of the server cluster.

[0081] When the same service is deployed on different servers, different computer rooms or different regions, the various service performance indicators (such as response time) are inconsistent. Therefore, we want to summarize and score the collected indicators and correspond them to these three regional dimensions, so that we can more intuitively map the performance of the same service on different servers, different computer rooms and different regions.

[0082] In an optional solution of the embodiment of the present invention, the region information includes the primary region, the secondary region and the tertiary region where the server is located;

[0083] In this embodiment, the first-level area is the geographical location of the server, such as a province or city; the second-level area is the computer room location, which refers to the specific data center location of the server; the third-level area is the IP of the server, which is the unique identifier of the server in the network.

[0084] The method for constructing the multidimensional prediction model comprises:

[0085] Obtain prediction rules for each level of regions where the server is located, respectively, wherein the prediction rules are weight allocation rules for executing parameters in the process of executing score prediction by the server;

[0086] In this embodiment, corresponding prediction rules are extracted from regions at each level to achieve accurate prediction of server performance, wherein the prediction rules specifically include the execution parameters used in the region where the server is located when making score predictions, and the weight distribution principle of the execution parameters; then, by analyzing the regional characteristics of the region where the server is located, the prediction rules of the server are obtained, and the prediction rules are used to provide a basis for the subsequent establishment of a prediction model.

[0087] Using the prediction rules of the regions at each level, respectively construct a first prediction sub-model, a second prediction sub-model, and a third prediction sub-model;

[0088] In this embodiment, according to the prediction rules of the first-level region, the second-level region and the third-level region, three different levels of prediction sub-models are constructed respectively. The first prediction sub-model is called the first prediction sub-model, which is mainly responsible for processing the prediction tasks related to the first region; the second prediction sub-model, namely the second prediction sub-model, is responsible for the prediction needs of the second region; the third prediction sub-model is mainly responsible for the prediction tasks of the third region. These three sub-models respectively allocate execution parameter weights for their respective regions, and construct prediction models to ensure that the most accurate prediction results can be provided in their respective fields.

[0089] By establishing three prediction sub-models, we can more accurately identify performance defects and optimization points, and then make targeted improvements. For example, if the server in a certain area performs poorly, its performance can be improved by adjusting parameter weights or optimizing server configurations. Similarly, if the network latency of a certain computer room is high, corresponding measures can be taken to reduce latency and improve the overall service quality, which can not only provide a comprehensive performance evaluation, but also provide a clear direction and basis for subsequent optimization work.

[0090] The first prediction sub-model, the second prediction sub-model, and the third prediction sub-model are combined to obtain a multi-dimensional prediction model.

[0091] The method for constructing a multi-dimensional prediction model provided in this embodiment first collects and analyzes relevant data of various servers, computer rooms and regions, and then assigns corresponding weights according to the different characteristics and importance of these data; in this way, the data of various dimensions can be effectively integrated and balanced; then, a comprehensive prediction is made using the weighted data to generate a prediction execution score, which can intuitively reflect the performance of the same service in different servers, different computer rooms and different regions, and better understand the differences in service performance in different environments, thereby providing strong data support for optimization and adjustment;

[0092] Comprehensive prediction and scoring are performed through the weights of multi-dimensional parameters such as different servers, different computer rooms, and different regions, so that the final predicted execution score can be more intuitively mapped to the performance of the same service on different servers, different computer rooms, and different regions; prediction and scoring are performed by comprehensively considering the weights of multi-dimensional parameters such as different servers, different computer rooms, and different regions.

[0093] Figure 2 A flowchart of a task execution method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the method specifically includes:

[0094] S201. Divide the server cluster into a first server sub-cluster and a second server sub-cluster according to the prediction score set, wherein the first server sub-cluster includes a first type of servers whose execution scores are lower than a normal execution threshold, and the second server sub-cluster includes a second type of servers whose execution scores are higher than or equal to the normal execution threshold;

[0095] In this embodiment, after obtaining the prediction score set, the normal execution threshold is first set according to the distribution of the execution scores in the prediction score set, and then the server cluster is divided into a first type of server whose execution score is lower than the normal execution threshold, that is, it is predicted that it will not be able to execute normally in the future, and a second type of server whose execution score is higher than or equal to the normal execution threshold, that is, it is predicted that it can execute normally in the future, according to the prediction score of each server.

[0096] It should be noted that the setting of the normal execution threshold can be based on a set of prediction scores, for example, the median, average or specific percentile in the prediction score set is used as the threshold; it can also be determined based on historical data and statistical analysis. For example, the average execution score of the server cluster over the past period of time can be calculated, and a reasonable threshold range can be set based on this average value. Servers below this range may need to adjust optimization measures, while servers above or equal to this range indicate that their performance is relatively good and does not require adjustment or optimization.

[0097] S202, determining a server to be adjusted corresponding to each of the first-category servers in the first server sub-cluster in the second server sub-cluster;

[0098] In this embodiment, the server types and performance parameters of the two server sub-clusters are analyzed and compared, and the server to be adjusted corresponding to each type of server in the first server sub-cluster in the second server sub-cluster is determined;

[0099] Specifically, firstly, the specific functions and characteristics of each server in the first server sub-cluster are clarified so as to find a matching server to be adjusted in the second server sub-cluster; then, by comparing the hardware configuration, software environment, network connection and other information of the servers in the two sub-clusters, it is ensured that the selected server to be adjusted can meet the requirements of the corresponding type of server in the first server sub-cluster.

[0100] In addition, when selecting the server to be adjusted, the server load and resource usage also need to be considered to ensure that the adjustment process will not have a negative impact on the stability and performance of the existing system, thereby laying a solid foundation for subsequent adjustment work.

[0101] S203, determining a split adjustment strategy for each first-class server according to the execution score of each first-class server corresponding to the server to be adjusted;

[0102] In this embodiment, after determining the servers to be adjusted corresponding to the first type of servers in the second server sub-cluster, the predicted scores of the servers to be adjusted are determined, and the split adjustment strategy of the first type of servers is determined according to the predicted scores;

[0103] In this embodiment, after determining the servers to be adjusted corresponding to a type of server in the second server sub-cluster by analyzing the performance data, load conditions, and historical operation records of the server, the predicted scores of the servers to be adjusted are obtained; the split adjustment strategy for a type of server is determined based on the predicted scores, and the predicted scores of the servers, the current workload, and the overall goals of the system need to be considered in the process of formulating the split adjustment strategy. For example, if the predicted score of a server is low, indicating that it may have performance defects or failures in the future, the tasks to be executed assigned to the server can be separated and its load can be transferred to other servers with better performance, which can not only avoid potential performance problems, but also ensure the efficient operation of the entire server cluster.

[0104] It should be noted that the split adjustment strategy includes dividing a type of servers into different groups according to the predicted execution scores, and taking different adjustment measures for each group of servers; it may include resource reallocation, load balancing, fault migration, etc. The split adjustment strategy can ensure that the overall performance of the server cluster is optimized, while also improving the reliability and stability of the system.

[0105] S204: Use the segmentation adjustment strategy of each type of server in the first server sub-cluster to adjust the segmentation rule.

[0106] In this embodiment, the segmentation adjustment strategy implemented on each specific type of server in the first server sub-cluster is used to adjust and optimize the existing segmentation rules accordingly to ensure that the segmentation process is more efficient and reasonable, more effectively utilize server resources, and improve the overall performance and stability of the system.

[0107] The task execution method provided in this embodiment, after obtaining the prediction score, divides the server cluster by using the prediction score, determines the server to be adjusted corresponding to the first class of servers from the second server cluster after the division, and then formulates a task splitting adjustment strategy between the server to be adjusted and the corresponding first class of servers based on the prediction score of the server to be adjusted, and adjusts the initial splitting rule according to the splitting adjustment strategy; by using this method, the tasks to be executed allocated in the first class of servers are separated and their loads are transferred to other servers with better performance, which can not only avoid potential performance problems, but also ensure the efficient operation of the entire server cluster, and in addition, it can also achieve more effective utilization of server resources and improve the overall performance and stability of the system.

[0108] In an optional solution of the embodiment of the present invention, the method for determining the segmentation rule includes:

[0109] Get the server type of each server in the server cluster, as well as the corresponding number of servers of each type;

[0110] According to the server types and the number of servers of each type, a division rule of the target task is determined so that the target task is divided into a set of target subtasks corresponding to the server cluster.

[0111] In this embodiment, in order to effectively manage and allocate tasks, it is necessary to determine the specific type information of each server in the server cluster and count the number of servers of each type, so as to provide a basis for subsequent task allocation.

[0112] Specifically, traverse each server in the server cluster and record the server type information, such as high-performance computing server, storage server, or load balancing server. Then, count each type of server by writing a script or using an existing monitoring tool to determine the number of each type of server in the cluster;

[0113] After obtaining the server type and its number, determine the target task segmentation rule based on the server information, and formulate a detailed task segmentation rule based on the server type and number to ensure that the target task can be efficiently segmented into a set of target subtasks corresponding to the server cluster; reasonably split a larger target task into multiple subtasks so as to make full use of the computing resources of the server cluster.

[0114] For example, if there are a large number of high-performance computing servers in the cluster, computing-intensive tasks can be assigned to these servers; and for tasks that require a large amount of storage space, they can be assigned to storage servers first; through the above methods, it is ensured that each subtask can run on the most suitable server type, thereby making full use of the cluster's computing resources and improving the efficiency of task execution and overall performance.

[0115] In an optional solution of the embodiment of the present invention, determining the server to be adjusted corresponding to each type of server in the first server sub-cluster in the second server sub-cluster includes:

[0116] Establishing a mapping relationship between the first server and the second server according to the server type of each server in the server cluster;

[0117] By using the mapping relationship, a second-category server corresponding to each first-category server in the first server sub-cluster in the second server sub-cluster is determined as a server to be adjusted corresponding to each first-category server.

[0118] In this embodiment, a mapping relationship between servers is established based on the specific type of each server in the server cluster, and the first server is associated with the second server based on the mapping relationship; then, the mapping relationship is used to determine the specific type of second-class server corresponding to each specific type of first-class server in the first server subcluster in the second server subcluster, thereby determining the adjustment server corresponding to each first-class server.

[0119] In an optional solution of the embodiment of the present invention, the method further includes:

[0120] According to the adjusted segmentation rules, re-predict the execution scores of a type of servers when executing the corresponding target subtasks;

[0121] When the execution score is lower than the normal execution threshold, the step of determining the partitioning adjustment strategy between each of the first-class servers and the corresponding adjustment servers is performed again.

[0122] In this embodiment, according to the adjusted and optimized segmentation rules, it is necessary to re-evaluate and predict the execution score of a certain type of server when executing its corresponding target subtask, and use the execution score as an important indicator to measure server performance and efficiency, reflecting the performance of the server when processing specific tasks. When the execution score obtained by re-evaluation is lower than the set normal execution threshold, it indicates that the server has performance bottlenecks or low efficiency when executing tasks. In order to solve this problem, it is necessary to re-execute the previously determined steps, that is, for each type of server and its corresponding adjustment server, formulate and implement corresponding segmentation adjustment strategies; optimize the server's resource allocation and task processing flow to improve its execution score, ensure that the server can complete the task efficiently and stably, and ensure that the server meets the expected performance standards when executing tasks, thereby improving the operating efficiency and reliability of the overall system.

[0123] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] Figure 3 A schematic diagram of a process for executing score prediction provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, in this embodiment, when the task segmentation rule needs to be adjusted, the corresponding execution scores are first obtained from multiple dimensions; then, the execution scores of each server in the server cluster are summarized and processed in detail, and the segmentation rules are adjusted accordingly based on these summary results to optimize the effect of task segmentation; finally, the segmentation process of the target task is re-executed according to the adjusted segmentation rules to ensure that the task can be more efficiently allocated and executed in the server cluster.

[0125] Figure 4 A schematic diagram of a task execution device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the device specifically includes:

[0126] Acquisition module 401, used to acquire target tasks to be processed by the server cluster;

[0127] A segmentation module 402 is used to segment the target task according to a preset segmentation rule to obtain a target subtask to be executed by each server in the server cluster;

[0128] Prediction module 403, used to predict the execution score of each server when executing the corresponding target subtask, and obtain a set of prediction scores corresponding to the server cluster;

[0129] The adjustment module 404 is configured to adjust the segmentation rule according to the prediction score set when the prediction score set meets a preset condition, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

[0130] In a possible implementation, the prediction module 403 is further used to query the regional information of each server and the execution parameters of each server for executing the corresponding target subtask;

[0131] The regional information and corresponding execution parameters of each server are input into the multidimensional prediction model to obtain the execution score of each server when executing the corresponding target subtask.

[0132] In a possible implementation, the device further includes a construction module 405, which is used to respectively obtain prediction rules for each level of regions where the server is located, wherein the prediction rules are weight allocation rules for execution parameters of the server during the score prediction process; using the prediction rules for each level of regions, respectively construct a first prediction sub-model, a second prediction sub-model, and a third prediction sub-model; and merge the first prediction sub-model, the second prediction sub-model, and the third prediction sub-model to obtain a multidimensional prediction model.

[0133] In one possible implementation, the segmentation module 402 is also used to divide the server cluster into a first server sub-cluster and a second server sub-cluster based on a set of prediction scores, wherein the first server sub-cluster includes a type of server whose execution score is lower than a normal execution threshold, and the second server sub-cluster includes a type of server whose execution score is higher than or equal to the normal execution threshold; determine the server to be adjusted corresponding to each type of server in the first server sub-cluster in the second server sub-cluster; determine the segmentation adjustment strategy of each type of server according to the execution score of each type of server corresponding to the server to be adjusted; and adjust the segmentation rule using the segmentation adjustment strategy of each type of server in the first server sub-cluster.

[0134] In one possible implementation, the splitting module 402 is also used to obtain the server type of each server in the server cluster and the corresponding number of servers of each type; based on the server type and the corresponding number of servers of each type, determine the splitting rule of the target task so that the target task is split into a set of target subtasks corresponding to the server cluster.

[0135] In a possible implementation, the segmentation module 402 is also used to establish a mapping relationship between the first server and the second server based on the server type of each server in the server cluster; using the mapping relationship, determine the second-class server corresponding to each first-class server in the first server subcluster in the second server subcluster as the server to be adjusted corresponding to each first-class server.

[0136] In one possible embodiment, the device also includes a detection module 406, which is used to re-predict the execution score of a class of servers when executing the corresponding target subtask based on the adjusted segmentation rules; when the execution score is lower than the normal execution threshold, re-execute the step of determining the segmentation adjustment strategy for each class of servers and the corresponding adjustment server.

[0137] The task execution device provided in this embodiment may be as follows Figure 4 The task execution device shown in can execute the following Figure 1-3 All the steps of the task execution in Figure 1-3 The technical effects of the tasks shown are shown in the table. Figure 1-3 For the sake of brevity, the relevant description is not repeated here.

[0138] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, an embodiment of the present application provides an electronic device, including a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504; the memory 503 is used to store computer programs; the processor 501 is used to execute the program stored in the memory 503, and implements the steps of the task execution method provided by any of the aforementioned method embodiments: obtaining a target task to be processed by a server cluster; according to a preset segmentation rule, segmenting the target task to be executed by each server in the server cluster; predicting the execution score of each server when executing the corresponding target subtask, and obtaining a set of prediction scores corresponding to the server cluster; when the set of prediction scores meets a preset condition, adjusting the segmentation rule according to the set of prediction scores, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

[0140] In one possible implementation, the regional information of each server and the execution parameters of each server executing the corresponding target subtask are queried; the regional information of each server and the corresponding execution parameters are input into a multidimensional prediction model to obtain the execution score of each server when executing the corresponding target subtask.

[0141] In one possible implementation, prediction rules for each level of regions where the server is located are obtained respectively, and the prediction rules are weight allocation rules for execution parameters of the server during the score prediction process; using the prediction rules for each level of regions, a first prediction sub-model, a second prediction sub-model, and a third prediction sub-model are constructed respectively; the first prediction sub-model, the second prediction sub-model, and the third prediction sub-model are merged to obtain a multidimensional prediction model.

[0142] In one possible implementation, the server cluster is divided into a first server sub-cluster and a second server sub-cluster based on a set of prediction scores, wherein the first server sub-cluster includes a type of server whose execution score is lower than a normal execution threshold, and the second server sub-cluster includes a type of server whose execution score is higher than or equal to the normal execution threshold; the server to be adjusted corresponding to each type of server in the first server sub-cluster in the second server sub-cluster is determined; the split adjustment strategy of each type of server is determined separately according to the execution score of each type of server corresponding to the server to be adjusted; and the split adjustment strategy of each type of server in the first server sub-cluster is used to adjust the split rule.

[0143] In one possible implementation, the server type of each server in the server cluster and the corresponding number of servers of each type are obtained; based on the server type and the corresponding number of servers of each type, a splitting rule for the target task is determined so that the target task is split into a set of target subtasks corresponding to the server cluster.

[0144] In one possible implementation, a mapping relationship between a first server and a second server is established based on the server type of each server in the server cluster; and using the mapping relationship, a corresponding type-2 server in the second server subcluster to each type-1 server in the first server subcluster is determined as the server to be adjusted corresponding to each type-1 server.

[0145] In one possible implementation, based on the adjusted segmentation rule, the execution score of a class of servers when executing the corresponding target subtask is re-predicted; when the execution score is lower than the normal execution threshold, the step of determining the segmentation adjustment strategy for each class of servers and the corresponding adjustment server is re-executed.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0147] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0148] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A task execution method, characterized in that: include: Get the target tasks to be processed by the server cluster; According to a preset segmentation rule, the target task is segmented to obtain a target subtask to be executed by each server in the server cluster; Predicting the execution score of each server when executing the corresponding target subtask, and obtaining a set of prediction scores corresponding to the server cluster; When the prediction score set meets a preset condition, the segmentation rule is adjusted according to the prediction score set, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

2. The method according to claim 1, characterized in that The predicting of the execution score of each server when executing the corresponding target subtask includes: Query the regional information of each server and the execution parameters of each server to execute the corresponding target subtask; The regional information and corresponding execution parameters of each server are input into the multidimensional prediction model to obtain the execution score of each server when executing the corresponding target subtask.

3. The method according to claim 2, characterized in that The area information includes the primary area, secondary area and tertiary area where the server is located; The method for constructing the multidimensional prediction model comprises: Obtain prediction rules for each level of regions where the server is located, respectively, wherein the prediction rules are weight allocation rules for executing parameters in the process of executing score prediction by the server; Using the prediction rules of the regions at each level, respectively construct a first prediction sub-model, a second prediction sub-model, and a third prediction sub-model; The first prediction sub-model, the second prediction sub-model, and the third prediction sub-model are combined to obtain a multi-dimensional prediction model.

4. The method according to claim 1, characterized in that: The adjusting the segmentation rule according to the prediction score set includes: According to the set of prediction scores, the server cluster is divided into a first server sub-cluster and a second server sub-cluster, wherein the first server sub-cluster includes a first type of servers whose execution scores are lower than a normal execution threshold, and the second server sub-cluster includes a second type of servers whose execution scores are higher than or equal to the normal execution threshold; Determine a server to be adjusted corresponding to each of the first-category servers in the first server sub-cluster in the second server sub-cluster; Determine the split adjustment strategy for each of the first-class servers according to the execution score of each of the first-class servers corresponding to the server to be adjusted; The segmentation rule is adjusted using the segmentation adjustment strategy of each type of server in the first server sub-cluster.

5. The method according to claim 1, characterized in that The method for determining the segmentation rule includes: Get the server type of each server in the server cluster, as well as the corresponding number of servers of each type; According to the server types and the number of servers of each type, a division rule of the target task is determined so that the target task is divided into a set of target subtasks corresponding to the server cluster.

6. The method according to claim 4, characterized in that The step of determining the server to be adjusted corresponding to each of the first-category servers in the first server sub-cluster in the second server sub-cluster includes: Establishing a mapping relationship between the first server and the second server according to the server type of each server in the server cluster; By using the mapping relationship, a second-category server corresponding to each first-category server in the first server sub-cluster in the second server sub-cluster is determined as a server to be adjusted corresponding to each first-category server.

7. The method according to claim 4, characterized in that The method further comprises: According to the adjusted segmentation rules, re-predict the execution scores of a type of servers when executing the corresponding target subtasks; When the execution score is lower than the normal execution threshold, the step of determining the partitioning adjustment strategy between each of the first-class servers and the corresponding adjustment servers is performed again.

8. A data processing device, characterized in that: include: An acquisition module is used to obtain target tasks to be processed by the server cluster; A segmentation module, used to segment the target task according to a preset segmentation rule to obtain a target subtask to be executed by each server in the server cluster; A prediction module is used to predict the execution score of each server when executing the corresponding target subtask, and obtain a set of prediction scores corresponding to the server cluster; An adjustment module is used to adjust the segmentation rule according to the prediction score set when the prediction score set meets a preset condition, so as to re-execute the segmentation of the target task based on the adjusted segmentation rule.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the task execution method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the task execution method according to any one of claims 1 to 7 are implemented.

Images