Task processing method, device, electronic device, storage medium and program product

Abstract

The present disclosure provides a task processing method, device, electronic device, storage medium and program product. The method includes: receiving a remote shuffle service request of a target task, responding to a remote system service request of the target task, generating a topology map of the target task according to the execution process information of the target task, wherein the topology map includes multiple target nodes, and each target node corresponds to at least one subtask of the target task; matching the information of the target node with the information of the historical node to obtain a first matching result; the historical node is a node of the topology map corresponding to the historical task; the first matching result includes at least one target historical task in the historical tasks that matches the target task; based on preset attribute conditions, screening the target historical tasks; based on the screening results, determining the execution parameter recommendation information of the target task as the processing result of the remote shuffle service process. The embodiment of the present disclosure helps to more accurately determine the execution parameters of the target task.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a task processing method, device, electronic device, storage medium, and program product. Background Art

[0002] With the advancement of computer technology, big data, as a branch of computer science, has also continued to develop. Big data computing engines are used to perform computations on data in data systems. Within big data computing engines, there are numerous similar task processing processes, and the large volume of tasks handled by big data computing engines consumes significant resources. To execute tasks, big data computing engines allocate resources to these tasks. Properly allocating resources for task processing can improve the efficiency of task processing within big data computing engines. Summary of the Invention

[0003] In view of this, embodiments of the present disclosure provide a task processing method, apparatus, electronic device, storage medium, and program product, which can enable users of a data table to conveniently obtain statistical information related to the content of the data table.

[0004] According to some embodiments of the present disclosure, the task processing method includes:

[0005] Generate a topology graph of the target task according to the execution process information of the target task, wherein the topology graph includes a plurality of target nodes, each target node corresponding to at least one subtask of the target task;

[0006] Matching the target node information with the historical node information to obtain a first matching result; the historical node is a node in the topological graph corresponding to the historical task; the first matching result includes at least one target historical task among the historical tasks that matches the target task;

[0007] Based on the preset attribute conditions, the above target historical tasks are filtered;

[0008] Based on the screening results, the recommended execution parameter information of the target task is determined as the processing result of the remote shuffling service process.

[0009] According to other embodiments of the present disclosure, a task processing device is provided, including:

[0010] a topology map generating module, configured to receive a remote shuffling service request for a target task, and in response to the remote system service request for the target task, generate a topology map for the target task based on the execution process information of the target task, wherein the topology map includes a plurality of target nodes, each target node corresponding to at least one subtask of the target task;

[0011] a first matching module, configured to match the target node information with the historical node information to obtain a first matching result; the historical node being a node in a topological graph corresponding to a historical task; the first matching result including at least one target historical task among the historical tasks that matches the target task;

[0012] A screening module is used to screen the above-mentioned target historical tasks based on preset attribute conditions;

[0013] The recommendation module is used to determine the execution parameter recommendation information of the above-mentioned target task based on the screening result as the processing result of the remote shuffling service process.

[0014] In addition, an embodiment of the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above method is implemented when the processor executes the program.

[0015] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions for causing a computer to execute the above method.

[0016] An embodiment of the present disclosure further provides a computer program product, comprising computer program instructions, which enable the computer to execute the above method when the computer program instructions are executed on a computer.

[0017] From the above content, it can be seen that the task processing method, device, electronic device, storage medium and program product provided by the embodiments of the present disclosure can filter out target historical tasks that are relatively similar to the target tasks currently received and not yet executed from the historical tasks that have been executed, and determine the execution parameter recommendation information of the target task based on the target historical tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of one application scenario of the task processing method provided by an embodiment of the present disclosure;

[0020] Figure 2 The implementation process of the task processing method of some embodiments of the present disclosure is shown;

[0021] Figure 3A schematic diagram of topology matching according to an embodiment of the present disclosure is shown;

[0022] Figure 4 A schematic diagram showing the matching process of target tasks and historical tasks according to an embodiment of the present disclosure is shown;

[0023] Figure 5 A schematic diagram of a task processing device according to an embodiment of the present disclosure is shown;

[0024] Figure 6 A schematic diagram showing the overall architecture of a task processing device according to some embodiments of the present disclosure is shown; and

[0025] Figure 7 A more specific schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0027] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0028] The modifications of "one", "multiple" or "at least one" mentioned in the embodiments of the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0029] As mentioned above, in general, business logic is completely coupled with the routing module. Every time a new routing rule is added, the routing module code must be modified. This causes inconvenience in routing module updates and code maintenance.

[0030] To this end, some embodiments of the present disclosure provide a task processing method. Figure 1, which is a schematic diagram of an application scenario of the task processing method provided by an embodiment of the present disclosure. The application scenario includes: a terminal device 101 and a storage device 102 for storing historical tasks. The terminal device 101 and the storage device 102 may be devices in a system for performing a shuffle operation.

[0031] In an embodiment of the present disclosure, the terminal device 101 can run multiple tasks and can be one of multiple devices on the server side.

[0032] In the embodiments of the present disclosure, the terminal device 101 includes, but is not limited to, a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA), or other electronic devices capable of implementing the above functions. The terminal device 101 can determine the parameters required to perform the task based on the received task.

[0033] Based on the above application scenarios and other possible application scenarios, some embodiments of the present disclosure provide a task processing method that can call the terminal device 101 and the storage device 102 to search for matching target historical tasks in the historical tasks that have been executed for the newly received target task, and determine the execution parameters of the target task based on the search results.

[0034] In the case where it is not necessary to obtain the content necessary for viewing the statistical information through a communication network, the embodiments of the present disclosure can also be applied to the above-mentioned terminal devices and storage devices that are not connected to the network.

[0035] Figure 2 The implementation process of the task processing method of the embodiment of the present disclosure is shown. Figure 2 As shown, the task processing method may include the following steps:

[0036] Step 201: receiving a remote shuffling service request for a target task, generating a topology graph of the target task based on execution process information of the target task in response to the remote system service request, wherein the topology graph includes a plurality of target nodes, each target node corresponding to at least one subtask of the target task;

[0037] Step 202: Match the target node information with the historical node information to obtain a first matching result; the historical node is a node in the topological graph corresponding to the historical task; the first matching result includes at least one target historical task among the historical tasks that matches the target task;

[0038] Step 203: Filter the target historical tasks based on the preset attribute conditions;

[0039] Step 204: Based on the screening results, the recommended execution parameter information of the target task is determined as the processing result of the remote shuffling service process.

[0040] In one embodiment of the present disclosure, the task processing method can be applied to a big data computing engine that calculates a target task. The big data computing engine is used to perform remote shuffling service processing for the target task received by the big data framework, and executes the remote shuffling service processing for the target task. Figure 2 The target task can be a task submitted to a task processing platform (big data computing engine) for processing. When the task processing platform receives a new task, it can use the new task as the target task and execute the task processing method provided by the embodiment of the present disclosure based on the newly received target task. The task processing platform can be a big data platform, such as MapReduce (Map Reduce), Apache Spark, and other big data frameworks that can be used for task processing.

[0041] In the disclosed embodiment, the target task to be processed by the task processing platform can be divided into multiple subtasks, each of which can correspond to an execution phase of the target task. For example, if the target task is divided into N execution phases to be executed sequentially, then during the execution of the target task, the N subtasks corresponding to the execution phases are executed.

[0042] In one embodiment of the present disclosure, the subtasks included in the target task can be first determined based on the target task, nodes corresponding to the subtasks can be constructed, and edges between nodes can be determined based on the order in which the subtasks are executed. The constructed nodes and the edges between the nodes form a topological graph of the target task. In the embodiment of the present disclosure, the topological graph can be represented by a directed acyclic graph or a tree diagram.

[0043] In the embodiment of the present disclosure, matching the target node information with the historical node information may be matching at least one target node in the topology map with at least one historical node corresponding to the historical task.

[0044] In the embodiment of the present disclosure, the historical task can be a completed task. Similar to the target task, the historical task can also be divided into multiple subtasks, and based on the subtask division, a topology map corresponding to the historical task is constructed. In order to distinguish it from the target task, the node in the topology map of the historical task is called a historical node. When matching the target node with the historical node, the depth of the target node is the same as the depth of the historical node. For example, if the matched target node is the root node in the topology map, the corresponding matched historical node is also the root node in the topology map. If the matched target node is the first-layer node in the topology map, the corresponding matched historical node is also the first-layer node in the topology map.

[0045] In the disclosed embodiment, the target node information is matched with the historical node information to obtain a first matching result. This can be done by screening the historical tasks to see if there are tasks that have a certain degree of similarity to the target task. The matching of different tasks can be determined from multiple perspectives. For example, the matching can be determined by whether the execution phases of the tasks are similar, whether the task operators are the same, or whether the operator parameters of the tasks are similar.

[0046] In an embodiment of the present disclosure, the above-mentioned target historical tasks are screened based on preset attribute conditions. The target historical tasks can be screened based on the preset attribute conditions and attribute information of each target historical task, so that at least one target historical task that meets the preset attribute conditions is screened.

[0047] In an embodiment of the present disclosure, based on the screening results, the recommended execution parameter information of the above-mentioned target task is determined. The value range of the execution parameter of the target task can be determined based on the target historical tasks included in the screening results, and the value range is used as the recommended execution parameter information.

[0048] Generally speaking, big data frameworks (MapReduce / Spark), big data computing engines, or the aforementioned task processing platforms all include a common computational process called shuffle. Shuffle performance largely determines the speed at which the big data framework (big data platform) executes tasks. To improve shuffle performance and achieve storage and computation separation, major vendors have launched Remote Shuffle services. As a remote shuffle service, it is necessary to elastically scale or reject high-load tasks based on the shuffle performance of the task. Tasks can be represented by Applications. Each task corresponds to an Application, which consists of multiple stages (which can correspond to the subtasks in the aforementioned embodiments). Stages release resources after completion, making this estimation approach particularly unfavorable for multi-stage tasks. In addition, the proportion of adhoc (ad hoc query) tasks is increasing. These tasks lack historical information for pre-estimation and analysis, necessitating a method for approximate shuffle performance estimation. The disclosed embodiments provide a method for estimating the performance of tasks such as shuffles at the stage granularity, treating stages as subtasks. The specific method involves graph matching based on the task's execution logic (e.g., a DAG). When the method provided by the embodiment of the present disclosure is applied to a big data framework, the historical information of the stage can be directly queried in the database according to the encoding. The stage encoding is divided into at least one level according to the matching accuracy, and the relevant situation of the shuffle execution parameters of the stage is estimated to the greatest extent.

[0049] Generally, data skew is prone to occur due to the parallel processing of multiple tasks on big data platforms. Data skew occurs when a single partition has significantly more data than other partitions during parallel data processing, resulting in uneven distribution. This causes a large amount of data to be concentrated on one or several computing nodes, causing the processing speed of this partition to be far lower than the average computing speed, becoming a bottleneck for the entire dataset processing and thus affecting overall computing performance. Based on the estimation of the stage-granularity shuffle, the disclosed embodiment can set reasonable execution parameters (number of partitions, cache size) for the shuffle to address the problem of partition data skew.

[0050] In this embodiment, the parameter recommendation information is the shuffle estimation result of the target task.

[0051] It can be seen that the task processing method provided by the embodiment of the present disclosure can filter out target historical tasks that are relatively similar to the target task currently received and not yet executed from the historical tasks that have been executed, and determine the execution parameter recommendation information of the target task based on the target historical tasks.

[0052] In an embodiment of the present disclosure, the execution parameter recommendation information can be used to provide a reference for the execution parameter configuration of the target task, for example, configuring the estimated quantity of various computing resources for the target task, applying for corresponding computing resources for the target task according to the estimated quantity, and setting an appropriate amount of computing resources for the target task while avoiding excessive application for resources, which may have an adverse effect on the execution of other tasks.

[0053] In one embodiment of the present disclosure, matching the target node information with the historical node information to obtain a first matching result includes:

[0054] Matching the first information of the target node with the first information of the historical node to obtain a first historical task whose topological graph structure similarity with the target task is greater than a first threshold;

[0055] Matching the second information of the target node with the second information of the historical node of the first historical task to obtain a second historical task whose operator information similarity with the target node is greater than a second threshold;

[0056] According to the second historical task, the first matching result is obtained.

[0057] In the disclosed embodiment, the topology of the target task reflects the execution plan of the target task. A unified task execution phase division method can be pre-set, and each historical task can be divided into multiple subtasks according to the unified division method. After receiving a new target task, the target task is divided into multiple subtasks according to the unified division method, so that the topology of the target task is comparable with the topology of the historical tasks.

[0058] In one embodiment of the present disclosure, the subtasks may be a plurality of pre-set subtasks. For example, a subtask set may be pre-set, the task set including all subtasks, and each task may be composed of a plurality of subtasks selected from the subtask set in an arrangement and combination.

[0059] In the disclosed embodiments, the first information may be information reflecting the structure of the topology. For example, information about each node in the topology may be summarized and used as the first information. For another example, information describing and expressing the structure of the topology may be generated and used as the first information. For another example, a code recording information about the root node of the topology and non-nodes at each layer may be generated and used as the first information.

[0060] The second information of the target node may be information related to the operator information that can represent the target node. For example, the operator type may be converted into a number, and the similarity between the target node and the historical node may be determined by the similarity between the numbers corresponding to the target node and the historical node.

[0061] The second information may also be related information reflecting multi-level operator information. For example, the second information may include information reflecting the operator type, or information reflecting the operator parameters.

[0062] In the embodiment of the present application, the second information is more detailed information than the first information, that is, the first information may be the framework information of the target task, and the second information may be the detailed information of the target task.

[0063] In the disclosed embodiment, the target historical tasks are screened from the historical tasks based on the first information and the second information, respectively. The second information is information that characterizes the task in more detail than the first information, so that the historical tasks can be screened more finely during the matching process, and historical tasks with large differences can be excluded early, and finally the historical task with the highest degree of matching can be selected as the target historical task.

[0064] In one embodiment of the present disclosure, the first information is obtained based on the number of the target node; the second information is obtained based on the number of the target node; the number of the target node is obtained by hash calculation based on the depth of the node and the parent node number of the node.

[0065] In one embodiment of the present disclosure, the first information and the second information are both the topological numbers of the target node. During matching, the target task is matched with the corresponding nodes in the topological graph of the historical tasks. For example, the root node of the target task topological graph is compared with the root node of the historical task topological graph, the first-level child nodes of the target task topological graph are compared with the first-level child nodes of the historical task topological graph, and so on.

[0066] By encoding the nodes of the topology map, the structure of the topology map can be reflected through digital content.

[0067] In one embodiment of the present disclosure, the target node includes a root node and a target descendant node of the target root node; the first information of the historical node includes the number of the historical node; and matching the first information of the target node with the first information of the historical node includes:

[0068] The number of the target root node is used as the first information of the target root node, and is matched with the number of the root node in the historical node to obtain the third historical task in the historical task; the number of the target root node is calculated based on the number of the target descendant node;

[0069] The number of the target descendant node is used as the first information of the target descendant node and matched with the number of the descendant node in the historical node of the third historical task; the number of the target descendant node is obtained based on the number of the child node of the target descendant node.

[0070] In the disclosed embodiments, when performing matching, the information to be matched can be further refined to achieve multi-level matching. Starting from a relatively general level, the target task information and historical task information are matched layer by layer, achieving a step-by-step screening of historical tasks. This ensures that target historical tasks that are close to the target task are screened, while also improving the matching efficiency of historical tasks.

[0071] In one embodiment, the second information includes operator name information and operator parameter information; and matching the second information of the target node with the second information of the historical node includes:

[0072] Match the operator name information of the target node with the operator name information of the historical node of the first historical task to obtain a fourth historical task; the operator type information of the target node is calculated based on the target node code, the operator name included in the subtask corresponding to the target node, and the input table name;

[0073] Match the operator parameter information of the above-mentioned target node with the operator parameter information of the above-mentioned fourth historical task. The operator parameter information of the above-mentioned target node is calculated based on the encoding of the above-mentioned target node, the operator name included in the subtask corresponding to the target node, the input table name included in the subtask corresponding to the target node, and the partition and operator parameters included in the subtask corresponding to the target node.

[0074] In the disclosed embodiments, operator parameter information differs from the target task's execution parameters. Operator parameter information refers to the parameters set in the operator when the target task is generated. However, the target task's execution parameters, such as the number of resources required and the number of resources requested, are unknown to the platform when the target task is received. Therefore, the platform needs to estimate these execution parameters, such as the number of resource requests, based on historical tasks and then execute the target task based on the estimated execution parameters.

[0075] In the embodiment of the present disclosure, the operator parameter information reflects the content of the target task in more detail than the operator name information. By matching the operator name information and the operator name information respectively, a layer-by-layer screening of multiple historical tasks is achieved.

[0076] In one embodiment of the present disclosure, the above-mentioned screening of the target historical tasks based on the preset attribute conditions includes:

[0077] Based on the preset attribute conditions, target historical tasks whose time difference between the task execution time and the current time is less than a preset time threshold are filtered out from the above target historical tasks as the filtering results.

[0078] In the embodiment of the present disclosure, considering that tasks with similar structures and operators may also have execution parameters changed during multiple executions, a historical task that is closer to the current time is selected as the target historical task.

[0079] In one embodiment, determining the recommended execution parameter information of the target task based on the screening results includes:

[0080] Refer to the execution parameters of the target historical tasks corresponding to the above screening results to set the execution parameter recommendation information of the above target tasks.

[0081] In one embodiment of the present disclosure, the parameters of the target historical task can be directly used as the parameters of the target task. Alternatively, the execution parameters of the target historical task can be calculated to obtain an estimated value of the execution parameters of the target task, and the estimated value can be used as the execution parameter recommendation information.

[0082] In one embodiment of the present disclosure, referring to the execution parameters of the target historical tasks corresponding to the screening results, setting the execution parameter recommendation information of the target tasks includes:

[0083] Reading group counter information of the target historical task, and if it is determined based on the group counter information that a memory overflow exists for the target historical task, increasing the partition setting based on the partition setting corresponding to the execution parameter of the target historical task according to a set step size; and determining the execution parameter recommendation information based on the increased partition setting;

[0084] Read the idle information of the executor resources of the target historical task. If, based on the idle information of the executor resources, the reading amount of the shuffle operation of the single task is determined to be less than the set threshold, then, based on the partition setting corresponding to the execution parameter of the target historical task, reduce the partition setting according to the set step size; and determine the recommended execution parameter information based on the reduced partition setting.

[0085] Common remote shuffle solutions use a default number of partitions. Setting it too high wastes Remote Shuffle service resources (including caches, slots, file handles, etc.), while setting it too low can lead to insufficient task parallelism and even memory overflow. This disclosed embodiment provides a method for setting a reasonable number of partitions based on the performance of the target historical task.

[0086] In one example of the present disclosure, online tasks of big data frameworks such as Spark can be classified into three categories: tasks with unique numbers, periodic tasks without numbers, and temporary query tasks. Among them, periodic tasks with unique numbers are generally hosted on a cloud platform. The platform will give the task a unique number and pass the number to the computing framework. Periodic tasks will also be updated, and some updates may bring changes to the execution logic. In the embodiment of the present disclosure, such changes can be discovered. Periodic tasks without numbers are generally started by users on a scheduled basis and have the characteristics of periodic tasks, but no information is passed to the computing framework. In the embodiment of the present disclosure, periodic tasks with a specific number can be identified as periodic tasks and historical information can be found. Temporary query tasks Adhoc are generally temporary queries by data analysts, and similar tasks may exist online. In the embodiment of the present disclosure, similar tasks can be matched to estimate the execution parameters of the shuffle. The encoding method of the directed acyclic graph DAG can be used, and this encoding method can achieve isomorphic matching of the DAG. In order to achieve stage granularity matching, a three-level encoding is constructed for the stage based on the DAG encoding. First, the encoding containing less framework information is matched, and then the encoding containing more stage details is matched.

[0087] The three-level codes constructed in the embodiment of the present disclosure can be: jobTopologySignature, stageTopologySignature, stagePlanSignature, and stageExecSignature.

[0088] Among them, JobTopologySignature, the topological encoding of the job, only considers the topological graph of the execution plan, which is equivalent to the first information of the target root node in the aforementioned embodiment. Periodic tasks will also be updated, and task updates may cause changes to the task execution plan. This encoding can match whether the topological graphs of the execution plan are the same. stageTopologySignature, the topological encoding of the stage, is equivalent to the first information of the target descendant node in the aforementioned embodiment. stagePlanSignature is equivalent to the operator name information of the target node in the aforementioned embodiment. The actual execution of the task will also have different inputs and operators. stagePlanSignature incorporates the input table name and operator name into the hash calculation. stageExecSignature is equivalent to the operator parameter information of the target node in the aforementioned embodiment. It incorporates the input partition and operator parameters into the hash calculation, which is a perfect match for the stage. In addition, the task also includes: recursiveUniqueId (a unique number for periodic tasks. Periodic tasks are generally hosted on a cloud platform, which assigns a globally unique number to the task. This number can identify different running instances as the same task), submitTime (task submission time), and stageInformation (stage-related information, including the number of partitions, shuffle size, tilted partition number, task running time, and executor memory usage).

[0089] In some embodiments of the present disclosure, after a task in a big data framework like Spark completes, it is necessary to save the stage encoding and shuffle information in the database. When a new task is submitted for execution, a stage-by-stage match is performed in the database based on the encoding. If a stageExecSignature is matched, the obtained stage information is completely accurate.

[0090] When matching jobTopologySignature and stageTopologySignature, the embodiment of the present disclosure adopts recursive matching. Figure 3To determine whether the execution plans of Job1 and Job2 are isomorphic, start from the root node of Job1 and traverse the child nodes B and C of node A to determine whether nodes b and c of Job2 are isomorphic to nodes B and C, and so on. This method is relatively complex and may not support complex query processes when querying the database.

[0091] In another embodiment, the nodes in the graph are numbered first, and the first information and the second information of each node of the task are obtained according to the number. Figure 3 As shown, the root node is numbered 0, and the numbers of other nodes are the number of the parent node plus the layer number of the node. When implemented algorithmically, the pre-order traversal can realize this process, and the number obtained by this process is recorded as preTopologyId. A node may have different layer numbers. For example, the number of H can be 0-1-2-3 and 0-1-2. In this case, the longest number is taken. The jobTopologySignature is calculated based on the preTopologyId (the jobTopologySignature of the root node is equivalent to the first information of the target root node). The specific process includes subsequent traversal of the node, that is, first visiting the child node to obtain the number of the child node; then visiting the parent node, and calculating the number of the parent node based on the number of the child node.

[0092] The calculation formula for the parent node number jobTopologySignature is: jobTopologySignature(parent) = hash(order(jobTopologySignature(child))). Order indicates the sorting operation of the parameters, and hash indicates the hash evaluation of the parameters. For example: jobTopologySignature(D) = hash(hash(F), hash(G), hash(H)). Figure 3The hash values ​​of nodes F, G, and H in are the same, and the order can be ignored. For another example, jobTopologySignature(A)=hash(jobTopologySignature(B), jobTopologySignature(C)), assuming that the hash value of node B is smaller than the hash value of node C. Through the above-mentioned pre-order and post-order traversal, if the calculated numbers of nodes A and a, jobTopologySignature, are equal, then it can be judged that nodes A and a are isomorphic, that is, tasks Job1 and Job2 are isomorphic in execution plan, using similar operator sets and similar execution orders. The calculation process of stageTopologySignature is similar to that of jobTopologySignature. In the calculation process of jobTopologySignature, the descendant nodes of the root node are used as the stage, and the descendant node numbers calculated in this process are used as the stageTopologySignature numbers of the stage (equivalent to the first information of the target descendant nodes in the aforementioned embodiment). stagePlanSignature is equivalent to the operator name information, taking into account the input table name and operator name. The calculation process is similar to jobTopologySignature, except that the hash function includes the input table name inputTables and the operator name operatorNames as parameters. The calculation formula can be: stagePlanSignature(parent) = hash(order(stagePlan Signature(child)), inputTables, operatorNames), which is calculated based on the node number. By comparing the stagePlanSignature of the root node of the task topology graph, it is possible to determine whether the periodic task has changed. stageExecSignature is equivalent to the operator parameter information, taking into account the input partitions and operator parameters. The calculation process is similar to stagePlanSignature, except that the hash function includes the input partitions (partitions) and operator parameters (operatorParameters) as parameters.stageExecSignature = hash(order(stageExecSignature(child)),inputTables,partitions,operatorNames,operatorParameters) The input partitions of periodic tasks generally change, while the operator parameters operatorParameters generally do not change. Therefore, you can include only operatorParameters in the hash calculation to determine whether the periodic task has changed. This judgment method is more accurate than the root node's stagePlanSignature.

[0093] Figure 4 The best matching process of the stage is shown. The specific process is as follows:

[0094] Step 401: Job submission. Specifically, when the job is submitted, the job execution plan DAG is passed to the SignatureSystem, and four numbers for each stage are calculated.

[0095] Step 402: Determine whether a unique number exists. Specifically, determine whether the job has a unique recursiveUniqueId. If so, proceed to step 4021; if not, proceed to step 4022.

[0096] Step 4021: If a unique ID exists, search the StageTable for an item that matches the recursiveUniqueId and the jobTopologySignature of the root node. If a match exists, proceed to step 403; if not, proceed to step 404.

[0097] Step 4022: If there is no unique number, check if there is an item in StageTable that matches the jobTopologySignature of the root node. If there is a match, go to step 403; if not, go to step 404.

[0098] Step 403: Traverse the stages. In this step, traverse all stages and match them in the order in which they were matched from the previous step (matching the stageTopologySignature, stagePlanSignature, and stageExecSignature of all nodes containing the jobTopologySignature). If a match is found, repeat step 403. If not, proceed to step 404.

[0099] Step 404: If no match exists, the previous match result is returned, which is considered the best match. If a match exists, the final match result is considered a perfect match. If the match result is empty, this indicates that no historical information exists for the job or stage, and the default configuration is used. The actual execution process involves database queries, and the matching result set gradually shrinks. This process is detrimental to database query performance. You can determine whether the previous result set exists. If it is empty, the previous match is the best match. You can directly execute the previous match to avoid slow database queries on large data volumes. The number of tasks typically fluctuates gradually, so recent historical information is the most accurate. Generally, the last 3-4 days of historical information is sufficient. During special periods such as holidays that impact business, estimates can be made based on month-over-month performance. Estimating the shuffle size and resource usage is crucial. To ensure task service quality, the historical maximum value from the last 3-4 days is generally used. Remote Shuffle maintains a quota status, elastically scaling the Remote Shuffle service based on the shuffle size and resource requirements, or denying access to tasks that impact cluster stability.

[0100] In the disclosed embodiment, the Shuffle parameter recommendation ConfigOpt partition number (optimal partition configuration) is the most important parameter affecting shuffle performance. A reasonable number of partitions can ensure task parallelism, avoid shuffle resource waste, and reduce small IO (Input / Output). The method for setting the number of partitions is as follows:

[0101] Initialize the number of partitions for the target task with an increase ratio of a = 1 and a decrease ratio of b = 0.5. 2. Read the stage history information. If the task GC (task group counter) is serious (i.e., the task GC takes a long time), there is a task memory overflow, and the partition setting is judged to be too small.

[0102] Get the partition number paritionNum of the target historical task and set the parition number to (1+a)×partitionNum;

[0103] If the task fails or slows down, return to the previous partition setting and set a = a / 2;

[0104] Read the stage history information. If the executor resources are idle, the shuffle read volume of a single task is very small, and the task execution time is very fast, it is judged that the parity setting is too large.

[0105] Get the last partition number paritionNum and set the partition number to (1-b)×partitionNum 1. If the task fails or slows down, return to the last parition setting and set b=b / 2.

[0106] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] Corresponding to the above method, an embodiment of the present disclosure further discloses a task processing device.

[0108] like Figure 5 As shown, the task processing device provided by the embodiment of the present disclosure includes:

[0109] A topology map generating module 501 is configured to receive a remote shuffling service request for a target task, and in response to the remote system service request for the target task, generate a topology map for the target task based on the execution process information of the target task, wherein the topology map includes a plurality of target nodes, each of which corresponds to at least one subtask of the target task;

[0110] A first matching module 502 is configured to match the target node information with the historical node information to obtain a first matching result; the historical node is a node in the topological graph corresponding to the historical task; the first matching result includes at least one target historical task among the historical tasks that matches the target task;

[0111] The screening module 503 is used to screen the above-mentioned target historical tasks based on preset attribute conditions;

[0112] The recommendation module 504 is used to determine the execution parameter recommendation information of the target task based on the screening result as the processing result of the remote shuffling service process.

[0113] The task processing device can be applied to a big data computing engine that calculates the target task. The big data computing engine is used to perform remote shuffling service processing for the received target task. The module of the task processing device used in the remote shuffling service processing process for the target task can be as follows: Figure 5 shown.

[0114] In one embodiment of the present disclosure, the first matching module includes:

[0115] A first historical task unit is configured to match the first information of the target node with the first information of the historical node to obtain a first historical task having a topological graph structure similarity with the target task that is greater than a first threshold;

[0116] A second historical task unit is configured to match the second information of the target node with the second information of the historical node of the first historical task to obtain a second historical task having an operator information similarity with the target node greater than a second threshold;

[0117] The second historical task processing unit is configured to obtain the first matching result according to the second historical task.

[0118] In one embodiment of the present disclosure, the above-mentioned first information is obtained based on the number of the above-mentioned target node; the above-mentioned second information is obtained based on the number of the above-mentioned target node; the number of the above-mentioned target node is obtained by hash calculation based on the depth of the above-mentioned node and the parent node number of the node.

[0119] In one embodiment of the present disclosure, the target node includes a root node and a target descendant node of the target root node; the first information of the historical node includes the number of the historical node; and the first historical task unit is further configured to:

[0120] The number of the target root node is used as the first information of the target root node, and is matched with the number of the root node in the historical node to obtain the third historical task in the historical task; the number of the target root node is calculated based on the number of the target descendant node;

[0121] The number of the target descendant node is used as the first information of the target descendant node and matched with the number of the descendant node in the historical node of the third historical task; the number of the target descendant node is obtained based on the number of the child node of the target descendant node.

[0122] In one embodiment of the present disclosure, the second information includes operator name information and operator parameter information; and the second historical task unit is further configured to:

[0123] Match the operator name information of the target node with the operator name information of the historical node of the first historical task to obtain a fourth historical task; the operator type information of the target node is calculated based on the target node code, the operator name included in the subtask corresponding to the target node, and the input table name;

[0124] Match the operator parameter information of the above-mentioned target node with the operator parameter information of the above-mentioned fourth historical task. The operator parameter information of the above-mentioned target node is calculated based on the encoding of the above-mentioned target node, the operator name included in the subtask corresponding to the target node, the input table name included in the subtask corresponding to the target node, and the partition and operator parameters included in the subtask corresponding to the target node.

[0125] In one embodiment of the present disclosure, the screening module includes:

[0126] The time screening unit is used to screen target historical tasks whose task execution time has a time difference from the current time that is less than a preset time threshold from the above target historical tasks based on preset attribute conditions, as screening results.

[0127] In one embodiment of the present disclosure, the recommendation module includes:

[0128] The reference unit is used to refer to the execution parameters of the target historical tasks corresponding to the above screening results and set the execution parameter recommendation information of the above target tasks.

[0129] In one embodiment of the present disclosure, the reference unit is further configured to:

[0130] Reading group counter information of the target historical task, and if it is determined based on the group counter information that a memory overflow exists for the target historical task, increasing the partition setting based on the partition setting corresponding to the execution parameter of the target historical task according to a set step size; and determining the execution parameter recommendation information based on the increased partition setting;

[0131] Read the idle information of the executor resources of the target historical task. If, based on the idle information of the executor resources, the reading amount of the shuffle operation of the single task is determined to be less than the set threshold, then, based on the partition setting corresponding to the execution parameter of the target historical task, reduce the partition setting according to the set step size; and determine the recommended execution parameter information based on the reduced partition setting.

[0132] Figure 6 This is an overall architecture of the task processing device of the embodiment of the present disclosure. Figure 6The illustrated embodiment uses Spark as an example to process multiple tasks. The task processing device includes: a DAG planner (DAGScheduler) 601, a shuffle manager (ShuffleManager) 602, a signature system (SignatureSystem) 603, a relational database management system (MySQL) 604, and a shuffle optimizer (ShuffleOpt) 605. The DAG planner 601 and the shuffle manager 602 are provided by the big data platform Spark. The shuffle optimizer 605 further includes a parameter optimization module (ConfigOpt), a remote shuffle access request module (RemoteCoordinator), and a signature matching module (SignatureMatch). DAGScheduler controls the execution plan of the task, and the ShuffleManager controls the shuffle plan of the Stage, which is used to submit the stage and configuration to the big data platform, as well as to submit stage matching information to the relational database management system 604. Before and after a job is executed, the job's execution plan (DAG) is passed to the signature system 603. The SignatureSystem calculates four codes (i.e., the jobTopologySignature, stageTopologySignature, stagePlanSignature, and stageExecSignature in the aforementioned embodiment) and stores them in an external database. When a job is submitted, the external database can be accessed to obtain historical stage information. The relational database management system 604 stores the task's stage list (StageTable).

[0133] Figure 6ShuffleOpt is the shuffle optimizer, consisting of three modules: the encoding matching module SignagureMatch, the Remote Shuffle access request module, and the parameter optimization module ConfigOpt. When a job is submitted, the encoding system SignatureSystem is first accessed to calculate the four encodings of the DAG, which are then optimally matched in the database by SignatureMatch. Based on the matching results, the shuffle's historical information and resource usage are obtained, and then access to the Remote Shuffle is requested through the RemoteCoordinator. Remote Shuffle elastically scales the Remote Shuffle service based on required resources or rejects tasks that affect cluster stability. Finally, ConfigOpt sets the appropriate number of partitions and cache size based on the shuffle size. Furthermore, StageTable stores the stage skew. If the partitioning function of a task remains unchanged, the same partition is likely to be skewed. Remote Shuffle can be notified to reserve multiple slots for the skewed partition to speed up writing to the skewed partition.

[0134] The specific implementation of each of the above modules can be referenced in the aforementioned methods and accompanying drawings, and will not be repeated here. For ease of description, the above device is described as functionally divided into various modules and described separately. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0135] The apparatus of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0136] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above-mentioned processor implements the above-mentioned method of any of the above-mentioned embodiments when executing the above-mentioned program.

[0137] Figure 7 1 shows a more specific hardware structure diagram of an electronic device provided by this embodiment. The device may include: a processor 2010, a memory 2020, an input / output interface 2030, a communication interface 2040, and a bus 2050. The processor 2010, the memory 2020, the input / output interface 2030, and the communication interface 2040 are connected to each other within the device via the bus 2050.

[0138] The processor 2010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0139] The memory 2020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 2020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 220 and is called and executed by the processor 2010.

[0140] The input / output interface 2030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0141] The communication interface 2040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0142] The bus 2050 comprises a path for transmitting information between the various components of the device (eg, the processor 2010 , the memory 2020 , the input / output interface 2030 , and the communication interface 2040 ).

[0143] It should be noted that although the above device only shows the processor 2010, the memory 2020, the input / output interface 2030, the communication interface 2040, and the bus 2050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0144] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0145] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0146] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0147] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the task processing method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0148] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0149] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the present disclosure, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present disclosure, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the purview of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0150] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0151] The embodiments of the present disclosure are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A task processing method, characterized in that: include: Receiving a remote shuffle service request of a target task, responding to the remote system service request of the target task, and generating a topology map of the target task according to the execution process information of the target task, wherein the topology map includes a plurality of target nodes, and each target node corresponds to at least one subtask of the target task; Matching the target node information with the historical node information to obtain a first matching result; The historical node is a node of a topological graph corresponding to a historical task; the first matching result includes at least one target historical task among the historical tasks that matches the target task, the information of the target node includes first information obtained based on the number of the target node, and the information of the historical node includes first information obtained based on the number of the historical node; Based on preset attribute conditions, the target historical tasks are screened; Based on the screening result, the recommended execution parameter information of the target task is determined as the processing result of the remote shuffling service request.

2. The method according to claim 1, characterized in that: The step of matching the target node information with the historical node information to obtain a first matching result includes: Matching the first information of the target node with the first information of the historical node to obtain a first historical task whose topological graph structure similarity with the target task is greater than a first threshold; Matching the second information of the target node with the second information of the historical node of the first historical task to obtain a second historical task whose operator information similarity with the target node is greater than a second threshold; The first matching result is obtained according to the second historical task.

3. The method according to claim 2, characterized in that The first information is obtained based on the number of the target node; the second information is obtained based on the number of the target node; the number of the target node is obtained by performing a hash calculation based on the depth of the node and the number of the node's descendant nodes.

4. The method according to claim 3, characterized in that The target node includes a root node and a target descendant node of the target root node; the first information of the historical node includes the serial number of the historical node; and matching the first information of the target node with the first information of the historical node includes: The number of the target root node is used as the first information of the target root node, and is matched with the number of the root node in the historical node to obtain the third historical task in the historical task; the number of the target root node is calculated based on the number of the target descendant node; The number of the target descendant node is used as the first information of the target descendant node, and matched with the number of the descendant node in the historical node of the third historical task; the number of the target descendant node is obtained based on the number of the child node of the target descendant node.

5. The method according to claim 3 or 4, characterized in that: The second information includes operator name information and operator parameter information; and matching the second information of the target node with the second information of the historical node includes: Matching the operator name information of the target node with the operator name information of the historical node of the first historical task to obtain a fourth historical task; the operator type information of the target node is calculated according to the code of the target node, the operator name included in the subtask corresponding to the target node, and the input table name; The operator parameter information of the target node is matched with the operator parameter information of the fourth historical task. The operator parameter information of the target node is calculated based on the code of the target node, the operator name included in the subtask corresponding to the target node, the input table name included in the subtask corresponding to the target node, and the partition and operator parameters included in the subtask corresponding to the target node.

6. The method according to claim 1, characterized in that The screening of the target historical tasks based on the preset attribute conditions includes: Based on the preset attribute condition, target historical tasks whose time difference between task execution time and current time is less than a preset time threshold are filtered out from the target historical tasks as filtering results.

7. The method according to claim 6, characterized in that The step of determining the recommended execution parameter information of the target task based on the screening result includes: Referring to the execution parameters of the target historical tasks corresponding to the screening results, the execution parameter recommendation information of the target tasks is set.

8. The method according to claim 7, characterized in that The step of setting the execution parameter recommendation information of the target task by referring to the execution parameter of the target historical task corresponding to the screening result includes: Reading the group counter information of the target historical task, if it is determined according to the group counter information that the target historical task has a memory overflow, then increasing the partition setting according to the set step size and on the basis of the partition setting corresponding to the execution parameter of the target historical task; determining the execution parameter recommendation information according to the increased partition setting; Read the executor resource idle information of the target historical task. If it is determined according to the executor resource idle information that the shuffle operation reading amount of a single task is less than a set threshold, then according to the set step size, on the basis of the partition setting corresponding to the execution parameter of the target historical task, reduce the partition setting; and determine the execution parameter recommendation information based on the reduced partition setting.

9. A task processing device, characterized in that: include: A topology map generating module, configured to receive a remote shuffle service request of a target task, and in response to the remote system service request of the target task, generate a topology map of the target task according to the execution process information of the target task, wherein the topology map includes a plurality of target nodes, and each target node corresponds to at least one subtask of the target task; A first matching module, used to match the information of the target node with the information of the historical node to obtain a first matching result; The historical node is a node of a topological graph corresponding to a historical task; the first matching result includes at least one target historical task among the historical tasks that matches the target task, the information of the target node includes first information obtained based on the number of the target node, and the information of the historical node includes first information obtained based on the number of the historical node; A screening module, used for screening the target historical tasks based on preset attribute conditions; The recommendation module is used to determine the execution parameter recommendation information of the target task based on the screening result, and use the parameter recommendation information as the processing result of the remote shuffling service process.

10. The device according to claim 9, characterized in that The first matching module comprises: A first historical task unit, used for matching the first information of the target node with the first information of the historical node to obtain a first historical task having a topological graph structure similarity with the target task greater than a first threshold; A second historical task unit, used for matching the second information of the target node with the second information of the historical node of the first historical task, so as to obtain a second historical task whose operator information similarity with the target node is greater than a second threshold; The second historical task processing unit is used to obtain the first matching result according to the second historical task.

11. The device according to claim 10, characterized in that The first information is obtained based on the number of the target node; the second information is obtained based on the number of the target node; The target node number is obtained by performing a hash calculation based on the depth of the node and the node's descendant node numbers.

12. The device according to claim 11, characterized in that The target node includes a root node and a target descendant node of the target root node; the first information of the historical node includes the serial number of the historical node; the first historical task unit is further used for: The number of the target root node is used as the first information of the target root node, and is matched with the number of the root node in the historical node to obtain the third historical task in the historical task; the number of the target root node is calculated based on the number of the target descendant node; The number of the target descendant node is used as the first information of the target descendant node, and matched with the number of the descendant node in the historical node of the third historical task; the number of the target descendant node is obtained based on the number of the child node of the target descendant node.

13. The device according to claim 11 or 12, characterized in that The second information includes operator name information and operator parameter information; the second historical task unit is further used for: Matching the operator name information of the target node with the operator name information of the historical node of the first historical task to obtain a fourth historical task; the operator type information of the target node is calculated according to the code of the target node, the operator name included in the subtask corresponding to the target node, and the input table name; The operator parameter information of the target node is matched with the operator parameter information of the fourth historical task. The operator parameter information of the target node is calculated based on the code of the target node, the operator name included in the subtask corresponding to the target node, the input table name included in the subtask corresponding to the target node, and the partition and operator parameters included in the subtask corresponding to the target node.

14. The device according to claim 9, characterized in that The screening module comprises: The time screening unit is used to screen target historical tasks whose time difference between task execution time and current time is less than a preset time threshold from the target historical tasks based on preset attribute conditions as screening results.

15. The device according to claim 14, characterized in that The recommendation module includes: The reference unit is used to refer to the execution parameters of the target historical task corresponding to the screening result and set the execution parameter recommendation information of the target task.

16. The device according to claim 15, characterized in that The reference unit is also used for: Reading the group counter information of the target historical task, if it is determined according to the group counter information that the target historical task has a memory overflow, then increasing the partition setting according to the set step size and on the basis of the partition setting corresponding to the execution parameter of the target historical task; determining the execution parameter recommendation information according to the increased partition setting; Read the executor resource idle information of the target historical task. If it is determined according to the executor resource idle information that the shuffle operation reading amount of a single task is less than a set threshold, then according to the set step size, on the basis of the partition setting corresponding to the execution parameter of the target historical task, reduce the partition setting; and determine the execution parameter recommendation information based on the reduced partition setting.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

18. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 8.

19. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images