A distributed hierarchical scheduling method and device based on a complex scene

Abstract

The application provides a distributed hierarchical scheduling method and device based on a complex scene, and belongs to the technical field of task scheduling, and specifically comprises the following steps: determining a shared control task type in a task type according to shared data of data flow of other task types in different analysis scenes, and construction data of a time-sensitive influence risk task in the different analysis scenes, determining a corresponding shared control task type of the data flow of a server in different running scenes, a time-sensitive preemption type of a running scene in which the shared control task type does not exist, determining a running scene in which the server performs flood storage control, and reducing the processing time efficiency of different time-sensitive requirement types of task types in a complex running scene.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of task scheduling, and particularly relates to a distributed hierarchical scheduling method and device based on a complex scenario. BACKGROUND

[0002] The credit institution processes a large amount of business data every day, and different task types have different timeliness requirements. The tasks with different timeliness requirements often have resource preemption problems, which seriously affect the scheduling of tasks with high timeliness requirements.

[0003] To solve the above technical problems, in the prior art, in the invention patent application CN202411735590.2 “Distributed computing resource scheduling system based on load balancing”, the running state of each distributed node is monitored, the load state of each distributed node is analyzed, and the most suitable running node of the subtask is selected for resource scheduling, so that the computing resources of the node can be fully utilized and the efficiency of task promotion can be improved. However, the above technical solution has the following technical defects:

[0004] In the optimization scheduling process of the distributed node, the existing technical solution ignores the similarity of resource flow between different task types. Specifically, when the same user performs different credit business application processing of different credit task types, the resource flow of different credit approval tasks of different credit task types often has certain similarities. Therefore, if the above similarities are ignored, the resource flow will be repeatedly read in different distributed nodes frequently, further increasing the pressure of data processing of the distributed node.

[0005] To solve the above technical problems, the application provides a distributed hierarchical scheduling method and device based on a complex scenario. SUMMARY

[0006] To achieve the object of the application, the application adopts the following technical solutions:

[0007] Specifically, the application provides a distributed hierarchical scheduling method based on a complex scenario, which specifically includes:

[0008] S1, based on the simultaneous processing data of the task type of data processing, dividing the running scenario, determining the timeliness preemption type of the running scenario according to the timeliness requirement data of the task type in the running scenario;

[0009] S2, taking the running scenario of the task type as an analysis scenario, determining whether the task type belongs to a timeliness impact risk task according to the timeliness preemption type corresponding to the analysis scenario and the distribution data of other task types in different analysis scenarios, and if not, entering the next step;

[0010] S3 determines a shared control task type in the task type according to shared data of data traffic of other task types in an analysis scenario and distribution of time-sensitive impact risk tasks.

[0011] S4 determines a running scenario of the server for flood storage control according to the shared control task type corresponding to the data traffic of the server in different running scenarios and the time-sensitive preemption type of the running scenario without the shared control task type.

[0012] The present application has the following beneficial effects:

[0013] In the present application, the shared control task type in the task type is determined according to shared data of data traffic of other task types in an analysis scenario and distribution data of other task types in different analysis scenarios, the analysis scenario of the time-sensitive impact risk task with reduced processing time requirement is excluded, the technical problem that the timeliness of data processing cannot meet the requirement caused by further flood storage control in the analysis scenario of the time-sensitive impact risk task with reduced processing time requirement is avoided, the matching between the number of analysis scenarios with simultaneous use of data traffic and the number of analysis scenarios meeting the requirement of the number of time-sensitive impact risk tasks is realized, the shared control task type is determined, and the technical problem that the timeliness of data processing cannot meet the requirement caused by the inability to timely and effectively switch to the analysis scenario with simultaneous use of data traffic due to the small number of analysis scenarios with simultaneous use of data traffic when flood storage control is performed is avoided.

[0014] The running scenario of the server for flood storage control is determined according to the shared control task type corresponding to the data traffic of the server in different running scenarios and the time-sensitive preemption type of the running scenario without the shared control task type, not only the number of running scenarios with shared control task types of the data traffic of the server is considered, but also the time-sensitive preemption type of the running scenario without the shared control task type is considered, the determination of the flood storage control method of the data traffic in different running scenarios is realized from the perspective of the difference of the time-sensitive preemption type and the matching with the number of running scenarios with shared control task types, that is, the risk of the technical problem that the timeliness of data processing cannot meet the requirement caused by the preemption of channels between different task types is reduced, and the efficiency of data processing is improved.

[0015] Further, the task type of the data processing includes customer qualification review, risk assessment, parameter optimization of a risk assessment model, and post-loan risk management of different credit approval task types.

[0016] Further, the simultaneous processing data of the task type includes a period in which the task type exists simultaneously in history.

[0017] Further, the method for dividing the running scene is:

[0018] Based on the simultaneous processing of different data processing task types, determine the period of simultaneous processing in history as the simultaneous processing period;

[0019] According to the number of simultaneous processing periods, determine whether the scene constructed by different data processing task types is a running scene.

[0020] Further, the method for determining the running scene of the server for flood control is:

[0021] Based on the shared control task type of the data flow of the server in different running scenes, determine the shared control task type of the data of the server in different running scenes as the shared association task, and determine the number of shared association tasks in different running scenes;

[0022] Determine the number of running scenes in a type of risk type of occupation in the running scene using the data of the server, which does not have a shared control task type and a time-sensitive impact risk task, and whose proportion meets the requirements, as the number of scenes in a type, based on the time-sensitive preemption type of the running scene without a shared control task type;

[0023] According to the number of scenes in a type and the number of shared association tasks in different running scenes, determine the running scene of the server for flood control.

[0024] In a second aspect, the application provides a distributed hierarchical scheduling device based on complex scenes, which adopts the above-mentioned distributed hierarchical scheduling method based on complex scenes, and specifically includes:

[0025] A priority classification module, a task scheduling module, and a flood control processing module;

[0026] The priority analysis module is responsible for determining the priority classification of the task type in the running scene;

[0027] The task scheduling module is responsible for the priority of different task types and the task scheduling processing of different task types;

[0028] The flood control processing module is responsible for determining the running scene of the server for flood control.

[0029] Specifically, the priority classification of the task type in the running scene includes:

[0030] According to the data processing time requirement of different task types, the priority classification of the task type is performed.

[0031] Specifically, the task type with a data processing time requirement less than 3 seconds, i.e. a processing time less than 3 seconds, is classified as a first priority, the task type with a data processing time requirement less than 10 seconds is classified as a second priority, the task type with a data processing time requirement less than 1 minute is classified as a third priority, the task type with a data processing time requirement less than 1 day is classified as a fourth priority, and the other task type is classified as a fifth priority.

[0032] Further, the task scheduling processing of different task types is performed, specifically including:

[0033] According to the priority of different task types, corresponding data processing channels are set, and the task scheduling processing of different task types is performed in different data processing channels and priority scheduling resource pools.

[0034] It can be understood that the task types with the same priority are classified into the same priority scheduling resource pool, the corresponding data processing channels are set for different priority scheduling resource pools, and the scheduling processing of the task types with different priorities is performed.

[0035] Other features and advantages will be set forth in the following description of the specification, and the objectives and other advantages of the present application will be achieved and obtained in the structure specifically pointed out in the specification and drawings.

[0036] In order to make the above-mentioned objectives, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described in detail below, and the accompanying drawings are referred to. BRIEF DESCRIPTION OF DRAWINGS

[0037] The above and other features and advantages of the present application will become more apparent by describing in detail example embodiments thereof with reference to the attached drawings.

[0038] Figure 1 is a flowchart of a distributed hierarchical scheduling method based on a complex scenario;

[0039] Figure 2 is a flowchart of a method for dividing a running scenario;

[0040] Figure 3 is a flowchart of a method for determining a time requirement preemption type of a running scenario;

[0041] Figure 4 is a flowchart of a method for determining a shared control task type in a task type;

[0042] Figure 5 is a framework diagram of a distributed hierarchical scheduling device based on a complex scenario. DETAILED DESCRIPTION

[0043] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described in the specification below in combination with the drawings in the specification. Obviously, the described embodiments are only part of the embodiments of the specification, not all. Based on the embodiments of the specification, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the specification.

[0044] In the present application, by the same use of the data of the server in the running scene of the simultaneous running of multiple data processing task types, the flood control processing of the server in the same use of different data processing task types is not performed, that is, the rate of data flow of the server is reduced, in one possible embodiment, to 0.7 to 0.8 times of the original, and the flood control processing in the running scene of the same use of different data processing task types is performed, in one possible embodiment, to more than 1.5 times of the original, thereby improving the efficiency of data processing under complex running scenes.

[0045] Embodiment 1

[0046] As Figure 1 shown, the present application provides a distributed hierarchical scheduling method based on complex scenes, specifically including:

[0047] S1, based on the simultaneous processing of data of different data processing task types, divides the running scene, and determines the time efficiency preemption type of the running scene according to the time efficiency requirement data of the task type in the running scene.

[0048] Further, the data processing task type includes customer qualification review, risk assessment, risk assessment model parameter optimization, and post-loan risk management of different credit approval task types.

[0049] Further, the simultaneous processing of data of the task type includes a time period in which the task type exists simultaneously in history.

[0050] Further, specifically, as Figure 2 shown, the method for dividing the running scene is:

[0051] Based on the simultaneous processing of data of different data processing task types, a time period in which the simultaneous processing exists in history is determined and is used as a simultaneous processing time period;

[0052] According to the number of the simultaneous processing time period, it is determined whether the scene constructed by different data processing task types is a running scene.

[0053] It can be understood that when the number of the simultaneous processing time periods meets the requirement, the scenario constructed by the different data processing task types is determined as a running scenario, and specifically, if the simultaneous processing time periods exist in different dates, it is determined that the number of the simultaneous processing time periods meets the requirement.

[0054] Optionally, the method for dividing the running scenario is:

[0055] Based on the simultaneous processing data of the different data processing task types, it is determined that the simultaneous processing time periods exist in the history and are taken as the simultaneous processing time periods.

[0056] According to the sum of the lengths of the different simultaneous processing time periods, it is determined whether the scenario constructed by the different data processing task types is a running scenario.

[0057] Further, when the sum of the lengths of the different simultaneous processing time periods is greater than a preset length threshold, the scenario constructed by the different data processing task types is determined as a running scenario.

[0058] Further, as shown in Figure 3 The method for determining the time-efficiency preemption type of the running scenario is:

[0059] Taking the task types under the running scenario as scenario matching task types, according to the time-efficiency requirement data of the different scenario matching task types under the running scenario, the data processing time-efficiency requirement of the different scenario matching task types is determined.

[0060] According to the data processing time-efficiency requirement, the scenario matching task type with the data processing time-efficiency requirement less than a preset processing length is determined and taken as a high time-efficiency task type.

[0061] The time-efficiency preemption type of the running scenario is determined through the constituent data of the high time-efficiency task type.

[0062] It can be understood that the data processing time-efficiency requirement is the length of time for the data of the scenario matching task type to complete data processing.

[0063] Specifically, the scenario matching task type with the data processing time-efficiency requirement less than 1 minute is taken as the high time-efficiency task type.

[0064] It should be noted that when the number of the high time-efficiency task types in the running scenario is greater than a preset type number threshold, the time-efficiency preemption type of the running scenario is determined as a first preemption risk type, and when the number of the high time-efficiency task types in the running scenario is not greater than the preset type number threshold, the time-efficiency preemption type of the running scenario is determined as a second preemption risk type, and in a possible embodiment, the preset type number threshold has a value range of 3 or more than 3.

[0065] S2 takes the running scene of the task type as an analysis scene, and determines that the task type does not belong to a time limit impact risk task according to a time limit preemption type corresponding to the analysis scene and distribution data of other task types in different analysis scenes, and enters the next step;

[0066] Further, the determination that the task type does not belong to a time limit impact risk task specifically includes:

[0067] determining a type of analysis scene of preemption risk type according to the time limit preemption type corresponding to the analysis scene of the task type, and taking the type of analysis scene as a type of analysis scene;

[0068] determining a number of the type of analysis scene in which other task types exist according to the composition data of the other task types in the type of analysis scene;

[0069] determining whether the task type belongs to a time limit impact risk task based on the number of the type of analysis scene in which the other task types exist.

[0070] It should be noted that the other task types are task types other than the task type.

[0071] It can be understood that the determination of whether the task type belongs to a time limit impact risk task based on the number of the type of analysis scene in which the other task types exist specifically includes:

[0072] When the number of the type of analysis scene in which the other task types exist does not meet the requirement, the task type is determined to belong to a time limit impact risk task. It can be understood that when the number of the other task types is more than 3, the number of the task types does not meet the requirement.

[0073] That is, when the number of the type of analysis scene in which the other task types exist in the type of analysis scene of the task type is not less than 2, the other task types are taken as impact task types, and the task type has an impact on the other task types in the type of analysis scene of the task type. Therefore, when the number of the impact task types is more than 3, the task type is determined to belong to a time limit impact risk task.

[0074] It should be noted that when the other task types are also in the type of analysis scene of the task type in the type of analysis scene of the task type, the task type has an impact on the execution time limit of the other task types in the type of analysis scene, and therefore the task type is taken as a time limit impact risk task.

[0075] S3 determines a shared control task type in the task type according to the sharing data of the data flow of other task types in the analysis scene and the distribution of the time-effect risk task;

[0076] Further, as shown in Figure 4 the method for determining the shared control task type in the task type is:

[0077] determining the business data shared by other task types in the analysis scene when performing data processing according to the sharing data of the data flow of other task types in the analysis scene of the task type, and taking other task types with the same business data as shared task types;

[0078] determining the shared optimization processing scene and the shared scene in the analysis scene according to the constituting data of the shared task type in different analysis scenes and the constituting data of the time-effect risk task;

[0079] determining whether the task type is a shared control task type based on the shared optimization processing scene and the shared scene data.

[0080] It should be noted that the shared optimization processing scene is an analysis scene in which the number proportion of time-effect risk tasks meets the requirement and there is no shared task type. It can be understood that when the number proportion of time-effect risk tasks in all business types in the analysis scene is less than 0.2, it is determined that the number proportion of time-effect risk tasks meets the requirement. The shared scene is an analysis scene in which there is a shared task type.

[0081] It can be understood that when the number of shared optimization processing scenes of the target multiple of the task type is less than the number of shared scenes, it is determined that the task type is a shared control task type.

[0082] In another embodiment, when the number of shared optimization processing scenes of 2 times is less than the number of shared scenes, the data flow of the task type is handled in the shared optimization processing scene at this time. Since the number of shared scenes at this time is large, it will not cause too much marketing to the execution efficiency of the task type, and it is determined that the task type is a shared control task type.

[0083] It should be noted that when the task type is a shared control task type, the flood storage processing of the service data of the service type during data processing can be performed in the shared optimization processing scene, that is, the speed of the data flow of the service data is reduced, and the flood discharge processing of the service data during data processing in the shared scene is performed, that is, the speed of the data flow of the service data is increased. In one embodiment, the speed of the data flow of the service data after being increased is at least 1.5 times or more than the speed of the data flow of the service data after being reduced.

[0084] S4 determines the running scene of the server for flood control according to the shared control task type corresponding to the data flow of the server in different running scenes and the time-sensitive preemption type of the running scene without the shared control task type.

[0085] Further, the method for determining the running scene of the server for flood control is:

[0086] Based on the shared control task type corresponding to the data flow of the server in different running scenes, the shared control task type of the server required in different running scenes is determined and is used as a shared associated task, and the number of shared associated tasks in different running scenes is determined.

[0087] Based on the time-sensitive preemption type of the running scene without the shared control task type, the number of running scenes of one type of preemption risk type in the running scene using the data of the server is determined, which meets the requirement that the number ratio of the time-sensitive impact risk task without the shared control task type meets the requirement, and is used as the number of one type of scene.

[0088] According to the number of one type of scene and the number of shared associated tasks in different running scenes, the running scene of the server for flood control is determined.

[0089] It can be understood that when the number of one type of scene is large, that is, the number of one type of scene is more than 4, the number of one type of scene is large, and therefore the flood control needs to be performed only in the scene with small impact of flood control, and specifically, the flood control of the data flow needs to be performed only in the running scene using the data of the server without the shared control task type, which meets the requirement that the number ratio of the time-sensitive impact risk task belongs to one type of preemption risk type.

[0090] In addition, it is necessary to point out that when the number of one type of scenarios is not more than 4, in one of the possible scenarios, the number of one type of scenarios is greater than the preset proportion of the number of running scenarios that share the associated tasks, that is, the number of one type of scenarios is greater than 0.5 times the number of running scenarios that share the associated tasks, at this time, the data flow flood control processing is only performed in the running scenarios that use the server data that do not share the control task type and belong to one type of preemption risk type.

[0091] If the number of one type of scenarios is not more than 4 and the number of one type of scenarios is not greater than the preset proportion of the number of running scenarios that share the associated tasks, at this time, the number of running scenarios that perform flood control processing is determined based on the preset proportion of the number of running scenarios that share the associated tasks, the number of running scenarios that perform flood control processing is subtracted from the number of one type of scenarios as the remaining number, and the determination of the running scenarios that perform flood control processing of data flow is performed according to the remaining number and the number of time-sensitive risk tasks in the running scenarios that use the server data that do not share the control task type and belong to the second type of preemption risk type and meet the requirement of the proportion of the number of time-sensitive risk tasks.

[0092] It can be understood that the determination of the running scenarios that perform flood control processing of data flow is performed according to the number of time-sensitive risk tasks from small to large under the constraint condition of the remaining number, that is, the remaining number of running scenarios that perform flood control processing of data flow is selected from small to large, and the preset proportion is one third.

[0093] In a specific embodiment:

[0094] Based on the simultaneous processing of different data processing task types, the time period in which simultaneous processing exists in history is determined and is taken as the simultaneous processing time period. If the simultaneous processing time period exists in different dates, it is determined that the number of simultaneous processing time periods meets the requirement.

[0095] According to the data processing time requirement, the scenario matching task type whose data processing time requirement is less than the preset processing time length is determined and is taken as the high time-sensitive task type. When the number of high time-sensitive task types is more than 3, it is determined that the time-sensitive preemption type of the running scenario is one type of preemption risk type, otherwise it belongs to the second type of preemption risk type.

[0096] Based on the timeliness preemption type corresponding to different analysis scenarios, a type of analysis scenario of preemption risk type is determined, and it is used as a type of analysis scenario. According to the association between a type of analysis scenario and different task types, the number of types of analysis scenarios that exist in different task types is determined. When the number of types of analysis scenarios that exist in the task type is 2 and the number of task types that are more than 2 is more than 3, it is determined that the task type belongs to a timeliness impact risk task.

[0097] The number of operating scenarios of a type of preemption risk in the operating scenarios using the data of the server, in which there is no shared control task type and the ratio of the number of time-limited risk tasks meets the requirements, is taken as the number of a type of scenario. When the number of a type of scenario is 4 or more, data flow flood storage control processing is performed only in the operating scenarios using the data of the server, in which there is no shared control task type, the ratio of the number of time-limited risk tasks meets the requirements, and the data flow belongs to a type of preemption risk;

[0098] When the number of a type of scenario is not more than 4, in one of the possible scenarios, when the number of a type of scenario is greater than the number of running scenarios with shared associated tasks in a preset proportion, that is, when the number of a type of scenario is not greater than 0.5 times the number of running scenarios with shared associated tasks, then only in the running scenarios of the data of the server that do not have a shared control task type, the proportion of time-limited risk tasks meets the requirements and belongs to a type of preemption risk type, the flood storage control processing of data traffic is performed; when the number of a type of scenario is greater than 0.5 times the number of running scenarios with shared associated tasks, the running scenarios of the data of the server that do not have a shared control task type, the proportion of time-limited risk tasks meets the requirements and belongs to a type of preemption risk type are determined according to the proportion of time-limited risk tasks from small to large, that is, 0.5 times the number of running scenarios with shared associated tasks are selected, and the running scenarios of the data of the server that do not have a shared control task type, the proportion of time-limited risk tasks meets the requirements and belongs to a type of preemption risk type are selected.

[0099] Example 2

[0100] Second, as Figure 5 As shown, in a second aspect, the present invention provides a distributed hierarchical scheduling device based on complex scenarios, which adopts the above-mentioned distributed hierarchical scheduling method based on complex scenarios, specifically including:

[0101] Priority classification module, task scheduling module, flood storage and processing module;

[0102] The priority analysis module is responsible for determining the priority classification processing of the task types in the operation scenario;

[0103] The task scheduling module is responsible for the priority of different task types and the task scheduling processing of different task types.

[0104] The flood storage processing module is responsible for determining the operation scenario of the server for flood control.

[0105] Specifically, the classification processing of the priority of the task type in the operation scenario is performed, specifically including:

[0106] According to the data processing time requirement of different task types, the classification processing of the priority of the task type is performed.

[0107] Specifically, the task type with a data processing time requirement less than 3 seconds, i.e., a processing time less than 3 seconds, is classified as a first priority, the task type with a data processing time requirement less than 10 seconds is classified as a second task type, the task type with a data processing time requirement less than 1 minute is classified as a third task type, the task type with a data processing time requirement less than 1 day is classified as a fourth task type, and the other data processing time requirement is classified as a fifth task type.

[0108] Further, the task scheduling processing of different task types is performed, specifically including:

[0109] According to the priority of different task types, the corresponding data processing channel is set, and the task scheduling processing of different task types is performed in different data processing channels and priority scheduling resource pools.

[0110] It can be understood that the same priority task type is divided into the same priority scheduling resource pool, the corresponding data processing channel is set for different priority scheduling resource pools, and the scheduling processing of different priority task types is performed.

[0111] Specifically, the following content is included:

[0112] I. Traffic priority division

[0113] According to the characteristics of the traffic, the priority is divided, and the high-priority traffic is preferentially scheduled (for example, according to the time efficiency definition, the priority of the time-consuming requirement is high)

[0114] II. "Seven-element control method"

[0115] 1. Task hierarchical registration: On the basis of self-defined granularity tasks, hot loading processors, dynamic task registration, and other technical capabilities are added to achieve general scheduling capabilities for stream and batch.

[0116] 2. Task routing matching: Multi-dimensional task matching strategy to ensure accurate hierarchical execution of tasks.

[0117] 3. Priority task scheduling: Abstracts multiple task scheduling modes such as FIXED, AUTO, and VIP, designs a task adaptive scheduling algorithm, and provides a VIP push mode channel and a flexible coexistence of non-VIP pull mode,

[0118] 4. Task work resource pool: Based on the basic work resource pool, a priority scheduling resource pool is designed to ensure absolute priority resource occupation of high-priority tasks.

[0119] 5. Task frequency execution: Abstracts OneOff and Cron scheduling strategies to support scheduling requirements in multiple business scenarios.

[0120] 6. Task timeout retry strategy: A task customization retry strategy and algorithm are designed to avoid intensive scheduling problems caused by abnormal fluctuations.

[0121] 7. Task flow control strategy: Based on resource and task dimensions, multiple flow control algorithms are integrated to achieve multi-dimensional accurate flow control.

[0122] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts of each embodiment can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0123] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or can be advantageous.

[0124] The above only describes one or more embodiments of the specification and does not limit the specification. One or more embodiments of the specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of one or more embodiments of the specification shall be included in the scope of the claims of the specification.

Claims

1. A distributed hierarchical scheduling method based on complex scenarios, characterized in that, Specifically comprising: Based on the simultaneous processing data of the task type of data processing, the running scene is divided, and the time efficiency preemption type of the running scene is determined according to the time efficiency requirement data of the task type under the running scene; The task type of the scene whose data processing time efficiency requirement is less than the preset processing duration is matched as a high time efficiency task type, and the time efficiency preemption type of the running scene is determined through the constituting data of the high time efficiency task type; The running scene of the task type is taken as an analysis scene, and when the task type does not belong to the time efficiency impact risk task according to the time efficiency preemption type corresponding to the analysis scene and the distribution data of other task types in different analysis scenes, the next step is entered, and the task type of data processing includes customer qualification review, risk assessment, parameter optimization of risk assessment model, and post-loan risk management of different credit approval task types; According to the sharing data of the data flow of the task type and other task types in the analysis scene, the distribution of the time efficiency impact risk task, the shared control task type in the task type is determined; According to the corresponding shared control task type of the data flow of the server in different running scenes, the time efficiency preemption type of the running scene without the shared control task type, the running scene of the server for flood storage control is determined, and the running scene for flood storage control is the running scene for reducing the speed of the data flow of the business data.

2. The distributed hierarchical scheduling method based on complex scenarios as claimed in claim 1, wherein, The simultaneous processing data of the task type includes the time period when the task type exists simultaneously in the history.

3. The method of claim 1, wherein the method is based on a complex scenario of distributed hierarchical scheduling, characterized by, The method for dividing the running scene is: Based on the simultaneous processing data of different task types of data processing, the time period when the simultaneous processing exists in the history is determined and taken as the simultaneous processing time period; According to the number of the simultaneous processing time period, whether the scene constructed by different data processing task types is a running scene is determined.

4. The method of claim 1, wherein, The data processing time efficiency requirement is the time length required for the data of the scene matching task type to complete data processing.

5. The distributed hierarchical scheduling method based on complex scenarios as claimed in claim 1, wherein, The method for determining the shared control task type in the task type is: According to the shared data of the data flow of other task types in different analysis scenes, the business data shared by other task types in the analysis scene when performing data processing is determined, and other task types existing the same business data are taken as shared task types; According to the constituting data of the shared optimization processing scene and the shared scene in different analysis scenes and the constituting data of the time efficiency impact risk task, the shared optimization processing scene and the shared scene in the analysis scene are determined; Based on the constituting data of the shared optimization processing scene and the shared scene in the analysis scene, whether the task type is a shared control task type is determined.

6. The distributed hierarchical scheduling method based on complex scenarios as claimed in claim 5, wherein, The shared optimization processing scene is an analysis scene in which the proportion of the number of time efficiency impact risk tasks meets the requirement and there is no shared task type.

7. The distributed hierarchical scheduling method based on complex scenarios as claimed in claim 1, wherein, The method for determining the running scene of the server for flood storage control is: Determine the shared control task type of the server data in different running scenarios based on the corresponding shared control task type of the server data flow, and determine the shared associated task as the shared control task type of the server data in different running scenarios, and determine the number of shared associated tasks in different running scenarios; Determine the number of running scenarios of one type of risk type in the running scenarios using the server data based on the time-sensitive preemption type of the running scenarios without the shared control task type, and determine the number of one type of scenarios as the number of running scenarios without the shared control task type and the time-sensitive impact risk task. Determine the running scenarios of the server for flood control according to the number of one type of scenarios and the number of shared associated tasks in different running scenarios.

8. A distributed hierarchical scheduling apparatus based on complex scenario, adopting the distributed hierarchical scheduling method based on complex scenario in any one of claims 1-7. Specifically includes: Priority classification module, task scheduling module, flood storage processing module; The priority analysis module is responsible for determining the priority classification of the task type in the running scenario; The task scheduling module is responsible for the priority of different task types and the task scheduling processing of different task types; The flood storage processing module is responsible for determining the running scenario of the server for flood control.

Images