Task scheduling method and device, equipment and medium

By obtaining detection tasks and resource information, and reasonably allocating tasks to detection nodes with sufficient resources, the problem of low detection resource utilization in the existing technology is solved, the efficiency of network resource detection is improved and server maintenance costs are reduced.

CN118945127APending Publication Date: 2024-11-12CHINA TELECOM NETWORK SECURITY TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410915468.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing cyberspace detection technology cannot make full use of detection resources when scheduling detection tasks, resulting in low detection efficiency.

Method used

By obtaining detection tasks and resource information, determining task types and priorities, reasonably allocating detection tasks to detection nodes with sufficient remaining resources to improve resource utilization and detection efficiency.

Benefits of technology

It improves the utilization rate of detection resources, reduces server maintenance costs, avoids the problem of excessive resource occupation of detection nodes, and improves the efficiency of network resource detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945127A_ABST
    Figure CN118945127A_ABST
Patent Text Reader

Abstract

The invention provides a task scheduling method and device, equipment and a medium, relates to the field of network technologies and security technologies, and is used for improving the utilization rate of detection resources so as to improve the detection efficiency of network resource detection. The method comprises the following steps that: first equipment determines a detection task with the highest priority of a task type from a plurality of obtained detection tasks according to the configured priority of the task type; the first device can determine the detection node corresponding to the detection task from the plurality of detection nodes according to the size relationship between the expected detection resource usage amount of the detection task and the residual resource amount of the plurality of detection nodes, and send the detection task to the detection node. Wherein the residual detection resource amount of the detection node is greater than or equal to the expected detection resource usage amount of the detection task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the fields of network technology and security technology, and in particular to a task scheduling method, device, equipment and medium. Background Art

[0002] Cyberspace detection technology is a technology that detects and identifies network resources. It can be used to sort out the information assets owned by enterprises and manage cyberspace efficiently. With the continuous development of network technology and the growth of enterprise demand for networks, the increase in network devices, the scale of enterprise networks is getting larger and larger, and the cyberspace is becoming more and more complex. In order to improve the utilization rate of enterprise cyberspace, enterprises usually use cyberspace detection technology to manage their cyberspace.

[0003] However, existing cyberspace detection technologies cannot fully utilize detection resources when scheduling detection tasks, resulting in low detection efficiency of cyberspace detection. Summary of the invention

[0004] The present application provides a task scheduling method, apparatus, device and medium for improving the utilization rate of detection resources, thereby improving the detection efficiency of network resource detection.

[0005] In a first aspect, the present application provides a task scheduling method, the method comprising: obtaining multiple detection tasks and detection resource information. The detection resource information is used to indicate the remaining resource amount of multiple detection nodes. The task types of multiple detection tasks are determined according to the corresponding relationship between the detection tasks and the task types. The expected detection resource usage of each task type in the multiple detection tasks is determined according to the saved expected resource usage. The expected resource usage is the expected detection resource usage of the detection task corresponding to the corresponding task type. According to the priority of the configured task type, the detection task with the highest priority of the task type is determined from the multiple detection tasks. According to the relationship between the expected detection resource usage of the detection task and the remaining resource amount of the multiple detection nodes, the detection node corresponding to the detection task is determined from the multiple detection nodes, and the remaining detection resource amount of the detection node is greater than or equal to the expected detection resource usage of the detection task. The detection task is sent to the detection node.

[0006] By adopting this method, the detection task to be dispatched in this application is the detection task with the highest priority among multiple detection tasks, so as to improve the flexibility of dispatching detection tasks. The detection node of the detection task is determined according to the expected detection resource usage of the detection task and the detection resource information, so as to reasonably dispatch the detection tasks, avoid the problem of excessive occupation of single detection resources of the detection node, improve the utilization rate of detection resources, and realize the dispatch of more detection tasks in the same detection resources, thereby improving the detection efficiency of network resource detection. In addition, the reasonable dispatch of detection tasks in this application can avoid server failures caused by large server resource consumption, thereby reducing the maintenance cost of the server.

[0007] In a possible embodiment, acquiring the detection resource information includes: receiving detection resource information from multiple detection nodes, wherein the detection resource information is obtained according to the total detection resource amount and the detection resource usage of the detection node.

[0008] Based on this embodiment, the total amount of detection resources and the detection resource usage of each detection node can be determined according to the acquired detection resource information of multiple detection nodes, so as to improve the rationality of detection task allocation and thus improve the utilization rate of detection resources.

[0009] In a possible embodiment, the expected resource usage corresponding to the task type is obtained according to the detection resource usage of multiple historical detection tasks, wherein the task type of the multiple historical detection tasks is the task type.

[0010] Based on this embodiment, this approach can improve the accuracy of obtaining the expected detection resource usage corresponding to the task type, so as to ensure the rationality of dispatching the detection task to the detection node.

[0011] In a possible embodiment, each task type corresponds to a task queue with a priority, and the multiple detection tasks are added to the corresponding task queues according to the task types of the multiple detection tasks.

[0012] In a possible embodiment, multiple coroutines are used to poll multiple task queues to obtain multiple detection tasks; and a detection task with the highest task type priority is determined from the multiple detection tasks according to the priority of the task type.

[0013] Based on this embodiment, the present application uses multiple coroutines to poll each task queue to obtain multiple detection tasks, which can improve the efficiency of obtaining multiple detection tasks. The detection task to be dispatched is the detection task with the highest priority among the task types of multiple detection tasks, which can give priority to dispatching detection tasks with higher priorities, so as to improve the flexibility of dispatching detection tasks.

[0014] In a second aspect, the present application provides a task scheduling device, which includes a communication module and a processing module.

[0015] The communication module is used to obtain multiple detection tasks and detection resource information, where the detection resource information is used to indicate the remaining resource amount of multiple detection nodes.

[0016] A processing module, configured to determine the expected detection resource usage of each task type in the plurality of detection tasks according to the saved expected resource usage. The expected resource usage is the expected detection resource usage of the detection task corresponding to the corresponding task type;

[0017] The processing module is further used to determine the detection task with the highest priority of the task type from the multiple detection tasks according to the configured priority of the task type;

[0018] The processing module is further used to determine the detection node corresponding to the detection task from the multiple detection nodes according to the relationship between the expected detection resource usage of the detection task and the remaining resource amounts of the multiple detection nodes, and the remaining detection resource amount of the detection node is greater than or equal to the expected detection resource usage of the detection task;

[0019] The communication module is also used to send detection tasks to the detection node.

[0020] In a possible embodiment, the communication module is specifically used to: receive detection resource information from the multiple detection nodes, where the detection resource information is obtained according to the total amount of detection resources and the detection resource usage of the detection nodes.

[0021] In a possible embodiment, the expected resource usage corresponding to the task type is obtained according to the detection resource usage of multiple historical detection tasks, wherein the task type of the multiple historical detection tasks is the task type.

[0022] In a possible embodiment, each task type corresponds to a task queue with a priority, and the processing module is further used to: add the multiple detection tasks to the corresponding task queues according to the task types of the multiple detection tasks.

[0023] In a possible embodiment, the communication module is specifically used to: poll multiple task queues in a multiple coroutine manner to obtain multiple detection tasks; and determine the detection task with the highest task type priority from the multiple detection tasks according to the priority of the task type.

[0024] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of the first aspect and the second aspect and any one of their designs is implemented.

[0025] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the first aspect and the second aspect and any one of the designed methods thereof.

[0026] The technical effects brought about by the second to fourth aspects and any one of their designs can refer to the technical effects brought about by the corresponding design in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0028] Figure 1 A schematic diagram of the structure of a task scheduling system provided in an embodiment of the present application;

[0029] Figure 2 A flowchart of a task scheduling method provided in an embodiment of the present application;

[0030] Figure 3 A flow chart of a state feedback method provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of a process for performing a detection task provided in an embodiment of the present application;

[0032] Figure 5 A flowchart of another task scheduling method provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of the structure of a task scheduling device provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of this application clearer, this application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the embodiments described in this application are only part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0036] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0037] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0038] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0039] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0041] Next, the method of detecting cyberspace assets is introduced in combination with the existing technology.

[0042] In the existing cyberspace asset detection technology, the cyberspace asset detection framework usually includes a task scheduling node and multiple detection nodes. The scheduling mechanism adopted usually includes a scheduling mechanism of a hash consistency algorithm and a scheduling mechanism based on the execution node load distribution scheduling task. The existing cyberspace asset detection technology usually evenly distributes the detection tasks to each detection node.

[0043] However, the resources consumed by detection tasks of different task types are often different, and the resources of each detection node and the occupied resources are also different. Therefore, the solution of evenly distributing detection tasks to each detection node cannot fully utilize the detection resources. For example, when multiple detection tasks with large central processing unit (CPU) consumption and small memory consumption are issued at the same time, although the CPU utilization rate of detection resources is high, the memory utilization rate is low, that is, the total utilization rate of detection resources is low, resulting in low detection efficiency of cyberspace asset detection.

[0044] Therefore, the existing cyberspace asset detection technology cannot reasonably utilize the utilization rate of detection resources, resulting in low detection efficiency of cyberspace asset detection.

[0045] In order to solve the above technical defects, the present application provides a task scheduling determination method and device for improving the utilization rate of detection resources, thereby improving the detection efficiency of network resource detection.

[0046] In the method, the first device can determine the detection task with the highest priority of the task type from the acquired multiple detection tasks according to the priority of the configured task type. The first device can determine the expected detection resource usage of each task type in the multiple detection tasks according to the saved expected resource usage. The expected resource usage is the expected detection resource usage of the detection task corresponding to the corresponding task type. The first device can determine the detection node corresponding to the detection task from multiple detection nodes according to the relationship between the expected detection resource usage of the detection task and the remaining resources of multiple detection nodes, and send the detection task to the detection node. The remaining detection resource amount of the detection node is greater than or equal to the expected detection resource usage of the detection task.

[0047] It is understandable that the detection task to be dispatched in the present application is the detection task with the highest priority among multiple detection tasks, so as to improve the flexibility of dispatching detection tasks. The detection node of the detection task is determined according to the expected detection resource usage of the detection task and the detection resource information, so as to reasonably allocate the detection tasks, avoid the problem of excessive occupation of single detection resources of the detection node, improve the utilization rate of the detection resources, and realize the allocation of more detection tasks in the same detection resources, thereby improving the detection efficiency of network resource detection. In addition, the reasonable allocation of detection tasks in the present application can avoid server failures caused by large server resource consumption, thereby reducing the maintenance cost of the server.

[0048] In addition, the first device may be a scheduling device for dispatching detection tasks, or the first device may be a partial device or module of a scheduling device for dispatching detection tasks. The first device may be included in a computer system for executing the method described in the present application, or the first device may be a processing device in a computer system for executing the method described in the present application, such as a processor or a processing module. For example, Figure 1 A structural diagram of a task scheduling system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the first device may be a scheduling node in the scheduling system, and the first device may also include multiple scheduling nodes.

[0049] Figure 2 A flowchart of a task scheduling method provided by an embodiment of the present invention. Taking the first device as the execution subject as an example, the process may include the following steps:

[0050] S201, a first device obtains a plurality of detection tasks and detection resource information, wherein the detection resource information is used to indicate the remaining resource amounts of a plurality of detection nodes.

[0051] In one or more embodiments, the first device may receive multiple detection tasks sent from other devices (such as a management device).

[0052] As an example, the present application also includes a management device, and the first device can send a task request for requesting a detection task to the management device. In addition, the task request can also be sent to the management device by other devices. The management device can send multiple detection tasks to the first device according to the task request. Accordingly, the first device receives multiple detection tasks from the management device.

[0053] As another example, the management device may generate multiple detection tasks according to business needs and send the multiple detection tasks to the first device. After generating the multiple detection tasks, the management device may send the detection tasks to the task queue corresponding to the first device. The first device may obtain the multiple detection tasks from the corresponding task queue according to needs. For example, Figure 1 As shown, the management node may represent a management device, the scheduling node may be represented as a first device, and the management node may send a detection task to the scheduling node.

[0054] In one or more embodiments, the first device may receive detection resource information from multiple detection nodes, wherein the detection resource information may be obtained based on the total detection resource amount and detection resource usage of the detection nodes.

[0055] The total amount of detection resources may refer to the total amount of detection resources of the detection engine included in the detection node. The detection resource usage may refer to the amount of detection resources currently used by the detection engine included in the detection node. The detection resources of the detection engine may include the detection engine host resource central processing unit (CPU), memory space, disk input (I) / output (O), number of network sessions and load, etc.

[0056] Based on this embodiment, the total amount of detection resources and the detection resource usage of each detection node can be determined according to the acquired detection resource information of multiple detection nodes, so as to improve the rationality of detection task allocation and thus improve the utilization rate of detection resources.

[0057] Optionally, the present application may also include a state feedback device for feeding back the resources of the detection engine, and the state feedback device may obtain the total amount of detection resources and the detection resource usage of the detection engines of multiple detection nodes.

[0058] For example, multiple detection nodes can send the detection resource usage of the current detection engine to the state feedback device according to a preset rule (e.g., timed transmission), and correspondingly, the state feedback device receives the detection resource usage of the detection engine. Alternatively, the state feedback device can obtain the current detection resource usage of multiple detection engines according to a preset rule (e.g., timed reading).

[0059] Furthermore, the state feedback device may generate detection resource information according to the total amount of detection resources and the usage of detection resources.

[0060] For example, the state feedback device can process the total amount of detection resources and the usage of detection resources to obtain detection resource information. For example, the detection resource information includes at least one of the usage rate of the detection engine host CPU, the memory space usage rate, the number of network sessions, and the disk usage rate. In addition, the detection resource information may also include the usage of other resources, which is not specifically limited in this application.

[0061] In addition, the state feedback device may send detection resource information to the first device.

[0062] In one or more embodiments, the detection node may send the execution status of the detection task of the detection node to the state feedback device, and the state feedback device forwards the execution status of the detection task to the first device. Alternatively, the detection node may directly send the execution status of the detection task to the first device. Accordingly, the first device receives the execution status of the detection task from the detection node. Among them, the execution status of the detection task can be used to indicate the progress of the detection task being executed. The execution status can be, for example, not executed, started, executing, completed, failed, timed out, and the like.

[0063] For example, Figure 3 A flow chart of a state feedback method provided in an embodiment of the present application. Figure 3 As shown, the task scheduling module can be represented as a first device, and the state feedback module can be represented as a state feedback device. The detection node can send detection resource information to the state feedback module through step S301, that is, the detection engine host CPU usage, the number of network sessions, the computing memory usage, the load, the disk I / O and other resource usage. The state feedback module can send the acquired detection resource information to the task scheduling module at a fixed time through step S302. The state feedback module can obtain the execution status of multiple detection tasks from the state channel through step S304, and send the execution status of multiple detection tasks to the task scheduling module through step S305.

[0064] Based on this embodiment, the state feedback device can obtain the detection resource information of multiple detection nodes and the execution status of the detection task, and feedback the detection resource information and the execution status of the detection task to the first device, so that the first device can dispatch the detection task according to the feedback information to improve the utilization rate of the detection resources. At the same time, it can also avoid the situation where the detection resources of the detection node are exhausted, and improve the stability of the operation of the detection node.

[0065] S202, the first device determines the task types of multiple detection tasks according to the correspondence between the detection tasks and the task types, and determines the expected detection resource usage of each task type in the multiple detection tasks according to the saved expected resource usage, wherein the expected resource usage is the expected detection resource usage of the detection task corresponding to the corresponding task type.

[0066] In one or more embodiments, the present application may include a plurality of different task types of detection tasks. The task type of the detection task may be set according to business requirements.

[0067] Alternatively, the task type of the detection task may also be determined based on the detection resource usage corresponding to the detection task. For example, detection tasks with the same detection resource usage or satisfying a certain threshold may be set to the same task type. Among them, the expected detection resource usage corresponding to detection tasks of different task types may be different.

[0068] In one or more embodiments, the expected resource usage corresponding to the task type may be obtained based on the detection resource usage of multiple historical detection tasks, wherein the task types of the multiple historical detection tasks are all the task type.

[0069] Exemplarily, the system log may record the detection resource usage of each detection task and the task type corresponding to the detection task. The first device may determine the expected detection resource usage corresponding to the task type based on the detection resource usage corresponding to multiple historical detection tasks of the specified task type recorded in the system log, and save the expected detection resource usage.

[0070] As an example, when the detection resource usages of multiple historical detection tasks corresponding to a task type are all the same value, the value of the detection resource usage may be used as the value of the expected detection resource usage corresponding to the task type.

[0071] As another example, when the detection resource usages of multiple historical detection tasks corresponding to a task type are different values, the maximum value of the detection resource usages corresponding to the multiple historical detection tasks may be used as the expected detection resource usage corresponding to the task type.

[0072] In addition, in the present application, detection tasks with the same detection resource usage can be classified into one task type. For example, if the detection resource usage of the first detection task is the same as the resource usage of the second detection task, it can be determined that the task type of the first detection task is the same as the task type of the second detection task.

[0073] Alternatively, in the present application, detection tasks whose detection resource usage meets a certain threshold may be classified into a task type. For another example, detection resources include CPU, network sessions, and computing memory. The expected detection resource usage corresponding to the first task type is that the CPU usage is less than the first threshold, the number of network sessions is less than the second threshold, and the computing memory usage is greater than the third threshold and less than the fourth threshold. When the detection resource usage of the first detection task and the detection resource usage of the second detection task both meet the expected detection resource usage corresponding to the first task type, it is determined that the task type of the first detection task and the task type of the second detection task are the same, that is, the task type of the first detection task and the task type of the second detection task are both the first task type.

[0074] It is understandable that the task type in this application may refer to a task whose detection resource usage meets the set requirements. The task type in this application may also refer to a task with similar business types and whose detection resource usage meets the set requirements. In other words, this application can determine the task type of the detection task based on the business requirements and the detection resource usage of the detection task.

[0075] Based on this embodiment, this approach can improve the accuracy of obtaining the expected detection resource usage corresponding to the task type, so as to ensure the rationality of dispatching the detection task to the detection node.

[0076] In one or more embodiments, after determining the task types of the multiple detection tasks, the first device adds the multiple detection tasks to corresponding task queues according to the task types of the multiple detection tasks.

[0077] Optionally, the priorities of multiple detection tasks of the same task type may be different. The first device may determine the order of the detection task in the task queue according to the priority of the detection task. For example, when the priority corresponding to the first detection task is the highest priority, the first device may determine that the first detection task is the detection task that is executed preferentially in the task queue.

[0078] At the same time, the priorities of multiple detection tasks of the same task type may also be the same. The first device may determine the order of the detection tasks in the task queue according to the priority and task time of the detection tasks. For example, when the priority of the first detection task is the same as the priority of the second detection task, the first device may determine the execution order corresponding to the first detection task and the second detection task according to the task time of the first detection task and the task time of the second detection task.

[0079] S203: The first device determines a detection task with the highest priority of the task type from among the multiple detection tasks according to the configured priorities of the task types.

[0080] In the present application, different priorities can be set for each task type according to business requirements to distinguish the importance and / or dispatch priority of different task types.

[0081] In one or more embodiments, the first device may poll multiple task queues in a multiple coroutine manner to obtain multiple detection tasks. The first device may determine the detection task with the highest task type priority from the multiple detection tasks according to the priority of the task type.

[0082] It is understandable that the queue for dispatching detection tasks can be in the form of a linked list array. The array controls task types of different priorities. When the queue performs a pop operation, it can obtain the detection task with the highest priority in the linked list, so that subsequent high-priority tasks can also be sent first, so as to improve the flexibility of scheduling detection tasks.

[0083] S204, the first device may determine a detection node corresponding to the detection task from the multiple detection nodes according to the relationship between the expected detection resource usage of the detection task and the remaining resource amounts of the multiple detection nodes, wherein the remaining detection resource amount of the detection node is greater than or equal to the expected detection resource usage of the detection task.

[0084] In one or more embodiments, the detection resource information may include the remaining detection resource amount of each detection node in the multiple detection nodes. Alternatively, the detection resource information may include the total detection resource amount and the used detection resource amount of each detection node in the multiple detection nodes, and the first device may determine the remaining detection resource amount of each detection node in the multiple detection nodes according to the detection resource information.

[0085] Furthermore, the first device may select a first detection node from multiple detection nodes according to the expected detection resource usage of the detection task and the remaining detection resource amount of each detection node in the detection resource information. The first detection node may refer to a detection node whose remaining detection resource amount is greater than or equal to the expected detection resource usage.

[0086] For example, if the usage of CPU resources in the expected detection resources of the website component detection task is high, the detection node with the lowest CPU usage and the remaining detection resources greater than or equal to the expected detection resource usage of the website component detection task can be determined from the multiple detection nodes based on the detection resource information of the multiple detection nodes.

[0087] It can be understood that the first detection node may refer to a plurality of detection nodes whose remaining detection resource amount is greater than or equal to the expected detection resource usage amount.

[0088] As an example, when the first detection node is a plurality of detection nodes, the first device may determine a detection node with the largest amount of remaining detection resources from the plurality of detection nodes, thereby ensuring the performance of each detection node.

[0089] As another example, when the first detection node is a plurality of detection nodes, the first device may determine a detection node with the smallest amount of remaining detection resources from the plurality of detection nodes, thereby improving the utilization rate of the detection resources.

[0090] In one or more embodiments, the present application can set a corresponding priority for a task type according to business needs. Different task types can correspond to the same priority. When the priorities of the task types are the same, the first device can determine the detection node that executes the task type according to the task time of the first detection task in the task queue corresponding to the task type. In other words, when the priorities of the task types are the same, the order of executing the detection tasks is determined according to the task time of the first detection task in the task queue corresponding to the task type. Among them, the first detection task can refer to the detection task that is executed first in the task type.

[0091] In addition, different task types may also correspond to different priorities. When the priorities of the task types are different, the first device may determine the detection node that executes the task type from multiple detection nodes based on the priorities of the task types, that is, when the priorities of the task types are different, the order of executing the detection tasks is determined based on the priorities of the task types. It is understandable that when the first device simultaneously dispatches and runs multiple detection tasks of different task types in parallel, the dispatch order of the detection tasks may be determined based on the priorities of the task types corresponding to the detection tasks, thereby improving the flexibility of dispatching multiple detection tasks. Alternatively, when the first device dispatches a new detection task, the insertion position of the detection task in the task queue to be dispatched may be determined based on the priority of the task type corresponding to the detection task, thereby improving the flexibility of dispatching detection tasks.

[0092] In addition, when the remaining resource amount of the detection node is less than the expected detection resource usage of the task type with a higher priority, a detection node corresponding to the task type with a lower priority is determined from multiple detection nodes. The remaining detection resource usage of the detection node is greater than or equal to the expected detection resource usage of the task type. For example, the priority of the first task type is higher than the priority of the second task type. When the remaining detection resource usage of each detection node in the multiple detection nodes is less than the expected detection resource usage of the first task type, and is greater than or equal to the expected detection resource usage of the second task type, a detection node corresponding to the second task type is determined from the multiple detection nodes.

[0093] It is understandable that when the remaining resource amount of the detection node is less than the expected detection resource usage of the task type with a higher priority, the first device can send a request for updating the detection resource information to the state feedback device. The state feedback device can send the updated detection resource information to the first device according to the request. Correspondingly, the first device can determine the detection node corresponding to the task type from multiple detection nodes according to the updated detection resource information.

[0094] Optionally, in the present application, when the remaining resources of the detection node are less than the expected detection resource usage of the task type with a higher priority, the first device may send a prompt message so that the operator can perform corresponding operations according to the prompt message. The method of sending the prompt message may be, for example, an alarm pop-up window, a log error, or sending an email.

[0095] S204: The first device sends a detection task to the detection node.

[0096] The first device and the detection node may be connected via a wired interface and / or a wireless interface. The first device may send a detection task to the detection node via a wired interface and / or a wireless interface. Correspondingly, the detection node may receive a detection task from the detection node via a wired interface and / or a wireless interface. For example, Figure 1 As shown, the scheduling node can be represented as a first device, and the scheduling node can be connected to the detection node through a wired interface. Accordingly, the scheduling node can send a detection task to the detection node through the wired interface.

[0097] In one or more embodiments, after receiving the detection task, the detection node may add the detection task to the task queue corresponding to the detection node. Optionally, the detection node may execute the detection task according to the order of the detection tasks in the task queue of the detection node.

[0098] For example, Figure 4 A schematic diagram of a process for executing a detection task provided in an embodiment of the present application. With the detection node as the execution subject, the process includes the following steps:

[0099] S401, the detection node receives a detection task and adds the detection task to a task queue.

[0100] The detection node may receive the detection task from the first device, and add the detection task to a task queue corresponding to the detection node.

[0101] In addition, the detection node can also set the execution order of the detection tasks in the task queue according to demand.

[0102] S402, the detection node starts the coroutine pool and polls the detection tasks in the execution task queue.

[0103] The detection node can poll the detection tasks in the task queue according to the remaining detection resources.

[0104] S403: The detection node determines a detection engine according to the task type of the detection task.

[0105] The detection node may determine a detection engine for executing the detection task from a plurality of detection engines according to the task type of the detection task. Accordingly, the detection engine may execute the detection task.

[0106] S404: The detection node sends the execution status of the detection task to the status channel.

[0107] After the detection node determines the detection engine corresponding to the detection task, it can send the execution status of the detection task to the status channel so that the status feedback device can obtain the execution status of the detection task from the status channel. The execution status of the detection task may include not executed, started, executing, completed, failed, timed out, etc. In addition, when the execution status of the detection task changes, the detection node can send the updated execution status of the detection task to the status channel. Figure 3 As shown, the detection node can send the execution status of the detection task to the status channel through step S303.

[0108] It is understandable that the detection node may also send the execution status of the detection task to the state feedback device. Correspondingly, the state feedback device receives the execution status of the detection task.

[0109] In one or more embodiments, the state feedback device can send the current execution status of the detection task to the first device. The first device can perform corresponding operations according to preset rules based on the current execution status of the detection task. For example, when the current execution status of the detection task is timed out or failed, the first device can add the detection task to the task queue corresponding to the task type so as to resend the detection task to avoid missing the execution of the detection task. The first device can also record the number of times the detection task is resent. When the number of resends exceeds a set threshold, the first device can stop sending the detection task to avoid the detection task occupying a position in the task queue for a long time, thereby ensuring the efficiency of dispatching detection tasks.

[0110] In addition, the first device may also send a prompt message indicating that the number of times the detection task is resent exceeds a set threshold, so that the operator can perform corresponding operations according to the prompt message. The prompt message may be sent in the form of a pop-up window, a log error, or an email.

[0111] In one or more embodiments, after the detection node starts to execute the detection task, the current detection resource information of the detection node can be fed back to the first device. For example, after the detection node starts to execute the detection task, it can send the current detection resource usage and total detection resource amount of the detection node to the state feedback device. The state feedback device can determine the detection resource information of the detection node based on the detection resource usage and total detection resource amount of the detection node, and send the detection resource information to the first device so that the first device can dispatch new detection tasks based on the updated detection resource information. It can be understood that this method can be used to cyclically dispatch multiple detection tasks, fully utilize detection resources, and improve the detection efficiency of cyberspace asset detection.

[0112] In one or more embodiments, the first device may start multiple coroutines to respectively poll the detection tasks in the task queues corresponding to the multiple task types to improve the utilization of detection resources. The specific manner in which the first device polls the detection tasks in the task queues corresponding to the multiple task types can be found in steps S201 to S204.

[0113] The following takes Example 1 as an example to introduce in detail the scheduling of detection tasks in this application. In Example 1, the detection resources are CPU and computing memory as an example, and the detection node of the detection task is determined according to the expected detection resource usage of the detection task. Figure 5A flowchart of a task scheduling method provided in an embodiment of the present application, with a scheduling node as the execution subject, may include the following contents:

[0114] S501, the scheduling node cyclically obtains detection tasks from the dispatch queue.

[0115] S502: The scheduling node determines the task type of the detection task according to the pre-saved correspondence relationship of the task types.

[0116] Exemplarily, the specified character or string of the task identifier of the detection task can represent the task type of the detection task. Different task types correspond to different character strings. For another example, the task type of the detection task is determined based on the correspondence between the name and / or identifier of the pre-selected and saved historical detection tasks and the task type, as well as the name and / or identifier of the detection task.

[0117] A task type may correspond to a task queue with a priority, and the priorities of task queues corresponding to different task types may be the same or different.

[0118] The scheduling node can start multiple coroutines to loop through the detection tasks in each task queue, and determine the dispatch or execution order of the detection tasks according to the priority of the task queue, thereby giving priority to dispatching or executing detection tasks with higher priority.

[0119] S503: The scheduling node determines whether the detection node can operate normally.

[0120] When the scheduling node determines that the detection node cannot operate normally, step S504 is executed.

[0121] When the scheduling node determines that the detection node can operate normally, step S505 is executed.

[0122] S504, the scheduling node clears the task queue.

[0123] When the detection node cannot operate normally, the scheduling node can delete the detection task in the task queue. The scheduling node can also send a prompt message to the management node indicating that the detection node cannot operate normally, so that the management node can perform corresponding operations according to the prompt message. For example, after receiving the prompt message, the management node can distribute the detection task in the task queue to other scheduling nodes according to the distribution strategy, thereby avoiding the loss of the detection task.

[0124] S505: The scheduling node determines that the detection task to be dispatched is a non-suspended task.

[0125] When the scheduling node determines that the detection task to be dispatched is a suspended task, the scheduling node can move the detection task to the suspension queue and reacquire other detection tasks. It is understandable that the management node can move the suspended detection task from the suspension queue to the dispatch queue to resume dispatching the detection task.

[0126] When the scheduling node determines that the detection task to be dispatched is a non-suspended task, the scheduling node executes step S506.

[0127] S506: The scheduling node may determine a corresponding detection node according to a task type of the detection task.

[0128] The scheduling node may determine, according to the task type of the detection task, whether the main detection resource used by the detection task is the CPU or the memory, and the detection resource usage of the detection task.

[0129] When the main detection resource used by the detection task is the CPU, the scheduling node can determine the detection node with the lowest CPU usage from multiple detection nodes based on the detection resource information, and determine whether the remaining detection resource amount of the detection node meets the detection resource usage of the detection task. If the remaining detection resource amount of the detection node does not meet the detection resource usage of the detection task, the detection task is moved to the task queue. If the remaining detection resource amount of the detection node meets the detection resource usage of the detection task, the detection task is sent to the detection node.

[0130] When the main detection resource used by the detection task is memory, the scheduling node can determine the detection node with the lowest memory usage from multiple detection nodes based on the detection resource information, and determine whether the remaining detection resource amount of the detection node meets the detection resource usage of the detection task. If the remaining detection resource amount of the detection node does not meet the detection resource usage of the detection task, the detection task is moved to the task queue. If the remaining detection resource amount of the detection node meets the detection resource usage of the detection task, the detection task is sent to the detection node.

[0131] S507: The scheduling node dispatches a detection task to the detection node.

[0132] Optionally, when the detection task fails to execute or times out, the detection task can be moved to a corresponding task queue and redispatched.

[0133] Understandably, when the CPU, memory and other resource usage of all detection nodes exceeds the upper limit, the scheduling node can send a message notification to the management node so that the management node dispatches detection task packages to other scheduling nodes, thereby avoiding high pressure on the scheduling node.

[0134] Based on the same technical concept, on the basis of the above embodiments, in the embodiment of the present application, a task scheduling device is provided. Figure 6 A schematic diagram of a task scheduling device provided in some embodiments of the present application. Figure 6 As shown, the device comprises:

[0135] The communication module 601 is used to obtain multiple detection tasks and detection resource information, where the detection resource information is used to indicate the remaining resource amount of multiple detection nodes.

[0136] The processing module 602 is used to determine the expected detection resource usage of each task type in the multiple detection tasks according to the saved expected resource usage. The expected resource usage is the expected detection resource usage of the detection task corresponding to the corresponding task type;

[0137] The processing module 602 is further configured to determine a detection task with the highest priority of the task type from among the multiple detection tasks according to the configured priority of the task type;

[0138] The processing module 602 is further used to determine a detection node corresponding to the detection task from multiple detection nodes according to the relationship between the expected detection resource usage of the detection task and the remaining resource amounts of the multiple detection nodes, and the remaining detection resource amount of the detection node is greater than or equal to the expected detection resource usage of the detection task;

[0139] The communication module 601 is also used to send the detection task to the detection node.

[0140] In a possible embodiment, the communication module 601 is specifically used to: receive the detection resource information from the multiple detection nodes, where the detection resource information is obtained according to the total amount of detection resources and the detection resource usage of the detection node.

[0141] In a possible embodiment, the expected resource usage corresponding to the task type is obtained according to the detection resource usage of multiple historical detection tasks, wherein the task type of the multiple historical detection tasks is the task type.

[0142] In a possible embodiment, each task type corresponds to a task queue with a priority, and the processing module 602 is further configured to: add the multiple detection tasks to the corresponding task queues respectively according to the task types of the multiple detection tasks.

[0143] In a possible embodiment, the communication module 601 is specifically used to: poll multiple task queues to obtain multiple detection tasks in a multiple coroutine manner; and determine the detection task with the highest task type priority from the multiple detection tasks according to the priority of the task type.

[0144] Based on the same inventive concept, an embodiment of the present application provides an electronic device, which can realize the functions of the device discussed above. Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.

[0145] The electronic device in the embodiment of the present application may include a processor 701. The processor 701 is the control center of the device, and various interfaces and lines can be used to connect various parts of the device, by running or executing instructions stored in the memory 703 and calling data stored in the memory 703. Optionally, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 703 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0146] The processor 701 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The method steps disclosed in the embodiments of the present application may be directly executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0147] In the embodiment of the present application, the memory 703 stores instructions that can be executed by at least one processor 701. The at least one processor 701 can be used to execute the method steps disclosed in the embodiment of the present application by executing the instructions stored in the memory 703.

[0148] The memory 703 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 703 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (RAM), a static random access memory (SRAM), a programmable read-only memory (PROM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 703 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 703 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0149] In the embodiment of the present application, the device may further include a communication interface 702 , through which the electronic device may transmit data.

[0150] Optional, can be Figure 7 The processor 701 (or the processor 701 and the communication interface 702) implements Figure 6 The processing module 602 and / or the communication module 601 shown, that is, the actions of the processing module 602 and / or the communication module 601 can be executed by the processor 701 (or the processor 701 and the communication interface 702).

[0151] Based on the same inventive concept, the embodiment of the present application also provides a computer-readable storage medium, which may store instructions. When the instructions are executed on a computer, the computer executes the operation steps provided in the above method embodiment. The computer-readable storage medium may be Figure 7 The memory 703 is shown.

[0152] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0153] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby executing the program on the computer or other programmable device.

Claims

1. A task scheduling method, characterized in that: The method comprises: Acquire multiple detection tasks and detection resource information, where the detection resource information is used to indicate the remaining detection resource amount of multiple detection nodes; Determine the task types of the multiple detection tasks according to the correspondence between the detection tasks and the task types, and determine the expected detection resource usage of each task type in the multiple detection tasks according to the saved expected resource usage, wherein the expected resource usage is the expected detection resource usage of the detection task corresponding to the corresponding task type; Determining, according to the priorities of the configured task types, a detection task with the highest priority of the task type from the multiple detection tasks; Determine, from the multiple detection nodes, a detection node corresponding to the detection task according to a relationship between an expected detection resource usage of the detection task and remaining resources of the multiple detection nodes, wherein the remaining detection resource of the detection node is greater than or equal to the expected detection resource usage of the detection task; The detection task is sent to the detection node.

2. The method according to claim 1, characterized in that The obtaining of detection resource information includes: Receive detection resource information from the multiple detection nodes, where the detection resource information is obtained based on the total amount of detection resources and the detection resource usage of the detection nodes.

3. The method according to claim 1, characterized in that The expected resource usage corresponding to the task type is obtained according to the detection resource usage of multiple historical detection tasks, and the task types of the multiple historical detection tasks are the task type.

4. The method according to claim 1, characterized in that Each task type corresponds to a task queue with a priority, and the method further includes: The multiple detection tasks are added to corresponding task queues respectively according to the task types of the multiple detection tasks.

5. The method according to claim 4, characterized in that The step of determining the detection task with the highest priority of the task type from the multiple detection tasks according to the priority of the task type includes: Use multiple coroutines to poll multiple task queues to obtain multiple detection tasks; A detection task with the highest task type priority is determined from the plurality of detection tasks according to the priority of the task type.

6. A task scheduling device, characterized in that: The device comprises: A communication module, used to obtain multiple detection tasks and detection resource information, wherein the detection resource information is used to indicate the remaining detection resource amount of multiple detection nodes; A processing module, configured to determine the task types of the plurality of detection tasks according to a correspondence between the detection tasks and the task types, wherein the expected detection resource usage of detection tasks of the same task type is the same, and different task types correspond to different priorities; The processing module is further configured to determine a detection task with the highest priority of the task type from among the multiple detection tasks according to the priority of the task type; The processing module is further used to determine the detection node corresponding to the detection task from the multiple detection nodes according to the relationship between the expected detection resource usage of the detection task and the remaining resource amounts of the multiple detection nodes, and the remaining detection resource amount of the detection node is greater than or equal to the expected detection resource usage of the detection task; The communication module is further used to send the detection task to the detection node.

7. The device according to claim 6, characterized in that The acquisition of detection resource information, the communication module is specifically used for: Receive detection resource information from the multiple detection nodes, where the detection resource information is obtained based on the total amount of detection resources and the detection resource usage of the detection nodes.

8. The device according to claim 6, characterized in that The expected resource usage corresponding to the task type is obtained according to the detection resource usage of multiple historical detection tasks, and the task types of the multiple historical detection tasks are the task type.

9. An electronic device, characterized in that: The electronic device comprises at least a processor and a memory, and the processor is configured to implement the steps of any one of the methods described in claims 1 to 5 when executing a computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the steps of any one of the methods described in claims 1-5.