Job Scheduling Method and Device for a Security Orchestration and Automation Response System

Abstract

The present invention provides a job scheduling method and device for a security orchestration and automation response system. The method includes: triggering and generating a call request through an alarm module; determining the generation speed of the call request, and calculating the queue length of script instances currently in the script scheduling queue when it is determined that the generation speed of the call request is less than the entrance speed limit threshold; when it is determined that the queue length of script instances currently in the script scheduling queue is less than the length threshold, generating a new script instance in response to the call request and adding the new script instance to the script scheduling queue; and scheduling the script instances in the script scheduling queue through a script scheduling center. The job scheduling method for the security orchestration and automation response system provided by the present invention realizes processing script instances at a rate adapted to system resources and load through the script scheduling center, thereby ensuring the stability of the security orchestration and automation response system.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a job scheduling method and device for a security orchestration and automation response system. Background Art

[0002] A playbook instance is a process strategy that combines data, technologies / tools, and people / teams, and is a guarantee for the security response standardization, consistency, and efficiency of a security orchestration and automation response system.

[0003] In daily security operations, the alert module triggers the invocation of a playbook instance manually or by configuring an alert response action to trigger automatically. When the configured rules for automatic triggering are unreasonable, or when there is a peak in security alert data, it will cause a large number of alerts to frequently invoke playbook instances, affecting the stability of the system.

[0004] Ensuring the stability of the security orchestration and automation response system is a technical problem that urgently needs to be solved at present. Summary of the Invention

[0005] In view of the problems in the prior art, embodiments of the present invention provide a job scheduling method and device to ensure the stability of a security orchestration and automation response system and improve the throughput of the system at the same time.

[0006] The present invention provides a job scheduling method for a security orchestration and automation response system, including:

[0007] Generating an invocation request through the alert module;

[0008] Determining the generation speed of the invocation request, and calculating the queue length of the playbook instances currently in the playbook scheduling queue when it is determined that the generation speed of the invocation request is less than the entry speed limit threshold;

[0009] When it is determined that the queue length of the playbook instances currently in the playbook scheduling queue is less than the length threshold, generating a new playbook instance in response to the invocation request and adding the new playbook instance to the playbook scheduling queue;

[0010] Scheduling the playbook instances in the playbook scheduling queue through the playbook scheduling center.

[0011] According to the job scheduling method for a security orchestration and automation response system provided by the present invention, after generating a new playbook instance in response to the invocation request, the method further includes:

[0012] Generating a unique identifier for the new playbook instance and returning the unique identifier to the alert module;

[0013] The alarm module queries the execution status of the new playbook instance from the playbook scheduling center according to the unique identifier of the new playbook instance.

[0014] According to a job scheduling method of a security orchestration and automation response system provided by the present invention, after generating a new playbook instance in response to the call request, the method further includes:

[0015] Save the new playbook instance to the database;

[0016] When the playbook scheduling center generates the execution status of the new playbook instance, the playbook scheduling center saves the execution status of the new playbook instance to the database;

[0017] When a failure causes the loss of the playbook instance in the queue, reload the playbook instance in the database into the queue.

[0018] According to a job scheduling method of a security orchestration and automation response system provided by the present invention, after determining the generation speed of the call request, the method further includes:

[0019] When it is determined that the generation speed of the call request is greater than or equal to the entry speed limit threshold, trigger the expansion of the security orchestration and automation response system, and return an information notice to the alarm module;

[0020] After calculating the queue length of the playbook instances currently in the playbook scheduling queue, the method further includes:

[0021] When it is determined that the queue length of the playbook instances currently in the playbook scheduling queue is greater than or equal to the length threshold, trigger the expansion of the security orchestration and automation response system, and return an information notice to the alarm module.

[0022] According to a job scheduling method of a security orchestration and automation response system provided by the present invention, the playbook instance includes multiple task instances, and each playbook instance corresponds to a main thread;

[0023] Scheduling the playbook instances in the playbook scheduling queue through the playbook scheduling center includes:

[0024] Submit the playbook instance to the main thread through the playbook scheduling center, and determine whether the number of core threads reaches the first threshold;

[0025] When the number of core threads does not reach the first threshold, create core threads to process the multiple task instances corresponding to the playbook instance;

[0026] When the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue does not reach the second threshold, submit the task instance to the task scheduling queue, and submit the task instances in the task scheduling queue to the core threads for processing in sequence.

[0027] According to a job scheduling method of a security orchestration and automated response system provided by the present invention, the method further includes:

[0028] When it is determined that the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue reaches the second threshold, judge whether the total number of threads corresponding to the task instance reaches the third threshold through the script scheduling center;

[0029] If it does not reach the third threshold, create a new thread and submit the task instance to the new thread for processing;

[0030] If it reaches the third threshold, determine the target task instance in the task scheduling queue and discard it according to the preset rejection policy.

[0031] According to a job scheduling method of a security orchestration and automated response system provided by the present invention, the rejection policy includes: a priority response policy or a response ratio policy; wherein, the response ratio is determined by the waiting time and the execution required time of the task instance;

[0032] Determining the target task instance in the task scheduling queue and discarding it according to the preset rejection policy includes:

[0033] Determine the response ratio of each task instance in the task scheduling queue, and according to the response ratio policy, determine the task instance with a response ratio lower than the response ratio threshold as the target task instance and discard it; or

[0034] Determine the priority of each task instance in the task scheduling queue, and according to the priority response policy, determine at least one task instance with a lower priority as the target task instance and discard it.

[0035] According to a job scheduling method of a security orchestration and automated response system provided by the present invention, during the process of processing the task instance by the core thread or the new thread, the method further includes:

[0036] When it is determined that the current task instance fails to execute, execute the corresponding retry of the current task instance, skip the current task instance, or terminate the script instance corresponding to the current task instance according to the preset task instance failure policy;

[0037] Wherein, the task instance failure policy includes: a failure skip task policy, a failure retry policy, or a failure terminate script policy.

[0038] According to a job scheduling method of a security orchestration and automated response system provided by the present invention, after executing the corresponding playbook instance for retrying the current task instance, skipping the current task instance, or terminating the current task instance, the method further includes:

[0039] Generating exception information and feeding back the exception information to the main thread of the playbook instance corresponding to the current task instance.

[0040] The present invention also provides a job scheduling device for a security orchestration and automated response system, including:

[0041] An alarm module for triggering and generating a call request;

[0042] A calculation module for determining the generation speed of the call request, and calculating the queue length of the playbook instances currently in the playbook scheduling queue when it is determined that the generation speed of the call request is less than the entry speed limit threshold;

[0043] A generation module for generating a new playbook instance in response to the call request and adding the new playbook instance to the playbook scheduling queue when it is determined that the queue length of the playbook instances currently in the playbook scheduling queue is less than the length threshold;

[0044] A scheduling module for scheduling the playbook instances in the playbook scheduling queue through a playbook scheduling center.

[0045] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the job scheduling method of the security orchestration and automated response system as described in any one of the above when executing the program.

[0046] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the job scheduling method of the security orchestration and automated response system as described in any one of the above.

[0047] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the job scheduling method of the security orchestration and automated response system as described in any one of the above.

[0048] The job scheduling method and device of the security orchestration and automated response system provided by the embodiments of the present invention control the invocation rate of playbook instances by setting an ingress rate limit threshold, and schedule the playbook instances in the playbook scheduling queue through a playbook scheduling center, so as to process the playbook instances at a rate adapted to the system resources and load through the playbook scheduling center, thereby ensuring the stability of the security orchestration and automated response system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0050] Figure 1 is one of the flowcharts of the job scheduling method of the security orchestration and automated response system provided by the present invention;

[0051] Figure 2 is the flowchart of the primary job scheduling method of the security orchestration and automated response system provided by the present invention;

[0052] Figure 3 is the second flowchart of the job scheduling method of the security orchestration and automated response system provided by the present invention;

[0053] Figure 4 is the third flowchart of the job scheduling method of the security orchestration and automated response system provided by the present invention;

[0054] Figure 5 is the flowchart of the secondary job scheduling method of the security orchestration and automated response system provided by the present invention;

[0055] Figure 6 is the fourth flowchart of the job scheduling method of the security orchestration and automated response system provided by the present invention;

[0056] Figure 7 is the structural diagram of the job scheduling device of the security orchestration and automated response system provided by the present invention;

[0057] Figure 8 is the structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0059] Before elaborating on the method of the present invention in detail, the noun terms involved in the present invention will be schematically explained first.

[0060] Security Orchestration, Automation and Response (SOAR): It combines the handling of security incidents with workflows and makes full use of threat intelligence to achieve orchestration and automated response, thereby improving the productivity of NGSOC and shortening the incident response time. Security Orchestration, Automation and Response can help organizations free security analysts from low-priority tasks and improve the overall efficiency of NGSOC in security incident response.

[0061] Playbook: A process & strategy that combines data, technology / tools, and people / teams, which is the guarantee of the standardization, consistency, and efficiency of security response; it is a process that can be followed, tracked, measured, and continuously improved.

[0062] Task: It is the basic component of a Playbook, atomic in terms of business, highly cohesive in terms of functional design, and is the basic orchestration unit. A task provides the ability to respond to security incidents. In addition to the security capabilities built by itself, it can also expand the security response capabilities by docking with third-party security devices or integrating services (task plugins).

[0063] Overload Protection: In the present invention, it refers to providing a corresponding protection mechanism for the security orchestration and automation response system, so that the incoming traffic of the system (i.e., the requests for continuously generating security alerts to invoke the playbook for automated handling) and the load capacity of the system reach a balance, ensuring that the system processes the most requests within its capacity range.

[0064] Rate Limiting: Considering the system stability, when the speed of processing requests exceeds a certain threshold, traffic control is taken.

[0065] Concurrency Limiting: By limiting the number of concurrent threads, when a working thread becomes unstable, such as a longer response time, the most direct impact on the system is that the number of threads will gradually accumulate. When the number of threads accumulates to a certain number, new requests will be rejected, and new requests will not be accepted until the accumulated threads complete their tasks. In the present invention, it refers to the concurrency control of the call of scripts and script tasks when handling security alerts.

[0066] QPS (Queries Per Second): The number of queries / requests per second, that is, the number of queries / requests that can be responded to per second, refers to the number of times the client sends a request and the server responds successfully.

[0067] RT (Response Time): The time difference from when the client initiates a request to when the server receives the request and responds. Generally, the average response time is taken.

[0068] Combine the following Figures 1 - 8 The present invention describes a method and device for scheduling jobs in a security orchestration and automated response system.

[0069] Figure 1 The flowchart of the job scheduling method of the security orchestration and automated response system provided by the present invention is as follows: Figure 1 As shown, the job scheduling method of the security orchestration and automated response system provided by the present invention includes:

[0070] Step 100: triggering generation of a call request through an alarm module.

[0071] In daily security operations, the alarm module can be triggered manually or automatically by configuring an alarm response action to generate a call request for the script instance.

[0072] Step 200: Determine the generation speed of the call request, and when it is determined that the generation speed of the call request is less than the entry speed limit threshold, calculate the queue length of the script instance currently in the script scheduling queue.

[0073] It should be noted that in the process of triggering the alarm module to generate call requests, the alarm module will generate call requests at a high frequency due to unreasonable alarm response rules configured by automatic triggering or peak values of security alarm data. In the process of script instance calling, many operations will involve third-party devices or platforms, such as requesting the threat intelligence platform to do data matching, linking the terminal or host security management platform to issue action instructions, linking the network boundary device to issue blocking strategies, calling the mail system / SMS gateway / instant messaging system to issue notifications, etc. The number of queries / requests per second (QPS) of such operations is not high, but the average response time (RT) is very high. In addition, in the complex network environment of customers, the operating status of the deployed third-party devices or platforms is often uncontrollable for the security orchestration and automated response system, which can easily lead to script instance scheduling blocking, thread accumulation, and resource occupation without release, further affecting the system throughput.

[0074] Based on this, this embodiment will preset an entry speed limit threshold at the entrance to control the speed at which the alarm module generates call requests, that is, provide overload protection for the system through the rate limit of the entrance, and the value of the entry speed limit threshold is automatically adapted to the size of the cluster. If the generation speed of the call request is less than the preset entry speed limit threshold, the call request is received and the subsequent calculation of the queue length of the script instance in the script scheduling queue is performed.

[0075] Step 300: When it is determined that the queue length of the script instance currently in the script scheduling queue is less than a length threshold, a new script instance is generated in response to the call request, and the new script instance is added to the script scheduling queue.

[0076] Step 400: Scheduling the script instances in the script scheduling queue through the script scheduling center.

[0077] Specifically, after receiving the call request, if it is determined that the queue length of the script instance currently in the script scheduling queue is less than the preset length threshold, a new script instance is generated in response to the call request, and the script instance is added to the script scheduling queue so that the script scheduling center can schedule it. Among them, the job scheduling for the script scheduling queue can adopt a "first come, first served" strategy, that is, based on the structure of the script scheduling queue, the first-in, first-out (FIFO) principle is adopted to achieve orderly scheduling of script instances.

[0078] The above is a step description of the first-level job scheduling method for the security orchestration and automated response system provided by the present invention. From the description of the above steps, it can be seen that according to the first-level job scheduling method for the security orchestration and automated response system provided by the present invention, by setting an entrance speed limit threshold to control the call rate of script instances, and by setting a script scheduling queue for script instances, the script scheduling center schedules the script instances in the script scheduling queue, so as to realize processing the script instances at a rate adapted to the system resources and load through the script scheduling center, thereby ensuring the stability of the security orchestration and automated response system.

[0079] Based on the above embodiments, in this embodiment, after generating a new script instance in response to the call request, the method further includes:

[0080] Generating a unique identifier for the new script instance and returning the unique identifier to the alarm module;

[0081] Through the alarm module, query the execution status of the new script instance from the script scheduling center according to the unique identifier of the new script instance.

[0082] Specifically, Figure 2 is a schematic flowchart of the first-level job scheduling method for the security orchestration and automated response system provided by the present invention. As Figure 2 shown, M is the entrance speed limit threshold, that is, the upper limit of the number of call requests generated per minute; N is the upper limit of the length of the script instances in the script scheduling queue.

[0083] Specifically, the alarm module is triggered to generate a call request manually or by configuring an alarm response action. It is necessary to determine whether the generation speed of the call request is less than the entrance speed limit threshold M. If the generation speed is greater than the entrance speed limit threshold M, no corresponding script instance will be generated according to the call request, avoiding thread congestion and affecting the system throughput; in addition, it is also necessary to determine whether the queue length of the script instances is less than the length threshold N. If the queue length is greater than the length threshold N, no corresponding script instance will be generated according to the call request, avoiding script instance scheduling blockage.

[0084] When it is determined that the generation speed of the call request is less than the entrance speed limit threshold M and the queue length of the script instances currently in the script scheduling queue is less than the length threshold N, a new script instance will be created. At the same time, a unique identifier corresponding to the script instance will be generated and returned to the alarm module, so that the alarm module can query the execution status of the script instance from the script scheduling center based on the unique identifier of the script instance.

[0085] The job scheduling method for the security orchestration and automated response system provided by the present invention can query the execution status of the corresponding script instance in real time through the unique identifier of the script instance, ensuring the effective scheduling of the script instance.

[0086] Based on the above embodiments, Figure 3 As another flowchart of the job scheduling method for the security orchestration and automation response system provided by the present invention, after generating a new playbook instance in response to the call request, the method further includes:

[0087] Step 310: Save the new playbook instance to the database.

[0088] Step 320: When the execution status of the new playbook instance is generated by the playbook scheduling center, save the execution status of the new playbook instance to the database through the playbook scheduling center.

[0089] Step 330: When a failure causes the loss of the playbook instance in the queue, reload the playbook instance in the database into the queue.

[0090] It should be noted that in the actual security operation process, the playbook instance can have three execution statuses: not executed, executing, and executed. The execution status of the newly created playbook instance is not executed. When this playbook instance enters the playbook scheduling queue, it will be persisted into the database; the execution status of the playbook instance in the playbook scheduling center is updated to executing, and the status during the execution of this playbook instance will be persisted into the database; after all task instances in the playbook instance are executed, the execution status of this playbook instance is updated to executed, and then the executed status of this playbook instance is persisted into the database.

[0091] Specifically, as Figure 2 shown, after the new playbook instance is generated, there are three output paths: the unique identifier of this playbook instance is returned to the alarm module, this playbook instance enters the playbook scheduling queue and waits for the playbook scheduling center to schedule, and this playbook instance is persisted into the database. In the actual security operation process, for the distributed cache, it is mainly implemented using a NoSQL database. Commonly used NoSQL databases include Redis, Memcached, MongoDB, etc. At the same time, if the playbook scheduling center schedules this playbook instance, the execution status of this playbook instance queried by the alarm module based on the unique identifier of this playbook instance will also be persisted into the database. If a failure occurs during the system operation, such as a device failure or a power outage, etc., resulting in data loss, then the data persisted into the database after the failure is repaired will be reloaded into the playbook scheduling queue. Specifically, the playbook instances with the execution status of not executed and executing will be reloaded into the playbook scheduling queue. Among them, the playbook scheduling center will schedule the playbook instances with the execution status of executing and continue to execute according to the execution status of this playbook instance stored in the database; the playbook instances with the execution status of executed will continue to be stored in the database in the executed state and will no longer be scheduled.

[0092] The job scheduling method of the security orchestration and automation response system provided by the present invention can effectively avoid resource occupation caused by repeated creation of playbook instances due to data loss by persisting playbook instances and their execution statuses into the database, thereby ensuring the stability of the security orchestration and automation response system.

[0093] Based on the above embodiments, in this embodiment, after determining the generation speed of the call request, the method further includes:

[0094] When it is determined that the generation speed of the call request is greater than or equal to the entry speed limit threshold, trigger the expansion of the security orchestration and automation response system, and return an informed message to the alarm module;

[0095] After calculating the queue length of the playbook instances currently in the playbook scheduling queue, the method further includes:

[0096] When it is determined that the queue length of the playbook instances currently in the playbook scheduling queue is greater than or equal to the length threshold, trigger the expansion of the security orchestration and automation response system, and return an informed message to the alarm module.

[0097] Specifically, as mentioned in the previous embodiments, when it is determined that the generation speed of the call request is less than the entry speed limit threshold and the queue length of the playbook instances currently in the playbook scheduling queue is less than the length threshold, new playbook instances will be created. However, as Figure 2 shown, if the generation speed of the call request is greater than or equal to the entry speed limit threshold or the generation speed of the call request is less than the entry speed limit threshold while the queue length of the playbook instances currently in the playbook scheduling queue is greater than or equal to the length threshold, new playbook instances cannot be created. In these two cases, in order to strengthen the self - protection against system overload and give priority to ensuring the availability of the overall system business and functions, the elastic expansion of the security orchestration and automation response system will be automatically triggered, and new servers of the security orchestration and automation response system will be deployed horizontally to increase the throughput.

[0098] The job scheduling method of the security orchestration and automation response system provided by the present invention controls the call rate of playbook instances by setting an entry speed limit threshold, and schedules the playbook instances in the playbook scheduling queue through the playbook scheduling center, so as to realize processing playbook instances at a rate adapted to the system resources and load through the playbook scheduling center, thereby ensuring the stability of the security orchestration and automation response system.

[0099] Based on the above embodiments, Figure 4Another process schematic diagram of the job scheduling method for the security orchestration and automated response system provided by the present invention, as Figure 4 shown, the scenario instance includes multiple task instances, and each scenario instance corresponds to a main thread;

[0100] It should be noted that a scenario is composed of several tasks arranged in combination with the business scenario, that is, each scenario instance includes multiple task instances, and each scenario instance has a corresponding main thread. When the task instances in the scenario instance are executed, they are controlled by the resource allocation of the task worker thread pool and executed in the order flow according to the arrangement logic of the scenario. When a parallel gateway appears in the process, the tasks will also be executed concurrently.

[0101] Specifically, in step 400, the scenario instances in the scenario scheduling queue are scheduled through the scenario scheduling center, including:

[0102] Step 410: Submit the scenario instance to the main thread through the scenario scheduling center, and determine whether the number of core threads reaches the first threshold.

[0103] Step 420: In the case where the number of core threads does not reach the first threshold, create core threads to process the multiple task instances corresponding to the scenario instance respectively.

[0104] Step 430: In the case where the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue does not reach the second threshold, submit the task instances to the task scheduling queue, and submit the task instances in the task scheduling queue to the core threads for processing in sequence.

[0105] It should be noted that Figure 5 A process schematic diagram of the secondary job scheduling method for the security orchestration and automated response system provided by the present invention, as Figure 5 shown, after the system is initialized, when a scenario instance is submitted to the main thread, core threads will be directly created and start working. The core threads will reside in memory permanently and will not trigger resource recycling due to idleness, which can better ensure the real-time performance of task instance scheduling and avoid additional performance overheads brought by thread creation, destruction, etc.

[0106] Based on the above embodiments, Figure 6 Another process schematic diagram of the job scheduling method for the security orchestration and automated response system provided by the present invention, as Figure 6 shown, the method further includes:

[0107] Step 610: In the case where it is determined that the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue reaches the second threshold, judge whether the total number of threads corresponding to the task instances reaches the third threshold through the scenario scheduling center;

[0108] Step 620: If the third threshold is not reached, create a new thread and submit the task instance to the new thread for processing;

[0109] Step 630: If the third threshold is reached, determine the target task instance in the task scheduling queue according to the preset rejection policy and discard it.

[0110] Next, the process of the secondary job scheduling method of the security orchestration and automation response system in the embodiments of the present invention will be described.

[0111] Specifically, as Figure 5 shown, N is the upper limit of the number of core threads in the core thread pool, that is, the first threshold; Q is the upper limit of the length of task instances in the task scheduling queue, that is, the second threshold; M is the upper limit of the number of all types of threads in the total thread pool, that is, the third threshold. During the operation of the system, when multiple task instances included in a script instance are submitted to the main thread, since the core threads are resident in memory, it is necessary to determine whether the core thread pool is full, that is, to determine whether the number of core threads reaches the preset first threshold N. If the first threshold N is not reached, core threads are created to process the multiple task instances corresponding to the script instance respectively.

[0112] In addition, the task scheduling queue provided in this embodiment is used to store task instances waiting to enter the core thread. When the number of core threads reaches the first threshold N, it is necessary to determine whether the number of task instances in the task scheduling queue reaches the preset second threshold Q. If not, the task instance enters the task scheduling queue and waits to enter the core thread for processing.

[0113] If the core thread pool is full and the task scheduling queue is full, it is determined whether the total number of threads corresponding to the task instance reaches the preset third threshold M. If the third threshold M is not reached, a new thread is created and the task instance is directly submitted to the new thread for processing. It should be noted that the new thread created here is created when the maximum number of task threads configured by the system does not reach the third threshold M, aiming to achieve full reuse of system computing resources. The new thread is different from the core thread. The core thread has a high priority and is resident in the thread pool to achieve the efficiency of task processing, and its corresponding resources will not be recycled by the system. The new thread can be idle, and after the thread work is completed, it is allowed to recycle resources. If the third threshold M is reached, the target task instance in the task scheduling queue is determined according to the preset rejection policy and discarded.

[0114] The job scheduling method of the security orchestration and automated response system provided by the present invention creates a core thread that resides in memory permanently and will not trigger resource recycling due to idleness, which can better ensure the real-time nature of task scheduling response and avoid additional performance overheads brought by thread creation, destruction, etc.; and when the core thread pool is full and the task scheduling queue is not full, task instances are allowed to be submitted to the task scheduling queue to queue up, improving the task scheduling response ability and the adaptability to the business; at the same time, by creating new threads that allow resource recycling, the system's adaptability to the business is improved; and the preset rejection policy and concurrency limit can guarantee the throughput rate of the system.

[0115] Based on the above embodiments, in this embodiment, the rejection policy includes: a priority response policy or a response ratio policy; wherein, the response ratio is determined by the waiting time and the required execution time of the task instance;

[0116] Determining and discarding the target task instance in the task scheduling queue according to the preset rejection policy includes:

[0117] Determining the response ratio of each task instance in the task scheduling queue, and according to the response ratio policy, determining the task instance with a response ratio lower than the response ratio threshold as the target task instance and discarding it; or

[0118] Determining the priority of each task instance in the task scheduling queue, and according to the priority response policy, determining at least one task instance with a lower priority as the target task instance and discarding it.

[0119] Specifically, the execution time of different task instances is different. In order to improve the user experience and quickly determine the target task instance to be discarded, the response ratio policy can be used. Specifically, the response ratio = waiting time / required execution time. For example, for task instance A, according to the historical execution time, it takes about 1 minute to execute, and it has been waiting in the task scheduling queue for 1 minute currently. The response ratio of task instance A is 1; for task instance B, it takes about 5 minutes to execute, and it has been waiting in the task scheduling queue for 2 minutes currently. The response ratio of task instance B is 0.4. At this time, if the response ratio of task instance B is also lower than the preset response ratio threshold, then task instance B is determined as the target task instance and discarded.

[0120] In addition, according to the business scenario to which the script instance is applied, the task instances included in the script instance can be marked with priorities, and then the task instances can be scheduled according to the priorities during scheduling, which is the priority response policy. Specifically, when the total thread pool is full, at least one task instance with a lower priority can be determined as the target task instance and discarded according to the priority response policy.

[0121] Generally speaking, if the priority response policy or response ratio policy is enabled, it is necessary to remove the task instance with the lowest priority or lower than the response ratio threshold from the task scheduling queue and add the new task instance to the end of the task scheduling queue.

[0122] The job scheduling method of the security orchestration and automation response system provided by the present invention adopts the priority response policy or response ratio policy, which is suitable for the security orchestration and automation response system to tilt the event handling capabilities towards certain business scenarios, giving priority to ensuring that the security alerts of such business scenarios are disposed of in a timely manner, and at the same time, the throughput rate of the system can be guaranteed.

[0123] Based on the above embodiments, in this embodiment, during the process of processing the task instance by the core thread or the new thread, the method further includes:

[0124] In the case where it is determined that the current task instance execution fails, according to the preset task instance failure policy, execute the corresponding retry of the current task instance, skip the current task instance, or terminate the script instance corresponding to the current task instance;

[0125] Among them, the task instance failure policy includes: failure skip task policy, failure retry policy, or failure terminate script policy.

[0126] Specifically, in addition to the rejection policy at the scheduling level for the execution of the task instance, when the task instance thread execution fails, in order to improve the fault tolerance of the system, corresponding task instance failure policies are designed, including: failure skip task policy, failure retry policy, or failure terminate script policy.

[0127] Specifically, the failure skip task policy means that if a task instance fails to execute due to reasons such as network environment, data integrity, equipment failure, etc., the task instance can be directly skipped and the subsequent task instances can be continued to be executed.

[0128] The failure retry policy means that if a task instance fails to execute due to reasons such as network environment, data integrity, equipment failure, etc., when the network is temporarily interrupted or there is a large time delay in the environment or the equipment is unresponsive during the business peak, in order to prevent the task instance from being retried frequently after execution fails, the failure retry policy can be selected. Specifically, the failure retry policy adopts the binary exponential backoff algorithm to avoid excessive retry times resulting in frequent requests, thereby avoiding a huge overhead of system resources. For example, when the task instance fails to execute at time t, it will be retried at times t1, t2, t3... tn until the execution is successful, and the time intervals of t1, t2, t3... tn are 2 n time units, such as milliseconds, seconds, minutes, and the maximum number of retries is declared during this process. After the retry execution is successful, the subsequent task instances will be continued to be executed.

[0129] The failure termination script strategy means that when a task instance fails to execute due to reasons such as network environment, data integrity, device failure, etc., the script instance corresponding to the task instance is directly terminated at this time, and other task instances in the script instance are no longer executed.

[0130] The job scheduling method of the security orchestration and automation response system provided by the present invention, supplemented by a task instance failure strategy during the execution of a task instance, can help the system release the resources occupied by blocked tasks and quickly recover from exceptions, ensuring the throughput rate of the system.

[0131] Based on the above embodiments, in this embodiment, after executing the corresponding retry of the current task instance, skipping the current task instance, or terminating the script instance corresponding to the current task instance, the method further includes:

[0132] Generating exception information and feeding back the exception information to the main thread of the script instance corresponding to the current task instance.

[0133] The job scheduling method of the security orchestration and automation response system provided by the present invention, supplemented by a task instance failure strategy during the execution of a task instance, can help the system release the resources occupied by blocked tasks and quickly recover from exceptions, ensuring the throughput rate of the system.

[0134] The job scheduling device of the security orchestration and automation response system provided by the present invention will be described below. The job scheduling device of the security orchestration and automation response system described below can be mutually referred to the job scheduling method of the security orchestration and automation response system described above.

[0135] Figure 7 It is a schematic diagram of the job scheduling device of the security orchestration and automation response system provided by the present invention, as Figure 7 shown, the job scheduling device of the security orchestration and automation response system provided by the present invention includes:

[0136] An alarm module 701 for triggering the generation of a call request;

[0137] A calculation module 702 for determining the generation speed of the call request, and calculating the queue length of the script instances currently in the script scheduling queue when it is determined that the generation speed of the call request is less than the entrance speed limit threshold;

[0138] A generation module 703 for generating a new script instance in response to the call request and adding the new script instance to the script scheduling queue when it is determined that the queue length of the script instances currently in the script scheduling queue is less than the length threshold;

[0139] A scheduling module 704, configured to schedule script instances in the script scheduling queue through a script scheduling center.

[0140] The job scheduling device of the security orchestration and automated response system provided by the present invention controls the invocation rate of script instances by setting an entry speed limit threshold, and schedules script instances in the script scheduling queue through the script scheduling center, so as to process script instances at a rate adapted to system resources and load through the script scheduling center, thereby ensuring the stability of the security orchestration and automated response system.

[0141] Based on the above embodiment, in this embodiment, the generation module 703 is further configured to:

[0142] After generating a new script instance in response to the invocation request, generate a unique identifier of the new script instance and return the unique identifier to the alarm module;

[0143] The device further includes:

[0144] A query module, configured to query the execution status of the new script instance from the script scheduling center through the alarm module according to the unique identifier of the new script instance.

[0145] The job scheduling device of the security orchestration and automated response system provided by the present invention queries the execution status of the corresponding script instance in real time through the unique identifier of the script instance, ensuring the effective scheduling of the script instance.

[0146] Based on the above embodiment, in this embodiment, the device further includes:

[0147] A storage module, configured to save the new script instance to a database after generating the new script instance in response to the invocation request;

[0148] In the case of generating the execution status of the new script instance through the script scheduling center, save the execution status of the new script instance to the database through the script scheduling center;

[0149] In the case of a failure resulting in the loss of script instances in the queue, reload the script instances in the database into the queue.

[0150] The job scheduling device of the security orchestration and automated response system provided by the present invention can effectively avoid the repeated creation of script instances due to data loss by persisting script instances and the execution status of script instances into the database, thereby ensuring the stability of the security orchestration and automated response system.

[0151] Based on the above embodiments, in this embodiment, the device further includes:

[0152] A call request discarding module, configured to trigger the expansion of the security orchestration and automation response system and return an informing message to the alarm module when it is determined that the generation speed of the call request is greater than or equal to the entrance speed limit threshold;

[0153] When it is determined that the queue length of the script instances in the script scheduling queue is greater than or equal to the length threshold, trigger the expansion of the security orchestration and automation response system and return an informing message to the alarm module.

[0154] The job scheduling device of the security orchestration and automation response system provided by the present invention controls the call rate of script instances by setting an entrance speed limit threshold, and schedules the script instances in the script scheduling queue through the script scheduling center, so as to process the script instances at a rate adapted to the system resources and load through the script scheduling center, thereby ensuring the stability of the security orchestration and automation response system.

[0155] Based on the above embodiments, in this embodiment, each script instance includes a plurality of task instances, and each script instance corresponds to a main thread; the device further includes:

[0156] A script instance submission module, configured to submit a script instance to the main thread through the script scheduling center and determine whether the number of core threads reaches a first threshold;

[0157] A core thread creation module, configured to create core threads to process the multiple task instances corresponding to the script instance when the number of core threads does not reach the first threshold;

[0158] A task instance submission module, configured to submit a task instance to the task scheduling queue and submit the task instances in the task scheduling queue to the core threads for processing in sequence when the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue does not reach a second threshold.

[0159] Based on the above embodiments, in this embodiment, the device further includes:

[0160] A determination module, configured to determine whether the total number of threads corresponding to the task instance reaches a third threshold through the script scheduling center when it is determined that the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue reaches the second threshold;

[0161] A new thread creation module, configured to create a new thread and submit a task instance to the new thread for processing when the third threshold is not reached;

[0162] A task instance discarding module, configured to determine and discard target task instances in the task scheduling queue according to a preset rejection policy when a third threshold is reached.

[0163] The job scheduling device of the security orchestration and automation response system provided by the present invention creates a core thread, and the core thread resides in memory all the time and will not trigger resource recycling due to idleness, which can better ensure the real-time performance of task scheduling response and avoid the additional performance overhead caused by thread creation, destruction, etc.; and when the core thread pool is full and the task scheduling queue is not full, task instances are allowed to be submitted to the task scheduling queue for queuing, improving the task scheduling response ability and the adaptability to services; at the same time, by creating new threads that allow resource recycling, the adaptability of the system to services is improved; and the preset rejection policy and concurrency limit can guarantee the throughput rate of the system.

[0164] Based on the above embodiments, in this embodiment, the rejection policy includes: a priority response policy or a response ratio policy; wherein, the response ratio is determined by the waiting time and the execution required time of the task instance;

[0165] Correspondingly, the task instance discarding module is further configured to:

[0166] Determine the response ratio of each task instance in the task scheduling queue, and according to the response ratio policy, determine and discard the task instances with a response ratio lower than the response ratio threshold as target task instances; or

[0167] Determine the priority of each task instance in the task scheduling queue, and according to the priority response policy, determine and discard at least one task instance with a lower priority as a target task instance.

[0168] The job scheduling device of the security orchestration and automation response system provided by the present invention adopts a rejection policy including a priority response policy or a response ratio policy, which is applicable to the security orchestration and automation response system to tilt the event handling capabilities towards certain business scenarios, giving priority to ensuring that the security alarms in such business scenarios are disposed of in a timely manner, and at the same time can guarantee the throughput rate of the system.

[0169] Based on the above embodiments, in this embodiment, the device further includes:

[0170] A failure policy adjustment module, configured to, during the process of processing the task instance through the core thread or the new thread, when it is determined that the current task instance execution fails, execute the corresponding retry of the current task instance, skip the current task instance, or terminate the script instance corresponding to the current task instance according to a preset task instance failure policy;

[0171] Among them, the task instance failure strategy includes: a failure skip task strategy, a failure retry strategy, or a failure terminate script strategy.

[0172] The job scheduling device of the security orchestration and automated response system provided by the present invention, supplemented by a task instance failure strategy during the execution of a task instance, can help the system release the resources occupied by blocked tasks and quickly recover from exceptions, ensuring the throughput rate of the system.

[0173] Based on the above embodiment, in this embodiment, the device further includes:

[0174] An exception information generation module, configured to generate exception information after executing the corresponding retry current task instance, skip current task instance, or terminate the script instance corresponding to the current task instance, and feedback the exception information to the main thread of the script instance corresponding to the current task instance.

[0175] The job scheduling device of the security orchestration and automated response system provided by the present invention, supplemented by a task instance failure strategy during the execution of a task instance, can help the system release the resources occupied by blocked tasks and quickly recover from exceptions, ensuring the throughput rate of the system.

[0176] Figure 8 Schematic diagram of the physical structure of an electronic device is exemplified, as Figure 8 shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the job scheduling method of the security orchestration and automated response system, and the method includes:

[0177] Trigger and generate a call request through the alarm module;

[0178] Determine the generation speed of the call request. When it is determined that the generation speed of the call request is less than the entry speed limit threshold, calculate the queue length of the script instances currently in the script scheduling queue;

[0179] When it is determined that the queue length of the script instances currently in the script scheduling queue is less than the length threshold, in response to the call request, generate a new script instance and add the new script instance to the script scheduling queue;

[0180] Schedule the script instances in the script scheduling queue through the script scheduling center.

[0181] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0182] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the job scheduling method of the security orchestration and automation response system provided by the above-mentioned various methods. The method includes:

[0183] Triggering a generation of a call request through an alarm module;

[0184] Determining the generation speed of the call request. When it is determined that the generation speed of the call request is less than the entry speed limit threshold, calculating the queue length of the script instances currently in the script scheduling queue;

[0185] When it is determined that the queue length of the script instances currently in the script scheduling queue is less than the length threshold, generating a new script instance in response to the call request and adding the new script instance to the script scheduling queue;

[0186] Scheduling the script instances in the script scheduling queue through a script scheduling center.

[0187] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the job scheduling method of the security orchestration and automation response system provided by the above-mentioned various methods. The method includes:

[0188] Triggering a generation of a call request through an alarm module;

[0189] Determining the generation speed of the call request. When it is determined that the generation speed of the call request is less than the entry speed limit threshold, calculating the queue length of the script instances currently in the script scheduling queue;

[0190] In the case where it is determined that the queue length of the script instance currently in the script scheduling queue is less than the length threshold, a new script instance is generated in response to the call request, and the new script instance is added to the script scheduling queue;

[0191] The script instances in the script scheduling queue are scheduled by the script scheduling center.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0193] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A job scheduling method for a security orchestration and automated response system, characterized in that, Including: Trigger the generation of a call request through an alarm module; Determine the generation speed of the call request. When it is determined that the generation speed of the call request is less than the entry speed limit threshold, calculate the queue length of the script instances currently in the script scheduling queue; When it is determined that the queue length of the script instances currently in the script scheduling queue is less than the length threshold, in response to the call request, generate a new script instance and add the new script instance to the script scheduling queue; Schedule the script instances in the script scheduling queue through a script scheduling center.

2. The job scheduling method according to claim 1, wherein After generating a new script instance in response to the call request, the method further includes: Generate a unique identifier for the new script instance and return the unique identifier to the alarm module; Through the alarm module, query the execution status of the new script instance from the script scheduling center according to the unique identifier of the new script instance.

3. The job scheduling method according to claim 1, wherein After generating a new script instance in response to the call request, the method further includes: Save the new script instance to a database; When the execution status of the new script instance is generated by the script scheduling center, save the execution status of the new script instance to the database through the script scheduling center; When a failure causes the script instances in the queue to be lost, reload the script instances in the database into the queue.

4. The job scheduling method according to claim 1, wherein After determining the generation speed of the call request, the method further includes: When it is determined that the generation speed of the call request is greater than or equal to the entry speed limit threshold, trigger the expansion of the security orchestration and automation response system and return an informing message to the alarm module; After calculating the queue length of the script instances currently in the script scheduling queue, the method further includes: When it is determined that the queue length of the script instances currently in the script scheduling queue is greater than or equal to the length threshold, trigger the expansion of the security orchestration and automation response system and return an informing message to the alarm module.

5. The job scheduling method according to claim 1, wherein The script instance includes multiple task instances, and each script instance corresponds to a main thread; Scheduling the script instances in the script scheduling queue through a script scheduling center includes: Submit a script instance to the main thread through the script scheduling center and determine whether the number of core threads reaches a first threshold; When the number of core threads does not reach the first threshold, create core threads to process the multiple task instances corresponding to the script instance respectively; When the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue does not reach a second threshold, submit the task instances to the task scheduling queue and submit the task instances in the task scheduling queue to the core threads for processing in sequence.

6. The job scheduling method according to claim 5, wherein The method further includes: When it is determined that the number of core threads reaches the first threshold and the number of task instances in the task scheduling queue reaches the second threshold, determine whether the total number of threads corresponding to the task instances reaches a third threshold through the script scheduling center; If it does not reach the third threshold, create a new thread and submit the task instances to the new thread for processing; If the third threshold is reached, determine the target task instance in the task scheduling queue and discard it according to the preset rejection policy.

7. The job scheduling method according to claim 6, wherein, The rejection policy includes: a priority response policy or a response ratio policy; wherein, the response ratio is determined by the waiting time and the required execution time of the task instance; Determine the target task instance in the task scheduling queue and discard it according to the preset rejection policy, including: Determine the response ratio of each task instance in the task scheduling queue, and according to the response ratio policy, determine the task instances with a response ratio lower than the response ratio threshold as target task instances and discard them; or Determine the priority of each task instance in the task scheduling queue, and according to the priority response policy, determine at least one task instance with a lower priority as a target task instance and discard it.

8. The job scheduling method according to claim 6, wherein During the process of processing the task instance by the core thread or the new thread, the method further includes: In the case where it is determined that the current task instance fails to execute, according to the preset task instance failure policy, execute the corresponding retry of the current task instance, skip the current task instance, or terminate the script instance corresponding to the current task instance; Wherein, the task instance failure policy includes: a failure skip task policy, a failure retry policy, or a failure terminate script policy.

9. The job scheduling method according to claim 8, wherein After executing the corresponding retry of the current task instance, skipping the current task instance, or terminating the script instance corresponding to the current task instance, the method further includes: Generate exception information and feedback the exception information to the main thread of the script instance corresponding to the current task instance.

10. A job scheduling device for a security orchestration and automated response system, characterized in that, Including: An alarm module for triggering the generation of a call request; A calculation module for determining the generation speed of the call request, and calculating the queue length of the script instances currently in the script scheduling queue when it is determined that the generation speed of the call request is less than the entry speed limit threshold; A generation module for generating a new script instance in response to the call request and adding the new script instance to the script scheduling queue when it is determined that the queue length of the script instances currently in the script scheduling queue is less than the length threshold; A scheduling module for scheduling the script instances in the script scheduling queue through a script scheduling center.

11. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the job scheduling method of the security orchestration and automation response system according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the job scheduling method of the security orchestration and automation response system according to any one of claims 1 to 9.

Images