Method, device, system and medium for collaboratively processing security operation tasks using multiple models

Abstract

A method, device, system and medium for collaboratively processing security operation tasks using multiple models, relating to the field of information security technology, the method comprising: receiving a user's security operation instruction, and determining the type of the security operation instruction using a pre-trained classification model; when the security operation instruction is determined to be an execution task instruction, using a first type of model to perform reasoning based on the security operation instruction to determine the target tool to be called in response to the security operation instruction; and, when the target tool is determined, calling the target tool to perform the corresponding security operation task; when the security operation instruction is determined to be a knowledge question and answer instruction, using a second type of model to perform reasoning based on the security operation instruction to determine the reply statement corresponding to the security operation instruction; the first type of model has a smaller number of parameters than the second type of model. The advantages of fast reasoning speed of small models and high reasoning accuracy of large models are fully utilized, which helps to improve the quality of security operation services.

Description

Technical Field

[0001] This article relates to the field of information security technology, and in particular to a method, device, system and medium for collaboratively processing security operation tasks using multiple models. Background Art

[0002] Security Orchestration, Automation and Response (SOAR) system is a system that orchestrates and calls the application interface of security devices during security operations to quickly respond to and handle security alerts automatically. With the development of artificial intelligence technology, using artificial intelligence technology to enhance the automation level of security operations has become an important way to improve the performance of SOAR systems. At present, using security operation corpus, small models (models with parameter scales within 100 million) trained by natural language processing and deep learning technology can more accurately match user instructions with training corpus, but for some user instructions that do not exist in the training corpus, small models usually have difficulty in accurately identifying their user intentions. Since large models (such as ChatGPT) have higher reasoning accuracy for natural language instructions, more and more SOAR systems have introduced large models to improve the accuracy of the system's understanding of security operation instructions issued by users.

[0003] In practice, since large models require higher computing power costs, in order to improve the utilization rate of large models, large models often need to undertake tasks other than reasoning security operation instructions, such as knowledge question answering, security analysis, etc. This makes it easy for large model reasoning tasks to queue up or even service timeouts when multiple tasks are concurrent, resulting in the SOAR system being unable to respond to security operation instructions issued by users in a timely manner. Summary of the invention

[0004] Embodiments of the present application provide a method, device, system, and medium for collaboratively processing security operation tasks using multiple models.

[0005] In the first aspect, an embodiment of the present application provides a method for collaboratively processing security operation tasks using multiple models, including: receiving security operation instructions from a user, and determining the type of the security operation instructions using a pre-trained classification model; when the security operation instruction is determined to be an execution task type instruction, using a first type of model to perform reasoning based on the security operation instruction to determine a target tool that needs to be called to respond to the security operation instruction; and, when the target tool is determined, calling the target tool to perform the corresponding security operation task; when the security operation instruction is determined to be a knowledge question and answer type instruction, using a second type of model to perform reasoning based on the security operation instruction to determine a reply statement corresponding to the security operation instruction; wherein the first type of model has a smaller number of parameters than the second type of model.

[0006] In the second aspect, an embodiment of the present application provides a device for multi-model collaborative processing of security operation tasks, including a processor and a memory storing a computer program, and is characterized in that when the computer program is executed by the processor, the method for multi-model collaborative processing of security operation tasks in the above embodiment can be implemented.

[0007] In the third aspect, an embodiment of the present application provides a security orchestration and automated response system, including: a command interface and a response module, wherein the command interface is configured to receive security operation instructions from users; the response module is configured to respond to security operation instructions through the method of multi-model collaborative processing of security operation tasks in the above-mentioned embodiment.

[0008] In a fourth aspect, an embodiment of the present application provides a non-volatile computer storage medium, which stores a computer program. When the computer program is executed by a processor, the method of multi-model collaborative processing of security operation tasks in the above embodiment is implemented.

[0009] The method for collaboratively processing security operation tasks using multiple models in the embodiment of the present application can use a classification model to determine the type of security operation instructions. For security operation instructions of the execution task type with low reasoning accuracy requirements, the first type of model with a small parameter amount can be used for reasoning to determine the target tool that needs to be called, and the target tool can be used to complete the corresponding security operation task; for security operation instructions of the knowledge question and answer type with high reasoning accuracy requirements, the second type of model with a large parameter amount can be used for reasoning to determine the reply statement of the security operation instruction. According to the type of security operation instruction, the first type of model (small model with a small parameter amount) and the second type of model (large model with a large parameter amount) can be called for processing respectively, making full use of the advantages of the small model's fast reasoning speed and the large model's high reasoning accuracy, reducing the adverse effects of the small model's low reasoning accuracy and the large model's low service throughput on the security operation automation response speed, and helping to improve the quality of security operation services.

[0010] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0012] Figure 1 A schematic diagram of the structure of an embodiment of the security orchestration and automated response system of the present application;

[0013] Figure 2 A flowchart of an embodiment of the method for multi-model collaborative processing of security operation tasks of the present application;

[0014] Figure 3 A schematic diagram of a flow chart of reasoning about execution task-type instructions in one embodiment of a method for multi-model collaborative processing of security operation tasks of the present application;

[0015] Figure 4 A schematic diagram of a process for reasoning on knowledge question-answering instructions in one embodiment of the method for multi-model collaborative processing of security operation tasks of the present application;

[0016] Figure 5 A flowchart of an embodiment of the method for multi-model collaborative processing of security operation tasks of the present application;

[0017] Figure 6 A schematic diagram of the reasoning process of the second type of model in one embodiment of the method for collaboratively processing security operation tasks by multiple models of the present application;

[0018] Figure 7 A flowchart of determining whether the task load of a high-precision large model meets preset conditions in one embodiment of the method for collaboratively processing security operation tasks by multiple models of the present application;

[0019] Figure 8 A flowchart of an embodiment of the method for collaboratively processing security operation tasks using multiple models of the present application. DETAILED DESCRIPTION

[0020] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0021] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0022] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0023] Figure 1 A schematic diagram of the structure of an embodiment of the security orchestration and automated response system of the present application is shown as follows: Figure 1 As shown, the security orchestration and automated response system may include a command interface and a response module, wherein the command interface is configured to receive security operation instructions from users; the response module is configured to respond to security operation instructions through the method of multi-model collaborative processing of security operation tasks in any embodiment of the present application.

[0024] like Figure 1 As shown, the response module may include multiple submodules such as a user instruction classification submodule, a large and small model collaboration engine, a tool execution engine, a small model reasoning service submodule, and a large model reasoning service submodule. Among them, the user instruction classification submodule is used to determine the type of security operation instructions. For example, the security operation instructions can be classified by a pre-trained classification model to predict whether the security operation instructions are execution task instructions or knowledge question and answer instructions. As an example, the classification model can be a natural language classification model such as BERT.

[0025] The security orchestration and automated response system in this embodiment responds to security operation instructions by using a method for collaboratively processing security operation tasks with multiple models. The system can call the first type of model (a small model with a small number of parameters) and the second type of model (a large model with a large number of parameters) for processing according to the type of security operation instructions, thereby fully utilizing the advantages of the fast reasoning speed of the small model and the high reasoning accuracy of the large model, reducing the adverse effects of the low reasoning accuracy of the small model and the low service throughput of the large model on the automated response speed of the security operation, and helping to improve the quality of security operation services.

[0026] In this embodiment, the type of security operation instruction can characterize the demand for reasoning accuracy of the security operation instruction. For example, the execution task type instruction has a lower requirement for the natural language understanding of the model, and the corresponding reasoning accuracy requirement is lower, while the knowledge question and answer type instruction has a higher requirement for the natural language understanding of the model, and the corresponding reasoning accuracy requirement is also higher. The reasoning accuracy of the model is usually positively correlated with the parameter scale of the model, that is, the reasoning accuracy of the first type of model (i.e., small model) is lower than that of the second type of model (i.e., large model), and the reasoning time of the model is negatively correlated with the parameter scale, that is, the reasoning time of the first type of model is less than that of the second type of model.

[0027] The small model inference service submodule can call the first type of model to perform inference tasks, and the large model inference service submodule can call the second type of model to perform inference tasks. As an example, the first type of model can be a natural language model with a smaller parameter scale (for example, BERT) obtained by fine-tuning the model based on the downstream task selected by the task tool, which is used to identify user intent based on security operation instructions and determine the tool that matches the user intent from the tool library. The second type of model can be a large model with a larger parameter scale (for example, ChatGPT).

[0028] In order to match the performance of different models, the large and small model collaboration engine can call the first type of model or the second type of model for reasoning according to the type of security operation instruction. For example, for knowledge question and answer instructions, they can be pushed to the large model reasoning service submodule to call the second type of model for reasoning and generate reply statements; for task execution instructions, they can be pushed to the small model reasoning service submodule to call the first type of model for reasoning, identify user intent to determine the tool that needs to be called, and then the tool execution engine calls the corresponding tool to perform the corresponding security operation task.

[0029] In addition, since the first type of model has a small number of parameters and a low ability to understand natural language, when the security operation instruction contains a field (Token) that does not exist in the training corpus or a large number of tools need to be called (for example, a tool chain consisting of multiple tools), it may not be able to recognize the user's intention and determine the tool that needs to be called. At this time, the large and small model collaboration engine can also push the security operation instruction to the large model reasoning service, and the second type of model will accurately identify the security operation instruction to determine the tool or tool chain that needs to be called, and then the tool execution engine will call the corresponding tool or tool chain to perform the corresponding security operation task.

[0030] In this embodiment, the description information of the tool can be generated in advance based on the tool's functions and applicable scenarios, so that the first model and / or the second model can determine the matching target tool based on the security operation instructions and the description information. In addition, the second model can also identify the security operation scenario of the security operation instruction based on the tool's description information, and then determine the execution time of the security operation task based on the identified execution scenario. For example, the real-time execution scenario of IP blocking can be executed immediately, while asynchronous execution scenarios such as vulnerability scanning can be delayed when the task load is large.

[0031] In this embodiment, in order to further improve the utilization rate of the second type of model, multiple large models of different precisions can be set to perform reasoning tasks of different precisions. As an example, the second type of large model can include a high-precision large model and a low-precision large model. In this way, the large model reasoning service submodule can provide high-precision large model reasoning services and low-precision large model reasoning services. In order to improve the accuracy of reasoning, the high-precision large model can be used by default to reason for all knowledge question and answer instructions. When the task load of the high-precision large model is large, some knowledge question and answer instructions with lower precision requirements can be assigned to the low-precision large model for reasoning. Under the premise of ensuring the accuracy of reasoning, the service throughput of the large model is improved.

[0032] In order to facilitate the scheduling of reasoning tasks of large models with different precisions, the response module can also include a task queue submodule and an reasoning routing control submodule, wherein the task queue submodule can build a corresponding task queue for each model. For example, the first queue is used to store execution task instructions, which are reasoned by the first type of large model; the second queue is used to store knowledge question and answer instructions, which are reasoned by the second type of large model. Furthermore, the second queue may include a high-precision task sub-queue and a low-precision task sub-queue, wherein the high-precision task sub-queue can store inference tasks with higher inference accuracy requirements, such as execution task type instructions for which the first type of model reasoning fails, and the execution task type instructions for which the first type of model reasoning fails can be pushed from the first queue to the high-precision large model or transferred to the high-precision task sub-queue through the inference routing control sub-module; the low-precision task sub-queue can store inference tasks with lower inference accuracy requirements, such as knowledge question and answer type instructions. When the computing power load of the second type of large model is low, the tasks stored in the high-precision task sub-queue and the low-precision task sub-queue can all be inferred by the high-precision large model to improve the inference accuracy. When the computing power load of the second type of model is high, the low-precision large model will infer the tasks in the low-precision task sub-queue to improve the service throughput of the large model.

[0033] In this embodiment, the response module may also include a load analysis and prediction submodule for real-time monitoring of the task load of the high-precision large model and scheduling the reasoning tasks of the second type of large model according to the task load. For example, the load analysis and prediction submodule can generate an early warning signal when the task load of the high-precision large model is too high, so that the task queue submodule and the reasoning routing control submodule push the tasks in the low-precision task subqueue to the low-precision large model for reasoning, so as to avoid excessive load and reduce the response speed; when the load analysis and prediction submodule detects that the task load of the high-precision large model is low, it generates an early warning release signal so that the task queue submodule and the reasoning routing control submodule push all the tasks in the high-precision task subqueue and the low-precision task subqueue to the high-precision large model for reasoning, so as to ensure the reasoning accuracy.

[0034] Figure 2 A flowchart of an embodiment of the method for multi-model collaborative processing of security operation tasks of the present application is shown. The method can be applied to Figure 1 The security orchestration and automated response system shown in Figure 1 is shown in Figure 2. Figure 2 As shown, the process may include the following steps.

[0035] Step 210: Receive a security operation instruction from a user, and use a pre-trained classification model to determine the type of the security operation instruction.

[0036] In this embodiment, the types of security operation instructions include task execution type instructions and knowledge question and answer type instructions, among which, task execution type instructions are instructions that need to call relevant tools to perform security operation tasks, and this type of instruction has lower requirements for reasoning accuracy; knowledge question and answer type instructions need to generate corresponding reply statements, and this type of instruction has higher requirements for reasoning accuracy.

[0037] Step 220: When the safety operation instruction is determined to be an execution task type instruction, use the first type of model to perform reasoning based on the safety operation instruction to determine the target tool that needs to be called in response to the safety operation instruction; and, when the target tool is determined, call the target tool to perform the corresponding safety operation task.

[0038] Step 230: When the safety operation instruction is determined to be a knowledge question and answer instruction, the second type of model is used to perform reasoning based on the safety operation instruction to determine a reply statement corresponding to the safety operation instruction.

[0039] Among them, the number of parameters of the first type of model is smaller than that of the second type of model.

[0040] In this embodiment, the first type of model refers to a model with a small number of parameters, which can also be called a small model, for example, a BERT model with a parameter number less than 100 million. The second type of model refers to a model with a large number of parameters, which can also be called a large model, and its parameter number is usually greater than 100 million.

[0041] The method of multi-model collaborative processing of security operation tasks in this embodiment can use the classification model to determine the type of security operation instructions. For security operation instructions of the execution task type with low reasoning accuracy requirements, the first type of model with a small parameter amount can be used for reasoning to determine the target tool that needs to be called, and the target tool can be used to complete the corresponding security operation task; for security operation instructions of the knowledge question and answer type with high reasoning accuracy requirements, the second type of model with a large parameter amount can be used for reasoning to determine the reply statement of the security operation instruction. According to the type of security operation instruction, the first type of model (small model with a small parameter amount) and the second type of model (large model with a large parameter amount) can be called for processing respectively, making full use of the advantages of the small model's fast reasoning speed and the large model's high reasoning accuracy, reducing the adverse effects of the small model's low reasoning accuracy and the large model's low service throughput on the security operation automation response speed, and helping to improve the quality of security operation services.

[0042] In some embodiments, the above step 220 can be performed by Figure 3 The process shown in the figure is to infer the execution task class instructions, such as Figure 3 As shown, the process may include the following steps.

[0043] Step 310: When the safety operation instruction is determined to be an execution task instruction, the safety operation instruction is added to the first task queue.

[0044] The first task queue is used to store execution task instructions to be inferred.

[0045] Step 320: extract the security operation instruction from the first task queue and push it to the first type of model.

[0046] Step 330: Use the first type of model to identify the intent of the safety operation instruction, and determine whether there is a tool in the tool library that matches the intent of the safety operation instruction.

[0047] In this embodiment, the tool library is used to store tools used to perform various security operation tasks and functional description information of the tools. The first type of model can match the functional description information of the tools according to the intention of the security operation instructions to determine whether there is a matching tool.

[0048] As an example, the first type of model can be a BERT model trained based on security operation task corpus. Assume that the security operation instruction is "What are the alarms in the last 7 days?", which corresponds to an API (Application Programming Interface) on a situational awareness system. The API is encapsulated as an alarm query tool. The BERT model can identify that the security operation instruction is to call the alarm query tool based on the training corpus.

[0049] Step 340: When it is determined that there is a tool in the tool library that matches the intent of the safety operation instruction, the tool is determined as the target tool.

[0050] In this embodiment, the first type of model with a smaller number of parameters can be used to infer the execution task type instruction, which can more quickly determine the target tool, which helps to improve the response speed of the execution task type instruction. Here, the target tool can be a single tool or a tool chain composed of multiple tools in a calling order.

[0051] In some embodiments, the above step 230 can be performed by Figure 4 The process shown in the figure is to infer the knowledge question answering instructions, such as Figure 4 As shown, the process may include the following steps.

[0052] Step 410: When the safety operation instruction is determined to be a knowledge question and answer instruction, the safety operation instruction is added to the second task queue.

[0053] The second task queue is used to store knowledge question and answer instructions to be inferred.

[0054] Step 420: extract the safety operation instruction from the second task queue and push it to the second type of model.

[0055] Step 430: Use the second type of model to perform reasoning based on the safety operation instruction to determine a response statement corresponding to the safety operation instruction.

[0056] In this embodiment, the second type of model with a larger number of parameters and stronger natural language understanding ability can be used to infer knowledge question and answer instructions, and more accurate reply sentences can be generated, thereby responding to safety operation instructions more accurately.

[0057] In some optional implementations of this embodiment, the second type of model includes a high-precision large model, and the above step 430 may include: using the high-precision large model to perform reasoning based on the safety operation instruction to determine the response statement corresponding to the safety operation instruction.

[0058] In this embodiment, a high-precision large model generally refers to a large model with a parameter amount far greater than 100 million, which has a strong natural language understanding capability, thereby being able to generate more accurate reply sentences, helping to improve the response accuracy of knowledge question-and-answer type instructions.

[0059] Reference below Figure 5 , Figure 5 A flowchart of an embodiment of the method for multi-model collaborative processing of security operation tasks of the present application is shown as follows: Figure 5 As shown, the process includes the following steps.

[0060] Step 500: Receive a security operation instruction from a user, and use a pre-trained classification model to determine the type of the security operation instruction.

[0061] Afterwards, step 590 or steps 510 to 580 may be executed according to the prediction result of the classification model.

[0062] Step 510: When the safety operation instruction is determined to be an execution task instruction, the safety operation instruction is added to the first task queue.

[0063] Step 590: When the safety operation instruction is determined to be a knowledge question and answer instruction, the second type of model is used to perform reasoning based on the safety operation instruction to determine a reply statement corresponding to the safety operation instruction.

[0064] Step 520: extract the security operation instruction from the first task queue and push it to the first type of model.

[0065] Step 530: Use the first type of model to identify the intent of the safety operation instruction, and determine whether there is a tool in the tool library that matches the intent of the safety operation instruction.

[0066] Step 540: When it is determined that there is a tool in the tool library that matches the intent of the security operation instruction, the tool is determined as a target tool, and the target tool is called to execute the corresponding security operation task.

[0067] In this embodiment, steps 510 to 540 are Figure 3 The embodiments shown correspond to those described above and will not be described in detail here.

[0068] Step 550: When it is determined by using the first type of model that there is no tool in the tool library that matches the intent of the safety operation instruction, the safety operation instruction is transferred from the first task queue to the high-precision task sub-queue.

[0069] In this embodiment, the second task queue includes a high-precision task sub-queue for storing tasks requiring higher reasoning accuracy, such as safety operation instructions for failure of reasoning of the first type of model.

[0070] Since the first type of model has a small number of parameters, its execution performance is much higher than that of the large model. However, due to its low natural language comprehension ability, its recall rate is low, and reasoning failure may occur. That is, the first type of model is used to determine that there is no tool in the tool library that matches the intent of the security operation instruction. At this time, the security operation instruction can be transferred from the first task queue to the high-precision task sub-queue.

[0071] Step 560: extract the safety operation instructions from the high-precision task sub-queue and push them to the high-precision large model.

[0072] Step 570: Use a high-precision large model to perform precise reasoning based on the safety operation instruction to determine whether there is a tool in the tool library that matches the safety operation instruction.

[0073] As an example, a high-precision large model can prompt engineering to select tools in the tool library with functional description information that matches the safety operation instructions according to the safety operation instructions.

[0074] Step 580: When it is determined that there is a tool in the tool library that matches the intent of the safety operation instruction, the tool is determined as the target tool.

[0075] In this embodiment, for task execution instructions, the first type of model can be used for reasoning first. If the first type of model reasoning fails, the high-precision large model can be used to accurately reason about the task execution instructions to determine the target tool. By scheduling different models, both reasoning speed and accuracy can be taken into account.

[0076] In some optional implementations of this embodiment, after step 580, the process may also include: using a high-precision large model to identify the security operation scenario applicable to the target tool; when the security operation scenario applicable to the target tool is a real-time execution scenario, calling the target tool to execute the corresponding security operation task; when the security operation scenario applicable to the target tool is an asynchronous execution scenario, determining the calling timing of the target tool and the execution timing of the security operation task according to the task load of the high-precision large model.

[0077] In this embodiment, the description information of the tool may include the scenarios in which the tool is applicable. The high-precision large model can use this to identify whether the security operation scenarios applicable to the target tool are real-time execution scenarios or asynchronous execution scenarios, and determine the execution timing of the security operation tasks based on the type of security operation scenarios, so as to take into account the efficiency of the large model and the timeliness of task execution.

[0078] As an example, for real-time execution scenarios, the target tool can be called immediately to ensure timely execution of security operation tasks; for asynchronous execution scenarios, when the task load of the high-precision large model is large, execution can be postponed until the task load of the high-precision large model is small.

[0079] In an optional example of this embodiment, when the security operation scenario applicable to the target tool is an asynchronous execution scenario, the waiting time is predicted based on the task load of a high-precision large model; when the waiting time is not less than a first preset time, the target tool is called to execute the security operation task.

[0080] In this example, the waiting time of the inference task can be predicted based on the task load of the high-precision large model. When the waiting time is not less than a first preset time (which can be set according to actual needs, such as 1 minute or 5 minutes), the target tool can be called immediately to perform the security operation task to avoid task timeout; when the waiting time is less than the first preset time, the security operation task can be put back to the end of the task queue and wait for the next call.

[0081] In some optional implementations of this embodiment, after determining the target tool using the high-precision large model, the high-precision large model can also be used to extract the parameters required by the target tool from the safety operation instructions, and push the parameters to the target tool so as to call the target tool to perform the corresponding safety operation tasks; and, in the process of using the target tool to perform safety operation tasks, the high-precision large model can be used to perform inference based on the data obtained by the target tool.

[0082] In this embodiment, the description information of the tool may also include a tag indicating whether to call a large model. When the description of the target tool includes a tag indicating whether to call a large model, a high-precision large model may be called to assist the target tool in performing safety operation tasks.

[0083] For example, when the security operation instruction is "Whether there are phishing emails among the 100 emails received recently", the corresponding target tool can be an analysis tool. The high-precision large model can extract "100" as a parameter from the security operation instruction and push it to the analysis tool. The analysis tool obtains the content of the 100 emails and then pushes the email content to the high-precision large model. The high-precision large model infers whether the email is a phishing email and returns the judgment reason and result to the analysis tool to complete the security operation task.

[0084] In this embodiment, the high-precision large model can also extract the parameters required by the target tool from the safety operation instructions to assist the target tool in performing safety operation tasks; and perform reasoning based on the data obtained by the target tool to assist the target tool in performing safety operation tasks.

[0085] Next reference Figure 6 ,exist Figure 6 In the illustrated embodiment, the second type of model also includes a low-precision large model; the second task queue also includes a low-precision task sub-queue. The low-precision large model refers to a model with a parameter amount greater than 100 million and less than the high-precision large model. For example, the parameter amount of the high-precision large model can be 70 billion, which can process about 10 inference tasks in parallel; the parameter amount of the low-precision large model can be 6 billion, which can process more than 50 inference tasks in parallel. Figure 6 As shown, the process of using the second type of model to reason about knowledge question and answer instructions may include the following steps.

[0086] Step 610: When the security operation instruction is determined to be a knowledge question and answer instruction, the security operation instruction is added to the low-precision task sub-queue.

[0087] When the task load of the high-precision large model meets the preset conditions, step 620 and step 630 are executed; when the task load of the high-precision large model does not meet the preset conditions, step 640 is executed.

[0088] In some optional implementations of this embodiment, when the target tool does not need to call the high-precision large model, the safety operation instruction can be transferred from the high-precision task subqueue to the low-precision task subqueue to reduce the task load of the high-precision large model and shorten the response time of the safety operation task.

[0089] Step 620: extract the safety operation instructions from the low-precision task sub-queue and push them to the low-precision large model.

[0090] Step 630: Use the low-precision large model to perform reasoning based on the safety operation instruction to determine the response statement corresponding to the safety operation instruction.

[0091] Step 640: When the task load of the high-precision large model does not meet the preset conditions, the safety operation instructions are extracted from the low-precision task sub-queue and the high-precision task sub-queue, and pushed to the high-precision large model, and the high-precision large model performs reasoning based on the safety operation instructions.

[0092] In this embodiment, the preset conditions can characterize the task load state of the high-precision large model. If the task load of the high-precision large model meets the preset conditions, it means that the task load of the high-precision large model is relatively high. At this time, the low-precision large model is used to infer the safety operation instructions in the low-precision task subqueue, thereby sharing the load pressure of the high-precision large model, and improving the service throughput of the second type of model; on the contrary, if the task load of the high-precision large model does not meet the preset conditions, it means that the task load of the high-precision large model is relatively low. At this time, the high-precision large model can be used to infer the safety operation instructions in the low-precision task subqueue and the high-precision task subqueue at the same time. The adaptive degradation of the accuracy of the inference task according to the load pressure of the high-precision large model is realized, which can improve the utilization rate of the second type of model.

[0093] Identifying the target tool or response statement separately can improve the accuracy of reasoning.

[0094] Next reference Figure 7 , Figure 7 A flowchart of determining whether the task load of a high-precision large model meets preset conditions in one embodiment of the method for multi-model collaborative processing of security operation tasks of the present application is shown, such as Figure 7 As shown, the process includes the following steps.

[0095] Step 710: Obtain the number of tasks to be inferred in the high-precision task subqueue within a unit time period and the latency information of a single task.

[0096] The delay information of a single task includes an average response delay of a first field of the single task and / or an average total delay of the single task.

[0097] In this embodiment, the average response delay of the first field (token) of a single task (i.e., a security operation instruction) and the average total delay of a single task can be calculated from the moment when the security operation instruction is added to the high-precision task subqueue. The unit time period can be set according to actual conditions, for example, 1 minute, 30 seconds or other.

[0098] Here, the delay information of a single task can be used as the waiting time of the reasoning task, and the execution timing of the asynchronous execution scenario is determined by comparing the delay information with the second preset time.

[0099] Step 720: Based on the number of tasks to be inferred and the latency information of a single task, a linear regression model is constructed, and the Gaussian distribution of the latency information of a single task is determined.

[0100] As an example, the latency information of a single task may include the average response latency of the first field of a single task and the average total latency of a single task. A linear regression model of the average response latency can be constructed based on the number of tasks to be inferred and the average response latency of the first field of a single task, and the Gaussian distribution of the average response latency can be determined. At the same time, a linear regression model of the average total latency can be constructed based on the number of tasks to be inferred and the average total latency of a single task, and the Gaussian distribution of the average total latency can be determined.

[0101] Alternatively, the delay information of a single task may include the average response delay of the first field of a single task or the average total delay of a single task. Then, a linear regression model of the average response delay or a linear regression model of the average total delay may be constructed according to the number of tasks to be inferred and the average response delay of the first field of a single task or the average total delay of a single task, and the Gaussian distribution of the average response delay or the Gaussian distribution of the average total delay may be determined accordingly.

[0102] Step 730: Determine that the task load of the high-precision large model meets the preset conditions when any of the following conditions is met: the delay information prediction value of the task most recently added to the high-precision task subqueue is greater than a first threshold, and the average of the delay information prediction values ​​of the tasks added to the high-precision task subqueue within a second preset time period is greater than a second threshold.

[0103] The first threshold and the second threshold are determined based on the mean value and standard deviation of the Gaussian distribution.

[0104] It should be noted that when none of the above conditions are met, it means that the task load of the high-precision large model does not meet the preset conditions.

[0105] As an example, the first threshold may be μ+2σ of the Gaussian distribution, and the second threshold may be μ+σ, where μ represents the mean value of the Gaussian distribution and σ represents the standard deviation of the Gaussian distribution. The second preset duration may be set according to actual conditions, for example, 2 minutes, 3 minutes or others.

[0106] In this embodiment, it is possible to estimate whether the task load of the high-precision large model meets the preset conditions based on the number of tasks and latency information in the high-precision task subqueue, and accurately and timely obtain the load pressure status of the high-precision large model, thereby scheduling the inference tasks of the high-precision large model and the low-precision large model, which helps to improve the load balancing and inference accuracy of the second type of model.

[0107] Figure 8 A flowchart of an embodiment of the method for multi-model collaborative processing of security operation tasks of the present application is shown as follows: Figure 8 As shown, the process includes the following steps.

[0108] Step 800: Receive a security operation instruction from a user, and use a pre-trained classification model to determine the type of the security operation instruction.

[0109] If the safety operation instruction is determined to be a task execution instruction, steps 802 to 818 are executed. If the safety operation instruction is determined to be a knowledge question answering instruction, steps 820 to 826 are executed.

[0110] Step 802: When the safety operation instruction is determined to be an execution task instruction, the safety operation instruction is added to the first task queue.

[0111] Step 804: extract the security operation instruction from the first task queue and push it to the first type of model.

[0112] Step 806: Use the first type of model to identify the intent of the safety operation instruction, and determine whether there is a tool in the tool library that matches the intent of the safety operation instruction.

[0113] Step 808: When it is determined that there is a tool in the tool library that matches the intent of the security operation instruction, the tool is determined as a target tool, and the target tool is called to execute the corresponding security operation task.

[0114] In this embodiment, steps 802 to 808 are Figure 3 The embodiments shown correspond to those described above and will not be described in detail here.

[0115] Step 810: When it is determined by using the first type of model that there is no tool in the tool library that matches the intent of the safety operation instruction, the safety operation instruction is transferred from the first task queue to the high-precision task sub-queue.

[0116] Step 812: extract the safety operation instructions from the high-precision task sub-queue and push them to the high-precision large model.

[0117] Step 814: Use a high-precision large model to perform precise reasoning based on the security operation instructions to determine whether there is a tool in the tool library that matches the security operation instructions.

[0118] Step 816: When it is determined that there is a tool in the tool library that matches the intent of the security operation instruction, the tool is determined as a target tool, and the target tool is called to execute the corresponding security operation task.

[0119] In this embodiment, steps 810 to 816 are Figure 5 Steps 550 to 580 in the illustrated embodiment correspond to each other and are not described in detail here.

[0120] Step 818, when the high-precision large model is used to determine that there is no tool in the tool library that matches the intent of the safety operation instruction, the safety operation instruction is pushed to the low-precision large model or the safety operation instruction is transferred to the low-precision task sub-queue, so that the low-precision large model can be used to perform reasoning based on the safety operation instruction to determine the response statement corresponding to the safety operation instruction.

[0121] In this embodiment, when the high-precision large model fails to infer the security operation task in the high-precision task sub-queue (i.e., it is determined that there is no tool in the tool library that matches the intent of the security operation instruction), it indicates that the security operation task may be a knowledge question and answer instruction. The low-precision large model then infers the security operation instruction to generate a reply statement.

[0122] Step 820: When the security operation instruction is determined to be a knowledge question and answer instruction, the security operation instruction is added to the low-precision task sub-queue.

[0123] When the task load of the high-precision large model meets the preset conditions, step 822 and step 824 are executed; when the task load of the high-precision large model does not meet the preset conditions, step 826 is executed.

[0124] Step 822: extract the safety operation instructions from the low-precision task sub-queue and push them to the low-precision large model.

[0125] Step 824: Use the low-precision large model to perform reasoning based on the safety operation instruction to determine the response statement corresponding to the safety operation instruction.

[0126] Step 826: When the task load of the high-precision large model does not meet the preset conditions, safety operation instructions are extracted from the low-precision task sub-queue and the high-precision task sub-queue, and pushed to the high-precision large model, which performs reasoning based on the safety operation instructions.

[0127] In this embodiment, the first type of model, the high-precision large model and / or the low-precision large model can be collaboratively processed according to the type of safety operation instructions. The safety operation instructions that failed to be identified by the first type of model are inferred by the high-precision large model, and the reasoning accuracy of the tasks in the second task queue is adaptively downgraded according to the task load pressure of the high-precision large model to coordinate the load pressure between the second type of models. Multiple models can be scheduled from multiple dimensions and multiple levels to collaboratively process safety operation tasks, which helps to improve the response speed and accuracy of safety operation tasks.

[0128] In a specific example, the first type of model is the BERT model; the high-precision large model is a large model that deploys 70 billion parameters and can support about 10 concurrent task sessions; the low-precision large model is a large model that deploys 6 billion parameters and can support more than 50 concurrent task sessions.

[0129] Users can Figure 1 The command interface in the security orchestration and automation system shown inputs the following three queries (i.e., security operation commands): perform a vulnerability scan on assets in network segment 192.168.56.0 (Q1), perform a vulnerability check on assets in network segment 192.168.56.0 (Q2), and what is the purpose of vulnerability scanning (Q3).

[0130] Through the classification model, it can be determined that Q1 and Q2 are execution task type instructions, and Q1 and Q2 are placed in the first task queue. Q3 is a knowledge question and answer type instruction and can be placed in a low-precision task sub-queue.

[0131] Afterwards, the BERT model is called to identify Q1 and Q2. Q1 is accurately identified as needing to call the vulnerability scanning tool to perform the task, and the corresponding tool is called through the tool execution engine to complete the vulnerability scanning task; Q2 cannot be matched with the corresponding tool, so Q2 is transferred to the high-precision task subqueue.

[0132] Call the high-precision large model to analyze the user intent of Q2 and match the description information of the tool to determine the tool chain required to perform the task, for example, it may include using the vulnerability scanning tool for the first scan, using the work order tool to notify the vulnerability repair, using the vulnerability scanning tool for a second scan to verify the repair structure, and using the report generation tool to generate a verification report. In this process, the high-precision large model can be used to extract the parameters in Q2, obtain the tool parameters of "Network segment: 192.168.56.0", and identify the tool chain as an asynchronous execution scenario.

[0133] In this process, it is assumed that the lengths of the high-precision task subqueues for three consecutive minutes are monitored to be 26, 28, and 25, the number of newly queued tasks is 25, and the second threshold is μ+σ of the Gaussian distribution. Based on the linear regression model, it is determined that the predicted value of the average response delay of the first field of a single task at this time is 12s, which is greater than the second threshold of 10s corresponding to the average response delay; the predicted value of the average total delay of a single task is 40s, which is greater than the second threshold of 35s corresponding to the average total delay. It can be determined that the task load of the high-precision large model meets the preset conditions. At this time, the security operation instructions in the low-precision task subqueue can be pushed to the low-precision large model for reasoning, that is, Q3 is switched to the low-precision large model for reasoning.

[0134] Determine whether the waiting time of Q2 (for example, it can be the delay information of a single task) is less than the second preset time. If it is less than the second preset time, put Q2 at the end of the task queue and continue waiting; if it is greater than the second preset time, execute Q2 immediately.

[0135] An embodiment of the present application also provides a device for collaboratively processing security operation tasks using multiple models, including a processor and a memory storing a computer program. When the computer program is executed by the processor, the method for collaboratively processing security operation tasks using multiple models in any of the above embodiments can be implemented.

[0136] An embodiment of the present application also provides a non-volatile computer storage medium, which stores a computer program. In some embodiments, when the computer program is executed by a processor, the method of multi-model collaborative processing of security operation tasks in any of the above embodiments is implemented.

[0137] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term "computer storage medium" includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A method for collaboratively processing security operation tasks using multiple models, characterized in that: include: Receive a security operation instruction from a user, and determine the type of the security operation instruction using a pre-trained classification model; In the case where the safety operation instruction is determined to be an execution task type instruction, reasoning is performed based on the safety operation instruction using the first type of model to determine a target tool that needs to be called in response to the safety operation instruction; and, in the case where the target tool is determined, calling the target tool to perform a corresponding security operation task; In the case where the safety operation instruction is determined to be a knowledge question and answer instruction, reasoning is performed based on the safety operation instruction using the second type of model to determine a reply sentence corresponding to the safety operation instruction; The first type of model has a smaller number of parameters than the second type of model; the second type of model includes a high-precision large model; The first type of model is used to perform reasoning based on the safety operation instruction to determine the target tool that needs to be called in response to the safety operation instruction, including: using the first type of model to identify the intention of the safety operation instruction, and determining whether there is a tool in the tool library that matches the intention of the safety operation instruction; if it is determined that there is a tool in the tool library that matches the intention of the safety operation instruction, the tool is determined as the target tool; The method also includes: when it is determined by using the first type of model that there is no tool in the tool library that matches the intention of the security operation instruction, using the high-precision large model to determine the target tool and identify the security operation scenario applicable to the target tool; when the security operation scenario applicable to the target tool is a real-time execution scenario, calling the target tool to execute the corresponding security operation task; when the security operation scenario applicable to the target tool is an asynchronous execution scenario, determining the calling timing of the target tool and the execution timing of the corresponding security operation task according to the task load of the high-precision large model.

2. The method according to claim 1, characterized in that: Before using the first type of model to identify the intention of the safety operation instruction, the method further includes: In the case where the safety operation instruction is determined to be an execution task type instruction, the safety operation instruction is added to a first task queue, where the first task queue is used to store the execution task type instructions to be inferred; The security operation instruction is extracted from the first task queue and pushed to the first type of model.

3. The method according to claim 2, characterized in that In the case where the safety operation instruction is determined to be a knowledge question and answer instruction, reasoning based on the safety operation instruction using the second type of model includes: In the case where the safety operation instruction is determined to be a knowledge question and answer instruction, the safety operation instruction is added to a second task queue, where the second task queue is used to store the knowledge question and answer instructions to be inferred; Extracting the security operation instruction from the second task queue and pushing it to the second type model; The second type of model is used to perform reasoning based on the safety operation instruction to determine a response statement corresponding to the safety operation instruction.

4. The method according to claim 3, characterized in that Using the second type of model to perform reasoning based on the safety operation instruction to determine the response statement corresponding to the safety operation instruction, including: using the high-precision large model to perform reasoning based on the safety operation instruction to determine the response statement corresponding to the safety operation instruction.

5. The method according to claim 4, characterized in that The second task queue includes a high-precision task sub-queue; and determining the target tool using the high-precision large model includes: In a case where it is determined by using the first type of model that there is no tool in the tool library that matches the intent of the safety operation instruction, transferring the safety operation instruction from the first task queue to the high-precision task sub-queue; Extracting the safety operation instruction from the high-precision task sub-queue and pushing it to the high-precision large model; Using the high-precision large model to perform precise reasoning based on the safety operation instruction, determining whether there is a tool matching the safety operation instruction in the tool library; When it is determined that there is a tool in the tool library that matches the intention of the safety operation instruction, the tool is determined as the target tool.

6. The method according to claim 5, characterized in that When the target tool is applicable to an asynchronous execution scenario, the calling time of the target tool and the execution time of the corresponding security operation task are determined according to the task load of the high-precision large model, including: When the target tool is applicable to an asynchronous execution scenario, the waiting time is predicted based on the task load of the high-precision large model; When the waiting time is not less than the first preset time, the target tool is called to execute the corresponding security operation task.

7. The method according to claim 5, characterized in that After determining the target tool using the high-precision large model, the method further includes: When the target tool needs to call the large model, the high-precision large model is used to extract the parameters required by the target tool from the safety operation instruction, and the parameters are pushed to the target tool so as to call the target tool; and In the process of using the target tool to perform a safe operation task, the high-precision large model is used to perform reasoning based on data obtained by the target tool so as to perform the safe operation task.

8. The method according to any one of claims 5 to 7, characterized in that The second type of model also includes a low-precision large model; the second task queue also includes a low-precision task sub-queue; When the security operation instruction is determined to be a knowledge question and answer instruction, adding the security operation instruction to the second task queue includes: when the security operation instruction is determined to be a knowledge question and answer instruction, adding the security operation instruction to the low-precision task sub-queue; The method further comprises: when the target tool does not need to call the high-precision large model, transferring the safety operation instruction to the low-precision task subqueue; The second type of model is used to perform reasoning based on the safety operation instruction to determine the reply statement corresponding to the safety operation instruction, including: when the task load of the high-precision large model meets the preset conditions, the safety operation instruction is extracted from the low-precision task subqueue and pushed to the low-precision large model; and the low-precision large model is used to perform reasoning based on the safety operation instruction to determine the reply statement corresponding to the safety operation instruction.

9. The method according to claim 8, characterized in that The method further comprises: When the high-precision large model is used to determine that there is no tool in the tool library that matches the intent of the safety operation instruction, the safety operation instruction is pushed to the low-precision large model or the safety operation instruction is transferred to the low-precision task subqueue, so that the low-precision large model can be used to perform reasoning based on the safety operation instruction to determine the reply statement corresponding to the safety operation instruction.

10. The method according to claim 8, characterized in that The method further includes: obtaining the number of tasks to be inferred in the high-precision task subqueue within a unit time period and latency information of a single task, wherein the latency information of the single task includes an average response latency of a first field of the single task and / or an average total latency of the single task; Based on the number of tasks to be inferred and the delay information of the single task, a linear regression model is constructed, and a Gaussian distribution of the delay information of the single task is determined; When any of the following conditions is met, it is determined that the task load of the high-precision large model meets the preset conditions: the delay information prediction value of the task most recently added to the high-precision task subqueue is greater than a first threshold, and the average of the delay information prediction values ​​of the tasks added to the high-precision task subqueue within a second preset time length is greater than a second threshold, wherein the first threshold and the second threshold are determined based on the mean and standard deviation of the Gaussian distribution.

11. A device for collaboratively processing security operation tasks using multiple models, characterized in that: It comprises a processor and a memory storing a computer program, and is characterized in that when the computer program is executed by the processor, it can implement the method of multi-model collaborative processing of security operation tasks as described in one of claims 1 to 10.

12. A security orchestration and automated response system, characterized in that: include: Command interface and response module, where: The command interface is configured to receive a user's security operation command; The response module is configured to respond to the safety operation instruction through the method of multi-model collaborative processing of safety operation tasks as described in one of claims 1 to 10.

13. A non-transitory computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for multi-model collaborative processing of security operation tasks as described in one of claims 1 to 10 is implemented.

Images