Model training method and apparatus, storage medium, and computer device

Abstract

Embodiments of the present application provide a model training method and apparatus, a storage medium, and a computer device. The method comprises: generating a plurality of first security task descriptions, and obtaining security task description clusters by means of clustering and dividing; acquiring security reference data, determining the similarity between each piece of security reference data and a description in each cluster, and determining a target security task description corresponding to each piece of reference data; on the basis of the foregoing, generating a first input instruction, the instruction requiring to generate question-and-answer data on the basis of the security reference data and the target security task description; inputting the instruction into a trained generation model, to obtain the corresponding question-and-answer data; and then inputting the question-and-answer data into a trained security task model for fine-tuning, so as to obtain a fine-tuned security task model.

Description

Model training methods, devices, storage media, and computer equipment

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 202510459014.8, filed on April 14, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, storage medium and computer equipment. Background Technology

[0004] As a critical department in cybersecurity, the Security Operations Center bears the heavy responsibility of detecting, analyzing, and responding to cybersecurity incidents using various tools, technologies, and processes. However, the center currently faces numerous serious challenges, such as a lack of security expertise, excessive time spent investigating alerts, and slow response times to advanced threats. With the emergence of large-scale models, leveraging their powerful understanding and generation capabilities to assist security operations has become possible. However, it is important to note that cybersecurity is a highly specialized field, and handling security issues requires profound professional knowledge and skills.

[0005] In related technologies, fine-tuning of large question-answering models with domain-specific data is particularly necessary to build models that fit security operation scenarios. However, the prominent problem at present is the lack of high-quality domain data, and most existing large model fine-tuning techniques are concentrated in general domains without fully considering the construction of fine-tuning data for specific domains. This results in poor accuracy of the model's answers for specific domains. Therefore, there is an urgent need to propose a model training method to solve the above-mentioned technical problems. Summary of the Invention

[0006] The main objective of this application is to provide a model training method, apparatus, storage medium, and computer device that can fine-tune the model using question-and-answer data from a specific domain, thereby improving the accuracy of the model's answers for that specific domain.

[0007] In a first aspect, embodiments of this application provide a model training method, including:

[0008] Multiple first security task descriptions are generated, and the multiple first security task descriptions are clustered to obtain multiple security task description clusters;

[0009] Acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster;

[0010] Based on the similarity between each security reference data and the corresponding first security task description, the target security task description corresponding to each security reference data is determined;

[0011] Based on each of the security reference data and the corresponding target security task description, a first input instruction is generated. The first input instruction is a command to generate input prompts for question-and-answer data according to the security reference data and the corresponding target security task description.

[0012] The first input instruction is input into the trained generative model to generate question-and-answer data corresponding to the first input instruction.

[0013] The question-and-answer data is input into the trained security task model to fine-tune the trained security task model, resulting in a fine-tuned security task model.

[0014] Secondly, embodiments of this application provide a model training apparatus, comprising:

[0015] A clustering unit is used to generate multiple first security task descriptions, and to perform clustering on the multiple first security task descriptions to obtain multiple security task description clusters.

[0016] The first determining unit is configured to acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster.

[0017] The second determining unit is used to determine the target security task description corresponding to each security reference data based on the similarity between each security reference data and the corresponding first security task description;

[0018] The generation unit is configured to generate a first input instruction based on each security reference data and the corresponding target security task description. The first input instruction is an input prompt for generating question-and-answer data according to the security reference data and the corresponding target security task description.

[0019] The input unit is used to input the first input instruction into the trained generative model to generate question-and-answer data corresponding to the first input instruction.

[0020] The fine-tuning unit is used to input the question-and-answer data into the trained security task model to fine-tune the trained security task model and obtain the fine-tuned security task model.

[0021] Thirdly, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the model training method as described above.

[0022] Fourthly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model training method as described above.

[0023] In this embodiment, multiple first security task descriptions are generated, and these descriptions are clustered to obtain multiple security task description clusters. Multiple security reference data are acquired, and the similarity between each security reference data and any first security task description in each security task description cluster is determined. Based on the similarity between each security reference data and its corresponding first security task description, a target security task description corresponding to each security reference data is determined. Based on each security reference data and its corresponding target security task description, a first input instruction is generated. This first input instruction is a prompt to generate question-and-answer data based on the security reference data and the corresponding target security task description. The first input instruction is then input into the training... The generated model generates question-and-answer data corresponding to the first input instruction. The question-and-answer data is then input into the trained security task model to fine-tune the model, resulting in a fine-tuned security task model. Compared to related technologies where the lack of high-quality domain data leads to poor accuracy in answers for specific domains, this embodiment of the application obtains a domain-specific security task description cluster by clustering multiple first security task descriptions and combining them with reference data to generate the first input instruction. The generated model then generates question-and-answer data for training the model in a specific domain. This question-and-answer data is used to train the security task model that needs fine-tuning after training, thereby improving the accuracy of the model's answers for specific domains.

[0024] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 is a schematic diagram of a model training system provided in an embodiment of this application.

[0027] Figure 2 is a flowchart illustrating the model training method provided in an embodiment of this application;

[0028] Figure 3 is a schematic diagram of the model training device provided in an embodiment of this application;

[0029] Figure 4 is a schematic diagram of the structure of the computer device provided in the embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] It should be noted that while some processes described in the specification, claims, and accompanying drawings contain multiple steps that appear in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not represent any particular order of execution. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0032] Please refer to Figure 1, which is a schematic diagram of a model training system provided in an embodiment of this application. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.

[0033] Terminal 140 includes, but is not limited to, pre-configured electronic devices with data reporting capabilities such as laptops, tablets, or desktop computers. Furthermore, it can be a single device or a collection of multiple devices. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.

[0034] Terminal 140 refers to a computer system capable of submitting data to server 110. Compared to ordinary terminals, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines).

[0035] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.

[0036] The model training method of this application embodiment can be implemented on server 110.

[0037] It should be noted that the schematic diagram of the model training system shown in Figure 1 is merely an example. The model training system and scenario described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided in the embodiments of this application.

[0038] In this embodiment, the description will be from the perspective of a model training device, which can be integrated into a computer device that has a storage unit and a microprocessor and thus computing power.

[0039] Please refer to Figure 2, which is a flowchart illustrating the model training method provided in an embodiment of this application. The model training method includes:

[0040] In step 201, multiple first security task descriptions are generated, and the multiple first security task descriptions are clustered to obtain multiple security task description clusters.

[0041] The first security task description refers to a textual description of a task within a specific security domain, such as "detecting the presence of malware intrusion behavior in the network" or "assessing the system's defense capabilities against DDoS attacks." It details the specific tasks that need to be completed within the security domain. A security task description cluster is a set obtained by clustering multiple first security task descriptions. First security task descriptions within the same cluster belong to the same security domain. For example, task descriptions related to network attack detection are grouped into one cluster.

[0042] In some implementations, the step of clustering the multiple first security task descriptions to obtain multiple security task description clusters includes:

[0043] (1) Perform vector transformation on each of the first security task descriptions to obtain a vector representation of each of the first security task descriptions;

[0044] (2) Select a first preset number of candidate security task descriptions from each of the first security task descriptions as the initial cluster centers of different initial security task description clusters;

[0045] (3) Calculate the distance between the vector representation of each of the other security tasks and the vector representation of each of the initial cluster centers;

[0046] (4) Each of the other security task descriptions is assigned to the initial security task description cluster corresponding to the nearest initial cluster center to obtain multiple updated security task description clusters;

[0047] (5) Determine the mean of the vector representations of other security task descriptions in each of the updated security task description clusters and the vector representations of the corresponding initial cluster centers to obtain the updated cluster centers of each of the updated security task description clusters;

[0048] (6) When the clustering termination condition is not met, the updated cluster center is determined as the initial cluster center, the updated security task description cluster is determined as the initial security task description cluster, and the step of calculating the distance between the vector representation of other security task descriptions and the vector representation of each initial cluster center is returned until the clustering termination condition is met, and the resulting multiple updated security task description clusters are determined as security task description clusters.

[0049] The vector conversion process transforms the text-based first-level security task description into a numerical vector form that the computer can process, enabling subsequent distance calculations and cluster analysis. Embedding models (such as Word2Vec, BERT, etc.) are used to convert each task description into a vector representation. Assume there are n task descriptions, denoted as T = {t1, t2, ..., tn}. n After processing by the embedding model, each task description t i It is converted into a d-dimensional vector x i =(x i1 ,x i2 ,…,x id ), where i = 1, 2, ..., n.

[0050] Specifically, the first preset quantity is a pre-defined value used to determine the number of initial cluster centers selected. This quantity affects the clustering results and computational efficiency, and usually needs to be adjusted reasonably based on the number and characteristics of the first security task descriptions. Candidate security task descriptions are selected from the set of first security task descriptions to serve as initial cluster centers. Initial security task description clusters are formed at the beginning of clustering, with each initial cluster center as the core. Other security task descriptions will be subsequently assigned to these clusters, forming the preliminary clustering results. The initial cluster center is the core of each initial security task description cluster; it is a vector representing the initial characteristics of that cluster. Other security task descriptions are the first security task descriptions other than those selected as initial cluster centers. Distance is used to measure the similarity or difference between two vectors; common distance metrics include Euclidean distance.

[0051] Specifically, to initialize clusters, k data points are randomly selected as initial cluster centers, denoted as μ1, μ2, ..., μ3. k Where, μ j ,=(μ j1 ,…,μ jd ),j=1,2,…,k.

[0052] For each other security task description x i Calculate its relationship with each cluster center μ j The distance is usually expressed using Euclidean distance:

[0053] Where, l∈[1, d], d(x) i ,μ j This refers to other security task descriptions x i With each cluster center μ j The distance. x il -μ jl Describe x for other security tasks i With each cluster center μ j The difference between vectors of the same dimension.

[0054] Based on the distances calculated in the previous step, for each other security task description, find the nearest initial cluster center and then assign that description to the initial security task description cluster corresponding to that initial cluster center. After partitioning all other security task descriptions, multiple updated security task description clusters are obtained. For each updated security task description cluster, the vector representations of all other security task descriptions within it are aggregated with the vector representations of the corresponding initial cluster centers (here, the mean is calculated). In this way, the updated cluster center of each updated security task description cluster is calculated.

[0055] The clustering termination condition is a pre-defined condition used to determine whether the clustering process can end. It checks whether the current clustering results meet the termination condition. If not, the updated cluster centers are used as new initial cluster centers, and the updated security task description clusters are used as new initial security task description clusters. This process is then repeated iteratively, recalculating the distances between other security task descriptions and the new initial cluster centers, performing description partitioning, and updating cluster centers. This process continues until the clustering termination condition is met. Finally, the multiple updated security task description clusters obtained when the termination condition is met are determined as the final security task description clusters.

[0056] Specifically, for each cluster, the mean of all data points within the cluster is calculated, and the cluster centers are updated accordingly. Let C be the cluster describing the j-th security task. j Then the updated cluster centers μ j for:

[0057] Among them, |C j | Represents the security task description cluster C j The number of data points in the cluster, i.e., the updated cluster centers μ are determined by calculating the mean. j Repeating the above steps, the first security task description is divided into k clusters, denoted as C1, C2, ..., Ck. k Each cluster contains a set of first security task descriptions within the same security domain.

[0058] Therefore, through clustering, numerous primary security task descriptions are grouped into different security task description clusters based on their similarity. The originally scattered and disordered security task descriptions are integrated into organized clusters with specific characteristics. In subsequent security task model training, these clustered clusters can serve as high-quality training data. Task descriptions within the same cluster share similarities, allowing the model to better learn the patterns and features of that type of task, thus improving training effectiveness and generalization ability. For example, when training a model for detecting network attacks, using security task description clusters related to network attack detection as training data allows the model to focus more intently on learning knowledge in that domain, improving detection accuracy and efficiency.

[0059] In some implementations, after obtaining the updated cluster center for each of the updated security task description clusters, the method further includes:

[0060] (1) Calculate the difference between the vector representation of the updated cluster center of each of the updated security task description clusters and the vector representation of the corresponding initial cluster center;

[0061] (2) Compare the difference value of each of the updated security task description clusters with the preset difference value;

[0062] (3) When there is an updated security task description cluster with a corresponding difference value greater than or equal to the preset difference value, it is determined that the clustering termination condition is not met.

[0063] (4) When the difference value of each of the updated security task description clusters is less than the preset difference value, it is determined that the clustering termination condition is met.

[0064] For each updated security task description cluster, a suitable difference metric (such as Euclidean distance or cosine distance) is used to calculate the difference between the vector representation of the updated cluster centers and the vector representation of the corresponding initial cluster centers. The preset difference value is a pre-defined threshold used as a standard to judge the degree of difference between the updated and initial cluster centers. This value usually needs to be adjusted based on the actual clustering requirements and data characteristics. The calculated difference value for each updated security task description cluster is then compared with the preset difference value.

[0065] Specifically, the comparison results of the difference values ​​of all updated security task description clusters with the preset difference value are checked. If the difference value of at least one updated security task description cluster is greater than or equal to the preset difference value, the current clustering result is considered unstable, and the cluster centers are still changing significantly. In this case, the clustering termination condition is not met, and the clustering operation needs to continue. When the difference values ​​of all updated security task description clusters are less than the preset difference value, it means that after this round of clustering operation, the difference between the updated cluster center and the initial cluster center of each cluster is relatively small, and the cluster centers have become relatively stable. At this point, the clustering result can be considered to have achieved the expected effect, and the clustering termination condition is met. The clustering process can be stopped, and the currently obtained multiple updated security task description clusters are determined as the final security task description clusters.

[0066] Therefore, by comparing the difference between the updated cluster centers and the initial cluster centers, and then comparing this difference with a preset value, premature termination of clustering before the cluster centers have stabilized can be effectively avoided. Without this judgment, the clustering results may be inaccurate because the cluster centers have not yet converged sufficiently to accurately represent the data characteristics within each cluster. For example, when the difference is large, it indicates that the cluster centers are still changing significantly. Stopping clustering at this point might result in similar security task descriptions being assigned to different clusters, or different security task descriptions being incorrectly assigned to the same cluster. This judgment mechanism ensures that the cluster centers are considered stable only when the difference values ​​of all clusters are small, thus yielding more accurate and reliable clustering results.

[0067] In some implementations, the generation of multiple first security task descriptions includes:

[0068] (1) Select a second preset number of security task descriptions from the first security task description set and the constructed second security task description set;

[0069] (2) Generate a second input instruction based on the selected security task description, the second input instruction being used to command the generation of a third preset number of generated security task descriptions according to the selected security task description;

[0070] (3) Input the second input instruction into the trained generative model to generate the corresponding generative safety task description;

[0071] (4) The generated security task description that meets the security task description conditions is identified as the first security task description and assigned to the first security task description set;

[0072] (5) When the total number of first security task descriptions in the first security task description set does not reach the fourth preset number, return to the step of selecting a second preset number of security task descriptions from the constructed second security task description set and the first security task description set, until the total number of first security task descriptions in the first security task description set reaches the fourth preset number, and obtain multiple first security task descriptions in the first security task description set.

[0073] The first security task description set is used to store the generated first security tasks, and the initial set is empty. The second security task description set is a task set constructed by technicians according to the needs of the security domain, and includes multiple second security task descriptions under at least one security domain. The second preset number is a pre-set value used to specify the total number of security task descriptions selected from the first and second security task description sets. Security task descriptions are randomly selected from the first and second security task description sets such that the total number of selected descriptions reaches the second preset number, and the number of security task descriptions selected from the second security task description set is greater than the number of security task descriptions selected from the first security task description set.

[0074] Specifically, the second input instruction is a command generated based on the selected security task description. Its function is to convey specific task requirements to the trained generative model, instructing the model to generate new security task descriptions using these selected descriptions as examples. For example, the selected security task descriptions are: Task 1, Task 2, and Task 3. Task 1 is: Threat Entity Extraction, described as: Please extract entities related to cyberattack events, such as threat organizations, malware, and attack methods, from the given text. The extracted information must be comprehensive and accurate, and the category of each entity must be labeled. Tasks 2 and 3 are similar to Task 1 and will not be elaborated here. The third preset quantity is no less than 3. Therefore, the second input instruction generated based on Task 1, Task 2, and Task 3 is:

[0075] Please generate at least 3 tasks and task descriptions related to security tasks:

[0076] Task 1: Threat Entity Extraction

[0077] Task Description: Extract entities related to cyberattacks, such as threat groups, malware, and attack methods, from the given text. The extracted information must be comprehensive and accurate, and each entity should be labeled with its category.

[0078] Task 2: XXX

[0079] Task 3: XXXX

[0080] In this way, the generated second input instruction is input into a trained generative model such as GPT-4o or ChatGPT, so that the trained generative model can generate a generative security task description corresponding to the second input instruction.

[0081] Since the specific content of the generated security task description may be unrelated to the security task, only the generated security task description based on the security task description conditions is identified as the first security task description and assigned to the first security task description set. The total number of first security task descriptions in the first security task description set is checked. If the total number of first security task descriptions in the first security task description set is less than the preset fourth number, the process returns to the step of selecting a second preset number of security task descriptions from the constructed second security task description set and the first security task description set, until the total number of first security task descriptions in the first security task description set reaches the fourth preset number, resulting in multiple first security task descriptions in the first security task description set.

[0082] Therefore, in the security field, high-quality task description data is often limited. This application's embodiment utilizes a trained generative model to generate new descriptions based on existing security task descriptions, effectively expanding the size of the first security task description set. As the number of iterations increases, the number of task descriptions in the set continuously grows, providing richer data support for subsequent security task analysis, model training, and other tasks. The generative model can expand, modify, and combine selected security task descriptions to varying degrees, thereby generating diverse new descriptions. These new descriptions may cover different security scenarios, task types, and expression methods, making the data in the first security task description set richer and more diverse.

[0083] Furthermore, as security operations evolve, the scope of security tasks may continuously expand or adjust. This solution allows technical personnel to build and update a second set of security task descriptions based on actual needs. Through an iterative generation process, the first set of security task descriptions can flexibly adapt to changes in the task scope. For example, when an enterprise decides to expand its security operations into new areas, it can add task descriptions for the new areas to the second set of security task descriptions, and then incorporate the relevant task descriptions into the first set of security task descriptions.

[0084] Furthermore, this embodiment of the application sets security task description conditions to filter the generated descriptions, identifying only those that meet the conditions as first security task descriptions and adding them to the set. This filtering mechanism can effectively exclude descriptions that are irrelevant to the security task or of low quality, ensuring the quality of data in the first security task description set.

[0085] In some implementations, determining the generated security task description that meets the security task description conditions as the first security task description includes:

[0086] (1) Generate a third input instruction based on each of the generated security task descriptions, wherein the third input instruction is used to command the answer as to whether the corresponding generated security task description is related to the security task;

[0087] (2) Input each of the third input instructions into the trained generative model to generate the answer result;

[0088] (3) The generated security task description corresponding to the third input instruction with the answer result of "yes" is determined as the first security task description.

[0089] The third input instruction is a specific instruction generated based on each generated security task description. Its main function is to enable the trained generative model to determine whether the description is relevant to the security task. This instruction clarifies that the task is to determine the relevance of the generated security task description. For each generated security task description, it is integrated into a single instruction to form the third input instruction. Each generated third input instruction is then sequentially input into the trained generative model. Upon receiving the instruction, the model understands and analyzes it, and based on its learned knowledge and patterns, determines the relevance of the generated security task description to the security task, generating the corresponding response.

[0090] Specifically, the response result for each third input instruction is checked. For responses of "yes," the generated security task description corresponding to that third input instruction is determined as the first security task description. These determined first security task descriptions are then assigned to the first security task description set.

[0091] For example, the security task description is: Malicious IP and Domain Identification. The task description is to identify malicious IP addresses and domains from given network traffic or DNS query logs. Based on existing threat intelligence databases, determine whether these IPs and domains are associated with known attackers or malicious activities. The third input instruction would then be:

[0092] Please determine whether the following content is related to a security task:

[0093] Malicious IP and Domain Identification

[0094] Task Description: Identify malicious IP addresses and domains from given network traffic or DNS query logs. Based on existing threat intelligence databases, determine whether these IPs and domains are associated with known attackers or malicious activities.

[0095] Please output "Yes" to indicate relevance and "No" to indicate irrelevance.

[0096] Output: "

[0097] Therefore, during the generation of security task descriptions, some content that is irrelevant or only weakly relevant to the security task may be generated. By having the trained generative model assess the relevance of each generated security task description, truly relevant descriptions can be effectively filtered out. Using the trained generative model for relevance assessment automates the process. Compared to manually assessing the relevance of each generated security task description, this automation significantly improves efficiency and saves labor costs. The generative model can quickly process a large number of third-party input instructions and generate corresponding responses, making it particularly suitable for situations involving a large number of security task descriptions.

[0098] In step 202, multiple security reference data are acquired, and the similarity between each security reference data and any first security task description in each security task description cluster is determined.

[0099] Among them, security reference data consists of data related to the security field, including network traffic data, system log data, security vulnerability report data, etc. This data forms the basis for subsequent analysis and question-and-answer data generation.

[0100] Specifically, security reference data is collected from multiple sources including ATT&CK, CVE, CWE, and open-source threat intelligence. This security reference data and the first security task descriptions are converted into vector form. Then, similarity calculation methods (such as cosine similarity) are used to calculate the similarity between each security reference data and any first security task description in each security task description cluster. In this way, the similarity value between each security reference data and each first security task description can be obtained.

[0101] Specifically, a dataset D of secure reference data is obtained through steps such as cleaning, long text segmentation, and deduplication. org ={t1,t2,…,t n}, regarding D org Each security reference data t i From each of the t clusters in T, sample one threat intelligence mission description and combine it with t... i Perform similarity calculation.

[0102] In step 203, the target security task description corresponding to each security reference data is determined based on the similarity between each security reference data and the corresponding first security task description.

[0103] Specifically, for each security reference data, the s first security task descriptions with the highest similarity are obtained from t first security task descriptions (s is a parameter defined by the fine-tuning data builder), and these s first security task descriptions are determined as the corresponding target security task descriptions.

[0104] In step 204, a first input instruction is generated based on each security reference data and the corresponding target security task description. The first input instruction is a command to generate input prompts for question-and-answer data based on the security reference data and the corresponding target security task description.

[0105] The first input instruction is a specific instruction that tells the subsequent generative model to generate question-and-answer data based on specific security reference data and the corresponding target security task description.

[0106] For example, security reference data is the specific data content of network traffic data. Based on the specific data content of network traffic data and the target security task description of detecting whether there is malware intrusion behavior in the network, question and answer data on how to detect malware intrusion based on the traffic data is generated.

[0107] Specifically, the details of each security reference data point and its corresponding target security task description are integrated and formatted to generate the first input instruction. The instruction format can be designed according to actual needs, but it usually clearly specifies the input data and task description, as well as the requirement to generate question-and-answer data.

[0108] Since the target security task description is a security task description for a specific domain, the question-and-answer data generated based on the first input instruction is also question-and-answer data related to that specific domain.

[0109] The first input command can be shown in the following example:

[0110] Task: Malicious IP and Domain Identification

[0111] Text: The security team discovered the following suspicious activity while analyzing network logs:

[0112] IP address 173.XXX made more than 10,000 connection requests to the internal server in the past 24 hours, all of which were HTTPS requests. This IP also attempted to access several common administrative backend paths such as / admin.

[0113] It was also discovered that the domain name hack.XXX was resolving to the IP address, and that the domain name was newly registered 48 hours ago.

[0114] According to the security platform query, this IP address has been marked as a malicious scanning source by multiple security vendors.

[0115] Based on the text content, please generate a task-related question and answer in the following format:

[0116] {

[0117] question:

[0118] Thought process:

[0119] answer:

[0120] }".

[0121] In step 205, the first input instruction is input into the trained generative model to generate question-and-answer data corresponding to the first input instruction.

[0122] The generated first input instruction is fed into the trained generative model. Based on its learned knowledge and patterns, the model understands and processes the first input instruction to generate corresponding question-and-answer data. During the generation process, the model considers the input security reference data and the target security task description, striving to generate question-and-answer data that meets the requirements as much as possible.

[0123] In step 206, the question-and-answer data is input into the trained security task model to fine-tune the trained security task model and obtain the fine-tuned security task model.

[0124] The trained security task model is a model that has already been trained to a certain extent for security tasks, but it still needs to be fine-tuned for specific security domains to improve its performance in those domains. Fine-tuning techniques are not limited to LoRA, QLoRA, P-Tuning v2, Prefix Tuning, Prompt Tuning, etc., and open-source models are not limited to LLaMA, Falcon, BLOOM, ChatGLM, Baichuan, InternLM, etc. The specific choice should be based on actual hardware resources and a comparison of fine-tuning effects to select the best option.

[0125] Specifically, the trained security task model is fine-tuned using question-and-answer data relevant to a specific domain, ultimately resulting in a fine-tuned security task model that performs better in that specific security domain.

[0126] In some implementations, the question-and-answer data includes questions, thought processes, and answers. The step of inputting the question-and-answer data into a trained security task model to fine-tune the trained security task model and obtain a fine-tuned security task model includes:

[0127] (1) Input the question into the trained security task model to obtain the first prediction probability of each word in the thinking process and the second prediction probability of each non-stop word in the answer;

[0128] (2) Determine the first loss value based on the first prediction probability of each word element in the thinking process and the second prediction probability of each non-stop word word element in the answer;

[0129] (3) Perform a first-stage fine-tuning on the trained security task model according to the first loss value to obtain the first-stage fine-tuned security task model.

[0130] (4) Input the question and the thought process into the security task model with adjustable parameters after the first stage of fine-tuning, and obtain the third prediction probability of each word in the answer and the first distribution probability of the answer from the security task model with adjustable parameters after the first stage of fine-tuning.

[0131] (5) Input the question and the thought process into the security task model with the frozen parameters after the first stage of fine-tuning to obtain the second distribution probability of the predicted answer;

[0132] (6) Determine the second loss value based on the third prediction probability of each word element in the answer, the first distribution probability of the answer, and the second distribution probability of the answer;

[0133] (7) Perform a second-stage fine-tuning on the adjustable parameter safety task model after the first-stage fine-tuning according to the second loss value to obtain the fine-tuned safety task model.

[0134] The question-and-answer data specifically includes: questions, thought processes, and answers. A question is a specific question posed in the question-and-answer data; a thought process is a description of the steps involved in analyzing, reasoning about, and thinking about the question; and an answer is the content of the response to the question. A token is the basic unit in text processing, typically a word or a meaningful segment of characters.

[0135] Current technical shortcomings in large-scale model fine-tuning include: (1) insufficient attention to key answer information when jointly training the thought process and the answer; (2) direct removal of the thought process for fine-tuning leads to knowledge forgetting; and (3) traditional single loss functions cannot balance the quality control of long text generation. Therefore, this application proposes to adopt a staged loss function framework when fine-tuning the model, dividing the training process into a reasoning enhancement stage (first-stage fine-tuning) and a refinement optimization stage (second-stage fine-tuning), and improving the model's reasoning ability and output simplicity through a multi-stage differentiated training strategy. For the reasoning enhancement stage, the main purpose is to improve the reasoning ability of the security task model; for the refinement optimization stage, the main purpose is to further improve the accuracy of the security task model for answering questions, on the premise that the reasoning ability of the security task model has been improved in the first-stage fine-tuning stage.

[0136] Specifically, for the first stage of fine-tuning, the input and output are defined as follows:

[0137] Input sequence: X = [CLS] Question: {question} [SEP]

[0138] Target output: Y = [STA] Thinking process: {reasoning} [SEP] Answer: {answer}

[0139] Wherein, [CLS] is the classification label used to aggregate feature information of the entire input sequence, [STA] is the sequence start label, [SEP] is the paragraph separator, and is the sequence terminator. Since the output of the first-stage fine-tuning includes both the thought process and the response, the loss function of the first-stage fine-tuning consists of sub-loss values ​​for both parts. For the sub-loss value of the thought process, it is determined by predicting the first prediction probability of each word in the thought process using the trained security task model. For the sub-loss value of the response, it is determined by predicting the second prediction probability of each non-stop word word in the thought process using the trained security task model. Finally, based on the first prediction probability of each word in the thought process and the second prediction probability of each non-stop word word in the response, the first loss value of the first-stage fine-tuning is determined. Therefore, the trained security task model is fine-tuned in the first stage based on the first loss value to obtain the security task model after the first stage fine-tuning.

[0140] For the second stage of fine-tuning, the input and output are defined as follows:

[0141] Input sequence: X' = [CLS] Question: {question} [SEP] Thought process: {reasoning} [SEP]

[0142] Target output: Y' = [STA] Answer: {answer}

[0143] The output of the second-stage fine-tuning only includes the answer. However, since the second-stage fine-tuning needs to prevent the model from forgetting previously learned knowledge during the answer optimization process, a knowledge distillation sub-loss value is also needed to balance answer generation and knowledge retention. Therefore, the loss function of the second-stage fine-tuning consists of sub-loss values ​​from both the answer and knowledge distillation. For the answer sub-loss value, the question and thought process are input into the adjustable-parameter safety task model after the first-stage fine-tuning, and the third prediction probability of each word in the answer is obtained from the adjustable-parameter safety task model after the first-stage fine-tuning. For the knowledge distillation sub-loss value, the question and thought process are input into both the adjustable-parameter safety task model after the first-stage fine-tuning and the frozen-parameter safety task model after the first-stage fine-tuning, respectively, and the first and second distribution probabilities of the predicted answer are obtained. The knowledge distillation sub-loss value is determined by using the first and second distribution probabilities. Finally, based on the third prediction probability of each word in the answer, the first distribution probability of the answer, and the second distribution probability of the answer, the second loss value is determined, and the second loss value for the second stage of fine-tuning is determined. Then, the trained security task model is fine-tuned in the second stage based on the second loss value to obtain the fine-tuned security task model.

[0144] Therefore, in the reasoning enhancement stage (first-stage fine-tuning), the input sequence is set to contain only questions, and the target output includes the thought process and the answer. This setting allows the model to focus more on reasoning from the questions and generating reasonable thought processes. This effectively improves the model's reasoning ability and avoids the problem of overemphasizing key answer information and neglecting reasoning when jointly training the thought process and the answer. Furthermore, since the loss function of the first-stage fine-tuning consists of sub-loss values ​​from both the thought process and the answer, determining the sub-loss values ​​based on the first predicted probability of each word in the thought process and the second predicted probability of each non-stop word in the answer provides a more comprehensive guide to model learning. This prompts the model to not only focus on the accuracy of the answer but also on the rationality of the thought process, further strengthening the training of reasoning ability.

[0145] Furthermore, in the refinement and optimization phase (second-stage fine-tuning), a safe task model with frozen parameters from the first-stage fine-tuning was introduced. The problem and thought process were input into this model and the adjustable-parameter safe task model. The knowledge distillation loss value was calculated to balance answer generation and knowledge retention. This ensures that the model does not forget the knowledge learned in the first stage while optimizing the answer. It guarantees that the generated answer meets the current task requirements while retaining previously learned knowledge, effectively avoiding the knowledge forgetting phenomenon caused by directly removing the thought process for fine-tuning.

[0146] Through two phases of fine-tuning working in synergy, the first phase lays the foundation for reasoning ability and knowledge for the second phase, while the second phase further optimizes the answer while retaining the existing knowledge. This multi-stage training approach enables the model to gradually learn and consolidate knowledge, improving its ability to memorize and apply knowledge.

[0147] In some implementations, determining the first loss value based on the first predicted probability of each word element during the thinking process and the second predicted probability of each non-stop word word element in the answer includes:

[0148] (1.1) Obtain the positional weight of each word element in the thinking process;

[0149] (1.2) Based on the position weight of each word in the thinking process and the corresponding first prediction probability, determine the first sub-loss value of the thinking process;

[0150] (1.3) Determine the answer sub-loss value based on the second predicted probability of each non-stop word lexical in the answer;

[0151] (1.4) The first sub-loss value and the response sub-loss value are weighted and summed to obtain the first loss value.

[0152] Positional weights represent the numerical value of the importance of each word in its position during the thinking process. Words in different positions have different impacts on the overall thinking logic and outcome, and positional weights are used to measure the degree of this impact. For example, words at the beginning of the thinking process may play an important role in guiding the direction, and their positional weights may be relatively high; while words in the middle or end will also have corresponding weights based on their role in the reasoning process.

[0153] Specifically, the determination of position weights can be referenced by the following formula: w t =1+γ·(t / Tr);

[0154] Where Tr represents the length of the token in the thinking process, and t represents the position index, used to traverse each token in the token sequence composed of multiple tokens in the thinking process. The position of the current token is calculated by t / Tr, which is a hyperparameter used to calculate the position weight. It can be set to 0.2. Through this formula, the position weight of each token in the thinking process can be calculated.

[0155] Specifically, the first sub-loss value of the thinking process The following loss function can be used as a reference for calculation:

[0156] in, This represents the actual word (or target word) associated with the predicted probability of the word at position t during the thought process. For example, when generating a sequence of thought processes, the model has a predicted word distribution for each position t. This refers to the word that should actually appear at this position, used to calculate the difference between the model's prediction and the actual situation. ＜t This represents the sequence of all words preceding position t during the thought process. In other words, it contains all word information from the beginning of the sequence to position t-1. When predicting words at position t, the model refers to previously generated word information, i.e., y. ＜t To calculate the predicted probability of the word at the current position. Here, X is the input sequence, i.e., the question. In this way, the model can utilize contextual information to make more accurate predictions.

[0157] Specifically, the answer sub-loss value The following loss function can be used as a reference for calculation:

[0158] Where T is the total number of all word units that make up the thought process and the answer. For lexical units that are not stop words, I() is an indicator function. hour, When y t When ∈S, This represents the true value corresponding to the word at position t in the answer portion. In calculating the answer loss function... When y represents the correct word that the model should predict at position t in the answer part, it is used to measure the accuracy of the model's prediction for each word in the answer. ＜t This represents the sequence of all words preceding position t when generating the answer. This is related to the thought process y. ＜t Similarly, but this applies to the answer portion. When predicting the word at position t in the answer, the model uses the previously generated answer word information y. ＜t and input sequence X to calculate The predicted probability, i.e. Utilize contextual information to improve the accuracy of answer generation.

[0159] The first sub-loss value of the thought process is obtained. And the answer sub-loss value Then, the first loss value of the first-stage fine-tuning is calculated using the following formula.

[0160] Where α is the first sub-loss value The weights, where 1-α is the loss value of the respondent. The weights are used to calculate the first loss value through a weighted summation.

[0161] Therefore, by assigning positional weights to words in the thinking process, the importance of words at different positions can be highlighted. This avoids the model treating all words equally when processing the thinking process, instead focusing on learning based on position and importance, thus improving the quality of the thinking process. When calculating the answer sub-loss value, a stop word masking mechanism is introduced (through an indicator function). This method calculates the loss only for the predicted probabilities of non-stop words (entity words). This avoids the model focusing excessively on stop words during training, instead concentrating on entity words with actual semantic meaning, thus increasing the focus on entity words and contributing to more meaningful and accurate answers. Similarly, it uses the lexical information y of the answer before position t. ＜t The predicted probability of the answer word is calculated using the input sequence X. By using contextual information to guide answer generation, the model can generate more logical and semantically sound answers based on preceding information, thereby improving the accuracy and relevance of the answers.

[0162] In some implementations, determining the second loss value based on the third predicted probability of each word in the answer, the first distribution probability of the answer, and the second distribution probability of the answer includes:

[0163] (1.1) Determine the second sub-loss value based on the third prediction probability of each word element in the response;

[0164] (1.2) Determine the KL divergence between the first probability distribution of the answer and the second probability distribution of the answer to obtain the knowledge distillation loss value;

[0165] (1.3) The second sub-loss value and the knowledge distillation sub-loss value are weighted and summed to obtain the second loss value.

[0166] The second sub-loss value is a numerical measure of the difference between the accuracy of the safety task model with adjustable parameters after the first stage in predicting word terms in the answer and the actual situation, and is part of the calculation of the second loss value.

[0167] Specifically, the second sub-loss value It can be calculated using the following loss function:

[0168] Where X′ is the input sequence for the second-stage fine-tuning, i.e., the problem and the thought process, and T a To answer the question about the total number of included lexical units, It is a conditional probability, representing the probability that, given an input sequence X′, the word at position t in the predicted answer is... The probability of. This represents the actual word at position t in the answer.

[0169] To avoid knowledge forgetting caused by directly removing the thought process for fine-tuning, the knowledge distillation loss value needs to be determined by the KL divergence between the first and second distribution probabilities of the answers. Knowledge distillation loss value It can be calculated using the following formula:

[0170] Among them, D θ (Y|X) represents the first probability distribution of the response predicted by the adjustable parameter safety task model after the first stage of fine-tuning, where θ is the adjustable parameter, and D... θ′ (Y|X) represents the second distribution probability of the answer predicted by the safety task model after the first-stage fine-tuning of the frozen parameters, where θ′ is the frozen parameter. Through D... KL ( ) Determine the KL divergence between the first probability distribution and the second probability distribution of the answer to obtain the knowledge distillate loss value to prevent knowledge forgetting.

[0171] Among them, the second loss value The following formula can be used to calculate it:

[0172] Where λ1 is the second sub-loss value The weight values, λ2 is the knowledge distillation loss value. The weight value is determined by the second sub-loss value. and knowledge distillation loss value We perform a weighted summation to obtain the final second loss value.

[0173] Therefore, the second sub-loss value measures the difference between the accuracy of the adjustable-parameter safety task model's prediction of word units in the answer and the actual situation after the first-stage fine-tuning. By evaluating the prediction probability of each word unit, the model can more accurately learn the correct expression of each word unit in the answer. The knowledge distillation sub-loss value is determined by calculating the KL divergence between the first and second probability distributions. The first probability distribution comes from the adjustable-parameter model, and the second probability distribution comes from the frozen-parameter model. In this way, the adjustable-parameter model can refer to the knowledge of the frozen-parameter model during the answer optimization process, avoiding the knowledge forgetting phenomenon caused by directly removing the thinking process for fine-tuning. The KL divergence measures the difference between the two probability distributions, and is calculated by minimizing... The adjustable-parameter model can approximate the predicted distribution of the frozen-parameter model as closely as possible, thus preserving the knowledge learned in the first stage. This ensures that when the model is fine-tuned in the second stage, it will not lose previously accumulated useful information due to excessive focus on optimizing the current answer, guaranteeing the model's stability and generalization ability.

[0174] In some implementations, after obtaining the fine-tuned security task model, its performance needs to be evaluated and tested. To ensure the fairness of the evaluation, a third-party evaluation set can be selected. Different quantitative calculation methods are used for different types of tasks, while fully considering the influence of reference examples. For evaluating the performance of the fine-tuned security task model under both reference example-less and reference example-within-reference-example modes, in the reference example-with-examples evaluation mode, reference examples are sampled from the dataset of the same task in the evaluation set. When assembling evaluation input prompts, reference examples are input as part of the model being evaluated; for classification tasks, accuracy needs to be calculated after rule matching based on the actual situation. For generation tasks, the similarity between the output of the fine-tuned model and the standard output is calculated; higher similarity indicates better generation performance. A pre-trained Transformer model can be used to vectorize the two input texts to be compared. The BERT model can map natural language text to a high-dimensional vector space, allowing the semantic information of the text to be fully expressed. In this embodiment, the SecBERT model, fine-tuned in the cybersecurity domain, can be used. After obtaining the vector representations of the two text segments, the cosine similarity between these two vectors is calculated. Specifically, let A be the vector representation of the output of the fine-tuned model, and B be the vector representation of the standard output. The formula is expressed as:

[0175] sim(A,B)=cos(θ)=(A·B) / (||A||·||B||);

[0176] Where A·B represents the dot product of two vectors, and ||A|| and ||B|| represent the Euclidean norms of the two vectors, respectively. Higher similarity indicates better output from the fine-tuned model. In practical implementation, the following two points should be noted:

[0177] (1) Vector extraction strategy: The output vector corresponding to the [CLS] tag can be used as the whole sentence representation or the average value of all token output vectors can be used. It is recommended to use the output features of the last layer or the last few layers.

[0178] (2) Calculation optimization: For long texts, it is recommended to segment them and then take the average value; it is recommended to normalize the vectors.

[0179] As described above, this embodiment generates multiple first security task descriptions, clusters these descriptions to obtain multiple security task description clusters; acquires multiple security reference data, determines the similarity between each security reference data and any first security task description in each security task description cluster; determines the target security task description corresponding to each security reference data based on the similarity between each security reference data and the corresponding first security task description; generates a first input instruction based on each security reference data and the corresponding target security task description, wherein the first input instruction is a prompt to generate question-and-answer data based on the security reference data and the corresponding target security task description; and inputs the first input instruction. The trained generative model generates question-and-answer data corresponding to the first input instruction. This question-and-answer data is then input into the trained security task model to fine-tune the model, resulting in a fine-tuned security task model. Compared to related technologies where the lack of high-quality domain data leads to poor accuracy in answers specific to a particular domain, this embodiment of the application obtains a domain-specific security task description cluster by clustering multiple first security task descriptions and combining them with reference data to generate the first input instruction. The generative model then generates question-and-answer data for training the model in the specific domain. This question-and-answer data is used to train the security task model, which requires fine-tuning after training, thereby improving the accuracy of the model's answers in the specific domain.

[0180] For details on the implementation of each of the above steps, please refer to the previous examples, which will not be repeated here.

[0181] To facilitate better implementation of the model training method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described model training method. The meanings of the terms used are the same as in the above-described model training method, and specific implementation details can be found in the descriptions in the method embodiments.

[0182] Please refer to Figure 3, which is a schematic diagram of the structure of the model training device provided in an embodiment of this application. The model training device is applied to a computer device. The model training device may include a clustering division unit 601, a first determination unit 602, a second determination unit 603, a generation unit 604, an input unit 605, and a fine-tuning unit 606, etc.

[0183] Clustering partitioning unit 601 is used to generate multiple first security task descriptions, and to perform clustering partitioning on the multiple first security task descriptions to obtain multiple security task description clusters;

[0184] The first determining unit 602 is used to acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster.

[0185] The second determining unit 603 is used to determine the target security task description corresponding to each security reference data based on the similarity between each security reference data and the corresponding first security task description;

[0186] The generation unit 604 is used to generate a first input instruction based on each of the security reference data and the corresponding target security task description. The first input instruction is an input prompt for generating question-and-answer data according to the security reference data and the corresponding target security task description.

[0187] The input unit 605 is used to input the first input instruction into the trained generative model to generate question-and-answer data corresponding to the first input instruction.

[0188] The fine-tuning unit 606 is used to input the question-and-answer data into the trained security task model to fine-tune the trained security task model and obtain the fine-tuned security task model.

[0189] In some embodiments, the clustering partitioning unit 601 includes:

[0190] The transformation subunit is used to perform vector transformation on each of the first security task descriptions to obtain a vector representation of each of the first security task descriptions.

[0191] The first selection subunit is used to select a first preset number of candidate security task descriptions from each of the first security task descriptions as the initial clustering centers of different initial security task description clusters.

[0192] The first computational subunit is used to calculate the distance between the vector representation of each of the other security task descriptions and the vector representation of each of the initial cluster centers;

[0193] The sub-unit is used to divide each of the other security task descriptions into the initial security task description cluster corresponding to the nearest initial cluster center, thereby obtaining multiple updated security task description clusters.

[0194] The first determining subunit is used to determine the mean of the vector representations of other security task descriptions and the vector representations of the corresponding initial cluster centers in each of the updated security task description clusters, so as to obtain the updated cluster centers of each of the updated security task description clusters.

[0195] The first execution subunit is configured to, when the clustering termination condition is not met, determine the updated cluster center as the initial cluster center, determine the updated security task description cluster as the initial security task description cluster, and return to the step of calculating the distance between the vector representation of other security task descriptions and the vector representation of each initial cluster center, until the clustering termination condition is met, and determine the resulting multiple updated security task description clusters as security task description clusters.

[0196] In some embodiments, the clustering partitioning unit 601 further includes:

[0197] The second calculation subunit is used to calculate the difference between the vector representation of the updated cluster center of each of the updated security task description clusters and the vector representation of the corresponding initial cluster center.

[0198] The comparison subunit is used to compare the difference value of each of the updated security task description clusters with a preset difference value;

[0199] The second determining subunit is used to determine that the clustering division termination condition is not met when there is an updated security task description cluster with a corresponding difference value greater than or equal to the preset difference value.

[0200] The third determining subunit is used to determine that the clustering termination condition is met when the difference value of each of the updated security task description clusters is less than the preset difference value.

[0201] In some embodiments, the clustering partitioning unit 601 further includes:

[0202] The second selection subunit is used to select a second preset number of security task descriptions from the first security task description set and the constructed second security task description set;

[0203] A generation subunit is used to generate a second input instruction based on the selected security task description. The second input instruction is used to command the generation of a third preset number of security task descriptions according to the selected security task description.

[0204] The first input subunit is used to input the second input instruction into the trained generative model to generate the corresponding generative safety task description;

[0205] The fourth determining subunit is used to determine the generated security task description that meets the security task description conditions as the first security task description and allocate it to the first security task description set;

[0206] The second execution subunit is used to return to the step of selecting a second preset number of security task descriptions from the constructed second security task description set and the first security task description set when the total number of first security task descriptions in the first security task description set does not reach a fourth preset number, until the total number of first security task descriptions in the first security task description set reaches a fourth preset number, thereby obtaining multiple first security task descriptions in the first security task description set.

[0207] In some embodiments, the fourth determining subunit is configured to:

[0208] A third input instruction is generated based on each of the generated security task descriptions, and the third input instruction is used to command the response to whether the corresponding generated security task description is related to the security task.

[0209] Each of the third input instructions is fed into the trained generative model to generate an answer result;

[0210] The security task description corresponding to the third input instruction with a yes answer is determined as the first security task description.

[0211] In some embodiments, the question-and-answer data includes questions, thought processes, and answers, and the fine-tuning unit 606 includes:

[0212] The second input subunit is used to input the question into the trained security task model to obtain the first prediction probability of each word in the thinking process and the second prediction probability of each non-stop word in the answer.

[0213] The fifth determining subunit is used to determine a first loss value based on the first predicted probability of each word element in the thinking process and the second predicted probability of each non-stop word word element in the answer;

[0214] The first fine-tuning subunit is used to perform a first-stage fine-tuning on the trained security task model according to the first loss value, so as to obtain the security task model after the first-stage fine-tuning.

[0215] The third input subunit is used to input the question and the thought process into the adjustable parameter security task model after the first stage of fine-tuning, so as to obtain the third prediction probability of each word in the answer and the first distribution probability of the answer predicted by the adjustable parameter security task model after the first stage of fine-tuning.

[0216] The fourth input subunit is used to input the question and the thought process into the security task model with the frozen parameters after the first stage of fine-tuning, so as to obtain the second distribution probability of the predicted answer;

[0217] The sixth determining subunit is used to determine the second loss value based on the third prediction probability of each word in the answer, the first distribution probability of the answer, and the second distribution probability of the answer;

[0218] The second fine-tuning subunit is used to perform a second-stage fine-tuning of the adjustable parameters of the safety task model after the first-stage fine-tuning according to the second loss value, so as to obtain the fine-tuned safety task model.

[0219] In some embodiments, the fifth determining subunit is configured to:

[0220] Obtain the positional weight of each word element during the thought process;

[0221] Based on the positional weight of each word in the thinking process and the corresponding first prediction probability, the first sub-loss value of the thinking process is determined.

[0222] The answer sub-loss value is determined based on the second predicted probability of each non-stop word lexical unit in the answer;

[0223] The first sub-loss value and the response sub-loss value are weighted and summed to obtain the first loss value.

[0224] In some embodiments, the sixth determining subunit is configured to:

[0225] The second sub-loss value is determined based on the third prediction probability of each word element in the response;

[0226] Determine the KL divergence between the first probability distribution of the answer and the second probability distribution of the answer to obtain the knowledge distillation loss value;

[0227] The second sub-loss value and the knowledge distillation sub-loss value are weighted and summed to obtain the second loss value.

[0228] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0229] As described above, in this embodiment, a clustering unit 601 generates multiple first security task descriptions, and clusters these descriptions to obtain multiple security task description clusters. A first determining unit 602 acquires multiple security reference data and determines the similarity between each security reference data and any first security task description in each security task description cluster. A second determining unit 603 determines the target security task description corresponding to each security reference data based on the similarity between each security reference data and the corresponding first security task description. A generation unit 604 generates a first input instruction based on each security reference data and the corresponding target security task description. The first input instruction is a command that generates input prompts for question-and-answer data based on the security reference data and the corresponding target security task description. An input unit 605 inputs the first input instruction into the trained generation model to generate question-and-answer data corresponding to the first input instruction. A fine-tuning unit 606 inputs the question-and-answer data into the trained security task model to fine-tune the trained security task model, resulting in a fine-tuned security task model. Compared to related technologies where the lack of high-quality domain data leads to poor accuracy in answers for specific domains, this application's embodiments describe multiple first security tasks and cluster them to obtain a security task description cluster for a specific domain. A first input instruction is generated by combining this cluster with reference data. Question-and-answer data for training the model for a specific domain is then obtained through the generation of the model. This question-and-answer data is used to train the security task model, which requires fine-tuning after training, thereby improving the accuracy of the model's answers for specific domains.

[0230] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0231] Referring to Figure 4, which is a partial structural block diagram of a computer device 1000 implementing an embodiment of this application, the computer device 1000 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 622 (e.g., one or more processors) and a memory 632, and one or more storage media 630 (e.g., one or more mass storage devices) storing application programs 642 or data 644. The memory 632 and storage media 630 may be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the server 600. Furthermore, the CPU 622 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the server 600.

[0232] Computer device 1000 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input / output interfaces 658, and / or one or more operating systems 641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0233] The central processing unit 622 in the computer device 1000 can be used to execute the model training method of the embodiments of this application, for example:

[0234] Multiple first security task descriptions are generated, and the multiple first security task descriptions are clustered to obtain multiple security task description clusters;

[0235] Acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster;

[0236] Based on the similarity between each security reference data and the corresponding first security task description, the target security task description corresponding to each security reference data is determined;

[0237] Based on each of the security reference data and the corresponding target security task description, a first input instruction is generated. The first input instruction is a command to generate input prompts for question-and-answer data according to the security reference data and the corresponding target security task description.

[0238] The first input instruction is input into the trained generative model to generate question-and-answer data corresponding to the first input instruction.

[0239] The question-and-answer data is input into the trained security task model to fine-tune the trained security task model, resulting in a fine-tuned security task model.

[0240] This application also provides a computer-readable storage medium for storing program code for executing the model training methods of the foregoing embodiments.

[0241] This application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the model training method described above. For example:

[0242] Multiple first security task descriptions are generated, and the multiple first security task descriptions are clustered to obtain multiple security task description clusters;

[0243] Acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster;

[0244] Based on the similarity between each security reference data and the corresponding first security task description, the target security task description corresponding to each security reference data is determined;

[0245] Based on each of the security reference data and the corresponding target security task description, a first input instruction is generated. The first input instruction is a command to generate input prompts for question-and-answer data according to the security reference data and the corresponding target security task description.

[0246] The first input instruction is input into the trained generative model to generate question-and-answer data corresponding to the first input instruction.

[0247] The question-and-answer data is input into the trained security task model to fine-tune the trained security task model, resulting in a fine-tuned security task model.

[0248] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0249] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0250] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0251] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0252] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0253] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0254] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0255] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0256] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0257] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A model training method, wherein, The model training method includes: Multiple first security task descriptions are generated, and the multiple first security task descriptions are clustered to obtain multiple security task description clusters; Acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster; Based on the similarity between each security reference data and the corresponding first security task description, the target security task description corresponding to each security reference data is determined; Based on each of the security reference data and the corresponding target security task description, a first input instruction is generated. The first input instruction is a command to generate input prompts for question-and-answer data according to the security reference data and the corresponding target security task description. The first input instruction is input into the trained generative model to generate question-and-answer data corresponding to the first input instruction. The question-and-answer data is input into the trained security task model to fine-tune the trained security task model, resulting in a fine-tuned security task model. 2.The model training method of claim 1, wherein, The clustering of multiple first security task descriptions yields multiple security task description clusters, including: Perform vector transformation on each of the first security task descriptions to obtain a vector representation of each of the first security task descriptions; A first preset number of candidate security task descriptions are selected from each of the first security task descriptions as the initial cluster centers of different initial security task description clusters; Calculate the distance between the vector representation of each of the other security task descriptions and the vector representation of each of the initial cluster centers; Each of the other security task descriptions is assigned to the initial security task description cluster corresponding to the nearest initial cluster center, resulting in multiple updated security task description clusters; The mean of the vector representations of other security task descriptions in each updated security task description cluster and the vector representations of the corresponding initial cluster centers is determined to obtain the updated cluster centers of each updated security task description cluster. When the clustering termination condition is not met, the updated cluster center is determined as the initial cluster center, the updated security task description cluster is determined as the initial security task description cluster, and the step of calculating the distance between the vector representation of other security task descriptions and the vector representation of each initial cluster center is returned until the clustering termination condition is met, and the resulting multiple updated security task description clusters are determined as security task description clusters. 3.The model training method of claim 2, wherein, After obtaining the updated cluster center for each of the updated security task description clusters, the method further includes: Calculate the difference between the vector representation of the updated cluster center of each of the updated security task description clusters and the vector representation of the corresponding initial cluster center; Compare the difference value of each updated security task description cluster with the preset difference value; When there is an updated security task description cluster with a corresponding difference value greater than or equal to the preset difference value, it is determined that the clustering termination condition is not met. When the difference value of each of the updated security task description clusters is less than the preset difference value, the clustering termination condition is determined to be met. 4.The model training method of claim 1, wherein, The generation of multiple first security task descriptions includes: Select a second preset number of security task descriptions from the first set of security task descriptions and the constructed second set of security task descriptions; A second input instruction is generated based on the selected security task description. The second input instruction is used to command the generation of a third preset number of security task descriptions based on the selected security task description. The second input instruction is input into the trained generative model to generate a corresponding generative safety task description; The generated security task descriptions that meet the security task description conditions are identified as the first security task descriptions and assigned to the first security task description set; When the total number of first security task descriptions in the first security task description set does not reach the fourth preset number, the process returns to the step of selecting a second preset number of security task descriptions from the constructed second security task description set and the first security task description set, until the total number of first security task descriptions in the first security task description set reaches the fourth preset number, thus obtaining multiple first security task descriptions in the first security task description set. 5.The model training method of claim 4, wherein, The step of determining the generated security task description that meets the security task description conditions as the first security task description includes: A third input instruction is generated based on each of the generated security task descriptions, and the third input instruction is used to command the response to whether the corresponding generated security task description is related to the security task. Each of the third input instructions is fed into the trained generative model to generate an answer result; The security task description corresponding to the third input instruction with a yes answer is determined as the first security task description. 6.The method of Claim 1, wherein, The question-and-answer data includes questions, thought processes, and answers. The step of inputting the question-and-answer data into the trained security task model to fine-tune the model and obtain a fine-tuned security task model includes: The question is input into the trained security task model to obtain the first predicted probability of each word in the thinking process and the second predicted probability of each non-stop word in the answer. Based on the first predicted probability of each word element in the thinking process and the second predicted probability of each non-stop word word element in the answer, a first loss value is determined; The trained security task model is fine-tuned in the first stage according to the first loss value to obtain the security task model after the first stage fine-tuning. The question and the thought process are input into the security task model with adjustable parameters after the first stage of fine-tuning. The security task model with adjustable parameters after the first stage of fine-tuning predicts the third prediction probability of each word in the answer and the first distribution probability of the answer. The question and the thought process are input into the security task model with frozen parameters after the first stage of fine-tuning, and the second distribution probability of the predicted answer is obtained. The second loss value is determined based on the third predicted probability of each word in the answer, the first distribution probability of the answer, and the second distribution probability of the answer; The adjusted parameter safety task model after the first stage of fine-tuning is fine-tuned according to the second loss value to obtain the fine-tuned safety task model. 7.The model training method of claim 6, wherein, The determination of the first loss value based on the first predicted probability of each word element in the thinking process and the second predicted probability of each non-stop word word element in the answer includes: Obtain the positional weight of each word element during the thought process; Based on the positional weight of each word in the thinking process and the corresponding first prediction probability, the first sub-loss value of the thinking process is determined. The answer sub-loss value is determined based on the second predicted probability of each non-stop word lexical unit in the answer; The first loss value is obtained by weighted summing of the first sub-loss value and the response sub-loss value. 8.The method of Claim 6, wherein, The step of determining the second loss value based on the third predicted probability of each word in the answer, the first distribution probability of the answer, and the second distribution probability of the answer includes: The second sub-loss value is determined based on the third prediction probability of each word element in the response; Determine the KL divergence between the first probability distribution of the answer and the second probability distribution of the answer to obtain the knowledge distillation loss value; The second sub-loss value and the knowledge distillation sub-loss value are weighted and summed to obtain the second loss value.

9. A model training apparatus, wherein, The model training device includes: A clustering unit is used to generate multiple first security task descriptions, and to perform clustering on the multiple first security task descriptions to obtain multiple security task description clusters. The first determining unit is configured to acquire multiple security reference data and determine the similarity between each security reference data and any first security task description in each security task description cluster. The second determining unit is used to determine the target security task description corresponding to each security reference data based on the similarity between each security reference data and the corresponding first security task description; The generation unit is configured to generate a first input instruction based on each security reference data and the corresponding target security task description. The first input instruction is an input prompt for generating question-and-answer data according to the security reference data and the corresponding target security task description. The input unit is used to input the first input instruction into the trained generative model to generate question-and-answer data corresponding to the first input instruction. The fine-tuning unit is used to input the question-and-answer data into the trained security task model to fine-tune the trained security task model and obtain the fine-tuned security task model.

10. A computer readable storage medium, wherein, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the model training method according to any one of claims 1 to 8.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the model training method according to any one of claims 1 to 8.

Images