Model training method, electronic equipment, medium and product

Abstract

The invention provides a model training method, electronic equipment, a medium and a product. The method comprises the following steps: acquiring sample data; the sample data comprises a feature matrix of sample vulnerabilities; training a long-short-term memory network layer of a network model based on the feature matrix to obtain a first training result; training a position coding layer of the network model based on the feature matrix to obtain a second training result; determining a loss function value of the network model based on the first training result and the second training result; adjusting parameters of the network model based on the loss function value to obtain a trained network model; through the newly applied and trained network model, the vulnerability can be accurately identified.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model training method, electronic device, medium, and product. Background Technology

[0002] With the continuous upgrading of technical defects, configuration errors, and attack methods, new vulnerabilities in the Linux operating system are constantly emerging. Therefore, quickly discovering vulnerabilities has become a core task of system security management.

[0003] In related technologies, vulnerability identification mainly relies on risk scanning, which cannot accurately identify vulnerabilities. Summary of the Invention

[0004] This application provides a model training method, electronic device, medium, and product that can accurately identify vulnerabilities.

[0005] The technical solution of this application embodiment is implemented as follows: This application provides a model training method, the method comprising: Acquire sample data; the sample data includes the feature matrix of sample vulnerabilities; The long short-term memory network layer of the network model is trained based on the feature matrix to obtain the first training result; The location encoding layer of the network model is trained based on the feature matrix to obtain a second training result; The loss function value of the network model is determined based on the first training result and the second training result; The parameters of the network model are adjusted based on the loss function value to obtain the trained network model.

[0006] In the above scheme, obtaining sample data includes: Obtain vulnerability information; the vulnerability information includes textual information about the sample vulnerability; The text information is vectorized to obtain multiple word vectors corresponding to the sample vulnerability; The feature matrix is ​​obtained based on multiple word vectors.

[0007] In the above scheme, training the long short-term memory network layer of the network model based on the feature matrix to obtain the first training result includes: The feature matrix is ​​input into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of the sample vulnerability; The long short-term memory network layer is trained based on the embedded vector sequence to obtain temporal features at different times in the embedded vector sequence; the first training result includes the temporal features.

[0008] In the above scheme, training the position encoding layer of the network model based on the feature matrix to obtain the second training result includes: The feature matrix is ​​input into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of the sample vulnerability; The position encoding layer is trained based on the embedding vector sequence to obtain position features at different positions in the embedding vector sequence; the second training result includes the position features.

[0009] In the above scheme, determining the loss function value of the network model based on the first training result and the second training result includes: The first training result and the second training result are input into the fusion layer of the network model to obtain the fused training result; The fused training results are input into the multi-head attention mechanism layer of the network model for weighted fusion to obtain a weighted result; the weighted result is used to indicate the correlation between the first training result and the second training result. The loss function value is determined based on the weighted result.

[0010] In the above scheme, determining the loss function value based on the weighted result includes: Based on the weighted result, the feedforward network layer of the network model is trained to obtain a third training result; Based on the third training result, predicted values ​​for multiple sample vulnerabilities are determined; the predicted values ​​are used to predict the classification results of the multiple sample vulnerabilities. The loss function value is determined based on the error between the predicted value and the target value.

[0011] The method in the above scheme further includes: Obtain monitoring text information from the vulnerability monitoring platform; The monitoring text information is input into the trained network model for identification to determine whether the monitoring text information includes vulnerability information; the vulnerability information includes vulnerability type. If so, generate vulnerability warning information based on the vulnerability type.

[0012] This application provides a model training apparatus, the apparatus comprising: An acquisition unit is used to acquire sample data; the sample data includes a feature matrix of sample vulnerabilities. The processing unit is used to train the long short-term memory network layer of the network model based on the feature matrix to obtain the first training result; The processing unit is used to train the position encoding layer of the network model based on the feature matrix to obtain a second training result; The processing unit is configured to determine the loss function value of the network model based on the first training result and the second training result; The processing unit is used to adjust the parameters of the network model based on the loss function value to obtain the trained network model.

[0013] This application provides an electronic device, including: a processor and a memory for storing computer programs capable of running on the processor. When the processor runs the computer program, it executes the model training method provided in the embodiments of this application.

[0014] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the model training method provided in this application when executed by a processor.

[0015] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the model training method provided in this application.

[0016] The embodiments of this application have the following beneficial effects: applying the trained network model to real-time vulnerability identification can shorten the time from vulnerability discovery to response, help to take timely protective measures, and reduce security risks. Attached Figure Description

[0017] Figure 1 A schematic flowchart illustrating a model training method provided in an embodiment of this application; Figure 2 This application provides a schematic flowchart of a Word2Vec algorithm processing method according to an embodiment of the present application. Figure 3 A schematic diagram of a network model provided in an embodiment of this application; Figure 4 A flowchart illustrating a vulnerability warning process provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0020] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0021] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0022] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0023] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0024] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0025] In related technologies, the vulnerability identification and handling process involves multiple stages and multiple participants. These stages not only increase processing time but may also be affected by information delays, poor communication, and human factors, thereby affecting the accuracy and efficiency of the entire vulnerability handling process.

[0026] Figure 1 This is an optional flowchart illustrating the model training method provided in the embodiments of this application. The following will combine... Figure 1 The steps shown are explained as follows: Figure 1 As shown, the method includes the following steps: Step S101: Obtain sample data.

[0027] The sample data includes the feature matrix of the sample vulnerabilities.

[0028] Understandably, sample data can be data collected and preprocessed from vulnerability alert platforms. For example, a vulnerability alert platform might collect text data about vulnerabilities that have appeared in Linux systems. (Reference) Figure 2 As shown, each word is vectorized using the Word2Vec model, and a feature matrix is ​​generated after unifying the sentence length. The feature matrix is ​​a two-dimensional array structure composed of multiple word vectors, used to represent the overall features of a text.

[0029] Step S102: Train the long short-term memory network layer of the network model based on the feature matrix to obtain the first training result.

[0030] In practical applications, network models can be constructed based on feature matrices. (Reference) Figure 3 As shown, the network model can include an embedding layer, where the feature matrix is ​​processed by the embedding layer to form an embedding vector sequence.

[0031] Long Short-Term Memory (LSTM) layers receive an embedded vector sequence as input, process the input sequence sequentially, and produce an output sequence. An LSTM layer can contain multiple LSTM units, each with its own weight parameters. The LSTM layers produce temporal features at different times within the embedded vector sequence; the first training result includes these temporal features.

[0032] Step S103: Train the position encoding layer of the network model based on the feature matrix to obtain the second training result.

[0033] In practical applications, refer to Figure 3 As shown, the positional encoding layer runs in parallel with the LSTM layer. It receives the embedded vector sequence as input, processes the input sequence sequentially, and enhances the network model's ability to understand and remember the features at different positions in the sequence. The positional encoding layer obtains the positional features at different positions in the embedded vector sequence. The second training result includes the positional features.

[0034] Step S104: Determine the loss function value of the network model based on the first training result and the second training result.

[0035] In practical applications, refer to Figure 3 As shown, the first training result from the LSTM layer and the second training result from the position encoding layer are fused through a fusion layer to obtain a fused training result. This fused training result is then sent to a multi-head attention layer. The multi-head attention layer uses self-attention to weight and fuse the inputs, producing a fused representation to capture the correlation between the first and second training results. After processing by the multi-head attention layer, its output is further fed into a feedforward layer for further processing. The feedforward layer consists of multiple fully connected layers and can map the input data into a higher-dimensional feature representation through a non-linear activation function.

[0036] The weighted fusion result can be used to train the Feed Forward layer to obtain a third training result. The third training result determines the predicted values ​​of multiple sample vulnerabilities through the output layer. The predicted values ​​are used to predict the classification results of multiple sample vulnerabilities, realizing the prediction of tasks such as classification and regression. Based on the error between the predicted value and the target value, the loss function value is determined.

[0037] Step S105: Adjust the parameters of the network model based on the loss function value to obtain the trained network model.

[0038] In practical applications, loss functions are used to measure the difference between the prediction results of a network model and the true labels. Cross-entropy loss function can be used to evaluate the performance of a network model on classification tasks, making the prediction results of the network model closer to the real situation.

[0039] To optimize the parameters of the network model, the Adaptive Moment Estimation Optimization Algorithm (Adam) can be used to update and adjust the parameters.

[0040] As can be seen from the above, the model training method provided in this application obtains the feature matrix of sample vulnerabilities, trains the long short-term memory network layer and the position encoding layer of the network model respectively, fuses the training results and calculates the loss function value, and adjusts the parameters of the network model to obtain the trained network model. This can effectively improve the network model's ability to identify sample vulnerability features and its classification accuracy, shorten the time from vulnerability discovery to response, help to take timely protective measures, and reduce security risks.

[0041] In some embodiments of this application, obtaining sample data includes: Obtain vulnerability information; vulnerability information includes textual information about sample vulnerabilities; The text information is vectorized to obtain multiple word vectors corresponding to the sample vulnerabilities; The feature matrix is ​​obtained based on multiple word vectors.

[0042] In practical applications, vulnerability information refers to descriptive content related to system security vulnerabilities, such as vulnerability ID, vulnerability name, vulnerability description, affected versions, and remediation suggestions. The textual information of sample vulnerabilities can come from official vulnerability databases or vulnerability notification platforms, representing the characteristics and background of a specific security vulnerability. By extracting the textual information of sample vulnerabilities, representative vulnerability features can be constructed, providing a basis for vulnerability classification and early warning.

[0043] Vectorization is a process of converting natural language text into numerical vectors. Its purpose is to map words in the text to a high-dimensional space, making semantically similar words appear closer together in that space. In this embodiment, the system uses the Word2Vec model to vectorize the vulnerability text information. The Word2Vec model learns the contextual relationships between words by training on a large-scale corpus, thereby generating a fixed-dimensional vector representation for each word.

[0044] A feature matrix is ​​a two-dimensional matrix formed by arranging and combining multiple word vectors according to certain rules. The feature matrix contains information about the entire text sequence. It not only reflects the vector representation of each word, but also preserves the word's position information in the original text, thus providing structured input data for subsequent sequence modeling.

[0045] The feature matrix assigns each row a time step and each column a dimension. For longer text information, truncation or padding can be used to standardize the length, ensuring all samples have the same input dimension. Converting vulnerability descriptions of varying lengths into a standardized feature matrix format facilitates batch training and prediction of the model.

[0046] In some embodiments of this application, the long short-term memory network layer of the network model is trained based on the feature matrix to obtain a first training result, including: The feature matrix is ​​input into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of sample vulnerabilities; The long short-term memory network layer is trained based on the embedded vector sequence to obtain the temporal features of different time sequences in the embedded vector sequence; the first training result includes the temporal features.

[0047] In practical applications, the embedding vector sequence of sample vulnerabilities refers to a sequence of vectors output after the feature matrix is ​​input into the embedding layer. The vectors in the embedding vector sequence of sample vulnerabilities not only retain the semantic information of the original words but also reflect the similarity and correlation between different words in the vector space. Through word embedding processing, network models can transform unstructured text data into structured vector data. This structured vector data enables subsequent deep learning models to effectively process and extract key information. (Reference) Figure 3 As shown, the output of the embedding layer can be expressed as Equation 1: (Formula 1) The LSTM layer receives the sequence of embedding vectors from the embedding layer as input, processes each vector step-by-step according to time steps, and records and updates the hidden state during processing to capture key features at different times in the sequence. (Reference) Figure 3 As shown, the output of the LSTM layer can be expressed as Equation 2: (Formula 2) By training with LSTM layers, temporal features related to vulnerability type, severity, and remediation difficulty can be extracted from vulnerability description text. For example, certain keywords such as remote code execution, privilege escalation, and unauthorized access may appear multiple times in a sequence and exhibit certain temporal distribution characteristics. These temporal distribution characteristics can help the model identify potential security risks.

[0048] As can be seen from the above, by training the embedding vector sequence of sample vulnerabilities through LSTM layers, the temporal dependencies in text data can be effectively modeled, and deep features that are helpful for vulnerability identification can be extracted, which can more accurately identify complex or implicit vulnerability information.

[0049] In some embodiments of this application, the location encoding layer of the network model is trained based on the feature matrix to obtain a second training result, including: The feature matrix is ​​input into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of sample vulnerabilities; The position encoding layer is trained based on the embedding vector sequence to obtain position features at different positions in the embedding vector sequence; the second training result includes position features.

[0050] In practical applications, positional features refer to the features extracted after encoding information at each position in the embedded vector sequence through the PositionalEncoding layer. Positional features reflect the location information of each word in the input sequence, helping the network model understand and remember the importance of features at different positions in the sequence. (Reference) Figure 3 As shown, the output of the PositionalEncoding layer can be expressed as Equation 3: (Formula 3) By introducing positional features, key positional information can be preserved when processing long text sequences, enabling more accurate modeling of text context and enhancing the network model's ability to identify and judge vulnerability-related information. When encoding positional features, one can choose to use sine / cosine functions or learnable parameters.

[0051] In some embodiments of this application, determining the loss function value of the network model based on the first training result and the second training result includes: The first and second training results are input into the fusion layer of the network model to obtain the fused training result; The fused training results are input into the multi-head attention mechanism layer of the network model for weighted fusion to obtain a weighted result; the weighted result is used to indicate the correlation between the first training result and the second training result. The loss function value is determined based on the weighted results.

[0052] In practical applications, refer to Figure 3 As shown, x is the output of the fusion layer, i.e. the fusion training result. The MuliheadAttention layer performs weighted fusion of the fusion training result to capture the correlation between the first training result and the second training result.

[0053] The fusion layer combines information from two or more vector spaces into a unified representation through combination methods (such as concatenation, weighted summation, etc.) to retain key features from each source and enhance overall expressive power. The first and second training results come from different processing paths. Through the processing of the fusion layer, these two types of information can be effectively integrated to generate richer feature representations, thereby improving the performance of the attention mechanism.

[0054] The weighted fusion result can be used to train the Feed Forward layer to obtain a third training result. The third training result determines the predicted values ​​of multiple sample vulnerabilities through the output layer. The predicted values ​​are used to predict the classification results of multiple sample vulnerabilities, realizing the prediction of tasks such as classification and regression. Based on the error between the predicted value and the target value, the loss function value is determined.

[0055] The calculation process can be as follows: Assuming the input sequence is X, the input is weighted and fused using a multi-head attention mechanism to produce a fused representation H.

[0056] The input sequence X is transformed linearly to obtain three matrices Q, K, and V, which are used to calculate the attention weights; Q = X * W_q; K = X * W_k; V = X * W_v.

[0057] Where W_q, W_k, and W_v are learnable weight matrices.

[0058] Attention score and attention weight calculation: The attention score matrix A is obtained by calculating the similarity between the query matrix Q and the key matrix K, as shown in Formula 4. This can be calculated using dot product attention, scaled dot product attention, or other methods. (Formula 4) d_k represents the dimension of each attention head.

[0059] Fusion of attention weights and values: By weighted summation of attention score matrix A and value matrix V, the weighted fused representation Z is obtained, Z=A*V.

[0060] Multi-head attention and linear transformation: The results of multiple independent attention heads are concatenated and mapped using a linear transformation to obtain the output result, as shown in Equation 5: (Formula 5) Where W_o is a learnable weight matrix.

[0061] The final expression for H is given by Formula 6: (Formula 6) Where LayerNorm represents the layer normalization operation, MHA_output is the representation after processing by the multi-head attention mechanism, and X is the input sequence.

[0062] Next, the output of the MuliheadAttention layer is fed into the FeedForward layer for processing. The FeedForward layer typically consists of two fully connected layers and an activation function, which can be represented by Equation 7: (Formula 7) In this model, FC1 and FC2 are fully connected layers, and activation is a non-linear activation function, such as ReLU. The final output of the model is FFN_output.

[0063] Finally, the output of the Feed Forward layer is normalized using the softmax activation function, the expression of which is shown in Equation 8: (Formula 8) Where i is an element in the vector, j is the index of all elements, and the softmax function ensures that all elements of the output vector are in the range [0, 1] and that the sum of all elements is equal to 1.

[0064] In some embodiments of this application, determining the loss function value based on the weighted result includes: The feedforward network layer of the network model is trained based on the weighted results to obtain the third training result; The predicted values ​​of multiple sample vulnerabilities are determined based on the third training results; the predicted values ​​are used to predict the classification results of multiple sample vulnerabilities. The loss function value is determined based on the error between the predicted value and the target value.

[0065] In practical applications, cross-entropy can be used as the loss function to calculate the error between the predicted result and the true label. Assuming the model's predicted result is y_pred and the true label is y_true (represented by one-hot encoding), the cross-entropy loss function is calculated as shown in Equation 9: (Formula 9) Here, log is the natural logarithm function. The goal of the cross-entropy loss function is to minimize the difference between the predicted result and the true label, making the model's predictions closer to reality.

[0066] To optimize the model parameters, a certain amount of vulnerability warning information and ordinary text information were used as a test set to evaluate the model. The adaptive moment estimation optimization algorithm (Adam) was used to update and adjust the parameters.

[0067] The update equations for the Adam algorithm are as follows: m = beta1 * m + (1 - beta1) * gradient v=beta2*v+(1-beta2)*gradient^2 theta=theta-learning_rate*m / (sqrt(v)+epsilon) Where m is the first moment estimate, v is the second moment estimate, beta1 and beta2 are the exponential decay rates of the mean and variance, learning_rate is the learning rate, and epsilon is a very small constant.

[0068] In some embodiments of this application, the method further includes: Obtain monitoring text information from the vulnerability monitoring platform; The monitoring text information is input into the trained network model for identification to determine whether the monitoring text information includes vulnerability information; the vulnerability information includes the vulnerability type. If so, generate vulnerability warning information based on the vulnerability type.

[0069] In practical applications, the monitoring address (URL) of the vulnerability monitoring platform is specified, the HTML content of the URL is requested and read, and the script is scheduled to run on a regular schedule using cron or service in the operating system. The monitoring frequency is set according to the requirements. Each time the script runs, the currently read vulnerability information is compared with the information saved last time. The newly added information is stored to obtain monitoring text information for subsequent analysis and processing. The trained network model identifies the vulnerability type and can generate vulnerability warning information based on the vulnerability type.

[0070] In a feasible scenario, refer to Figure 4 As shown, the network model in this application embodiment can be applied to vulnerability early warning, and the implementation method may include the following steps: 1. Collect and analyze vulnerability data, generate word vectors, and further encapsulate them into a feature matrix.

[0071] 2. Use the feature matrix to train the model and obtain the trained network model.

[0072] 3. Use automated scripts to periodically extract newly added text information and determine whether it includes vulnerability information based on the trained network model.

[0073] 4. Conduct a risk assessment based on the vulnerability information to confirm the potential threat level to system security; classify and prioritize the vulnerabilities based on the assessment results in order to implement further countermeasures and arrange remediation work.

[0074] 5. Develop response strategies: Based on the priority and impact of vulnerabilities, develop corresponding response strategies, and determine the time cycle for vulnerability remediation and updates based on risk level and priority.

[0075] 6. Vulnerability Notification and Warning: After determining the response strategy, issue vulnerability warning information to remind users of potential security threats.

[0076] 7. Continuous monitoring and updates: Use configured automated scripts to regularly monitor updates to domestic vulnerability warning platforms to obtain the latest vulnerability information, thus forming a closed loop.

[0077] Based on the same inventive concept as described above, Figure 5 This is a schematic diagram of a model training device provided in an embodiment of the present invention. The device includes: Acquisition unit 501 is used to acquire sample data; the sample data includes the feature matrix of sample vulnerabilities. The processing unit 502 is used to train the long short-term memory network layer of the network model based on the feature matrix to obtain the first training result; Processing unit 502 is used to train the position encoding layer of the network model based on the feature matrix to obtain the second training result.

[0078] Processing unit 502 is used to determine the loss function value of the network model based on the first training result and the second training result.

[0079] The processing unit 502 is used to adjust the parameters of the network model based on the loss function value to obtain the trained network model.

[0080] In some embodiments of this application, the acquisition unit 501 is used to acquire vulnerability information; the vulnerability information includes text information of sample vulnerabilities; The text information is vectorized to obtain multiple word vectors corresponding to the sample vulnerabilities; The feature matrix is ​​obtained based on multiple word vectors.

[0081] In some embodiments of this application, the processing unit 502 is used to input the feature matrix into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of sample vulnerabilities. The long short-term memory network layer is trained based on the embedded vector sequence to obtain the temporal features of different time sequences in the embedded vector sequence; the first training result includes the temporal features.

[0082] In some embodiments of this application, the processing unit 502 is used to input the feature matrix into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of sample vulnerabilities. The position encoding layer is trained based on the embedding vector sequence to obtain position features at different positions in the embedding vector sequence; the second training result includes position features.

[0083] In some embodiments of this application, the processing unit 502 is used to input the first training result and the second training result into the fusion layer of the network model to obtain the fused training result; The fused training results are input into the multi-head attention mechanism layer of the network model for weighted fusion to obtain a weighted result; the weighted result is used to indicate the correlation between the first training result and the second training result. The loss function value is determined based on the weighted results.

[0084] In some embodiments of this application, the processing unit 502 is used to train the feedforward network layer of the network model based on the weighted result to obtain a third training result; The predicted values ​​of multiple sample vulnerabilities are determined based on the third training results; the predicted values ​​are used to predict the classification results of multiple sample vulnerabilities. The loss function value is determined based on the error between the predicted value and the target value.

[0085] In some embodiments of this application, the processing unit 502 is used to obtain monitoring text information from the vulnerability monitoring platform; The monitoring text information is input into the trained network model for identification to determine whether the monitoring text information includes vulnerability information; the vulnerability information includes the vulnerability type. If so, generate vulnerability warning information based on the vulnerability type.

[0086] Based on the foregoing embodiments, embodiments of this application provide an electronic device. Figure 6 This is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention. The electronic device 600 includes at least one processor 601 and a memory 602. Optionally, the electronic device 600 may further include at least one communication interface 603. The various components in the electronic device 600 are coupled together through a bus system 604. It is understood that the bus system 604 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general designated all buses as Bus System 604.

[0087] Based on the hardware implementation of the above program modules, the communication interface 603 is able to interact with other communication devices. The processor 601 is connected to the communication interface 603 to enable information exchange with other communication devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program; The memory 602 contains the computer program.

[0088] Specifically, processor 601 is used to acquire sample data; the sample data includes a feature matrix of sample vulnerabilities. The first training result is obtained by training the long short-term memory network layer of the network model based on the feature matrix. The location encoding layer of the network model is trained based on the feature matrix to obtain the second training result; The loss function value of the network model is determined based on the first and second training results; The parameters of the network model are adjusted based on the loss function value to obtain the trained network model.

[0089] In some embodiments of this application, processor 601 is used to acquire vulnerability information; the vulnerability information includes textual information about sample vulnerabilities; The text information is vectorized to obtain multiple word vectors corresponding to the sample vulnerabilities; The feature matrix is ​​obtained based on multiple word vectors.

[0090] In some embodiments of this application, processor 601 is used to input the feature matrix into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of sample vulnerabilities. The long short-term memory network layer is trained based on the embedded vector sequence to obtain the temporal features of different time sequences in the embedded vector sequence; the first training result includes the temporal features.

[0091] In some embodiments of this application, processor 601 is used to input the feature matrix into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of sample vulnerabilities. The position encoding layer is trained based on the embedding vector sequence to obtain position features at different positions in the embedding vector sequence; the second training result includes position features.

[0092] In some embodiments of this application, processor 601 is used to input the first training result and the second training result into the fusion layer of the network model to obtain the fused training result; The fused training results are input into the multi-head attention mechanism layer of the network model for weighted fusion to obtain a weighted result; the weighted result is used to indicate the correlation between the first training result and the second training result. The loss function value is determined based on the weighted results.

[0093] In some embodiments of this application, processor 601 is used to train the feedforward network layer of the network model based on the weighted result to obtain a third training result; The predicted values ​​of multiple sample vulnerabilities are determined based on the third training results; the predicted values ​​are used to predict the classification results of multiple sample vulnerabilities. The loss function value is determined based on the error between the predicted value and the target value.

[0094] In some embodiments of this application, processor 601 is used to acquire monitoring text information from a vulnerability monitoring platform; The monitoring text information is input into the trained network model for identification to determine whether the monitoring text information includes vulnerability information; the vulnerability information includes the vulnerability type. If so, generate vulnerability warning information based on the vulnerability type.

[0095] It is understood that memory 602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 602 described in this embodiment of the invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0096] The memory 602 in this embodiment of the invention is used to store various types of data to support the operation of the electronic device 600. Examples of such data include any computer program for operation on the electronic device 600, and programs implementing the methods of this embodiment of the invention may be included in the memory 602.

[0097] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 601. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0098] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the methods described above.

[0099] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the model training method described above in this application.

[0100] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the model training method provided in this application. For example, ... Figure 1 The model training method is shown.

[0101] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0102] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0103] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0104] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0105] In summary, the network model trained in this application can be applied to real-time vulnerability identification, which can shorten the time from vulnerability discovery to response, help to take timely protective measures, and reduce security risks.

[0106] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A model training method, characterized in that, The method includes: Acquire sample data; the sample data includes the feature matrix of sample vulnerabilities; The long short-term memory network layer of the network model is trained based on the feature matrix to obtain the first training result; The location encoding layer of the network model is trained based on the feature matrix to obtain a second training result; The loss function value of the network model is determined based on the first training result and the second training result; The parameters of the network model are adjusted based on the loss function value to obtain the trained network model.

2. The method according to claim 1, characterized in that, The acquisition of sample data includes: Obtain vulnerability information; the vulnerability information includes textual information about the sample vulnerability; The text information is vectorized to obtain multiple word vectors corresponding to the sample vulnerability; The feature matrix is ​​obtained based on multiple word vectors.

3. The method according to claim 1, characterized in that, The step of training the long short-term memory network layer of the network model based on the feature matrix to obtain a first training result includes: The feature matrix is ​​input into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of the sample vulnerability; The long short-term memory network layer is trained based on the embedded vector sequence to obtain temporal features at different times in the embedded vector sequence; the first training result includes the temporal features.

4. The method according to claim 1, characterized in that, The second training result obtained by training the position encoding layer of the network model based on the feature matrix includes: The feature matrix is ​​input into the embedding layer of the network model for word embedding processing to obtain the embedding vector sequence of the sample vulnerability; The position encoding layer is trained based on the embedding vector sequence to obtain position features at different positions in the embedding vector sequence; the second training result includes the position features.

5. The method according to claim 1, characterized in that, Determining the loss function value of the network model based on the first training result and the second training result includes: The first training result and the second training result are input into the fusion layer of the network model to obtain the fused training result; The fused training results are input into the multi-head attention mechanism layer of the network model for weighted fusion to obtain a weighted result; the weighted result is used to indicate the correlation between the first training result and the second training result. The loss function value is determined based on the weighted result.

6. The method according to claim 5, characterized in that, Determining the loss function value based on the weighted result includes: Based on the weighted result, the feedforward network layer of the network model is trained to obtain a third training result; Based on the third training result, predicted values ​​for multiple sample vulnerabilities are determined; the predicted values ​​are used to predict the classification results of the multiple sample vulnerabilities. The loss function value is determined based on the error between the predicted value and the target value.

7. The method according to claim 1, characterized in that, The method further includes: Obtain monitoring text information from the vulnerability monitoring platform; The monitoring text information is input into the trained network model for identification to determine whether the monitoring text information includes vulnerability information; the vulnerability information includes vulnerability type. If so, generate vulnerability warning information based on the vulnerability type.

8. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images