A CPE identification system based on ChatGPT large model and incremental learning technology

Abstract

This invention discloses a CPE identification system based on the ChatGPT large-scale model and incremental learning technology, relating to the field of information technology security. It includes a software security vulnerability information database, a preprocessing layer, a semantic encoding layer, and a label prediction layer. The semantic encoding layer comprises a ChatGPT pre-trained large-scale model module and a gated recurrent unit network module. The ChatGPT pre-trained large-scale model module performs preliminary processing on the preprocessed data to obtain text feature data. The gated recurrent unit network module performs sequence analysis on the text feature data to obtain dynamic features. The label prediction layer includes a layer-by-layer improved label attention mechanism module and an anti-forgetting incremental learning module. The layer-by-layer improved label attention mechanism module refines the dynamic features. The anti-forgetting incremental learning module updates the CPE identification system in a timely manner. This invention effectively improves the accuracy and training efficiency of vulnerability information identification.

Description

Technical Field

[0001] This invention relates to the field of information technology security technology, and more specifically to a CPE identification system based on the ChatGPT large model and incremental learning technology. Background Technology

[0002] The software industry in the digital age is growing rapidly, but many related software products are based on open-source code libraries and components; therefore, many organizations are focusing on the security risks of open-source software. Among them, the National Vulnerability Database (NVD), supported by the globally recognized National Institute of Standards and Technology (NIST) in the United States, records a large number of new security vulnerabilities, and the number continues to grow. Because many organizations rely on the NVD to update its vulnerability database, rapid analysis and response to security vulnerabilities in open-source software is crucial for every organization.

[0003] Therefore, improving the model's ability to identify rare CPE entities and enhancing its generalization capabilities to more effectively address evolving security challenges is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a CPE identification system based on the ChatGPT large model and incremental learning technology, which can effectively improve the accuracy of vulnerability information identification and training efficiency.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A CPE identification system based on ChatGPT large model and incremental learning technology includes: a software security vulnerability information database, a preprocessing layer, a semantic encoding layer and a label prediction layer connected in sequence;

[0007] The preprocessing layer is used to preprocess the data in the software security vulnerability information database;

[0008] The semantic coding layer includes a ChatGPT pre-trained large model module and a gated recurrent unit network module. The ChatGPT pre-trained large model module is used to perform preliminary processing on the pre-processed data to obtain text feature data after language understanding. The gated recurrent unit network module is used to perform sequence analysis on the text feature data to obtain dynamic features.

[0009] The label prediction layer includes a progressively improved label attention mechanism module and an anti-forgetting incremental learning module; the progressively improved label attention mechanism module is used to refine the dynamic features; the anti-forgetting incremental learning module is used to update the CPE recognition system in a timely manner.

[0010] Preferably, the ChatGPT pre-trained large model module includes a Transformer model; the Transformer model adopts an encoder-decoder architecture, including an input embedding layer, a multi-head self-attention layer, a feedforward neural network layer, a residual connection layer, and layer normalization.

[0011] Preferably, the gated loop unit network module includes an update gate, a reset gate, candidate hidden states, and a final hidden state; the update gate is used to control the transmission of information flow and capture long-term dependencies; the reset gate is used to combine new input with past memories and forgotten irrelevant historical information; the candidate hidden state is used to provide candidate values ​​for updating the hidden state; and the final hidden state is used to combine current and past information to form the current state.

[0012] Preferably, the layer-by-layer improved label attention mechanism module includes a label attention layer, a channel attention layer, a self-attention layer, and a global feature aggregation module; the process of refining the dynamic features includes:

[0013] The label attention layer captures the relationship between labels and features, assigning different attention weights to different labels; the channel attention layer enhances the model's feature representation ability by learning the attention weights of each channel feature; the self-attention layer handles the irregularities of the text feature data and obtains the correlation between elements; the global feature aggregation module obtains an enhanced feature map by aggregating local features and global context information.

[0014] Preferably, the layer-by-layer improved label attention mechanism module converts the enhanced feature map into the final recognition result through an output layer to determine whether the software security vulnerability information database has security vulnerabilities and the severity of the vulnerabilities.

[0015] Preferably, the anti-forgetting incremental learning module includes a memory replay mechanism, a regularization strategy, feature sparsity, and contrastive learning;

[0016] The memory replay mechanism is used to store some old category samples and replay them during incremental training; the regularization strategy adds a penalty term to limit the excessive updates of the CPE recognition system; the feature sparsity and contrastive learning are used to reduce noise or irrelevant information in the feature data and to determine the category to which the feature data belongs.

[0017] Preferably, it also includes a method for a CPE identification system based on the ChatGPT large model and incremental learning technology, characterized in that it includes:

[0018] Data is obtained from a software security vulnerability information database; the obtained data is preprocessed through a preprocessing layer; the preprocessed data is then input into a semantic coding layer, which includes a ChatGPT pre-trained large model module and a gated recurrent unit network module. The ChatGPT pre-trained large model module is used to perform preliminary processing on the preprocessed data to obtain text feature data after language understanding. Then, the gated recurrent unit network module performs sequence analysis on the text feature data to obtain dynamic features.

[0019] The obtained dynamic features are then input into the label prediction layer, which includes a progressively improved label attention mechanism module and an anti-forgetting incremental learning module. The progressively improved label attention mechanism module is used to refine the dynamic features.

[0020] Finally, the CPE recognition system is updated in a timely manner through the anti-forgetting incremental learning module.

[0021] As can be seen from the above technical solution, compared with the prior art, this invention discloses a CPE identification system based on the ChatGPT large model and incremental learning technology. This system supports continuous model learning and retains and utilizes existing knowledge while learning new knowledge, eliminating the need to repeatedly process previous training data and saving significant computational resources and storage space. Simultaneously, existing knowledge is corrected and enhanced based on new vulnerability data to match it. This dynamic adjustment mechanism helps improve the accuracy and generalization ability of the CPE identification model. Furthermore, ChatGPT is pre-trained on a large amount of unlabeled data to learn the general features of textual language, and then fine-tuned for specific vulnerability information identification tasks to better adapt it to CPE tasks. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 A schematic diagram of the AI-enabled CPE identification system provided by the present invention;

[0024] Figure 2 This is a schematic diagram of the structure of the ChatGPT pre-trained large model module provided by the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of the gated loop unit network module provided by the present invention;

[0026] Figure 4 This is a flowchart illustrating the workflow of the anti-forgetting incremental learning module provided by the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This invention discloses a CPE identification system based on the ChatGPT large model and incremental learning technology.

[0029] like Figure 1 As shown, it includes: a software security vulnerability information database, a preprocessing layer, a semantic encoding layer, and a label prediction layer connected in sequence.

[0030] The preprocessing layer is used to preprocess the data in the software security vulnerability information database.

[0031] The semantic coding layer includes a ChatGPT pre-trained large model module and a gated recurrent unit network module. The ChatGPT pre-trained large model module is used to perform preliminary processing on the pre-processed data to obtain text feature data after language understanding. The gated recurrent unit network module is used to perform sequence analysis on the text feature data to obtain dynamic features.

[0032] The label prediction layer includes a progressively improved label attention mechanism module and an anti-forgetting incremental learning module; the progressively improved label attention mechanism module is used to refine the dynamic features; the anti-forgetting incremental learning module is used to update the CPE recognition system in a timely manner.

[0033] The ChatGPT pre-trained large model module, the Gated Recurrent Unit (GRU) network module, the layer-by-layer improved label-based attention mechanism module (HLAN module), and the anti-forgetting incremental learning module form a pipeline. Each module further processes and improves the CPE recognition results based on the previous module. The ChatGPT pre-trained large model module provides basic language understanding capabilities, the GRU network module processes sequence features, the HLAN module focuses on key information through the attention mechanism, and the anti-forgetting incremental learning module ensures the model's ability to continuously learn and adapt to new data. This modular design enables the system to handle complex CPE recognition tasks and to continuously evolve and improve over time.

[0034] Specifically, such as Figure 2 As shown, the ChatGPT pre-trained large model module includes a Transformer model; the Transformer model adopts an encoder-decoder architecture, including an input embedding layer, a multi-head self-attention layer, a feedforward neural network layer, a residual connection layer, and layer normalization.

[0035] Specifically, the input embedding layer mainly includes word embedding, position encoding, and the embedding layer. The input embedding layer is used to convert the text data describing the vulnerability information into a numerical form that the model can process, and retain the necessary position information to provide a foundation for subsequent Transformer model processing.

[0036] Furthermore, in the embodiments provided by the present invention, the input embedding layer is the first step in processing sequence data in the deep learning model, where the input discrete symbols (software vulnerability security information characters) are converted into continuous vector representations; this layer uses the pre-trained Word2Vec word embedding method, which is randomly initialized during training and learned during model training. The corresponding mathematical calculation process is: Embedding(W,x) = Wx; where W is the embedding matrix and x is the input index vector.

[0037] Specifically, in the multi-head self-attention layer, multiple attention heads allow the model to focus on different parts of the vulnerability input information simultaneously, improving the CPE identification system's ability to understand the context of the vulnerability input information.

[0038] Furthermore, in the embodiments provided by this invention, the multi-head self-attention layer allows the model to capture dependencies between different positions in the vulnerability information sequence, regardless of their position in the sequence; by transforming the query vector, key vector, and value vector through different linear layers, and then calculating the attention score and weights, the corresponding mathematical calculation process is as follows:

[0039]

[0040] Where Q, K, and V represent the query vector, key vector, and value vector, respectively, and d k It is the dimension of the key.

[0041] Specifically, the feedforward neural network layer further processes and extracts features from the output of the self-attention layer in the Transformer module. By introducing nonlinear transformation, the CPE recognition system's ability to capture complex features is enhanced.

[0042] Furthermore, in the embodiments provided by this invention, the feedforward neural network layer is another layer after the self-attention layer, which further processes the output of the self-attention layer; typically, it is a combination of two linear transformations with a ReLU or other activation function in between, and the corresponding mathematical calculation process is as follows:

[0043] FFN(x)=max(0,xW1+b1)W2+b2;

[0044] Where W1, W2 and b1, b2 are the parameters of the layer.

[0045] Specifically, the residual connections and normalization layers are used to alleviate the vanishing gradient problem, accelerate model training, improve the stability and flexibility of the CPE recognition system, and prevent network degradation, enabling deep networks to be trained and learned effectively.

[0046] Furthermore, in the embodiments provided by this invention, residual connections help information flow through deep networks by directly adding the input to the layer's output, while layer normalization normalizes the activation of each sample to accelerate training and improve model stability. The corresponding mathematical calculation process is: LayerNorm(x+Sublayer(x)).

[0047] Where x is the input of the layer, and Sublayer(x) is the output of the feedforward network layer or the self-attention layer.

[0048] Specifically, the encoder consists of multiple identical layers stacked together, each containing the aforementioned self-attention mechanism and feedforward neural network. This stacked structure allows the model to capture and learn complex features of the input vulnerability data at different levels, providing a basis for the decoder to generate output sequences.

[0049] Furthermore, the encoder typically includes a multi-head self-attention layer, a feedforward neural network layer, residual connections, and layer normalization; the encoder layer processes the input sequence and generates a continuous representation for subsequent processing, and the corresponding mathematical calculation process is as follows:

[0050] EncoderLayer(x)=LayerNorm(x+Sublayer(x));

[0051] Sublayer(x) can be a multi-head self-attention layer or a feedforward network layer.

[0052] Specifically, the decoder layer generates output CPE recognition text step by step based on the internal representation generated by the encoder. It captures the dependencies between the input sequence and the generated sequence through a self-attention sublayer and an encoder-decoder attention sublayer, and processes this information through a feedforward neural network sublayer to finally generate the CPE output sequence.

[0053] Furthermore, in the embodiments provided by this invention, the decoder typically comprises three main parts: a multi-head self-attention layer for processing the target sequence, a multi-head self-attention layer for processing the encoder output and the output of the previous decoder layer, and a feedforward neural network layer. Each part is accompanied by residual connections and layer normalization. The corresponding mathematical calculation process for this is as follows:

[0054] DecoderLayer(x,y)=LayerNorm(x+Attention(x,y)+FFN(x))

[0055] Where x is the input of the decoder layer and y is the output of the encoder.

[0056] Specifically, such as Figure 3 As shown, the Gated Cyclic Unit (GRU) network module includes an update gate, a reset gate, candidate hidden states, and a final hidden state; the update gate is used to control the transmission of information flow and capture long-term dependencies; the reset gate is used to combine new input with past memories and forgotten irrelevant historical information; the candidate hidden state is used to provide candidate values ​​for updating the hidden state; and the final hidden state is used to combine current and past information to form the current state.

[0057] Furthermore, in the embodiments provided by this invention, the main principle by which the GRU network captures long-term dependencies in CPE vulnerability identification is through its unique gating mechanism. GRU is specifically designed to process sequential data and is suitable for modeling long-term dependencies. Unlike LSTM, GRU has a relatively simple structure, containing only two gates (update gate and reset gate) instead of three gates (input gate, forget gate, and output gate). This simplification of structure allows GRU to improve computational efficiency while maintaining effectiveness. Specifically, it is the synergistic effect of the update gate and the reset gate. The following is a detailed explanation of how these two gating mechanisms work.

[0058] The update gate determines the extent to which the hidden state should be updated with new information. It outputs a value between 0 and 1 through a σ (sigmoid function), representing the weight of retaining the hidden state from the previous time step. The update gate z... t The calculation process is as follows:

[0059] z t =σ(W z ·[x t ,h t-1 ]+b z );

[0060] Where x is t The input at the current time step, h t-1It is the hidden state of the previous time step, W z and b z These are the weight matrix and bias vector of the update gate, respectively.

[0061] The reset gate determines the relationship between the input of the current time step and the hidden state of the previous time step; it also outputs a value between 0 and 1 through a sigmoid function, which represents the weight of retaining the hidden state of the previous time step. The calculation process of the reset gate is as follows:

[0062] r t =σ(W r ·[x t ,h t-1 ]+b r );

[0063] Among them, W r and b r These are the weight matrix and bias vector of the reset gate, respectively.

[0064] In the candidate hidden state, the candidate hidden state combines information from the current input and the previous time step (adjusted through a reset gate). Its calculation process is as follows:

[0065]

[0066] Where ⊙ represents the Hadamard product (element-wise product), W is the weight matrix of the candidate hidden state, b is the bias term of the candidate hidden state, and tanh is the hyperbolic tangent function, whose output value is between -1 and 1.

[0067] The current hidden state is the final output, combining information from the previous time step and candidate hidden states; its calculation formula is as follows:

[0068]

[0069] Among them, update gate z t The hidden state h of the previous time step is controlled. t-1 and candidate hidden state h t In the current hidden state The proportion of [something].

[0070] Through these gating mechanisms, GRU can dynamically adjust the flow of information when processing sequential data, enabling the model to capture long-term dependencies in vulnerability data. In CPE vulnerability identification tasks, this means that GRU can identify time-series patterns in software vulnerability reports. For example, a vulnerability may persist across multiple versions of software, and GRU can learn these long-term patterns to identify potential security vulnerability CPE information.

[0071] Specifically, the layer-by-layer improved label attention mechanism module includes a label attention layer, a channel attention layer, a self-attention layer, and a global feature aggregation module; the process of refining the dynamic features includes:

[0072] The label attention layer captures the relationship between labels and features, assigning different attention weights to different labels; the channel attention layer enhances the model's feature representation ability by learning the attention weights of each channel feature; the self-attention layer handles the irregularities of the text feature data and obtains the correlation between elements; the global feature aggregation module obtains an enhanced feature map by aggregating local features and global context information.

[0073] Furthermore, in the embodiments provided by this invention, the label attention layer aims to capture the relationship between labels and features, assigning different attention weights to different labels. Its core idea is that different labels have different importance to features; by learning these weights, the model can better identify features related to specific labels. The calculation process of the label attention layer is as follows:

[0074]

[0075] ContextVector=AttentionWeights·V;

[0076] Here, Q is the query vector, typically the embedding representation of the target label; K is the key vector, corresponding to the embedding representation of each element in the input data; and V is the value vector, also corresponding to the embedding representation of each element in the input data. The attention weight for each element is obtained by calculating the dot product of the query vector and all key vectors, then applying the softmax function. Finally, these weights are used to sum the value vector to obtain the context vector.

[0077] The channel attention layer focuses on the importance of different channels (features) and enhances the model's feature representation ability by learning the weights of each channel. The calculation process of the channel attention layer is as follows:

[0078] ChannelWeights=σ(FC(GlobalAvgPool(X)));

[0079] WeightedFeatures=X·ChannelWeights;

[0080] Where X is the feature map, GlobalAvgPool is global average pooling, FC represents a fully connected layer, and σ is the sigmoid activation function used for output channel weights.

[0081] The self-attention layer, also known as the internal attention layer, allows the CPE recognition model to dynamically adjust its attention to each element when processing sequence data, and allows the model to directly capture dependencies between different positions in the sequence, regardless of distance. The self-attention layer dynamically adjusts its attention to each element by calculating the correlation score between elements at different positions in the sequence. This mechanism enables the model to capture complex dependencies within the sequence without relying on the absolute position of the elements in the sequence.

[0082]

[0083] Here, Q, K, and V are the query vector, key vector, and value vector, respectively, which are usually different linear transformations of the input features.

[0084] The global feature aggregation module, by aggregating local features and global context information, helps the model capture the global structure and patterns of the entire vulnerability information data. This is crucial for identifying complex vulnerabilities spanning multiple code segments. The calculation process of the global feature aggregation module is as follows:

[0085] GlobalFeature=Concat(GlobalAvgPool(X),GlobalMaxPool(X));

[0086] Where X is the feature map, GlobalAvgPool is global average pooling, GlobalMaxPool is global max pooling, and Concat represents the join operation.

[0087] Finally, the HLAN module transforms the extracted features into a final identification result through an output layer, such as determining the existence and severity of security vulnerabilities. The output layer typically contains a softmax layer, which converts the model's output into a probability distribution, assigning a probability value to each possible label (e.g., vulnerabilities of different types and severity). In this way, the model can provide corresponding signal sequence labels, identifying potential CPE software security vulnerabilities. The formula for calculating the softmax function is:

[0088]

[0089] Where p is the probability distribution after processing by the softmax function, z is the output of the fully connected layer, and C is the total number of categories. Through the above steps, the model can provide corresponding signal sequence labels and identify potential security vulnerabilities. This enables the HLAN module to effectively extract security vulnerability information related to CPE from a large amount of software vulnerability data and provide corresponding signal sequence labels, thereby improving the accuracy and efficiency of software security vulnerability identification tasks.

[0090] Specifically, the layer-by-layer improved label attention mechanism module (HLAN module) converts the enhanced feature map into the final recognition result through an output layer to determine whether the software security vulnerability information database has security vulnerabilities and the severity of the vulnerabilities.

[0091] Specifically, such as Figure 4 As shown, the anti-forgetting incremental learning module includes a memory replay mechanism, a regularization strategy, feature sparsity, and contrastive learning;

[0092] The memory replay mechanism is used to store some old category samples and replay them during incremental training; the regularization strategy adds a penalty term to limit the excessive updates of the CPE recognition system; the feature sparsity and contrastive learning are used to reduce noise or irrelevant information in the feature data and to determine the category to which the feature data belongs.

[0093] Furthermore, in the embodiments provided by this invention, the application of incremental learning in the CPE vulnerability information identification task is reasonable. It provides an effective mechanism to handle continuously growing and changing data while maintaining the memory of old knowledge, thereby improving the accuracy and efficiency of CPE vulnerability information identification. Incremental learning allows the model to continuously improve its performance as it receives new data, which is a significant advantage for CPE vulnerability information identification because it ensures that the model always stays up-to-date to address evolving software vulnerability security threats. Simultaneously, incremental learning reduces the risk of forgetting old task knowledge when learning new tasks, which is particularly important for CPE vulnerability information identification because it is necessary to retain the memory of old vulnerability information for association and comparison with new vulnerability information.

[0094] During the model initialization phase, initial vulnerability information data D0 is collected and organized to form a dataset. This data will be used to train the initial model; the dataset should contain feature vectors x. i and the corresponding label y i The labels represent the category or attribute of the vulnerability; they are used to train the initial CPE identification system based on the ChatGPT large model and incremental learning techniques; the initial training dataset D0 and the selected gradient descent optimization algorithm are used to train the model; during the initial training process, the model parameters θ0 are adjusted according to the gradient of the loss function to minimize the loss, and the validation set is used to evaluate the model's performance; this helps prevent overfitting and ensures that the CPE identification model performs well on unseen data; an appropriate loss function is selected to measure the difference between the model's predictions and the true values. For linear models, the mean squared error (MSE) is typically used as the loss function.

[0095]

[0096] Where m is the number of samples, hθ(x) is the model prediction, and y is the actual value.

[0097] During the incremental learning phase, the model absorbs new knowledge, namely, the ability to identify new categories of vulnerability information. After the CPE identification model is deployed, new software vulnerability information data is monitored, which may represent previously unseen vulnerability types. For the newly emerging vulnerability information, relevant training data is collected to form a new dataset.

[0098] Calculate the gradient of the loss function using the new dataset Dnew, and update the model parameters θ0 to θ1, i.e. To achieve incremental learning that resists forgetting, an L2 regularization term is added to the loss function. The L2 regularization term is the sum of squares of the model parameters, and its purpose is to limit the size of the model parameters, thereby reducing overfitting and helping the model learn on new tasks while reducing the forgetting of knowledge from old tasks. This approach helps improve the model's generalization ability and adaptability.

[0099]

[0100] Where λ is the regularization parameter, controlling the strength of regularization; n is the number of model parameters; θ j These are model parameters; the data loss L data and regularization loss L reg By merging, we obtain the total loss function:

[0101]

[0102] In the model parameter update stage, gradient descent is used in incremental learning to update the model parameters. First, the gradient of the loss function with respect to the parameter θ is calculated:

[0103]

[0104] The gradient calculation process for the L2 regularization term can be expressed as:

[0105]

[0106] Therefore, the gradient values ​​of the summarized model can be expressed as:

[0107]

[0108] Where X is the feature matrix and y is the target value vector. For the gradient descent update rule, in each iteration, the parameters need to be updated according to the gradient descent rule:

[0109]

[0110] Here, α is the learning rate; this process is iterative, and as new vulnerability information continues to emerge, the model needs to continuously perform incremental learning to maintain the effectiveness of its identification ability.

[0111] During the model validation and testing phase, after updating the CPE identification model, the performance of the CPE identification system model based on the ChatGPT large model and incremental learning technology was verified using an independent software vulnerability information test set, Dtest. Through these steps, the incremental learning model can continuously update its knowledge base while receiving new data to adapt to new identification tasks, while maintaining its ability to identify older tasks as much as possible.

[0112] Specifically, it also includes a CPE identification method based on the ChatGPT large model and incremental learning technology. The method is characterized by: acquiring data from a software security vulnerability information database; preprocessing the acquired data through a preprocessing layer; inputting the preprocessed data into a semantic coding layer, which includes a ChatGPT pre-trained large model module and a gated recurrent unit network module. The ChatGPT pre-trained large model module is used to perform preliminary processing on the preprocessed data to obtain text feature data after language understanding. Then, the gated recurrent unit network module performs sequence analysis on the text feature data to obtain dynamic features.

[0113] The obtained dynamic features are then input into the label prediction layer, which includes a progressively improved label attention mechanism module and an anti-forgetting incremental learning module. The progressively improved label attention mechanism module is used to refine the dynamic features.

[0114] Finally, the CPE recognition system is updated in a timely manner through the anti-forgetting incremental learning module.

[0115] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0116] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A CPE identification system based on the ChatGPT large model and incremental learning technology, characterized in that, include: The software security vulnerability information database, preprocessing layer, semantic encoding layer, and label prediction layer are connected sequentially. The preprocessing layer is used to preprocess the data in the software security vulnerability information database; The semantic coding layer includes a ChatGPT pre-trained large model module and a gated recurrent unit network module. The ChatGPT pre-trained large model module is used to perform preliminary processing on the pre-processed data to obtain text feature data after language understanding. The gated recurrent unit network module is used to perform sequence analysis on the text feature data to obtain dynamic features. The label prediction layer includes a progressively improved label attention mechanism module and an anti-forgetting incremental learning module; the progressively improved label attention mechanism module is used to refine the dynamic features; the anti-forgetting incremental learning module is used to update the CPE recognition system in a timely manner. The anti-forgetting incremental learning module includes a memory replay mechanism, a regularization strategy, feature sparsity, and contrastive learning. The memory replay mechanism is used to store some old category samples and replay them during incremental training; the regularization strategy adds a penalty term to limit the excessive updates of the CPE recognition system; the feature sparsity and contrastive learning are used to reduce noise or irrelevant information in the feature data and to determine the category to which the feature data belongs.

2. The CPE identification system based on ChatGPT large model and incremental learning technology according to claim 1, characterized in that, The ChatGPT pre-trained large model module includes a Transformer model; the Transformer model adopts an encoder-decoder architecture, including an input embedding layer, a multi-head self-attention layer, a feedforward neural network layer, a residual connection layer, and layer normalization.

3. The CPE identification system based on ChatGPT large model and incremental learning technology according to claim 1, characterized in that, The gated loop unit network module includes an update gate, a reset gate, candidate hidden states, and a final hidden state; the update gate is used to control the transmission of information flow and capture long-term dependencies; the reset gate is used to combine new input with past memories and forgotten irrelevant historical information; the candidate hidden state is used to provide candidate values ​​for updating the hidden state; and the final hidden state is used to combine current and past information to form the current state.

4. The CPE identification system based on ChatGPT large model and incremental learning technology according to claim 1, characterized in that, The layer-by-layer improved label attention mechanism module includes a label attention layer, a channel attention layer, a self-attention layer, and a global feature aggregation module; The process of refining the dynamic features includes: The label attention layer is used to capture the relationship between labels and features, and different attention weights are assigned to different labels. The channel attention layer enhances the model's feature representation ability by learning the attention weights of each channel feature; the self-attention layer is used to handle the irregularities of the text feature data and obtain the correlation between elements; the global feature aggregation module obtains an enhanced feature map by aggregating local features and global context information.

5. The CPE identification system based on ChatGPT large model and incremental learning technology according to claim 4, characterized in that, The layer-by-layer improved label attention mechanism module converts the enhanced feature map into the final recognition result through an output layer, and determines whether there are security vulnerabilities in the software security vulnerability information database and the severity of the vulnerabilities.

6. A method for applying to a CPE identification system based on ChatGPT large model and incremental learning technology as described in any one of claims 1-5, characterized in that, include: Retrieve data from a software security vulnerability information database; The acquired data is preprocessed through a preprocessing layer; The preprocessed data is input into the semantic coding layer, which includes a ChatGPT pre-trained large model module and a gated recurrent unit network module. The ChatGPT pre-trained large model module is used to perform preliminary processing on the preprocessed data to obtain text feature data after language understanding. Then, the gated recurrent unit network module is used to perform sequence analysis on the text feature data to obtain dynamic features. The obtained dynamic features are then input into the label prediction layer, which includes a progressively improved label attention mechanism module and an anti-forgetting incremental learning module. The progressively improved label attention mechanism module is used to refine the dynamic features. Finally, the CPE recognition system is updated in a timely manner through the anti-forgetting incremental learning module.

Images