Large model, method, system, medium and device for script file detection

By comparing the learning mechanism and the optimized CodeBERT model, the problem of high false alarm and omission rate in malicious script detection is solved, and the in-depth analysis and feature extraction of complex script behavior is realized, which improves detection accuracy and robustness. Enhanced detection capabilities for unknown threats.

CN118761062BActive Publication Date: 2024-11-19MINGYANG SHICHUANG (BEIJING) TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411225458.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-19
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

The prior art has high false alarm and omission rates when detecting complex and diverse malicious scripts, and it is difficult to effectively capture and identify hidden features of malicious scripts, especially when facing code obfuscation, encryption and segmented loading strategies. ,The limitations of traditional methods lead to insufficient detection accuracy and robustness.

Method used

By introducing a comparison learning mechanism and an optimized CodeBERT model, we train a large model for script file detection, enhance our understanding of the internal logical relationships of the script, and use the comparison loss function and the classification loss function to optimize the model parameters, combining the multi-head self-attention mechanism and the front Feed neural network, introduces ReLU activation function and dropout layer, and performs deep feature extraction and classification.

Benefits of technology

Improve the accuracy and robustness of malicious script detection, reduce false positives and missed reports, enhance the model's ability to capture and classify complex script behaviors, improve the detection ability of unknown threats, and improve the accuracy of feature extraction and detection effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118761062B_ABST
    Figure CN118761062B_ABST
Patent Text Reader

Abstract

The present invention discloses a large model, method, system, medium and device for script file detection, wherein the system comprises a data acquisition module, a data preprocessing module and a large model for script file detection, the data acquisition module is communicatively connected with the data preprocessing module, and the data preprocessing module is communicatively connected with the large model for script file detection. The present invention enhances the understanding of the internal logical relationship of the script by the large model for script file detection by introducing a contrast learning mechanism and an optimized large model for script file detection, thereby improving the accuracy and robustness of detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of script detection, and in particular to a large model, method, system, medium and equipment for script file detection. Background Art

[0002] In recent years, with the continuous evolution of network attack methods, the complexity and diversity of malicious scripts have increased significantly, and traditional detection methods face huge challenges. Existing technologies mainly rely on feature matching and rule detection, but these methods are insufficient in dealing with the obfuscation techniques and dynamic changes of malicious scripts. In particular, when malicious scripts adopt strategies such as code obfuscation, encryption, or segmented loading, traditional static and dynamic analysis methods often cannot effectively detect them. The most similar technical solutions at present are based on machine learning and deep learning methods, which automatically extract script features through training models. However, these methods usually rely on a large amount of annotated data, and have limited understanding of the internal structure and contextual associations of the script, resulting in high false positive and false negative rates.

[0003] The shortcomings of existing technologies in detecting malicious script files stem from their limitations in the face of evolving and complex attack methods. Since malicious scripts often use strategies such as obfuscation, encryption, and segmented loading, traditional feature matching and rule detection methods rely on static or dynamic analysis, which makes it difficult to effectively capture and identify these hidden features, resulting in frequent false positives and false negatives. At the same time, although machine learning and deep learning methods have improved detection capabilities through automatic feature extraction, they rely on a large amount of annotated data, which not only increases the difficulty of data preparation, but also limits the model's deep understanding of the script's internal structure and contextual associations, further leading to insufficient detection accuracy and robustness. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to provide a big model, method, system, medium and equipment for script file detection. By introducing a comparative learning mechanism and an optimized big model for script file detection, the big model for script file detection enhances its understanding of the internal logical relationship of the script, thereby improving the accuracy and robustness of detection.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] A large model for script file detection, wherein the large model for script file detection is a large model obtained by training the primary large model through the following steps:

[0007] S110) Data preprocessing, the specific operations are as follows:

[0008] S111) Script segmentation: segmenting the malicious script file into n script sections P;

[0009] S112) Construct positive and negative sample pairs: For the script paragraphs in the malicious script file , randomly select a script paragraph from the script file of the same category as the malicious script file as a positive sample pair, and randomly select a script paragraph from the script file of a different category from the malicious script file as a negative sample pair, where i is a natural number less than n;

[0010] S120) Optimize and adjust the initial large model parameters using the contrast loss function and the classification loss function. The specific operations are:

[0011] S121) Feature extraction: Use the initial large model to process each script segment P independently and map the input into a dense vector space;

[0012] S122) calculating the similarity between the script paragraph P and the positive sample pair and the negative sample pair;

[0013] S123) The parameters of the initial large model are optimized and adjusted by maximizing the similarity between the script paragraph P and the positive sample pair and minimizing the similarity between the script paragraph P and the negative sample pair through the contrast loss function. At the same time, the parameters related to the script paragraph classification in the initial large model are optimized and adjusted using the classification loss function, so as to obtain a large model for script file detection.

[0014] The above-mentioned large model for script file detection includes an encoder network and a classifier network; the contrast loss function is used to optimize the training of the encoder network, and the classification loss function is used to optimize the training of the classifier network.

[0015] The above-mentioned large model for script file detection also includes a newly added fully connected layer, and the newly added fully connected layer is provided with a new linear layer that uses a ReLU activation function to introduce a nonlinear transformation.

[0016] The above large model for script file detection has a dropout layer in the newly added fully connected layer, and the dropout rate of the dropout layer is 0.1.

[0017] The above-mentioned large model for script file detection is a primary large model, which is a CodeBERT model that adopts a Transformer architecture and contains multiple stacked encoder layers. Each encoder layer contains a multi-head self-attention mechanism sublayer and a feedforward neural network sublayer. The feedforward neural network sublayer performs nonlinear conversion and mapping on the input through two layers of linear transformation combined with the ReLU activation function.

[0018] The script file detection method includes the following steps:

[0019] S210) Data preprocessing, specifically: segment the script file into natural segments to obtain script paragraphs ;

[0020] S220) The script segments obtained by segmentation A large model for script file detection is input for detection analysis and the detection analysis result is output, wherein the large model for script file detection is the above-mentioned large model for script file detection.

[0021] The script file detection method further includes the following steps:

[0022] S230) uses the detection analysis results obtained in step S220) and steps S110) and S120) to iteratively train the large model used for script file detection.

[0023] Script file detection system, including:

[0024] Data collection module, used to collect script files to be detected;

[0025] The data preprocessing module is used to segment the script file into natural segments and construct positive and negative sample pairs;

[0026] The large model used for script file detection is the large model used for script file detection mentioned above;

[0027] The data acquisition module is communicatively connected with the data preprocessing module, and the data preprocessing module is communicatively connected with the large model used for script file detection.

[0028] A computer-readable storage medium stores a computer program, which implements the above method when executed by a processor.

[0029] A computer device includes a readable storage medium, a processor, and a computer program stored in the readable storage medium and executable on the processor, wherein the computer program implements the above method when executed by the processor.

[0030] The technical solution of the present invention achieves the following beneficial technical effects:

[0031] 1. By introducing a contrastive learning mechanism, the big model used for script file detection can enhance its understanding of the internal logical relationship of the script, and use the optimized big model for script file detection to perform effective feature extraction, thereby improving the accuracy and robustness of detection.

[0032] 2. The purpose of the present invention is to solve the deficiencies of the prior art in processing complex and diverse malicious scripts, and to achieve accurate identification and classification of malicious scripts through a deeper feature extraction and comparative learning mechanism.

[0033] 3. The present invention also introduces multiple data enhancement strategies in supervised contrastive learning to further improve the adaptability and robustness of the model in different environments. These enhancement strategies can not only effectively deal with the obfuscation and deformation methods of malicious scripts, but also enhance the model's ability to detect unknown threats.

[0034] 4. Using the optimized CodeBERT model, the present invention takes into account the contextual association and semantic information of the script during the feature extraction process, so that the model can more accurately identify potential threats in the script. This optimization process includes further fine-tuning of the pre-trained model to make it more suitable for malicious script detection tasks, thereby improving the accuracy of feature extraction and detection effect.

[0035] 5. The detection method of the present invention achieves effective capture and classification of complex script behaviors by conducting in-depth analysis of the structure and behavior characteristics of malicious scripts. This not only improves the accuracy of detection, but also reduces the false positive rate and missed negative rate, making the method more reliable and efficient in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a working principle diagram of the script file detection system in the present invention;

[0037] Figure 2 A flowchart of large model training optimization for script file detection in the present invention;

[0038] Figure 3 A flowchart of script file detection in the present invention;

[0039] Figure 4 A schematic diagram of a computer device capable of performing script file detection. DETAILED DESCRIPTION

[0040] The present invention is further described below with reference to examples.

[0041] like Figure 1 As shown, the script file detection system of the present invention includes a data acquisition module, a data preprocessing module and a large model for script file detection, the data acquisition module is connected to the data preprocessing module, and the data preprocessing module is connected to the large model for script file detection. Among them, the data acquisition module is used to collect the script file to be detected; the data preprocessing module is used to segment the script file according to natural segments and construct positive and negative sample pairs.

[0042] like Figure 2 As shown, the large model used for script file detection in the present invention is a large model obtained by training the primary large model through the following steps:

[0043] S110) Data preprocessing, the specific operations are as follows:

[0044] S111) Script segmentation: segmenting the malicious script file into n script sections P; wherein the script file may be segmented in a natural section or an operation manner;

[0045] S112) Construct positive and negative sample pairs: For the script paragraphs in the malicious script file , randomly select a script paragraph from the script file of the same category as the malicious script file as the script paragraph A positive sample pair is obtained, and a script paragraph is randomly selected from a script file of a different category from the malicious script file as the script paragraph Negative sample pairs, where i is a natural number less than n;

[0046] S120) Optimize and adjust the initial large model parameters using the contrast loss function and the classification loss function. The specific operations are:

[0047] S121) Feature extraction: Use the initial large model to process each script segment P independently and map the input into a dense vector space;

[0048] S122) calculating the similarity between the script paragraph P and the positive sample pair and the negative sample pair;

[0049] S123) The parameters of the initial large model are optimized and adjusted by maximizing the similarity between the script paragraph P and the positive sample pair and minimizing the similarity between the script paragraph P and the negative sample pair through the contrast loss function. At the same time, the parameters related to the script paragraph classification in the initial large model are optimized and adjusted using the classification loss function, so as to obtain a large model for script file detection.

[0050] In the present invention, the initial large model and the large model for script file detection are iteratively trained and optimized by constructing and optimizing positive and negative sample pairs.

[0051] In step S121), when extracting features from the script segment P, two special tag separators [SOS] and [EOS] are inserted at the start and end of the input sequence, respectively, and the hidden state of the last layer corresponding to the special tag [EOS] is regarded as the embedded representation of the input sequence. For example, the high-dimensional features of different samples are shown in the following two equations:

[0052]

[0053] In the formula, Represented as a sample The high-dimensional features of E are represented as feature extraction functions. Represented as a sample , ⊕ represents the operation of concatenating two strings together.

[0054] Through experiments, it is found that using different separators can make the training more stable. For example, use "[" as , use "]" as , for the sample For example, use "{" as , use "}" as .

[0055] For example, suppose the sample The sample is "Everything is possible." and the next adjacent paragraph is "Everything is possible." , specifically:

[0056] “Everything is possible.

[0057] Every thing is possible.”

[0058] According to the above sample processing method, the sample can be expressed as [Everything is possible.], and the sample It can be expressed as {Every thing is possible.}.

[0059] After feature extraction of the script segment, the cosine similarity is used to calculate the similarity between the positive and negative sample pairs in the high-dimensional feature space. In the present invention, the cosine similarity formula used is:

[0060]

[0061] In the formula, For sample High-dimensional features; For samples Different high-dimensional features of another sample.

[0062] By extracting the features of the script paragraph P and using the features and cosine similarity formula to quantify the similarity between the script paragraphs, a basis is provided for optimizing the parameters of the large model used for script file detection using the contrast loss function and the classification loss function in step S123).

[0063] The primary large model in the present invention is a CodeBERT model that uses a Transformer architecture and includes multiple stacked encoder layers, wherein each encoder layer contains a multi-head self-attention mechanism sublayer and a feedforward neural network sublayer, and the feedforward neural network sublayer performs nonlinear transformation and mapping on the input through two layers of linear transformation combined with a ReLU activation function. The large model for script file detection obtained by training and optimization of the primary large model has an encoder network and a classifier network built in.

[0064] The self-attention mechanism sublayer uses query ( ) and keys ( ), combined with the normalization factor And softmax function, calculate the attention weight of each position to other positions ( ), these attention weights are used to weight the values ​​of all positions in the input sequence ( ), and generates output for each location .

[0065] The feedforward neural network sublayer is used to process the output of the self-attention mechanism sublayer. The feedforward neural network sublayer uses two layers of linear transformation combined with the ReLU activation function to perform nonlinear transformation and mapping on the input to generate higher-level features, which is specifically expressed as FFN(x)=ReLU(x*W1+b1)*W2+b2.

[0066] In order to facilitate the preprocessing of script files, in the data preprocessing stage, the large model for script files introduces position encoding to embed position information into the word representation of the input sequence. Position encoding uses sine and cosine functions to generate position vectors, and uses the exponential term i / 10000 (2k / d) To control the rate of change of position encoding and ensure the uniqueness and distinctiveness of different positions, the large model used for script file detection can better understand the semantics and contextual relationships of different positions in the code sequence.

[0067] In order to perform classification task output, an additional fully connected layer is introduced in the large model for script file detection. Specifically, a new linear layer with 100 features is first added, and the ReLU activation function is used to introduce nonlinear transformation. In order to prevent overfitting, a dropout layer is introduced after activating the new linear layer, where the dropout rate of the dropout layer is 0.1. The final output uses a linear layer with four features to generate the final classification label.

[0068] When optimizing and adjusting the parameters of the initial large model and the parameters of the large model used for script file detection, the contrast loss function and the classification loss function are used to optimize the initial model and the large model used for script file detection. Specifically, the contrast loss function is used to maximize the similarity of positive sample pairs in the feature space and minimize the similarity of negative sample pairs in the feature space through a contrast learning mechanism to optimize the encoder network. At the same time, the classification loss function is used to optimize the classifier network to improve the classification capabilities of the initial large model and the large model used for script file detection.

[0069] The contrast loss function used in the present invention is:

[0070]

[0071] In the formula, is the temperature parameter, Is with the sample The set of indices of all positive samples of the same type, is its cardinality, Indicates except The collection of all samples except .

[0072] The classification loss function is:

[0073]

[0074] Among them, M is the number of samples, C is the total number of categories, It is a sample The one-hot encoding of the true label of Belongs to category c, then =1, otherwise =0, The model is a sample The predicted probability of belonging to class c.

[0075] In each training iteration of the large model for script file detection, batches of negative sample pairs are used for efficient training, supervision contrast loss and classification loss are calculated, and model parameters are updated. In this way, the model performance is gradually optimized through continuous iteration.

[0076] The script file detection system is used to detect the script file. The specific process is as follows: Figure 3 As shown, the following steps are included:

[0077] S210) Data preprocessing, specifically: segment the script file into natural segments to obtain script paragraphs ;

[0078] S220) The script segments obtained by segmentation Input a large model for script file detection to perform detection analysis and output the detection analysis result, wherein the large model for script file detection is the large model for script file detection described above;

[0079] S230) uses the detection analysis results obtained in step S220) and steps S110) and S120) to iteratively train the large model used for script file detection.

[0080] In the present invention, positive and negative sample pairs are constructed by supervised contrastive learning, and the ability of the large model used for script file detection to accurately capture malicious script features is enhanced by using tag information, which effectively improves the accuracy and robustness of detection. When the ability of the large model used for script file detection to accurately capture malicious script features is enhanced by using tag information, the script paragraphs constructed from malicious script files of the same class are marked as positive sample pairs, and the script paragraphs constructed from malicious script files of different classes are marked as negative sample pairs. Secondly, the optimized large model for script file detection can capture the semantics and structural information of the script file more comprehensively through embedded representation and feature extraction, thereby improving the adaptability of the large model used for script file detection to complex script variants. Finally, combining the advantages of supervised contrastive learning and the large model used for script file detection, the present invention is fully trained and verified on a large-scale data set to ensure that the large model used for script file detection has high detection efficiency and reliability in practical applications. These innovative technical combinations make the present invention have significant advantages in improving malicious script detection capabilities and responding to new threats.

[0081] Based on the above script file detection method, accordingly, a computer-readable storage medium storing a computer program is also provided in this example. When the computer program is executed by a processor, the following steps are implemented: the script file is segmented according to natural segments to obtain script paragraphs, and then the script paragraphs obtained after segmentation are input into a large model for script file detection for detection and analysis and output the detection and analysis results, and the detection and analysis results are also used to iteratively train the large model for script file detection.

[0082] like Figure 4As shown, based on the above-mentioned script file detection method and computer-readable storage medium, in this embodiment, a computer device is also provided, which includes a readable storage medium, a processor, and a computer program stored on the readable storage medium and executable on the processor, wherein the readable storage medium and the processor are both arranged on a bus, and the processor implements the following steps when executing the computer program: the script file is segmented according to natural segments to obtain script paragraphs, and then the script paragraphs obtained after segmentation are input into a large model for script file detection for detection and analysis and the detection and analysis results are output, and at the same time, the detection and analysis results are used to iteratively train the large model for script file detection.

[0083] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the claims of this patent application.

Claims

1. A large model for script file detection, characterized in that: The large model used for script file detection is a large model obtained by training the primary large model through the following steps: S110) Data preprocessing, the specific operations are as follows: S111) Script segmentation: segmenting the malicious script file into n script sections P; S112) Construct positive and negative sample pairs: For the script paragraphs in the malicious script file , randomly select a script paragraph from the script file of the same category as the malicious script file as a positive sample pair, and randomly select a script paragraph from the script file of a different category from the malicious script file as a negative sample pair, where i is a natural number less than n; S120) Optimize and adjust the initial large model parameters using the contrast loss function and the classification loss function. The specific operations are: S121) Feature extraction: Use the initial large model to process each script segment P independently and map the input into a dense vector space; S122) calculating the similarity between the script paragraph P and the positive sample pair and the negative sample pair; S123) Optimizing and adjusting the parameters of the initial large model by maximizing the similarity between the script paragraph P and the positive sample pair and minimizing the similarity between the script paragraph P and the negative sample pair through a contrast loss function, and optimizing and adjusting the parameters related to the script paragraph classification in the initial large model by using a classification loss function, thereby obtaining a large model for script file detection; The primary large model is a CodeBERT model that uses a Transformer architecture and contains multiple stacked encoder layers. Each encoder layer contains a multi-head self-attention mechanism sublayer and a feedforward neural network sublayer. The feedforward neural network sublayer performs nonlinear transformation and mapping on the input through two layers of linear transformation combined with a ReLU activation function. The large model for script file detection obtained by training and optimization of the primary large model has an encoder network and a classifier network built in. The contrast loss function is: In the formula, is the temperature parameter, Is with the sample The set of indices of all positive samples of the same type, is its cardinality, Indicates except The collection of all samples except The classification loss function is: Among them, M is the number of samples, C is the total number of categories, It is a sample The one-hot encoding of the true label of Belongs to category c, then ,otherwise , The model is a sample The predicted probability of belonging to category c; In each training iteration of the large model for script file detection, batches of negative sample pairs are used for efficient training, the supervised contrast loss and classification loss are calculated, and the model parameters are updated.

2. The large model for script file detection according to claim 1, characterized in that: The large model for script file detection includes an encoder network and a classifier network; the contrast loss function is used to optimize the training of the encoder network, and the classification loss function is used to optimize the training of the classifier network.

3. The large model for script file detection according to claim 2, characterized in that: The large model for script file detection also includes a new fully connected layer, which contains a new linear layer that uses the ReLU activation function to introduce nonlinear transformations.

4. The large model for script file detection according to claim 3, characterized in that: A dropout layer is also added in the newly added fully connected layer, and the dropout rate of the dropout layer is 0.

1.

5. The script file detection method is characterized in that: The steps include: S210) Data preprocessing, specifically: segment the script file into natural segments to obtain script paragraphs ; S220) The script segments obtained by segmentation A large model for script file detection is input for detection analysis and the detection analysis result is output, wherein the large model for script file detection is the large model for script file detection described in claim 1.

6. The script file detection method according to claim 5, characterized in that: The following steps are also included: S230) uses the detection analysis results obtained in step S220) and steps S110) and S120) to iteratively train the large model used for script file detection.

7. The script file detection system is characterized by: include: Data collection module, used to collect script files to be detected; The data preprocessing module is used to segment the script file into natural segments and construct positive and negative sample pairs; The large model for script file detection is the large model for script file detection described in claim 1; The data acquisition module is communicatively connected with the data preprocessing module, and the data preprocessing module is communicatively connected with the large model used for script file detection.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claim 5 or 6 is implemented.

9. A computer device comprising a readable storage medium, a processor, and a computer program stored on the readable storage medium and executable on the processor, characterized in that: When the computer program is executed by a processor, the method according to claim 5 or 6 is implemented.