A third-party malicious package detection method based on large model feature generation

The generation of feature sets and machine learning models for detecting third-party malicious packets is solved by generating a large language model, and the problem of high false alarm rates and missed alarm rates in the existing technology is solved, and malicious packet detection with high accuracy and scalability is achieved.

CN117852034BActive Publication Date: 2024-11-19HUAZHONG UNIV OF SCI & TECH +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202311778051.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-20
Publication Date
2024-11-19
Estimated Expiration
2043-12-20

AI Technical Summary

Technical Problem

Existing third-party malicious package detection methods rely on expert-defined features, resulting in high false positive and missed rate, and it is difficult to adapt to new types of malicious code.

Method used

A feature generation method based on large language models is adopted, and feature sets are automatically generated through abstract syntax trees and machine learning models, false positives and false negatives are reduced, and detectors are updated through incremental learning to adapt to new types of malicious packages.

Benefits of technology

It achieves high accuracy and high scalability, can effectively detect third-party malicious packages, reduce false positives and missed reports, and adapt to the ever-changing new types of malicious code in the third-party package ecosystem.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117852034B_ABST
    Figure CN117852034B_ABST
Patent Text Reader

Abstract

The present invention discloses a third-party malicious package detection method based on large model feature generation, which belongs to the field of program package detection. The method utilizes a large language model to guide the recognition of malicious code fragments of third-party packages and the generation of features, automatically generates a feature set, does not rely on features defined by experts, and can effectively overcome the shortcomings of existing machine learning-based detectors, so that the feature set can better reflect the difference between malicious samples and benign samples, and has stronger scalability; adopts an update prompt polling method and constructs a malicious fragment set, so that the accuracy is improved; at the same time, the false alarm rate and missed alarm rate of malicious package detection can be significantly reduced, and the malicious code location can be given.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of program package detection, and more specifically, relates to a third-party malicious package detection method based on large model feature generation. Background Art

[0002] Dynamic languages ​​are currently popular programming languages, such as JavaScript, Python, and Ruby. Their continued growth is supported by the third-party package ecosystem, such as npm, PyPI, and RubyGems, which is a repository with reusable packages. The popularity of third-party packages has made the third-party package ecosystem a favorite target for attackers for a long time. Attackers try to release malicious versions of packages that can tamper with or leak data on the machines where they are installed, or other malicious activities. Recent malicious package attacks have occurred frequently, including the eslint-scope leak, in which attackers managed to steal the credentials of the maintainer of a popular package, allowing them to release a new malicious version of the package that uploads user credentials to the server when installed; the event-stream backdoor incident, which used social engineering techniques to obtain the maintainer identity and then launched similar attacks; at the same time, some small incidents are also happening. These malicious packages have seriously threatened the information security of users. Being able to detect and discover malicious packages in advance is of great help to both corporate production and personal privacy.

[0003] Existing detectors follow two approaches: program analysis and machine learning. The program analysis approach mainly detects malicious packages through pattern matching, clone detection, and dynamic analysis. The rules used to detect malicious behavior in this approach are relatively general and may produce many false positives. In addition, some tools are heavyweight and difficult to deploy and meet actual needs. Machine learning methods use expert-defined packet feature representations to train detectors. The effectiveness of this approach depends on the quality of features, which is difficult to define (for example, when obfuscation techniques are involved), subjective (i.e. different experts may define different feature sets), and cumbersome to define. Summary of the invention

[0004] In response to the above defects or improvement needs of the prior art, the present invention provides a third-party malicious packet detection method based on large model feature generation. The method does not rely on expert-defined features, and can effectively reduce the false alarm rate and missed alarm rate, thereby achieving more effective malicious third-party packet detection.

[0005] To achieve the above object, according to a first aspect of the present invention, a third-party malicious packet detection method based on large model feature generation is provided, comprising:

[0006] Training phase:

[0007] Inputting the training package into the large language model for malicious code identification to obtain the malicious code in the training package, extracting the features of the malicious code to construct a feature set; wherein the training package is a third-party malicious package;

[0008] Generate an abstract syntax tree AST corresponding to the training package, traverse the nodes of the AST to determine a grammatical structure corresponding to the feature in the feature set, and encode the code segment corresponding to the grammatical structure to obtain a feature vector of the malicious code;

[0009] Inputting the feature vector into a machine learning model for supervised training;

[0010] Application phase:

[0011] Generate an abstract syntax tree AST corresponding to the third-party package to be detected, traverse the nodes of the AST to determine the grammatical structure corresponding to the features in the feature set, encode the code segment corresponding to the grammatical structure to obtain the feature vector to be detected, and input it into the trained machine learning model to obtain the detection result.

[0012] According to a second aspect of the present invention, there is provided a third-party malicious packet detection system based on large model feature generation, comprising: a computer-readable storage medium and a processor;

[0013] The computer-readable storage medium is used to store executable instructions;

[0014] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0015] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a processor to execute the method as described in the first aspect.

[0016] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0017] 1. Automatically generate feature sets: Compared with existing machine learning-based detectors, the present invention fully utilizes the advantages of the Large Language Model (LLM) and uses LLM to guide the recognition of third-party malicious code fragments and the generation of features. It does not rely on expert definitions, so that the feature set can better reflect the differences between malicious samples and benign samples and has stronger scalability.

[0018] 2. High accuracy: By polling and prompting to update and optimize the results of LLM, the confidence issue is taken into account, so that the accuracy of the analysis results of the large model is improved. At the same time, the present invention will summarize the malicious code fragments of the same malicious behavior in the data set, so that the feature extractor can capture malicious features more accurately, which is helpful for the extraction of feature vectors.

[0019] 3. High scalability: Considering the third-party package ecosystem, such as npm, PyPI, RubyGems, etc., there are tens of thousands of new packages released by third-party users or organizations every day, and new types of malicious code may appear at the same time. In order to ensure that new malicious behaviors can be learned, malicious packages will be expanded to the data set in an incremental manner. This method detects malicious packages based on the features summarized by the large model. The continuously updated increments enable the large model to summarize more and richer feature sets, and update the detector in a timely manner through incremental learning. It also has strong detection capabilities for new types of malicious packages. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of a third-party malicious packet detection method based on LLM feature generation provided in an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a flow chart for generating a feature set based on LLM for identifying malicious code and malicious behavior provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a process for polling malicious behavior based on LLM update prompts provided in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of a process for performing malicious packet detection on a target sample provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] The embodiment of the present invention provides a third-party malicious packet detection method based on large model feature generation, such as Figure 1 As shown, including:

[0026] Training phase:

[0027] Input the training package into the large language model for malicious code identification to obtain the malicious code in the training package; extract the features of the malicious code to construct a feature set; wherein the training package is a third-party malicious package;

[0028] Generate an abstract syntax tree AST corresponding to the training package, traverse the nodes of the AST to determine a grammatical structure corresponding to the feature in the feature set, and encode the code segment corresponding to the grammatical structure to obtain a feature vector of the malicious code;

[0029] The feature vector is input into the machine learning model for supervised training.

[0030] As a further preferred embodiment of the present invention, the large language model identifies malicious code in the training package under a preset prompt;

[0031] The preset prompt is: determine the malicious code fragment, analyze its malicious behavior, and give the confidence level of the malicious behavior.

[0032] As a further preferred embodiment of the present invention, the large language model uses an iterative query method to identify malicious code in the training package under a preset prompt;

[0033] When performing the t+1th query, the recognition result of the tth query is added to the preset prompt in the form of an annotation.

[0034] As a further preferred embodiment of the present invention, the step of extracting features of the malicious code and constructing a feature set includes:

[0035] The malicious code corresponding to the malicious behavior that meets the confidence requirement is retained;

[0036] Extracting keywords from malicious codes corresponding to each malicious behavior respectively as type identifiers of the malicious behaviors;

[0037] The code fragment with the highest frequency in the malicious code corresponding to each malicious behavior is used as the feature of the malicious code corresponding to each malicious behavior to obtain a feature set, which includes the features of the malicious code corresponding to each malicious behavior.

[0038] As a further preferred embodiment of the present invention, the large language model is ChatGPT, mT0 or Flan-PaLM.

[0039] Specifically, in the training phase, the offline malicious packet detector is trained: based on the LLM, the malicious behavior of the training packet is identified to construct a malicious code snippet set, and a feature set is generated according to the malicious code snippet set. The feature vector of the malicious code in the training packet is extracted according to the feature set, and the vector is input into the machine learning model for training to obtain the trained machine learning model, which is used as a malicious packet detector. Specifically, the following steps are included:

[0040] (1.1) Identify malicious code and malicious behavior in training packages based on LLM: Build prompts and use LLM to identify malicious code and corresponding malicious behavior from third-party training packages. Update prompt content based on the results and iterate multiple queries.

[0041] That is, use the training package as the input of LLM, guide LLM to focus on malicious code fragments through prompts, analyze their malicious behaviors, and give the confidence of the conclusion. Update the prompts according to the results: add the analysis results of the previous round to the prompt content in the form of comments, and conduct a new round of queries (i.e., a new round of malicious code identification) to solve the input limitation problem of the large model; output the results when the output results of LLM are close to fitting.

[0042] For example, step (1.1) requires LLM to give confidence in its conclusion by adding the following prompt: "Each malicious behavior analysis must be marked as 'confident' or 'uncertain'. If you are confident about a malicious behavior, you should mark it as 'confident'. Otherwise, if you are uncertain about a malicious behavior, you should mark it as 'uncertain'." LLM may overestimate its confidence in the results, but rarely underestimate it.

[0043] (1.2) Constructing a malicious code snippet set: Process the results after fitting in step (1.1), retain the results with high confidence, extract the keywords of malicious behavior (used to represent the type of malicious behavior) and malicious code for each training package, and then summarize and construct a malicious code snippet set.

[0044] That is, the results with a confidence level of "confident" are retained, and by using a keyword extractor to extract keywords as identifiers of malicious behaviors, the results of LLM are constructed into a malicious code snippet set with the following structure: "(third-party package name, malicious code snippet, malicious behavior identifier)".

[0045] (1.3) Generate feature set: Merge similar malicious behaviors in the malicious code snippets constructed based on step (1.2) and summarize their behavior characteristics, and generate a feature set (also called malicious feature set) based on the frequency of malicious behaviors appearing in the code snippet set;

[0046] It is understandable that each training package can extract code snippets corresponding to multiple malicious behaviors, and the code snippets corresponding to the same malicious behavior of all training packages are counted, and the snippet with the highest frequency is used as the feature of the malicious behavior.

[0047] (1.4) Extract feature vectors of malicious code in the training package: parse the third-party package through the compiler, convert the file (i.e., all code files in the third-party package) into an abstract syntax tree (AST), traverse the nodes and match the malicious feature set generated in step (1.3), find the grammatical structure corresponding to the malicious features in the malicious feature set, and then encode the extracted code segment corresponding to the grammatical structure into a feature vector.

[0048] The code snippet with the highest frequency in the malicious code snippet set corresponding to each malicious behavior is used as the feature of each malicious behavior to construct a feature set of all malicious behaviors (also called a malicious feature set), and is matched with the code snippet contained in each node in the syntax tree. If the code snippet contained in a node is the same as the malicious feature in the malicious feature set, then the node is a malicious node (i.e., a node containing malicious features). For code files, focus on accessing files and network systems, executing sensitive functions, creating new processes, etc. For metadata files of packages, focus on custom script instructions in the files.

[0049] (1.5) Training the machine learning model: The vector obtained in step (1.4) and the corresponding type label are input into the machine learning model training. By participating in comparative experiments, the best malicious packet detector is obtained.

[0050] There are two types of the above type labels: 0 or 1; 0 represents a non-malicious packet, and 1 represents a malicious packet.

[0051] As a further preferred embodiment of the present invention, the machine learning model adopts any one of Naive Bayes (NB), Multilayer Perceptron (MLP), Random Forest (RF) and Support Vector Machine (SVM); among them, NB, RF and SVM have ideal interpretability properties, MLP has advanced expression ability, and RF is preferred.

[0052] As a further preferred embodiment of the present invention, a powerful K-fold cross validation is used to train and validate the model.

[0053] The following takes ChatGPT as the LLM for analysis as an example to further explain the above training stage.

[0054] (1.1) Identify malicious code and malicious behavior through LLM. Use the training package as the input of LLM. Guide LLM to focus on malicious code fragments through prompts, analyze its malicious behavior, and summarize its characteristics. Figure 2The “Input” section shows a malicious JavaScript code that calls a function to obtain local information, packages the stolen information into JSON format data, and sends it to a remote server controlled by the attacker. Figure 2 The "Identifying Malicious Code in npm Packages" section shows the results of LLM's analysis of this malicious JavaScript code, extracting the code with malicious behavior and summarizing the behavior characteristics. LLM can not only detect malicious behavior (i.e. packaging user data and sending the package to the server), but also identify potential dangers of referencing third-party packages.

[0055] Furthermore, when using ChatGPT to analyze malicious behavior, an update prompt polling method is adopted. For a specific implementation method, refer to Figure 3 As shown: a third-party training package is taken as input, and ChatGPT's analysis results of its malicious behavior are output. ChatGPT is iteratively queried to analyze the malicious behavior of the training package. Specifically, prompts are first pre-generated for each file in the input training package, and then the generated prompts are used to query through the ChatGPT API, one file at a time. After obtaining a response from ChatGPT, the natural language output is parsed. "Program analysis" (result processing in the figure) is then applied to propagate the analysis results between files to address the input limitations of large models. The propagated results are further used to construct the next round of queries for ChatGPT, thereby improving over time. After convergence, the most appropriate set of malicious behaviors (such as Figure 2 for further processing as shown in the "Identifying Malicious Code in npm Packages" section in the previous section.

[0056] (1.2) Constructing a malicious code snippet set: By using a keyword extractor, the results of LLM are constructed into a malicious code snippet set with the following structure: “(third-party package name, malicious code snippet, malicious behavior)”. Figure 2 The "Building a Malicious Code Snippet Set" section in the 2013 paper shows how to use YAKE! (a commonly used keyword extraction method) to process Figure 2 Based on the results shown in the "Identifying Malicious Code in npm Packages" section, we extract the keywords "data leakage" and "remote code execution". The keywords "data leakage" and "remote code execution" effectively reflect the two types of malicious behaviors present in the sample. These keywords and their corresponding code snippets are included in the malicious code snippet set.

[0057] (1.3) Generate feature set: Merge similar malicious behaviors and summarize their behavioral characteristics, and generate feature sets based on the frequency of malicious behaviors in the code snippet set. Figure 2The "Summary Features" section in the figure shows the processing of the malicious code snippet set, by merging similar malicious behaviors (such as "process execution", "child process", "create process") and summarizing their behavioral characteristics. Most of the malicious code snippets of "data leakage" contain operating system-related APIs, and exploiting the operating system is an important feature. The code snippets of "remote code execution" generally contain strings such as IP addresses, and whether they contain IP addresses is a feature that needs attention.

[0058] (1.4) Extract feature vectors from malicious code in the training package: parse the third-party package through the compiler, convert the file into AST, traverse the nodes and match the malicious feature set, find the grammatical structure corresponding to the generated features, and then encode the extracted code segment into a vector.

[0059] The third-party package is parsed by the compiler. Since a set of malicious code snippets is constructed, similar code snippets (i.e., focus points) of all malicious codes with certain characteristics can be easily and systematically summarized, making the feature extractor more accurate in extracting malicious behaviors such as sensitive APIs. For example, for the malicious behavior of remote code execution in the malicious code snippet set, as many as 312 code snippets containing the eval function were identified. Therefore, eval is used as a focus point to extract the feature vector of useEval. Similarly, for the useNetwork feature, focus points that frequently appear in the malicious code snippet set are selected, such as http, request, node-fetch, etc. (Among them, eval, http, request, node-fetch, etc. are nodes in AST, and useEval and useNetwork are malicious features)

[0060] (1.5) Training the machine learning model: Input the vector obtained in step (1.4) and the corresponding type label into the machine learning model training, select different machine learning models and training parameters, select the best model through K-fold cross validation, and obtain the trained malicious packet detector.

[0061] Application phase:

[0062] Generate an abstract syntax tree AST corresponding to the third-party package to be detected, traverse the nodes of the AST to determine the grammatical structure corresponding to the features in the feature set, encode the code segment corresponding to the grammatical structure to obtain the feature vector to be detected, and input it into the trained machine learning model to obtain the detection result.

[0063] As a further preferred embodiment of the present invention, in the application stage, when the detection result of the third-party packet to be detected is a third-party malicious packet, it also includes:

[0064] Output the type of malicious behavior and the line where the malicious code is located;

[0065] The behavior where the malicious code is located is the row where the code segment corresponding to the grammatical structure corresponding to the feature in the feature set in the abstract syntax tree AST corresponding to the third-party package to be detected is located.

[0066] Specifically, based on AST, the malicious features contained in the target package (i.e., the third-party package to be detected) are identified, feature vectors of the malicious code are extracted, and the trained malicious package detector is used to classify the target package and locate the code segment containing malicious behavior, and a detection report is output, including:

[0067] (2.1) Based on AST, malicious features contained in the target package are identified. The AST is extracted by compiler parsing for the target package, the nodes are traversed and matched with the malicious feature set, and the grammatical structure corresponding to the generated features is found. It can be understood that the grammatical structure is composed of one or more nodes. The matching of the grammatical structure refers to the code fragments that frequently appear in the malicious fragment set, that is, the features in the feature set, that is, it is determined whether there are codes or code fragments in the code fragments contained in each node that are the same as the features in the feature set. If so, the grammatical structure composed of such nodes is the grammatical structure corresponding to the generated features.

[0068] (2.2) Extraction of feature vectors is similar to step (1.4), except that it is targeted at the target package;

[0069] (2.3) Output the detection report. According to the trained malicious packet detector outputted in step (1), classify the vector obtained in step (2.2). If the classification result is "1", it means that the corresponding packet contains malicious behavior. Finally, output all malicious code segments. If the classification result is "0", it means that the corresponding packet does not have malicious behavior.

[0070] The specific detection process of malicious packages is as follows: Figure 4 As shown. The target package has two files, package.json and index.js. When processing the package.json file, focus on the installation and execution of the script, and preinstall is recorded as a malicious feature value. index.js imports exec from the module child_process and then executes the shell command string. This command collects sensitive information and sends it to an external server. First, the API and strings that perform malicious behavior are extracted from the AST. Then the malicious behavior (i.e., creating processes and accessing domain names) is identified and feature vectors are extracted based on the feature set. Finally, the trained classification model receives the feature vector and outputs the detection result. While extracting the feature vector, the corresponding code lines are recorded. If it is a malicious package, all malicious code segments will be output.

[0071] When extracting the feature vector, the corresponding code lines are recorded. If it is a malicious package, the type of malicious behavior contained in it will be given in the detection report, and the corresponding malicious code lines will be marked.

[0072] In summary, the method provided by the present invention uses LLM to guide the identification of third-party malicious code fragments and the generation of features, which does not rely on expert definition, so that the feature set can better reflect the difference between malicious samples and benign samples, and has stronger expansibility. At the same time, a malicious code fragment set is constructed, and keywords corresponding to malicious behaviors are obtained through keyword extraction. A malicious feature set is constructed by extracting the code fragments with the highest frequency in the malicious code fragment set of each malicious behavior, and feature vectors are further extracted based on the malicious feature set, so that the extracted feature vectors are more accurate, and malicious behaviors in target packages are detected with lower false alarm rates and false negative rates, and the location of malicious code can be located.

[0073] An embodiment of the present invention provides a third-party malicious packet detection system based on large model feature generation, including: a computer-readable storage medium and a processor;

[0074] The computer-readable storage medium is used to store executable instructions;

[0075] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method described in any of the above embodiments.

[0076] An embodiment of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method described in any of the above embodiments.

[0077] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A third-party malicious packet detection method based on large model feature generation, characterized in that: include: Training phase: Inputting the training package into the large language model for malicious code identification to obtain the malicious code in the training package, extracting the features of the malicious code to construct a feature set; wherein the training package is a third-party malicious package; Generate an abstract syntax tree AST corresponding to the training package, traverse the nodes of the AST to determine a grammatical structure corresponding to the feature in the feature set, and encode the code segment corresponding to the grammatical structure to obtain a feature vector of the malicious code; Inputting the feature vector into a machine learning model for supervised training; Application phase: Generate an abstract syntax tree AST corresponding to the third-party package to be detected, traverse the nodes of the AST to determine the grammatical structure corresponding to the features in the feature set, encode the code segment corresponding to the grammatical structure to obtain a feature vector to be detected, and input it into the trained machine learning model to obtain a detection result; The large language model identifies malicious code in the training package under a preset prompt; The preset prompt is: determine the malicious code fragment, analyze its malicious behavior, and give the confidence level of the malicious behavior; The large language model uses an iterative query method to identify malicious code in the training package under preset prompts; Wherein, when performing the t+1th query, the recognition result of the tth query is added to the preset prompt in the form of an annotation; The extracting the features of the malicious code to construct a feature set includes: The malicious code corresponding to the malicious behavior that meets the confidence requirement is retained; Extracting keywords from malicious codes corresponding to each malicious behavior respectively as type identifiers of the malicious behaviors; The code fragment with the highest frequency of occurrence in the malicious code corresponding to each malicious behavior is used as a feature of the malicious code corresponding to each malicious behavior to obtain a feature set.

2. The method according to claim 1, characterized in that In the application stage, when the detection result of the third-party package to be detected is a third-party malicious package, it also includes: Output the type of malicious behavior and the line where the malicious code is located; The behavior where the malicious code is located is the row where the code segment corresponding to the grammatical structure corresponding to the feature in the feature set in the abstract syntax tree AST corresponding to the third-party package to be detected is located.

3. The method according to claim 1, characterized in that The large language model is ChatGPT, mT0 or Flan-PaLM.

4. The method according to claim 1, characterized in that The machine learning model is naive Bayes, multi-layer perceptron, random forest or support vector machine.

5. A third-party malicious packet detection system based on large model feature generation, characterized in that: include: A computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Malicious JavaScript code detection model based on Bi-LSTM network and attention mechanism

    CN112307473A

  • Method and device for generating malicious file detection rule and method and device for detecting malicious file

    CN115168851A

  • Data processing method and device, electronic equipment and storage medium

    CN117079299A

  • Multi-representational learning models for static analysis of source code

    US20210240825A1