A malicious code intelligent detection method and system based on large language model

Abstract

The present invention belongs to the field of network security technology, and its purpose is to provide a method and system for intelligent detection of malicious code based on a large language model. The method includes: obtaining a pre-trained large language model, and using the pre-trained large language model as an initial malicious code detection model; obtaining a malicious code detection sample task to fine-tune the parameters of the initial malicious code detection model to obtain a fine-tuned malicious code detection model; obtaining a specified prompt template to fine-tune the fine-tuned malicious code detection model to obtain a final malicious code detection model; obtaining the malicious code to be detected and its additional information, and constructing a malicious code detection task based on the malicious code to be detected and its additional information, so as to obtain a malicious code detection result that matches the malicious code to be detected with the final malicious code detection model. The present invention can reduce the cost of malicious code detection, and at the same time can realize real-time detection of network attacks, with a better detection rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular relates to a malicious code intelligent detection method and system based on a large language model. Background Art

[0002] With the in-depth development of computer technology, the Internet has become increasingly integrated into people's lives and brought great convenience, but it also provides many lawless elements with opportunities to exploit network vulnerabilities to create and spread Trojan viruses, steal users' privacy and sensitive information, and cause network paralysis, which has brought great obstacles to the development of the information society. Malicious code detection technology collects data generated during the use of operating systems, applications, etc., and analyzes and monitors the code in the running program to find behaviors that violate security policies or endanger system and network security in the system or local area network. It can detect hacker network attacks in a timely and effective manner, provide users with early warning information, and take timely security protection measures.

[0003] Since the malicious code detection technology was proposed and applied, it has evolved into a rich detection method after decades of development. At present, the most widely used malicious code detection technology is mainly based on deep learning technology. In 2023, PBhat et al. proposed a static analysis method that uses various machine learning and deep learning algorithms to detect malicious and benign Android applications. The results of the experiment found that the deep learning algorithms of DBN and MLP are superior to traditional machine learning algorithms in detecting malicious Android applications. PG Balikcioglu et al. used a combination of long short-term memory (LSTM) and natural language processing (NLP) to solve the problem of malware detection. They experimented with different basic units (instructions, basic blocks, methods, and classes) and source code obtained from three commonly used disassembly tools (JEB, IDA, and Apktool) and achieved ideal results. SC Ngwobia et al. trained the model proposed by the author based on a synthetic DNA dataset. The synthetic DNA dataset consists of salient features (dynamic and static) extracted from the behavior of non-malicious programs (.exe, .dll, etc.) before and after malware infection using reverse engineering and tracking tools. Experiments show that the model achieves 99% and 99.6% accuracy in detecting and classifying PE-oriented malware, achieving very good results.

[0004] However, in the process of using the prior art, the inventors found that the prior art has at least the following problems:

[0005] Deep learning technology requires a massive amount of data as input to achieve the ideal state of deep learning algorithms, and its complex network structure and a large number of parameters will consume a lot of computing power and system resources during model training and parameter adjustment, making the initial cost of applying this technology to the field of malicious code detection too high. At the same time, most of the current malicious code detection models lack the ability to continuously learn and can only detect the network attack types inherent in the training resources, but lack good detection capabilities for new attacks, making it difficult to follow up on the increasingly complex and diverse attack styles in the network. Summary of the invention

[0006] The present invention aims to solve the above technical problems at least to a certain extent. The present invention provides a malicious code intelligent detection method, system, device and product based on a large language model.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a malicious code intelligent detection method based on a large language model, comprising:

[0009] Obtain a pre-trained large language model, and use the pre-trained large language model as an initial malicious code detection model;

[0010] Acquire a malicious code detection sample task, and fine-tune the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model;

[0011] Obtaining a specified prompt template, and fine-tuning the fine-tuned malicious code detection model based on the specified prompt template to obtain a final malicious code detection model;

[0012] Acquire the malicious code to be detected and its additional information, and construct a malicious code detection task according to the malicious code to be detected and its additional information;

[0013] The malicious code detection task is input into the final malicious code detection model to obtain a malicious code detection result that matches the malicious code to be detected.

[0014] In a possible design, the pre-trained large language model adopts the CodeBERT model.

[0015] In a possible design, the malicious code detection sample task uses the EMBER dataset.

[0016] In a possible design, fine-tuning parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model includes:

[0017] Adding a bypass module to the self-attention head part of the Transformer layer of the initial malicious code detection model;

[0018] Freezing the model parameters of the initial malicious code detection model, and training the bypass module based on the malicious code detection sample task to obtain the trained bypass parameters of the bypass module;

[0019] Combining the trained bypass parameters with the model parameters to obtain fine-tuned model parameters;

[0020] The model parameters of the initial malicious code detection model are replaced with the fine-tuned model parameters to obtain a fine-tuned malicious code detection model.

[0021] In one possible design, the post-training bypass parameters of the bypass module include a post-training dimension reduction matrix and a post-training dimension increase matrix;

[0022] The model parameters after fine-tuning are:

[0023] ;

[0024] In the formula, are the model parameters, is the number of matrix rows of the model parameters, is the number of matrix columns of the model parameters; is the post-training bypass parameter; is the post-training dimension reduction matrix in the post-training bypass parameters; is the post-training dimension-raising matrix in the post-training bypass parameter; is a hyperparameter, is the dimension reduction matrix after training And the dimension-enhanced matrix after training The preset rank.

[0025] In a possible design, after obtaining the final malicious code detection model, the method further includes:

[0026] A specified function code is obtained so that the final malicious code detection model calls the specified function code.

[0027] In a possible design, the malicious code detection result includes the code type, attack type and judgment basis of the malicious code to be detected.

[0028] In a second aspect, the present invention provides a malicious code intelligent detection system based on a large language model, which is used to implement the malicious code intelligent detection method based on a large language model as described in any one of the above; the malicious code intelligent detection system based on a large language model includes:

[0029] An initial model acquisition module, used to acquire a pre-trained large language model and use the pre-trained large language model as an initial malicious code detection model;

[0030] A model fine-tuning module, which is in communication with the initial model acquisition module, is used to acquire a malicious code detection sample task, and fine-tune the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model;

[0031] A model fine-tuning module, which is in communication with the model fine-tuning module and is used to obtain a specified prompt template and fine-tune the fine-tuned malicious code detection model based on the specified prompt template to obtain a final malicious code detection model;

[0032] A task acquisition module to be detected, which is in communication with the model fine-tuning module, is used to obtain the malicious code to be detected and its additional information, and construct a malicious code detection task according to the malicious code to be detected and its additional information;

[0033] The detection module is in communication with the to-be-detected task acquisition module and is used to input the malicious code detection task into the final malicious code detection model so as to obtain a malicious code detection result that matches the to-be-detected malicious code.

[0034] In a third aspect, the present invention provides an electronic device, comprising:

[0035] a memory for storing computer program instructions; and,

[0036] A processor is used to execute the computer program instructions to complete the operation of the malicious code intelligent detection method based on a large language model as described in any one of the above.

[0037] In a fourth aspect, the present invention provides a computer program product, including a computer program or instructions, which, when executed by a computer, implements the intelligent malicious code detection method based on a large language model as described in any one of the above.

[0038] The beneficial effects of the present invention are:

[0039] The present invention discloses a method, system, device and product for intelligent detection of malicious code based on a large language model, which can reduce the cost of malicious code detection, realize real-time detection of network attacks and have a better detection rate. Specifically, in view of the problem that network attacks mutate quickly and traditional models are difficult to update knowledge and detect new attacks, the present invention fine-tunes and adjusts the parameters of the pre-trained large language model to obtain the final malicious code detection model. In this process, the acquisition cost of the final malicious code detection model is low. At the same time, the advantages of the large language model technology in context understanding and feature capture are utilized, and the pre-trained large language model is fine-tuned using incremental knowledge, and the model is further fine-tuned using a specified prompt template, so that the final malicious code detection model can have an evolution speed comparable to the attack and has a good detection effect on new attacks.

[0040] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the malicious code intelligent detection method based on a large language model in Example 1;

[0042] Figure 2 It is a module block diagram of the malicious code intelligent detection system based on the large language model in Example 2;

[0043] Figure 3 This is a module block diagram of the electronic device in Example 3. DETAILED DESCRIPTION

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0045] Embodiment 1:

[0046] This embodiment discloses a malicious code intelligent detection method based on a large language model, which can be executed by, but is not limited to, a computer device or a virtual machine with certain computing resources, such as a personal computer, a smart phone, a personal digital assistant, or a wearable device, or by a virtual machine.

[0047] like Figure 1As shown, a malicious code intelligent detection method based on a large language model may include but is not limited to the following steps:

[0048] S1. Obtain a pre-trained large language model, and use the pre-trained large language model as an initial malicious code detection model.

[0049] It should be noted that pre-training a large language model usually requires a massive data set and a large amount of computing resources so that the malicious code detection model generated based on the large language model in this embodiment can learn the semantics of words, the grammatical structure of sentences, basic knowledge and contextual understanding capabilities. In order to effectively improve the adaptability of the malicious code detection model, in this embodiment, the pre-trained large language model adopts the CodeBERT model. In this embodiment, the CodeBERT model uses a multi-layer bidirectional Transformer as the model structure, including 12 Transformer layers, each layer has 12 self-attention heads, the size of each self-attention head is 64, the hidden dimension is 768, the internal hidden layer size of the feedforward layer is 3072, and the total parameters of the model are 125 million.

[0050] It should be noted that the CodeBERT model is an extension of the BERT (Bidirectional Encoder Representations from Transformers) model developed by Microsoft in 2020. It is a bimodal pre-trained large language model for programming languages ​​(PL) and natural languages ​​(NL). It can perform downstream NL-PL (natural language-programming language) tasks. It uses 6 programming languages ​​(Python, Java, JavaScript, PHP, Ruby and Go) for NL-PL matching training. The CodeBERT model has achieved good results in both natural language code search tasks and code document generation tasks, and although the CodeBERT model is only pre-trained on code languages ​​such as Ruby, JavaScript, Go, Python, Java, and PHP, its pre-trained model can be generalized to other code language tasks. Based on this, the excellent performance of the CodeBERT model is related to the malicious code detection task of this embodiment, and selecting it as a pre-trained large language model can improve the performance of the malicious code detection model in this embodiment.

[0051] S2. Obtain a malicious code detection sample task, and fine-tune the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model.

[0052] The malicious code detection sample task uses the EMBER dataset. It should be noted that the EMBER dataset is a large open source dataset released by the cybersecurity company Endgame. It is extracted from a large corpus of malicious and benign Windows Portable Executable (PE) files and contains more than 1 million training samples. Based on this, the efficiency of fine-tuning the parameters of the initial malicious code detection model can be improved.

[0053] Specifically, in step S2, fine-tuning the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model includes:

[0054] S201. Add a bypass module to the self-attention head part of the Transformer layer of the initial malicious code detection model;

[0055] S202. Freeze the model parameters of the initial malicious code detection model, and train the bypass module based on the malicious code detection sample task to obtain the bypass parameters of the bypass module after training; it should be noted that the bypass parameters of the bypass module are divided into a dimension reduction matrix and a dimension increase matrix, and the number of rows and columns of the product of the dimension reduction matrix and the dimension increase matrix are the same as the number of rows and columns of the model parameters to ensure that the data dimension of the input data and the output data to the bypass model does not change. In addition, before training the bypass module, this embodiment pre-initializes the dimension reduction matrix of the bypass module with a random Gaussian distribution, and initializes the dimension increase matrix of the bypass module with a 0 matrix to ensure that the bypass parameter is still a 0 matrix at the beginning of training.

[0056] Specifically, in step S202, the post-training bypass parameters of the bypass module include a post-training dimension reduction matrix and a post-training dimension increase matrix;

[0057] Correspondingly, in step S203, the fine-tuned model parameters are:

[0058] ;

[0059] In the formula, are the model parameters, is the number of matrix rows of the model parameters, is the number of matrix columns of the model parameters; is the post-training bypass parameter; is the post-training dimension reduction matrix in the post-training bypass parameters; is the post-training dimension-raising matrix in the post-training bypass parameter; is a hyperparameter, is the dimension reduction matrix after training And the dimension-enhanced matrix after training of the preset rank, and .

[0060] That is, in the process of fine-tuning the parameters of the initial malicious code detection model in this embodiment, the model parameters It is fixed and unchanged, and only the dimension reduction matrix and dimension increase matrix in the bypass parameter are updated to obtain the dimension reduction matrix after training And the dimension-enhanced matrix after training .

[0061] S203. Combining the trained bypass parameters with the model parameters to obtain fine-tuned model parameters;

[0062] S204. Replace the model parameters of the initial malicious code detection model with the fine-tuned model parameters to obtain a fine-tuned malicious code detection model.

[0063] It should be noted that, in this embodiment, the bypass module is mainly used to perform dimensionality reduction and then dimensionality increase operations, so that the initial malicious code detection model can better adapt to new data sets in specific tasks; specifically, in the process of fine-tuning the parameters of the initial malicious code detection model, this embodiment only updates the bypass parameters of the bypass module, without updating the parameters of the entire initial malicious code detection model.

[0064] In order to make the initial malicious code detection model applicable to malicious code detection scenarios, it is necessary to adjust its parameters. At this time, if full parameter adjustment is adopted, the cost of adjusting the parameters of the initial malicious code detection model is relatively high due to the large number of parameters that can be trained in the initial malicious code detection model. Based on the above steps S201-S204, this embodiment can use LORA (Low-Rank Adaptation) technology to fine-tune the parameters of the initial malicious code detection model. In this process, only the dimension reduction matrix and dimension increase matrix in the bypass parameter need to be updated to obtain the dimension reduction matrix after training. And the dimension-enhanced matrix after training , and then the dimension reduction matrix after training And the dimension-enhanced matrix after training The matrix product of the model parameters The original model parameters in the initial malicious code detection model can be replaced by merging them into one as new model parameters, without adding additional computing resources, and the number of trainable parameters can be significantly reduced.

[0065] S3. Obtain a specified prompt template, and fine-tune the fine-tuned malicious code detection model based on the specified prompt template to obtain a final malicious code detection model.

[0066] It should be noted that in the process of fine-tuning the fine-tuned malicious code detection model using the specified prompt template, the focus is on adjusting the input prompt rather than modifying the model parameters. Specifically, in this embodiment, a variety of prompt templates suitable for network attacks in different scenarios can be prepared in advance according to the specific fields and attack methods to be detected for malicious code. When fine-tuning the fine-tuned malicious code detection model based on the specified prompt template, the optimization of the prompt template and the induction of the fine-tuned malicious code detection model to focus on the key information related to the task can make the understanding and answer of the fine-tuned malicious code detection model more biased, better exert the capabilities related to the target task, and thus better adapt to the downstream malicious code detection tasks.

[0067] As an example, the specified prompt template in this embodiment is as follows:

[0068] You are a cybersecurity expert tasked with detecting software to determine whether it is legitimate or malware. To complete this task, perform the following subtasks:

[0069] (1) Learn the rules of specific software features: We will provide you with multiple sets of specific software feature sets and their judgment labels. If it is a feature of benign software, the label is 0, otherwise there will be a corresponding malware type name. You can learn the rules of this set of feature sets based on your own existing knowledge and give the analysis results.

[0070] (2) Analyze the given software features: We will provide you with a set of features of the software to be tested without label identification. Please analyze the feature patterns of the software samples to be tested and give the analysis results. If you do not find any pattern that matches what you know, please answer "unknown".

[0071] (3) Determine the nature of the software: Please give the classification results based on the characteristics of the software to be tested and explain your basis. If there is not enough evidence to make a judgment, answer "unknown".

[0072] (4) Output your judgment results in JSON format, which contains the following keywords:

[0073] Is_true: boolean, used to indicate whether you are sure of your answer. If you are sure that your answer has sufficient evidence and is logically reasonable, output "True". If you are not sure of your answer or cannot judge at all, output "False".

[0074] Software_label: string, used to indicate your judgment on the type of the software sample to be tested. If it is benign, output "benign software", otherwise output "malware {}", and fill in the specific type name of the malware in {}.

[0075] Analysis_report: string, used to output your analysis report. Please explain in the analysis report the type of software sample you have determined and the basis for your judgment; for the uncertain software sample, explain your doubts; for the software sample that cannot be judged at all, explain the evidence you think is missing.

[0076] In addition, after obtaining the final malicious code detection model, the inventor conducted a detection effect experiment on the final malicious code detection model. Specifically, in this embodiment, the detection effect experiment can be divided into two stages, wherein the first stage of the experiment is used to obtain the detection effect of the final malicious code detection model on labeled data, at which time known malicious code data can be input. Through the first stage of the experiment, the inventor found that the final malicious code detection model can accurately determine the type of attack with high accuracy, and can provide detailed judgment reasons and a certain degree of attack process restoration, which is more helpful for the next step of resisting and defending against the attack while ensuring the detection quality; the second stage of the experiment is used to obtain the detection effect of the final malicious code detection model on network code data in real scenarios. In this process, the running code and real experimental code of the application captured by the experimental server within a certain period of time are input to the final malicious code detection model. The experiment proves that for streaming data within a long period of time, the final malicious code detection model obtained based on this embodiment can still show good continuous detection capabilities, can continuously analyze the logic and purpose of the input code within a reasonable delay range, and make timely analysis and warnings for possible malicious code attacks, and provide corresponding protection mechanisms, achieving good detection expectations.

[0077] In step S3, after obtaining the final malicious code detection model, the method further includes:

[0078] A specified function code is obtained so that the final malicious code detection model calls the specified function code.

[0079] Specifically, the designated function code includes functions such as autonomous Internet search for relevant information, PDF text output, and knowledge crawling. The final malicious code detection model can call the pre-set designated function code according to the needs in the subtask output information, such as consulting the latest information related to the subtask as new input for information supplement according to its own needs, so that the final malicious code detection model has the auxiliary capabilities that match the designated function code, and the final malicious code detection model can be updated in real time.

[0080] In this embodiment, in order to enable the final malicious code detection model to have the ability to detect new network attacks and timely update the data reserve of the final malicious code detection model, the input and output data in step S2 and step S3 are processed as fine-tuning data, and the specified function code is used to allow the final malicious code detection model to search the Internet for the latest relevant valid knowledge or data set as a new input fine-tuning model. Compared with the fine-tuning of the final malicious code detection model in step S2, this fine-tuning stage has fewer rounds, a simpler process, and consumes fewer resources, but can effectively update the knowledge base of the model, further enhancing the professional ability and detection efficiency of the final malicious code detection model in the field of malicious code detection.

[0081] In order to verify the incremental knowledge learning effect of the final malicious code detection model, the final malicious code detection model was also subjected to experiments in the real-time incremental knowledge learning phase and the variant attack detection phase. The inventor used the variant attack of the malicious code in the previous phase of the experiment as input. The experiment found that the final malicious code detection model can still identify the type and target of new attacks with a high accuracy, and can also be associated with the original attack means of the variant attack. It can not only ensure the accuracy of the detection of new attacks, but also sort out the variant process of the attack and even predict the attacks that may be encountered. The final malicious code detection model in this experimental phase achieved good expected detection results.

[0082] S4. Obtain the malicious code to be detected and its additional information, and construct a malicious code detection task based on the malicious code to be detected and its additional information; it should be noted that by pre-constructing the malicious code detection task, the final malicious code detection model can clearly understand the intention of malicious code detection and make full use of its malicious code detection function.

[0083] Specifically, in this embodiment, in the process of performing malicious code detection using the final malicious code detection model, the malicious code to be detected and its additional information are merged in advance to construct the malicious code detection task for this time, and then the malicious code detection task is uploaded to the final malicious code detection model to realize the malicious code detection function.

[0084] Specifically, in this embodiment, the example code block when constructing a malicious code detection task is as follows:

[0085] ai_goals:

[0086] - Perform logic analysis and malware detection on the code in the uploaded malicious code files

[0087] - Write the code detection report to security.doc

[0088] ai_name: Malicious code detection task 1

[0089] ai_role: A professional malware analyst. You can use the necessary commands in "ai_commonds" to assist in malware detection. The format of your reply content must follow the format requirements in "ai_format".

[0090] ai_commonds:

[0091] 1. analyze_code: Analyze Code, args: "code": "<full_code_string> "

[0092] 2. execute_python_file: Execute Python File, args: "filename": " <filename>"

[0093] 3. append_to_file: Append to file, args: "filename": " <filename>", "doc": " <doc>"

[0094] 4. delete_file: Delete file, args: "filename": " <filename>"

[0095] 5. list_files: List Files in Directory, args: "directory": " <directory>"

[0096] 6. read_file: Read file, args: "filename": " <filename>"

[0097] 7. write_to_file: Write to file, args: "filename": " <filename>", "text": " <text>"

[0098] 8. google: Google Search, args: "query": " <query>"

[0099] 9. improve_code: Get Improved Code, args: "suggestions": "<list_of_suggestions>", "code": "<full_code_string>"

[0100] 10. browse_website: Browse Website, args: "url": " <url>", "question":"<what_you_want_to_find_on_website>"

[0101] 11. get_text_summary: Get text summary, args: "url": " <url>", "question": " <question>"

[0102] 12. Task Complete (Shutdown): "task_complete", args: "reason": " <reason>"

[0103] ai_format:

[0104] Response Format:

[0105] {"thoughts": {

[0106] "text": "thought",

[0107] "reasoning": "reasoning",

[0108] "plan": "- short bulleted\n- list that conveys\n- long-term plan",

[0109] "finished": "finished? Output report and Task Complete: Continuetask"

[0110] },

[0111] "command": {

[0112] "name": "command name",

[0113] "args": {

[0114] "arg name": "value"

[0115] }}}

[0116] Ensure the response can be parsed by Python json.loads

[0117] The parameters in the above code block are explained as follows:

[0118] ai_goals: indicates the goals to be achieved in this malicious code detection task.

[0119] ai_name: The name of this malicious code detection task.

[0120] ai_role: In order to give full play to the detection performance of the final malicious code detection model and set necessary requirements for it, the role of the final malicious code detection model is set so that the final malicious code detection model can successfully complete the detection task and output the detection report.

[0121] ai_commonds: This part is the function code we programmed ourselves. We set the command name, main function and input format requirements in English. When the final malicious code detection model is performing detection, it can call the corresponding command interface according to its specific needs, such as searching the Internet for content, executing Python files, etc. The returned results will be input into the final malicious code detection model again to help it verify its judgment results and continue the detection task.

[0122] ai_format: This part is the set return format of the final malicious code detection model result. During the analysis process of the final malicious code detection model, it will divide the malicious code analysis into multiple subtasks. Different subtasks may correspond to different command call requirements. Therefore, it is necessary to standardize the single output format of the final malicious code detection model and make necessary judgments such as whether the detection task is completed. If it is completed, the detection report is output; if the task is not completed, the obtained results will continue to be input into the final malicious code detection model to continue the detection task.

[0123] S5. Input the malicious code detection task into the final malicious code detection model to obtain a malicious code detection result that matches the malicious code to be detected. In this embodiment, the malicious code detection result can be output in the form of a detection report for user viewing.

[0124] It should be noted that in the analysis module of the final malicious code detection model, it is necessary to further add a necessary caching mechanism, so that the final malicious code detection model can obtain the context before and after the execution of different subtasks, so as to obtain better and more global results. At the same time, it can also avoid the problem of repeated execution and ensure the continuity of the detection process.

[0125] In step S5 of this embodiment, the malicious code detection result includes the code type, attack type and judgment basis of the malicious code to be detected. Specifically, in this embodiment, the judgment basis can also be bound to the data features of the malicious code to be detected. According to the setting of the malicious code detection result in this embodiment, compared with the traditional result display type answer, it can assist security maintenance personnel to more accurately know the cause of the attack and locate the attacked target, and then make precise protection.

[0126] This embodiment can reduce the cost of malicious code detection, and can also realize real-time detection of network attacks, with a better detection rate. Specifically, in view of the problem that network attacks mutate quickly and traditional models are difficult to update knowledge and detect new attacks, this embodiment fine-tunes and fine-tunes the parameters of the pre-trained large language model to obtain the final malicious code detection model. In this process, the acquisition cost of the final malicious code detection model is low. At the same time, the advantages of the large language model technology context understanding and feature capture are utilized, and the pre-trained large language model is fine-tuned using incremental knowledge, and the model is further fine-tuned using a specified prompt template, so that the final malicious code detection model can have an evolution speed comparable to the attack, and has a good detection effect on new attacks. In addition, based on the characteristics of the large language model, the final malicious code detection model of this embodiment can not only provide malicious code detection results, but also provide information such as judgment basis and protection measures, and make predictions to a certain extent, which can effectively help security maintenance personnel resist and protect against immediate network attacks.

[0127] Embodiment 2:

[0128] This embodiment discloses a malicious code intelligent detection system based on a large language model, which is used to implement the malicious code intelligent detection method based on a large language model in Embodiment 1; Figure 2 As shown, the malicious code intelligent detection system based on the large language model includes:

[0129] An initial model acquisition module, used to acquire a pre-trained large language model and use the pre-trained large language model as an initial malicious code detection model;

[0130] A model fine-tuning module, which is in communication with the initial model acquisition module, is used to acquire a malicious code detection sample task, and fine-tune the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model;

[0131] A model fine-tuning module, which is in communication with the model fine-tuning module and is used to obtain a specified prompt template and fine-tune the fine-tuned malicious code detection model based on the specified prompt template to obtain a final malicious code detection model;

[0132] A task acquisition module to be detected, which is in communication with the model fine-tuning module, is used to obtain the malicious code to be detected and its additional information, and construct a malicious code detection task according to the malicious code to be detected and its additional information;

[0133] The detection module is in communication with the to-be-detected task acquisition module and is used to input the malicious code detection task into the final malicious code detection model so as to obtain a malicious code detection result that matches the to-be-detected malicious code.

[0134] It should be noted that the working process, working details and technical effects of the malicious code intelligent detection system based on a large language model provided in this embodiment 2 can be found in embodiment 1 and will not be repeated here.

[0135] Embodiment 3:

[0136] Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smart phone, a tablet computer, a laptop computer, or a desktop computer. The electronic device may be referred to as a user terminal, a portable terminal, a desktop terminal, etc. Figure 3 As shown, the electronic equipment includes:

[0137] a memory for storing computer program instructions; and,

[0138] A processor is used to execute the computer program instructions to complete the operation of the malicious code intelligent detection method based on a large language model as described in any one of Embodiment 1.

[0139] Specifically, the processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen.

[0140] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 302 is used to store at least one instruction, which is used to be executed by the processor 301 to implement the malicious code intelligent detection method based on a large language model provided in Example 1 of the present application.

[0141] In some embodiments, the terminal may further optionally include: a communication interface 303 and at least one peripheral device. The processor 301, the memory 302 and the communication interface 303 may be connected via a bus or a signal line. Each peripheral device may be connected to the communication interface 303 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 304, a display screen 305 and a power supply 306.

[0142] The communication interface 303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the communication interface 303 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0143] The radio frequency circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 304 communicates with a communication network and other communication devices through electromagnetic signals.

[0144] The display screen 305 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof.

[0145] The power supply 306 is used to supply power to various components in the electronic device.

[0146] Embodiment 4:

[0147] Based on any one of Examples 1 to 3, this embodiment discloses a computer program product, including a computer program or instructions, which, when executed by a computer, implements the intelligent malicious code detection method based on a large language model as described in any one of Example 1.

[0148] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / reason> < / question> < / url> < / url> < / query> < / text> < / filename> < / filename> < / directory> < / filename> < / doc> < / filename> < / filename>

Claims

1. A malicious code intelligent detection method based on a large language model, characterized in that: include: Obtain a pre-trained large language model, and use the pre-trained large language model as an initial malicious code detection model; Acquire a malicious code detection sample task, and fine-tune the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model; Obtaining a specified prompt template, and fine-tuning the fine-tuned malicious code detection model based on the specified prompt template to obtain a final malicious code detection model; Acquire the malicious code to be detected and its additional information, and construct a malicious code detection task according to the malicious code to be detected and its additional information; Inputting the malicious code detection task into the final malicious code detection model to obtain a malicious code detection result matching the malicious code to be detected; The pre-trained large language model adopts the CodeBERT model; Fine-tuning the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model, including: Adding a bypass module to the self-attention head part of the Transformer layer of the initial malicious code detection model; Freezing the model parameters of the initial malicious code detection model, and training the bypass module based on the malicious code detection sample task to obtain the trained bypass parameters of the bypass module; Combining the trained bypass parameters with the model parameters to obtain fine-tuned model parameters; Replacing the model parameters of the initial malicious code detection model with the fine-tuned model parameters to obtain a fine-tuned malicious code detection model; The post-training bypass parameters of the bypass module include a post-training dimension reduction matrix and a post-training dimension increase matrix; The model parameters after fine-tuning are: ； ； In the formula, W 0 is the model parameter, d is the number of matrix rows of the model parameters, k is the matrix column number of the model parameters; Δ W is the post-training bypass parameter; A is the post-training dimension reduction matrix in the post-training bypass parameters; B is the post-training dimension-raising matrix in the post-training bypass parameter; r is a hyperparameter, is the dimension reduction matrix after training A And the dimension-enhanced matrix after training B The preset rank.

2. According to claim 1, a malicious code intelligent detection method based on a large language model is characterized in that: The malicious code detection sample task uses the EMBER dataset.

3. According to claim 1, a malicious code intelligent detection method based on a large language model is characterized in that: After obtaining the final malicious code detection model, the method further includes: A specified function code is obtained so that the final malicious code detection model calls the specified function code.

4. According to claim 1, a malicious code intelligent detection method based on a large language model is characterized in that: The malicious code detection result includes the code type, attack type and judgment basis of the malicious code to be detected.

5. A malicious code intelligent detection system based on a large language model, characterized in that: include: An initial model acquisition module, used to acquire a pre-trained large language model and use the pre-trained large language model as an initial malicious code detection model; A model fine-tuning module, which is in communication with the initial model acquisition module, is used to acquire a malicious code detection sample task, and fine-tune the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model; A model fine-tuning module, which is in communication with the model fine-tuning module and is used to obtain a specified prompt template and fine-tune the fine-tuned malicious code detection model based on the specified prompt template to obtain a final malicious code detection model; A task acquisition module to be detected, which is in communication with the model fine-tuning module, is used to obtain the malicious code to be detected and its additional information, and construct a malicious code detection task according to the malicious code to be detected and its additional information; A detection module, which is in communication with the to-be-detected task acquisition module and is used to input the malicious code detection task into the final malicious code detection model so as to obtain a malicious code detection result that matches the to-be-detected malicious code; The pre-trained large language model adopts the CodeBERT model; Fine-tuning the parameters of the initial malicious code detection model based on the malicious code detection sample task to obtain a fine-tuned malicious code detection model, including: Adding a bypass module to the self-attention head part of the Transformer layer of the initial malicious code detection model; Freezing the model parameters of the initial malicious code detection model, and training the bypass module based on the malicious code detection sample task to obtain the trained bypass parameters of the bypass module; Combining the trained bypass parameters with the model parameters to obtain fine-tuned model parameters; Replacing the model parameters of the initial malicious code detection model with the fine-tuned model parameters to obtain a fine-tuned malicious code detection model; The post-training bypass parameters of the bypass module include a post-training dimension reduction matrix and a post-training dimension increase matrix; The model parameters after fine-tuning are: ； ； In the formula, W 0 is the model parameter, d is the number of matrix rows of the model parameters, k is the matrix column number of the model parameters; Δ W is the post-training bypass parameter; A is the post-training dimension reduction matrix in the post-training bypass parameters; B is the post-training dimension-raising matrix in the post-training bypass parameter; r is a hyperparameter, is the dimension reduction matrix after training A And the dimension-enhanced matrix after training B The preset rank.

Images