A malicious sample encryption process tracking method based on a large language model guide

Abstract

A malicious sample encryption process tracking method based on large language model guidance, potential encryption features in a binary malicious sample are extracted through static analysis technology, and the related function call relationship is identified, then, the large language model is used to automatically screen and reason the disassembled code, and the irrelevant execution path is eliminated, and the key function subset related to key generation and encryption operation is screened out. Then, the screened function subset is taken as input and is given to the angr symbolic execution framework for further analysis, through the marking of the key function parameters, dynamic taint analysis and path exploration are carried out, and important links in the encryption process are accurately identified. Finally, the automatic identification of the whole process of malicious sample encryption is realized, and the storage positions of the encryption key and sensitive data can be accurately located. The present application effectively solves the "path explosion" problem in the traditional symbolic execution method, and significantly improves the analysis efficiency of the complex encryption type of confrontation behavior.

Description

Technical Field

[0001] This invention belongs to the field of software security technology, specifically relating to binary program analysis, malicious code detection, and the application of Large Language Models (LLM) in automated reverse engineering, and particularly to a method for using Large Language Models to guide symbolic execution to identify the encryption process of malicious samples. Background Technology

[0002] With the evolution of malware countermeasures, an increasing number of malicious software programs (such as ransomware and data-stealing Trojans) employ high-strength encryption algorithms to protect their critical instructions, configuration information, and communication data, evading static scanning and signature detection by antivirus software. Current analysis methods face the following challenges: First, traditional manual reverse engineering relies heavily on analyst experience, resulting in low efficiency when dealing with massive numbers of malware variants and hindering automation. Second, when using tools like symbolic execution for automated analysis, the complex conditional branches within binary programs can lead to an exponential increase in execution paths, easily triggering a "path explosion" problem, exhausting computing resources and making it difficult to reach the core encryption logic. Third, traditional automated analysis tools lack an understanding of the macroscopic semantics of disassembled code, making it difficult to directly reconstruct complex encryption logic and algorithmic intent without source code. Therefore, how to combine artificial intelligence technology to improve the automated tracking and identification capabilities of the entire malware encryption process has become a critical issue that urgently needs to be addressed in the current cybersecurity field. Summary of the Invention

[0003] The purpose of this invention is to provide a method for tracing the encryption process of malicious samples based on a large language model, enabling automated identification and key extraction of malicious sample encryption adversarial behaviors. This addresses the problems of "path explosion" caused by the excessively large execution paths in traditional symbolic execution analysis of malicious samples, as well as the low efficiency of manual reverse engineering.

[0004] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0005] In a first aspect, the present invention provides a method for tracking the encryption process of malicious samples based on a large language model, the method comprising the following steps:

[0006] S1. Static Feature Extraction and Localization: Static scanning is performed on binary malicious samples to extract static features related to encryption algorithms, including encryption constants, S-Box lookup tables and API call information, and candidate encryption functions are identified based on the static features;

[0007] In step S1, the extraction of encrypted constants includes identifying single-valued constants and continuous S-Box / T-Table lookup tables. The identification process is implemented by matching a predefined encryption algorithm feature library, which contains constant signatures and table structure features of common encryption algorithms such as AES, TEA, RC4, and ChaCha20. For the identified S-Box / T-Table, cross-reference analysis is further used to associate it with the functions that reference the table.

[0008] S2. LLM Semantic Guidance and Blueprint Construction: Static data flow analysis is performed on the candidate encryption functions described in step S1 to construct variable passing paths within and across functions; at the same time, semantic reasoning is performed on the disassembled code and data flow context of the functions using a large language model to filter out a subset of target functions involving key processing and core encryption operations, and to generate a guidance blueprint;

[0009] In step S2, the static data flow analysis is performed using an intermediate language in the form of Static Single Assignment (SSA), including: constructing a Def-Use chain inside the function to trace the definition and usage relationship of variables; establishing a data flow propagation path in non-stack memory by analyzing the address expressions of memory storage (STORE) and loading (LOAD) instructions; tracing the caller upwards along the function call graph and prioritizing it according to the number of cross-references of the caller, giving priority to analyzing call paths with close business logic; the generated bootstrapping blueprint includes the target function address, key execution path nodes, taint source (Source), and sink (Sink) information;

[0010] The process of using a large language model to perform semantic reasoning on the disassembled code and data flow context of a function includes: constructing a prompt containing a function disassembled code fragment, data flow evidence (Source / Sink API category), and preliminary algorithm identification results; inputting the prompt into the large language model, and requiring the model to output a judgment result on the encryption algorithm type and malicious behavior scenario classification; the malicious behavior scenario classification includes at least payload decryption loading, C2 command execution, data exfiltration, and ransomware encryption.

[0011] S3. Guided Dynamic Symbolic Execution: Based on the guiding blueprint described in step S2, the target function is dynamically analyzed using a symbolic execution engine. The path explosion problem is mitigated by actively triggering detection at the guiding address, limiting the number of loop unrolls, and pruning low-value states based on a scoring machine. Simultaneously, memory read / write operations are monitored during symbolic execution to extract encryption keys and confirm data flow. This includes: initializing the symbolic execution state and injecting symbolic taints into the input parameters of the target function; during symbolic execution, real-time checks are performed to see if the instruction address of the current execution state hits a critical path node in the guiding blueprint; if so, a full sink check is performed on the current state and a high-priority score is given; the number of symbolic executions of the loop structure is limited using the LoopSeer mechanism, and the access frequency of basic blocks is statistically analyzed; when the access frequency exceeds a preset threshold, the exploration of the current path is forcibly terminated; the active state set is dynamically pruned based on a state scoring mechanism, which weights states according to whether they hit an API, whether they are on the guiding path, and the novelty of the path exploration.

[0012] The extraction of the encryption key includes: registering a memory write monitoring hook during symbolic execution; capturing the written data content and target address when a memory write operation is detected; performing entropy calculation and character distribution analysis on the written data content; and marking the data as a candidate key if the data length, entropy value, and character distribution match the key characteristics and the write location is close to the key management API call point.

[0013] S4. Comprehensive analysis of the attack chain: Combining the static features of step S1, the semantic filtering results of step S2, and the dynamic symbolic execution evidence of step S3, the encryption algorithm type is determined through multi-layer cross-validation, and a malicious behavior attack chain is constructed based on data flow connectivity.

[0014] In step S4, the multi-layer cross-validation to determine the encryption algorithm type specifically includes: Layer A, a first-layer voting based on the YARA rule hits, S-Box matching, and constant features extracted in step S1; Layer B, a second-layer voting based on the instruction opcode histogram and SSA data stream features extracted in step S2; the opcode histogram statistically analyzes the distribution ratio of instructions such as XOR, SHIFT, and ROTATE, and matches it with a preset encryption algorithm fingerprint template; Layer C, a third-layer voting based on the runtime constants and S-Box access behavior detected during dynamic execution in step S3; the results of the above three layers of voting are weighted and summarized to determine the final encryption algorithm type and confidence level;

[0015] The construction of the malicious attack chain includes: defining the three-segment structure of the attack chain as: Source → Crypto → Sink; checking whether there is a complete data flow connectivity from the source API to the crypto function and then to the sink API in the dynamic symbolic execution path of step S3; if a complete connectivity exists, the attack chain is marked as verified, and the attack behavior is classified into a specific malicious scenario according to the API category combination of the source and sink.

[0016] Secondly, the present invention provides a malicious sample encryption process tracking system based on a large language model to implement the method of the present invention. The system includes: a static feature extraction module, used to execute step S1, extracting static features related to the encryption algorithm and identifying candidate encryption functions; a semantic analysis and blueprint generation module, used to execute step S2, combining SSA data flow analysis and large language model inference to generate a guiding blueprint; a dynamic symbol execution module, used to execute step S3, exploring paths and extracting encryption keys based on the guiding blueprint; and a comprehensive judgment module, used to execute step S4, outputting a report on the encryption algorithm type and malicious behavior attack chain.

[0017] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the malicious sample encryption process tracking method based on a large language model as described in the first aspect of the present invention.

[0018] The present invention has the following advantages:

[0019] (1) Solving the path explosion problem of symbolic execution: This invention introduces a large language model as an intelligent aid, and performs pre-screening and pruning of the control flow of binary functions through semantic understanding. The generated bootstrapping blueprint forces the symbolic execution engine to focus only on the core encryption logic path, effectively avoiding the exponential expansion of the state space when the engine processes obfuscated code or complex branches, and significantly improving analysis efficiency.

[0020] (2) Automated key extraction in a source code-free environment: This invention innovatively combines dynamic memory monitoring and data entropy analysis technology. During symbolic execution, the Shannon entropy of the data written to memory is calculated in real time. This enables key extraction for various symmetric encryption algorithms (such as AES, RC4, etc.) without relying on the reverse code of a specific decryption algorithm and without source code, greatly reducing the cost of manual reverse analysis.

[0021] (3) High-confidence malicious behavior assessment: Unlike traditional signature-based static detection, this invention determines malicious behavior by constructing a complete data flow attack chain of "Source → Crypto → Sink". Malicious behavior is only confirmed when dynamic taint analysis proves that the data flow is closed-loop connected. This verification method based on mathematical logic provides strong evidence and effectively reduces the false alarm rate. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0023] Figure 2 This is a detailed flowchart of static feature extraction and association localization in this invention;

[0024] Figure 3 This is a flowchart of the large language model guidance path selection process in this invention;

[0025] Figure 4 This is a flowchart illustrating the guided symbolic execution and precise key extraction in this invention.

[0026] Figure 5 This is a schematic diagram illustrating the principle of key extraction and flow verification in this invention;

[0027] Figure 6 This is a flowchart of the entire encryption process identification and result output in this invention. Detailed Implementation

[0028] To better understand the purpose, technical solution, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings. However, this invention can be implemented in many different ways depending on the provided technical solution. The accompanying drawings, which constitute a part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0029] like Figure 1 The flowchart of the method of the present invention is shown below.

[0030] Step 1: Static Feature Extraction and Localization

[0031] This invention utilizes a pre-built rule base of encryption algorithm features (YARA rules, S-Box, etc.) to perform a comprehensive static scan of binary samples. By identifying hard-coded encryption constants and lookup tables within the samples, and combining this with cross-reference analysis techniques from disassemblers, it quickly locates code references to these features, thereby filtering out candidate encryption functions that may contain encryption logic. This process eliminates a large amount of irrelevant code in the early stages of analysis, significantly improving the focus of subsequent analysis.

[0032] Specifically, the collected binary malicious samples are first scanned from multiple dimensions using feature matching tools such as Yara to identify potential encryption logic within the samples. Specifically, this invention pre-constructs a rule base containing features of various publicly available encryption and decryption algorithms. These features include, but are not limited to, S-box constants in symmetric encryption algorithms (such as AES, DES, RC4, and ChaCha20), round constants in block ciphers, large prime number fingerprints in asymmetric encryption algorithms (such as RSA), and specific shift-and-XOR loop structures. After accurately locating the memory offset address of the feature through feature matching, the logical relationships between functions are further analyzed using disassembly techniques, mainly including the following two steps:

[0033] (1) Script-based candidate function localization: Using a custom-developed automated analysis script in a disassembler, starting from the identified feature addresses, the script traces upwards through the control flow graph (CFG). The script automatically analyzes the function call stack and cross-referencing (Xref) relationships to locate all functions that directly or indirectly call the feature address as "candidate encryption functions". This process, through recursive search, can identify the top-level functions in the sample responsible for key expansion, initial permutation, and other key behaviors, thus initially delineating the potential encryption logic boundaries in the malicious sample.

[0034] (2) Instruction Sequence Tracing and Topology Construction: After locating the candidate encryption function, a customized script traces the instruction sequence affected by the feature data downwards, recording the registers and memory units through which the data flows. This invention extracts the disassembly logic inside the candidate function, transforming the originally discrete assembly instructions into a topological call relationship graph containing function dependencies and call order. This graph can clearly show the execution chain of the encryption adversarial function inside the sample, and the extracted disassembly code fragments will serve as the basic data input for semantic analysis of the large language model in subsequent steps.

[0035] The core of this step lies in:

[0036] A. Feature Scan: Traverse the read-only data segment (.rdata) and data segment (.data) of the binary file, and match the S-Box (replacement table), T-Table, or initialization vector (IV) specific to the encryption algorithm.

[0037] B. Reference Backtracking: Once a feature is matched, the cross-references (Xrefs) of that address are immediately queried through the disassembler engine. The system traces all instructions that reference the address with the feature, backtracking upwards to the function to which they belong, and marks these functions as "candidate encryption functions," while recording their entry address and physical offset.

[0038] The process of static feature extraction and association localization in this step is as follows: Figure 2 As shown.

[0039] Step 2: LLM Semantic Guidance and Blueprint Construction

[0040] The candidate encryption function code logic extracted in the first step is encapsulated into a preset prompt template and input into a large language model for semantic recognition. To ensure the rigor of the selection, this invention uses the large model to prune the complex control flow, eliminating error-checking branches, logging logic, or dead code that are irrelevant to the encryption process. Through semantic reasoning of the large language model, this invention significantly reduces the search space for subsequent dynamic analysis, fundamentally avoiding the path explosion problem caused by the blind exploration of the symbolic execution engine.

[0041] The specific implementation involves the following two steps:

[0042] (1) Construction of Prompts for Semantic Recognition: This invention constructs a prompt template specifically for binary reverse engineering scenarios, which structures the disassembly logic. The template consists of three parts: first, contextual background information, which informs the large language model of the current analysis task objective (identifying encryption logic); second, code fragment carrier, which encapsulates the candidate encryption function code obtained in the first step into basic blocks; and third, analysis task instructions, which explicitly require the model to identify: ① whether the function contains core encryption operation logic (such as key scheduling, XOR transformation); ② the behavioral intent of the function (such as payload decryption, C2 communication); ③ the storage location of key parameters (such as key pointer, ciphertext buffer).

[0043] (2) Semantic Reasoning-Based Guided Blueprint Generation: Based on the analysis results returned by the large language model, the system generates a structured "Guided Blueprint". This blueprint contains the determined address of the target function, the "critical basic block paths" that symbolic execution should prioritize exploring, and the suggested taint source injection locations. This blueprint will serve as a hard constraint on the symbolic execution engine in subsequent steps, forcing the engine to focus only on valid paths.

[0044] The path selection and blueprint construction process guided by the large language model in this step is as follows: Figure 3 As shown.

[0045] In this step, a Large Language Model (LLM) is creatively introduced as an auxiliary agent for reverse analysis. By extracting intermediate language (MLIL) code fragments and related control flow information from candidate functions, specific prompts are constructed to guide the LLM in semantic reasoning. The LLM can identify the behavioral intent of a function (such as decryption or communication) and determine the importance of code blocks. Based on the feedback from the LLM, the system generates a "guidance blueprint," which clearly identifies the critical path nodes that symbolic execution should focus on, thereby eliminating dead code and irrelevant logic and effectively avoiding the path explosion problem in traditional symbolic execution.

[0046] This step is primarily performed by the HybridSemanticAnalyzer engine, as detailed below:

[0047] A. Static Tag Pre-analysis: Based on the SSA (Static Single Assignment) form of Binary Intermediate Language (MLIL), a Def-Use chain is constructed inside the function. The system automatically identifies potential input sources (such as recv, ReadFile) and potential sinks (such as VirtualAlloc, WriteFile) within the function.

[0048] B. Prompt Engineering and Reasoning: Construct a Prompt containing a function control flow summary, a list of key API calls, and core MLIL code snippets. The Prompt is designed to guide the LLM to ignore auxiliary code such as logging and error checking, and focus on identifying the core logic related to key scheduling and data transformation.

[0049] C. Blueprint Generation: Based on the semantic analysis results output by the LLM, the system generates a "bootstrapping blueprint" in JSON format. The blueprint details the "critical basic block addresses" that symbolic execution should prioritize exploring, as well as suggested "taint injection locations".

[0050] Step 3: Guided Dynamic Symbolic Execution and Key Extraction

[0051] This invention proposes a controlled dynamic symbolic execution method. A "bootstrapping blueprint" is injected as a hard constraint into the symbolic execution engine (such as Angr), restricting the engine to explore only along predetermined paths. Simultaneously, a LoopSeer mechanism is introduced to dynamically limit the number of loop iterations, and a Lazy Solves strategy is employed to optimize constraint solving efficiency. More importantly, this invention embeds a memory write monitoring hook during symbolic execution, capturing and analyzing the entropy value of memory data in real time, thereby accurately extracting the decryption key without fully executing malicious code.

[0052] Appendix Figure 4 The logical flow of guided symbolic execution and precise key extraction is illustrated, including:

[0053] (1) Guided Path Exploration and State Pruning: This invention first sets the starting point of symbolic execution based on the guiding blueprint and marks the key parameters of the objective function as symbolic variables (taint sources). During the path exploration process, the LoopSeer mechanism is used to limit the number of iterations of the loop structure to prevent the engine from getting stuck in the complex loops unique to cryptographic algorithms (such as the 256 initialization loops of RC4); at the same time, a dynamic scoring mechanism is introduced, which adds points and gives high priority to the execution state that hits the blueprint guiding address, and deducts points and performs fast pruning for the state that gets stuck in an invalid loop or deviates from the guiding path, thereby concentrating computing resources on the core logic.

[0054] (2) Dynamic Key Extraction Based on Entropy: During symbolic execution, this invention registers a memory write monitoring hook. When the program performs a write operation in the context of the encryption function, the written data content is captured in real time and its Shannon entropy is calculated. If the length of the written data meets the key characteristics (e.g., 16 bytes or 32 bytes) and the entropy value exceeds a preset threshold (e.g., 3.5), the data is determined to be a decryption key or round key, and its memory address is recorded. This method can capture the generated key in dynamic simulation without needing to understand the specific details of the key generation algorithm.

[0055] The guided symbolic execution and key extraction method of this invention is implemented in WorkerEngine and consists of the following steps:

[0056] (1) State initialization and taint injection based on bootstrap blueprint

[0057] The large language model bootstrapping blueprint generated in step two is analyzed to understand the parameter structure and critical path nodes of the target function. The blueprint records the virtual address of the target function in the binary file, the list of critical basic blocks, and suggested taint injection locations. To track data flow during symbolic execution, the state of the symbolic execution engine needs to be initialized based on the blueprint information, and symbolic variables need to be injected. State initialization involves three steps. First, the binary file is loaded and a symbolic execution project is created, setting the execution start point to the entry address of the target function specified in the blueprint. Second, the memory model of symbolic execution is configured, enabling a lazy solves strategy to optimize performance, and, according to the suggestions in the blueprint, the input parameters of the target function (such as the ciphertext buffer pointer and data length) are replaced with symbolic vectors (BVS) instead of specific numerical values. Finally, the taint propagation context is initialized, and symbolic variables are marked as taint sources to facilitate subsequent tracking of their propagation paths in registers and memory.

[0058] (2) Heuristic path exploration and loop control

[0059] Based on the critical path nodes in the bootstrapping blueprint, active states during symbolic execution are filtered and pruned, and controlled exploration is performed within the complex control flow. Heuristic algorithms are used to deeply manage the state machine (SimGr) generated by the symbolic execution engine. The state machine represents all possible execution branches of the program; by analyzing the instruction address of the current state, it can be identified whether the state hits the cryptographic logic path predicted by the large language model. Path exploration and control mainly include two mechanisms. First, a LoopSeer loop control mechanism is introduced to monitor the loop structure in the execution flow in real time. When a tight loop specific to the cryptographic algorithm (such as the 256 initialization loops in RC4) is detected, its unrolling count is forcibly limited (e.g., limited to 8 times) to prevent symbolic execution from falling into an infinite loop. Second, a score-based state pruning is implemented. A state scoring function is defined, giving high scores (e.g., +40 points) to states that hit bootstrapping blueprint addresses, rewarding states that hit API entry points (+50 points), and deducting points (-50 points) for states trapped in infinite loops. After each step of execution, only the highest-scoring states (e.g., Top-12) are retained, thus effectively avoiding the path explosion problem.

[0060] (3) Dynamic key capture based on memory monitoring

[0061] Registering a memory monitoring hook in the symbolic execution engine allows for real-time capture of data write operations during program execution, thereby extracting potential decryption keys. The core of this step lies in leveraging the high-entropy data generated during the key scheduling process of encryption algorithms. The specific key capture operation is as follows: First, a breakpoint is set on the memory write event (mem_write) in the symbolic execution engine; when the program executes any memory write instruction, execution is paused, and the data content to be written to memory and its target address are extracted; then, Shannon entropy is calculated on the captured data. Encryption keys typically exhibit extremely high randomness, with entropy values ​​significantly higher than ordinary program data (such as strings or pointers). The system sets a threshold (e.g., entropy > 3.5 and length 16 or 32 bytes). If the captured data meets this threshold condition, and the write operation occurs within the execution context of an encryption function, it is identified as a decryption key and recorded. This operation successfully captures dynamically generated key data by monitoring general memory behavior, without relying on reverse engineering of specific algorithms. Once the capture is complete, symbolic execution continues to verify whether the data stream associated with the key ultimately flows into the sensitive API (Sink).

[0062] The principle behind key extraction and flow verification in this step is as follows: Figure 5 As shown.

[0063] This invention receives the guiding blueprint output from the second step and injects it into symbolic execution engines such as Angr. By combining the semantic guidance provided by the large language model, it significantly reduces the path exploration depth when analyzing complex programs and can efficiently reach the core encryption logic.

[0064] Step 4: Comprehensive analysis of the attack chain

[0065] In this step, based on the dynamic analysis results obtained in step three, and combined with static evidence, the encryption adversarial behavior patterns of malicious samples are identified. By integrating static features, semantic analysis conclusions, and dynamic taint tracing paths, this invention can accurately reconstruct the data flow logic of malicious code and generate a high-confidence analysis report.

[0066] Appendix Figure 6 This step illustrates the entire process of encryption identification and result output, including:

[0067] (1) Multi-layer cross-validation of the algorithm: The summary is the result of static feature scanning (YARA / S-Box), static code fingerprint analysis (Opcode Histogram) and dynamic runtime constant detection. The final encryption algorithm type is determined through a weighted voting mechanism, which solves the problem of high false alarm rate of single detection method.

[0068] (2) Attack chain closed-loop verification: Check the symbolic execution path record (Trace) to verify whether there is a complete data flow connectivity from the "taint source (Source, such as network receiving API)" through the "encryption operation (Crypto function)" and finally to the "sensitive operation (Sink, such as memory execution API)". Only the path with closed-loop data flow connectivity is confirmed as a valid malicious attack chain, thereby generating a final analysis report containing the malicious behavior type, confidence level, encryption algorithm and extraction key.

[0069] In this step, the present invention constructs a multi-layered evidence fusion framework. It integrates the feature fingerprints from the static scan in the first step, the source / sink information from the static data flow analysis in the second step, and the path verification results from the dynamic symbol execution in the third step. A three-layer cross-validation mechanism (Layer-A / B / C) is used to confirm the type of encryption algorithm, and a complete "Source-Crypto-Sink" attack chain is reconstructed based on the connectivity of the data flow. Finally, a high-quality analysis report containing malicious behavior determination, confidence assessment, and key parameters is generated.

[0070] Specifically, this step is completed by the Behavior Synthesizer, as detailed below:

[0071] A. Algorithm Voting: The results of static YARA matching, static code fingerprint analysis (Opcode Histogram), and dynamic runtime constant detection are aggregated and weighted for voting. For example, static features have higher weights, and dynamic verification serves as confirmation, ultimately determining the encryption algorithm type.

[0072] B. Attack Chain Reconstruction: Examine the symbolic execution trace to confirm the existence of a complete data path from the Source API (such as recv) to the Crypto function, and then to the Sink API (such as WinExec). Only paths with connected data flows will be marked as "Verified," thus constructing a complete malicious attack chain.

[0073] The core principle of this invention in achieving the above technical solutions is:

[0074] (1) Static Feature Extraction Principle

[0075] To ensure obfuscation and diffusion effects, encryption algorithms typically rely on specific mathematical constants or lookup tables (S-Boxes). These constants usually exist as consecutive byte sequences in the compiled binary file and remain stable across different versions of the code. By matching these unique "fingerprints," the scope of reverse engineering can be quickly narrowed down, precisely locating the core algorithm code segment.

[0076] (2) Path pruning and LLM guidance principle

[0077] The biggest challenge facing symbolic execution is path explosion, where the number of program branches grows exponentially with depth. Large Language Models (LLMs) possess a high-level understanding of assembly code semantics, enabling them to quickly determine, like experienced reverse engineers, which branches are the "main trunk" of cryptographic logic and which are "branches" of error handling or irrelevant functions. Using the LLM's judgments as "landmarks" to constrain the search space of the symbolic execution engine allows computational resources to be concentrated on effective paths, thus enabling the analysis of complex functions within a limited time.

[0078] (3) Key extraction principle

[0079] The key scheduling process in cryptographic programs typically involves writing the original key into a memory buffer after complex mathematical transformations to generate round keys. The data generated in this process exhibits extremely high randomness, resulting in a high entropy value. Symbolic execution engines can precisely simulate the execution of CPU instructions. By monitoring memory write operations and calculating the entropy of the data, the key generation can be captured at the instant it occurs. This method is independent of specific key generation algorithm details and possesses strong versatility.

[0080] (4) Attack chain verification principle

[0081] Malicious behavior (such as ransomware encryption and C2 callbacks) manifests at the underlying level as a specific data flow. For example, the behavior pattern of ransomware is: ReadFile -> Encrypt -> WriteFile. Through taint analysis, if it can be proven that the data written by WriteFile is directly derived from the data read by ReadFile and has been processed by an encryption function, then the existence of this malicious behavior can be rigorously proven mathematically, thus ruling out coincidence or false positives.

[0082] Based on the above technical solutions, compared with the prior art, the present invention significantly improves the success rate of analyzing complex and obfuscated malicious samples, and can accurately extract decryption keys in static or semi-static environments without running samples, thus having higher security and efficiency.

[0083] Example 1

[0084] This embodiment uses a typical malware loader sample (SHA256: 0000de7eb061559767e4f9d5f545cff203c2a77f149fade7812c8c467eaa31c1) as an example to demonstrate the specific execution process of the method of the present invention. This sample uses code obfuscation technology, and the encryption function has been customized, making it difficult to directly identify using traditional static analysis.

[0085] 1. Static Feature Extraction and Association Localization (Step 1)

[0086] The system first starts the static scanning module (corresponding code crypto_discovery.py).

[0087] Feature scanning: The system loads the pre-configured YARA rule base to perform a full scan of the sample binary file. The scan results show that no standard AES S-Box was found at offset 0x403020 in the read-only data segment (.rdata) of the file, but a continuous data segment with a suspected high entropy value was found, and no standard encryption constants were hit.

[0088] Cross-reference backtracking: Although no strong feature was hit, the system, through API analysis using the disassembler engine (BinaryNinja), discovered that the high-entropy data was referenced by the function sub_4021f3. Further analysis of the function's instruction histogram revealed that XOR instructions accounted for as much as 15% of the instruction set and contained a double loop structure.

[0089] Candidate determination: Based on the above characteristics, the system marks sub_4021f3 as a "high-priority candidate encryption function" and maps its physical address to a virtual address, storing it in the candidate list.

[0090] 2. Path selection guided by the large language model (Step 2)

[0091] The system starts the hybrid analysis module (corresponding code: hybrid_analysis.py).

[0092] Intermediate Language Extraction: The system extracts MLIL (Intermediate Intermediate Language) code snippets from sub_4021f3. Due to the large amount of obfuscated junk instructions (such as invalid arithmetic operations) within the functions, direct reading is extremely difficult.

[0093] Semantic reasoning: The system builds prompts, sending the function's control flow graph summary, key API calls (APIs not called internally by the function), and MLIL code to the large language model.

[0094] Example of a Prompt: "Analyze the following assembly logic to determine if it is an encryption function. If so, indicate the location of the key and ciphertext parameters."

[0095] LLM Feedback and Blueprint Generation: The large language model returns the analysis results: "This function has typical stream cipher characteristics, and its structure is similar to the KSA (key scheduling) and PRGA (pseudo-random generation) stages of RC4. Parameter 1 is presumed to be the key buffer, and parameter 2 is the data length."

[0096] Blueprint output: The system generates a "Boot Blueprint" based on this, marking the function entry point 0x4021f3 and suggesting that arg1 be marked as a taint source for tracing, while also pointing out the entry address of the loop body as a critical path node.

[0097] 3. Guided Symbolic Execution and Key Extraction (Step 3)

[0098] The system starts the dynamic analysis module (corresponding code symbolic_execution.py), loads the blueprint generated in step 2, and uses the Angr framework to perform guided symbolic execution.

[0099] Environment initialization: The system sets the starting point for symbolic execution to 0x4021f3. Based on the blueprint, the system constructs a symbolic memory region as a "key" and passes a pointer to this region as a function parameter.

[0100] Anti-path explosion treatment:

[0101] LoopSeer Mechanism: The RC4 algorithm contains an initialization loop that runs 256 times. The system detects the loop counter increasing and automatically triggers the LoopSeer mechanism, forcibly limiting the loop unrolling count to 8 times. This allows the symbolic execution engine to escape a loop that would normally take minutes to complete within seconds, successfully avoiding path explosion.

[0102] State pruning: Approximately 40 branch states were generated during execution. The system uses the scoring function _prune_states to award +40 points to states that hit key nodes in the blueprint and -50 points to states that get stuck in an infinite loop, dynamically eliminating 30 low-value states and retaining only 10 core states for further exploration.

[0103] Precise key extraction:

[0104] During execution, the memory write hook captured the program writing 16 bytes of data to the stack space while executing at address 0x4021f3.

[0105] Entropy verification: The system calculates the Shannon entropy of the data in real time, and the result is 3.85 (close to the maximum value of 4.0).

[0106] Extraction result: The system determined that the data is the decryption key, and the content is 1F 2B 3C 4D ....

[0107] Tainted data flow verification: Symbolic execution continues, tracing the data flow after the function returns. The system finds that the data processed by sub_4021f3 (i.e., the decrypted payload) is directly passed as a parameter to the memory region with the PAGE_EXECUTE_READWRITE attribute allocated by VirtualAlloc, and then the program jumps (jmp) to that region for execution.

[0108] 4. Encryption-based end-to-end identification and result output (Step 4)

[0109] The system initiates the comprehensive analysis module (corresponding code step4_behavior_synthesis.py) to fuse the above evidence.

[0110] Algorithm determination: Although the static features did not explicitly match RC4, the LLM analysis (Opcode fingerprint) in Step 2 and the dynamic behavior in Step 3 (256 loop structures and XOR operations) strongly point to the RC4 algorithm. The three-layer cross-validation system determines the algorithm to be RC4.

[0111] Attack chain reconstruction: The system detected a complete data flow path: Resource segment data (Source) -> sub_4021f3 (Crypto: RC4) -> VirtualAlloc (Sink) -> Execution. Since the data flow is fully connected, the system marks this attack chain as Verified.

[0112] Final Report: The system generates a JSON report that clearly states:

[0113] Malicious behavior: Payload_Decryption_Loading.

[0114] Encryption algorithm: RC4.

[0115] Decryption key: 1F 2B 3C 4D ... (Hex), located at memory offset 0x12ff80.

[0116] Confidence level: 95 / 100.

[0117] As can be seen from this embodiment, the present invention can efficiently and accurately locate the encryption logic of malicious samples, extract keys, and identify behavioral intent without source code or code obfuscation, thus verifying the efficiency and accuracy of the method.

[0118] Through the above specific examples, the embodiments of the present invention have been verified in the following ways:

[0119] (1) Solving the path explosion problem: By using the semantic understanding capabilities of large language models for pre-screening and path guidance, the symbolic execution engine does not need to blindly explore all branches, thus enabling it to handle logically complex malicious samples, significantly improving analysis efficiency and success rate.

[0120] (2) Automated key extraction: Combining dynamic memory monitoring and entropy analysis, it can extract keys for a variety of symmetric encryption algorithms without relying on the reverse code of a specific decryption algorithm, which greatly reduces the cost of manual analysis.

[0121] (3) High-confidence behavior judgment: Unlike traditional signature-based detection, this invention judges malicious behavior based on complete data flow connectivity (from source to sink), providing mathematical evidence and greatly reducing the false alarm rate.

[0122] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for tracing a malicious sample encryption process based on a large language model guide, characterized in that, The method includes the following steps: S1. Static Feature Extraction and Localization: Static scanning is performed on binary malicious samples to extract static features related to encryption algorithms, including encryption constants, S-Box lookup tables and API call information, and candidate encryption functions are identified based on the static features; S2. LLM Semantic Guidance and Blueprint Construction: Static data flow analysis is performed on the candidate encryption functions described in step S1 to construct variable passing paths within and across functions; at the same time, semantic reasoning is performed on the disassembled code and data flow context of the functions using a large language model to filter out a subset of target functions involving key processing and core encryption operations, and to generate a guidance blueprint; S3. Guided Dynamic Symbolic Execution: Based on the guiding blueprint described in step S2, the objective function is dynamically analyzed using a symbolic execution engine; the path explosion problem is mitigated by actively triggering detection at the guiding address, limiting the number of loop unrollings, and pruning low-value states based on a scoring machine; simultaneously, memory read / write operations are monitored during symbolic execution to extract encryption keys and confirm data flow; including: Initialize the symbolic execution state and inject symbolic taints into the input parameters of the target function; during symbolic execution, check in real time whether the instruction address of the current execution state hits a critical path node in the bootstrap blueprint; if it does, perform a full convergence point detection on the current state and assign a high-priority score; limit the number of symbolic executions of loop structures through the LoopSeer mechanism and count the access frequency of basic blocks; when the access frequency exceeds a preset threshold, forcibly terminate the exploration of the current path; dynamically prune the active state set based on a state scoring mechanism, which weights the state according to whether it hits an API, whether it is on the bootstrap path, and the novelty of the path exploration; S4. Comprehensive analysis of the attack chain: Combining the static features of step S1, the semantic filtering results of step S2, and the dynamic symbolic execution evidence of step S3, the encryption algorithm type is determined through multi-layer cross-validation, and a malicious behavior attack chain is constructed based on data flow connectivity. In step S3, the extraction of the encryption key includes: registering a MemoryWrite monitoring hook during symbolic execution; capturing the written data content and target address when a memory write operation is detected; performing entropy calculation and character distribution analysis on the written data content; if the data length, entropy value, and character distribution meet the key characteristics, and the writing location is close to the key management API call point, then marking it as a candidate key. In step S4, the multi-layer cross-validation determines the encryption algorithm type, including: Layer A, a first-layer vote based on YARA rule hits, S-Box matching, and constant features extracted in step S1; Layer B, a second-layer vote based on the instruction opcode histogram and SSA data stream features extracted in step S2; the opcode histogram statistically analyzes the distribution ratio of XOR, SHIFT, and ROTATE instructions and matches them with a preset encryption algorithm fingerprint template; Layer C, a third-layer vote based on runtime constants and S-Box access behavior detected during dynamic symbol execution in step S3; the results of the above three layers of voting are weighted and summarized to determine the final encryption algorithm type and confidence level; In step S4, constructing the malicious attack chain includes: defining the three-segment structure of the attack chain as: taint source → encryption operation → convergence point; checking whether there is a complete data flow connectivity from the taint source API to the encryption function and then to the convergence point API in the dynamic symbolic execution path of step S3; if a complete connectivity exists, the attack chain is marked as verified, and the attack behavior is classified into a specific malicious scenario based on the API category combination of the taint source and the convergence point.

2. The method of claim 1, wherein, In step S1, the extraction of encrypted constants includes identifying single-valued constants and continuous S-Box / T-Table lookup tables. The identification process is achieved by matching a predefined encryption algorithm feature library, which contains constant signatures and table structure features of common encryption algorithms such as AES, TEA, RC4, and ChaCha20. For the identified S-Box / T-Table, cross-reference analysis is used to associate it with the functions that reference the table.

3. The method according to claim 1, characterized in that, In step S2, the static data flow analysis is performed using an intermediate language based on static single assignment, including: constructing a Def-Use chain within a function to trace the definition and usage relationships of variables; establishing a data flow propagation path in non-stack memory by analyzing the address expressions of memory storage and loading instructions; tracing the caller upwards along the function call graph and prioritizing the analysis of call paths with tightly coupled business logic based on the number of cross-references to the caller; and generating a bootstrapping blueprint containing the target function address, critical execution path nodes, taint sources, and convergence point information.

4. The method according to claim 1, characterized in that, In step S2, the semantic reasoning of the disassembled code and data flow context of the function using a large language model includes: constructing a prompt word containing the disassembled code fragment of the function, data flow evidence, and preliminary algorithm identification results; inputting the prompt word into the large language model, and requiring the model to output the determination result of the encryption algorithm type and malicious behavior scenario classification; the malicious behavior scenario classification includes at least payload decryption and loading, C2 command execution, data theft, and ransomware encryption.

5. A malicious sample encryption process tracking system based on a large language model, implementing the method of any one of claims 1-4, characterized in that, The system includes: a static feature extraction module, used to execute step S1, extracting static features related to the encryption algorithm and identifying candidate encryption functions; a semantic analysis and blueprint generation module, used to execute step S2, combining static single-assignment data flow analysis and large language model inference to generate a guiding blueprint; a dynamic symbol execution module, used to execute step S3, exploring paths and extracting encryption keys based on the guiding blueprint; and a comprehensive judgment module, used to execute step S4, outputting a report on the encryption algorithm type and malicious attack chain.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 4.

Images