Non-killing PE program generation method based on Shellcode encryption

By converting PE programs into shellcode and encrypting them, an undetectable PE program is generated, which solves the problem that modifying PE source code is complex and cannot evade detection in existing technologies. This achieves evasion of traditional antivirus engines and improves the malware detection system.

CN121030768APending Publication Date: 2025-11-28SICHUAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511104365.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-08-07
Publication Date
2025-11-28

AI Technical Summary

Technical Problem

Existing methods for constructing antivirus-evading PE programs mainly involve directly modifying the PE source code. However, it is difficult to obtain the original source code, the construction process is complex, and it cannot effectively circumvent the static feature analysis and conventional file detection mechanisms of traditional antivirus engines and terminal detection systems.

Method used

The original PE program is converted into position-independent shellcode, and the shellcode and key loading parameters are encrypted to generate an undetectable PE program. This program is then compiled using the shellcode loader configuration file and the main program file, thereby improving stealth and reliability.

Benefits of technology

The generated undetectable PE program can bypass static feature analysis and regular file detection, significantly improving stealth and reliability, and promoting the generalization of malware detection systems and the optimization of defense strategies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121030768A_ABST
    Figure CN121030768A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of network security, and discloses a non-killing PE program generation method based on Shellcode encryption, which comprises the following steps of: converting an original PE program into a position-independent Shellcode, respectively encrypting the Shellcode and a key loading parameter, and writing the Shellcode and the key loading parameter into a loader configuration file; a loader configuration file and a Shellcode loader main program file used for decrypting and executing a Shellcode are compiled through a compiling tool, and a non-killing PE program is generated, so that the PE program can bypass static feature analysis and a conventional file detection mechanism on the basis of not modifying a source code; meanwhile, the Shellcode loader directly executes the Shellcode in a system call (syscale) mode, and behavior characteristic monitoring caused by a common API call path is avoided. According to the method, the PE program effectively avoids a traditional antivirus engine and a terminal detection system, the concealment and reliability of the antivirus-free PE program sample are remarkably improved, and then the generalization of an existing malicious program detection system is improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of network security, and particularly relates to a Shellcode encryption-based PE program generation method. BACKGROUND

[0002] With the rapid development of informatization, malicious software has become a major threat in the field of network security. Especially in the Windows operating system environment, it is widely used in office, industry and government scenarios, and has become the platform that attackers focus on. Attackers often bypass traditional protection by constructing malicious programs with concealment, destructiveness and persistence to implement attacks such as data theft, remote control and service interruption. Especially in advanced persistent threats (APTs) and targeted attacks, malicious software on the Windows platform often has strong anti-detection ability and self-hiding mechanism, which brings great challenges to security defense.

[0003] Shellcode is a piece of machine code (usually binary instructions) used to exploit software vulnerabilities, often used as a payload for vulnerability attacks, usually written in hexadecimal or assembly form, with the purpose of performing specific operations (such as obtaining a shell, gaining privileges, reverse connection, etc.) through vulnerabilities in the target system.

[0004] PE program refers to a program designed using the Portable Executable (PE) format under the Windows operating system.

[0005] PE program refers to a program designed using the Portable Executable (PE) format under the Windows operating system.

[0006] Currently, the development of malicious software shows a high level of antagonistic trend. Attackers are constantly exploring new anti-virus means to break through the detection mechanism on the Windows platform, such as code obfuscation, DLL hijacking, API replacement, fileless execution, syscall bypass, etc. At the same time, detection systems are constantly introducing deep learning, behavior tracing, context analysis and other methods to improve accuracy and real-time performance. However, the asymmetry between attack and defense still exists, and the defense side often finds itself in a passive response position, lacking proactive countermeasures.

[0007] The increasing diversity and evasiveness of malware samples prompt detection techniques to optimize feature extraction and model identification strategies through actual confrontation with samples. By constructing PE samples with certain resistance and representativeness, the coverage and identification accuracy of existing detection systems can be tested to evaluate their adaptability to complex threat samples and further improve and evolve detection algorithms and security mechanisms.

[0008] However, existing methods for constructing PE programs mainly use direct modification of PE source code, but in practical applications, it is often difficult to obtain the original source code. Even if the PE file structure is modified to achieve immunity, the construction process is complex, there is a risk of destroying the original program logic or running stability, and it cannot effectively avoid static feature analysis and conventional file detection mechanisms of traditional antivirus engines and terminal detection systems.

[0009] Therefore, a Shellcode encryption-based PE program generation method is proposed to solve the above problems. SUMMARY

[0010] To solve the above problems, the purpose of the present application is to provide a Shellcode encryption-based PE program generation method, which converts the original PE program into position-independent Shellcode, encrypts the Shellcode and key loading parameters and writes them into a configuration file, and compiles the configuration file and the loader main program file using a compilation tool to generate an immune PE program, which effectively avoids traditional antivirus engines and terminal detection systems, improves the stealthiness and reliability of immune PE program samples, and further improves the generalization of existing malicious program detection systems. The technical solution is as follows:

[0011] A Shellcode encryption-based PE program generation method, comprising the following steps:

[0012] Step S1, converting the original PE program into Shellcode;

[0013] Step S2, encrypting the Shellcode to obtain first encrypted configuration data and encrypted Shellcode ciphertext;

[0014] Step S3, encrypting the key loading parameters to obtain second encrypted configuration data and encrypted key loading parameter ciphertext; wherein the key loading parameters are key feature data in the Shellcode loader;

[0015] Step S4, writing the first encrypted configuration data, the second encrypted configuration data, the Shellcode ciphertext and the key loading parameter ciphertext into the Shellcode loader configuration file;

[0016] Step S5, write a Shellcode loader main program file for decrypting and executing the Shellcode;

[0017] Step S6, compile the Shellcode loader configuration file and the Shellcode loader main program file using a compiling tool to generate an anti-killing PE program.

[0018] Further, in the step S1, the original PE program is converted into the Shellcode using the Donut tool.

[0019] Further, in the step S3, the key loading parameters are encrypted, including encrypting the name string of the key function in the Shellcode loader.

[0020] Further, the encryption of the key loading parameters also includes encrypting the name string of the dynamic link library to which the key function in the Shellcode loader belongs.

[0021] Further, the encryption of the name string of the key function in the Shellcode loader includes encrypting the name string of the VirtualAlloc function.

[0022] Further, the encryption of the name string of the key function in the Shellcode loader also includes encrypting the name string of the dynamic link library kernel32.dll to which the VirtualAlloc function belongs.

[0023] Further, the step S5 includes the following steps:

[0024] S41, read the first encrypted configuration data, the second encrypted configuration data, the encrypted Shellcode and the key loading parameter ciphertext from the Shellcode loader configuration file;

[0025] S42, decrypt and restore the Shellcode and the key loading parameter according to the first encrypted configuration data and the second encrypted configuration data;

[0026] S43, based on the decrypted and restored key loading parameter, obtain the name of the VirtualAlloc function and the dynamic link library kernel32.dll to which it belongs, and dynamically analyze and obtain the base address of the dynamic link library kernel32.dll and the actual address of the exported function VirtualAlloc through the API query mechanism;

[0027] S44. Call the VirtualAlloc function based on the actual address of VirtualAlloc to dynamically allocate a memory region with executable permissions;

[0028] S45. Write the decrypted and recovered Shellcode into the memory area allocated in step S44;

[0029] S46. By calling the syscall interface and using the memory address written to the Shellcode in step S45 as the entry point, the Shellcode is executed directly.

[0030] Furthermore, both step S2, which encrypts the Shellcode, and step S3, which encrypts the key loading parameters, use the Advanced Encryption Standard (AES) combined with the Cipher Block Chaining (CBC) mode for encryption.

[0031] Furthermore, the first encryption configuration data and the second encryption configuration data in steps S2 and S3 both include the key and initialization vector IV required for encryption, and the key and the initialization vector IV are dynamically generated by a secure random number generation algorithm.

[0032] Furthermore, based on the architecture bitness of the original PE program, a corresponding architecture compilation tool is selected for adaptive compilation.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] This invention transforms the original PE program into position-independent shellcode, encrypts the shellcode and key loading parameters, and writes them into a loader configuration file. A compilation tool then compiles the loader configuration file and the main shellcode loader program file used to decrypt and execute the shellcode, generating an undetectable PE program. This allows the PE program to bypass static signature analysis and conventional file detection mechanisms without modifying the source code. Simultaneously, the shellcode loader executes the shellcode directly via a system call (syscall), avoiding behavioral signature monitoring triggered by common API call paths. This method effectively evades traditional antivirus engines and endpoint detection systems, significantly improving the stealth and reliability of undetectable PE program samples, thereby enhancing the generalization capabilities of existing malware detection systems. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall process of this method.

[0036] Figure 2 This is a flowchart of the Shellcode loader's operation. Detailed Implementation

[0037] The technical solutions of the present invention will now be clearly and completely described in conjunction with the accompanying drawings, so as to provide a better understanding of the concept of the present invention, the technical problem solved, the technical features constituting the technical solution, and the technical effects brought about.

[0038] To improve the detection and defense capabilities against novel malicious code execution techniques and further enhance the security of network and endpoint systems, this paper proposes a technical solution based on shellcode encryption and Go language-driven execution to construct highly covert and adversarial evasion-resistant PE program samples. Real-world attack behaviors are simulated to evaluate the adaptability of malware detection mechanisms to complex samples, thereby promoting the continuous optimization of detection systems and defense strategies.

[0039] The overall flowchart of this method is as follows: Figure 1 As shown, the specific steps include:

[0040] S1. Convert the original PE program that needs to be evaded by antivirus software into Shellcode;

[0041] The Donut tool is used to convert malicious PE programs into position-independent shellcode, enabling them to be loaded and executed at arbitrary memory addresses. This frees them from file structure limitations, providing a foundation for subsequent encryption and covert execution. At the same time, it allows malicious PE programs to bypass static feature analysis and conventional file detection mechanisms, avoiding detection risks caused by static feature matching and sensitive function names in import tables, significantly improving the stealth and execution reliability of the generated undetectable malicious PE programs.

[0042] S2. Encrypt the Shellcode obtained in step S1 to obtain the first encrypted configuration data and the encrypted Shellcode ciphertext.

[0043] S3. Encrypt the key loading parameters and obtain the second encrypted configuration data and the encrypted key loading parameter ciphertext.

[0044] Among them, the key loading parameters are the key feature data in the Shellcode loader, including: the names of the key functions in the Shellcode loader and the names of the dynamic link libraries to which they belong;

[0045] It can encrypt only the name strings of key functions in the Shellcode loader, hiding the name information of key functions in the Shellcode loader, effectively avoiding the static scanning and identification of existing detection systems, and significantly improving the stealth of the generated anti-virus evasion malicious PE program;

[0046] It can also encrypt the name strings of key functions in the Shellcode loader and the name strings of their respective dynamic link libraries, hiding the name information of key functions in the Shellcode loader and the name information of their respective dynamic link libraries, further improving the stealth of the generated anti-virus evasion malicious PE program;

[0047] The name string of the VirtualAlloc function in the Shellcode loader is encrypted. During the execution of the Shellcode loader, the VirtualAlloc function is called to request a memory region with executable permissions for the malicious PE program. Therefore, encrypting the name string of the VirtualAlloc function can prevent the malicious PE program from being identified by the static scanning of the existing detection system when requesting the memory region, thus improving the stealth of the evaded malicious program.

[0048] It can also encrypt the name string of the VirtualAlloc function in the Shellcode loader and the name string of its system dynamic link library kernel32.dll together, which better prevents malicious PE programs from being identified by the static scanning of existing detection systems when requesting memory areas, and further improves the stealth of anti-virus evasion malware.

[0049] The encryption of Shellcode in step S2 and the encryption of key loading parameters in step S3 both use the Advanced Encryption Standard (AES) combined with Cipher Block Link Mode (CBC) for encryption.

[0050] The encrypted configuration data (first encrypted configuration data and second encrypted configuration data) includes the key and initialization vector (IV) required for symmetric encryption, and both the key and the initialization vector (IV) used for encryption are dynamically generated by a secure random number generation algorithm; it should be noted that other encryption and encoding methods can also be used in this invention;

[0051] In addition, to ensure the compatibility of encrypted data during program transmission and processing, the encryption results (Shellcode ciphertext and key loading parameter ciphertext) are further Base64 encoded to avoid generating unprintable characters and improve portability and stability.

[0052] S4. Write the first encrypted configuration data, the second encrypted configuration data, the Shellcode ciphertext, and the key loading parameter ciphertext into the Shellcode loader configuration file;

[0053] Write the encryption results (Shellcode ciphertext and key loading parameter ciphertext) obtained in steps S2 and S3, along with the encryption configuration data (first encryption configuration data and second encryption configuration data), into the main_var.go file.

[0054] S5. Write the main program file for the Shellcode loader, which is used to decrypt and execute Shellcode;

[0055] The main program file main.go, written in Go, is used to read encrypted configuration data and decrypt and execute shellcode. At the execution level, the invention uses Go to implement a shellcode loader, which reduces the possibility of undetectable malicious PE programs being identified by security products by leveraging its static compilation and structural differences.

[0056] like Figure 2 As shown, the Go main program file main.go includes the following steps:

[0057] S51. Read the first encrypted configuration data, the second encrypted configuration data, the Shellcode ciphertext, and the key loading parameter ciphertext from the main_var.go file.

[0058] S52. Decrypt and recover the Shellcode and key loading parameters based on the first encrypted configuration data and the second encrypted configuration data.

[0059] S53. Based on the key loading parameters recovered from decryption (including the name of the VirtualAlloc function and the name of its system dynamic link library kernel32.dll), the base address of the dynamic link library kernel32.dll and the actual address of its exported function VirtualAlloc are dynamically resolved and obtained through the API query mechanism.

[0060] Key loading parameters (such as function names and dynamic link library names) are stored in an encrypted manner and dynamically decrypted and invoked at runtime, further reducing the possibility of malicious PE programs being identified by static scanning and improving the stealth of generated anti-virus evasion malicious PE programs.

[0061] S54. Call the VirtualAlloc function to dynamically allocate a memory region with executable permissions (i.e., the executable permission "X" in RWX) for loading shellcode.

[0062] S55. Write the Shellcode decrypted and recovered in step S52 into the executable memory area allocated in step S54.

[0063] S56. Call the syscall.Syscall interface and use the memory address of the Shellcode written in step S55 as the entry point to directly execute the Shellcode, so as to complete the running logic of the initial malicious PE program.

[0064] The shellcode loader executes shellcode directly via system calls (syscalls), bypassing behavioral monitoring triggered by common API call paths, thus improving the stealth and reliability of undetectable malicious PE programs.

[0065] S6. Use a compilation tool to compile the Shellcode loader configuration file and the Shellcode loader main program file to generate an unprotected PE program;

[0066] The Go compiler is used to compile the main.go and main_var.go files to generate an undetectable malicious PE program;

[0067] Specifically, based on the architecture bitness (32-bit or 64-bit) of the PE program to be evaded by antivirus software, a Go compiler corresponding to the architecture is selected for compatibility compilation to ensure that the generated executable file runs compatiblely on the target platform, thereby improving the reliability of the generated evaded malicious PE program.

[0068] This method encrypts the key features of the loader based on shellcode, without modifying the original PE source code. The construction process is fast and highly reproducible, making it suitable for generating highly adversarial evasion PE program samples for detection system evaluation. It helps to reveal the blind spots in the current detection mechanism when dealing with advanced evasion samples, providing support for the improvement and optimization of malicious code detection technology and promoting the continuous development of network security protection capabilities.

[0069] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating unprotected PE programs based on shellcode encryption, characterized in that, Includes the following steps: Step S1: Convert the original PE program into Shellcode; Step S2: Encrypt the Shellcode to obtain the first encrypted configuration data and the encrypted Shellcode ciphertext; Step S3: Encrypt the key loading parameters to obtain the second encrypted configuration data and the encrypted key loading parameter ciphertext; wherein, the key loading parameters are the key feature data in the Shellcode loader; Step S4: Write the first encrypted configuration data, the second encrypted configuration data, the Shellcode ciphertext, and the key loading parameter ciphertext into the Shellcode loader configuration file; Step S5: Write the main program file for the Shellcode loader, which is used to decrypt and execute Shellcode; Step S6: Use a compilation tool to compile the Shellcode loader configuration file and the Shellcode loader main program file to generate an unprotected PE program.

2. The method for generating an undetectable PE program based on shellcode encryption as described in claim 1, characterized in that, In step S1, the Donut tool is used to convert the original PE program into Shellcode.

3. The method for generating an undetectable PE program based on shellcode encryption as described in claim 1, characterized in that, In step S3, key loading parameters are encrypted, including encrypting the name strings of key functions in the Shellcode loader.

4. The method for generating an undetectable PE program based on shellcode encryption as described in claim 3, characterized in that, The encryption of key loading parameters also includes encrypting the name string of the dynamic link library to which the key functions in the Shellcode loader belong.

5. The method for generating an undetectable PE program based on shellcode encryption as described in claim 3, characterized in that, The encryption of the name strings of key functions in the Shellcode loader includes the encryption of the name string of the VirtualAlloc function.

6. The method for generating an undetectable PE program based on shellcode encryption as described in claim 5, characterized in that, The encryption of the name strings of key functions in the Shellcode loader also includes encrypting the name string of the dynamic link library kernel32.dll to which the VirtualAlloc function belongs.

7. The method for generating an undetectable PE program based on shellcode encryption as described in claim 6, characterized in that, Step S5 includes the following steps: S41. Read the first encrypted configuration data, the second encrypted configuration data, the shellcode ciphertext, and the key loading parameter ciphertext from the Shellcode loader configuration file; S42. Decrypt and recover the Shellcode and key loading parameters based on the first and second encrypted configuration data; S43. Based on the key loading parameters recovered from decryption, obtain the name of the VirtualAlloc function and its associated dynamic link library kernel32.dll, and dynamically resolve and obtain the base address of the dynamic link library kernel32.dll and the actual address of its exported function VirtualAlloc through the API query mechanism. S44. Call the VirtualAlloc function based on the actual address of VirtualAlloc to dynamically allocate a memory region with executable permissions; S45. Write the decrypted and recovered Shellcode into the memory area allocated in step S44; S46. By calling the syscall interface and using the memory address written to the Shellcode in step S45 as the entry point, the Shellcode is executed directly.

8. The method for generating an undetectable PE program based on shellcode encryption as described in claim 1, characterized in that, The encryption of the Shellcode in step S2 and the encryption of the key loading parameters in step S3 both use the Advanced Encryption Standard (AES) combined with the Cipher Block Chaining (CBC) mode for encryption.

9. The method for generating an undetectable PE program based on shellcode encryption as described in claim 1, characterized in that, The first encryption configuration data and the second encryption configuration data in steps S2 and S3 both include the key and initial vector IV required for encryption, and the key and the initial vector IV are dynamically generated by a secure random number generation algorithm.

10. The method for generating an undetectable PE program based on shellcode encryption as described in claim 1, characterized in that, Step S5 includes: Based on the architecture bitness of the original PE program, select the corresponding architecture compilation tool for adaptive compilation.