A private protocol analysis method and device based on memory data read-write monitoring and a computer readable storage medium

Abstract

The application provides a private protocol analysis method and device based on memory data read-write monitoring and a computer readable storage medium. The method comprises the following steps: monitoring a data receiving function of communication software, obtaining a storage address and size of a received data packet in memory; setting a memory area corresponding to the data packet as a restricted access state; triggering an exception when a program accesses the memory area, extracting instructions for accessing the memory area in exception processing, and forming an instruction set; analyzing address offset and access granularity of each instruction in the instruction set for accessing memory; determining a field boundary of the data packet according to the address offset and the access granularity, and obtaining a protocol format of the data packet. The application automatically extracts and analyzes memory access instructions by monitoring the actual access behavior of the program to the data packet memory, without complex code tracking and manual debugging, thereby significantly improving the efficiency and accuracy of private protocol analysis, and particularly having better performance in identifying dynamic fields.

Description

Technical Field

[0001] This invention relates to the field of software reverse engineering technology, specifically to a method, apparatus, and computer-readable storage medium for analyzing proprietary protocols based on memory data read / write monitoring. Background Technology

[0002] Network protocols are a set of rules that must be followed when peer entities in a computer network exchange information. In specific application scenarios such as industrial control systems, where the application scope is limited to within an enterprise and requires a certain degree of isolation, many manufacturers use custom proprietary protocols for communication. Because the protocol specifications of proprietary protocols are not publicly available, reverse engineering is often necessary to understand their design details when conducting security research, compatibility development, or fault diagnosis. Protocol reverse engineering refers to the process of extracting the protocol format during communication by monitoring and analyzing the network packets of protocol entities without relying on the protocol description.

[0003] Currently, there are two main technical approaches to analyzing proprietary protocols. The first method involves disassembling the client software that uses the proprietary protocol for communication and dynamically debugging key functions in the assembly code to obtain the message type, field length, and construction characteristics of the proprietary protocol. These key functions include the constructors for message packets, functions for sending and receiving message packets, and encryption and decryption functions. However, this method requires a high level of reverse engineering ability from the analyst, a deep understanding of the assembly code's execution logic, and a significant investment of time and effort in step-by-step software tracing and debugging. This approach is inefficient and fails to meet the needs of rapid proprietary protocol analysis in practical applications.

[0004] The second method analyzes and compares data packets of a proprietary protocol through sequence alignment. It identifies the commonalities and differences between multiple data packets to define segmentation points and thus determine the overall structure of the data packet. This method is highly accurate in identifying static, immutable fields in the protocol, such as fixed identifiers and version numbers in the protocol header. However, it has significant limitations in identifying dynamic fields with variable content, resulting in lower accuracy. This is because the content of dynamic fields changes according to the specific business logic during communication, and relying solely on sequence alignment is insufficient to accurately determine the boundaries and meanings of these fields, easily leading to errors in field boundary identification or semantic misunderstanding.

[0005] In summary, existing methods for analyzing private protocols either require excessively high levels of expertise from analysts and are inefficient, or lack sufficient accuracy in identifying dynamic fields, thus failing to adequately meet the current practical needs for efficient and accurate analysis of private protocols. Summary of the Invention

[0006] The purpose of this invention is to provide a private protocol analysis method, device, and computer-readable storage medium based on memory data read / write monitoring, which can improve analysis efficiency, ensure analysis accuracy, and perform better in identifying dynamic fields.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A proprietary protocol analysis method based on memory data read / write monitoring includes: S1: Monitor the data receiving function of the communication software to obtain the storage address and size of the received data packets in memory; S2: Set the memory region corresponding to the data packet to a restricted access state; S3: When the program accesses the memory region, an exception is triggered. The instructions for accessing the memory region are extracted in the exception handling and formed into an instruction set. S4: Analyze the address offset and access granularity of memory access for each instruction in the instruction set; S5: Determine the field boundaries of the data packet based on the address offset and access granularity to obtain the protocol format of the data packet.

[0008] Further: The monitoring of the data receiving function of the communication software in step S1 specifically includes: intercepting the network data receiving function of the communication software through HOOK technology, and extracting the memory address parameter and length parameter of the data packet from the parameters of the network data receiving function, wherein the memory address parameter indicates the storage location of the data packet in memory, and the length parameter indicates the size of the memory area.

[0009] Further: In step S2, setting the memory region corresponding to the data packet to a restricted access state specifically includes: calling the memory protection function provided by the system to set the access attribute of the memory region where the data packet is located to inaccessible, so that any read, write or execution operation on the memory region will trigger an access exception.

[0010] Furthermore, step S3 also includes: After extracting the instruction to access memory, temporarily remove the access restrictions on the memory region and set a breakpoint at the next instruction immediately following that instruction. When the program reaches the breakpoint, the memory region is reset to a restricted access state, and subsequent memory access operations are monitored.

[0011] Furthermore, the access granularity in step S4 includes at least one of byte, word, and double word, and the granularity of memory access for each instruction is determined by parsing the opcode and operands of the instruction.

[0012] Furthermore, step S5 also includes: When multiple instructions access the same memory address with the same granularity, the granularity size corresponding to that memory address is determined as the size of the protocol field; When multiple instructions access the same memory address at different granularities, the memory address is identified as the starting position of the structure. The instruction with the largest access granularity accesses the entire size of the structure, while the other smaller granularities access the fields inside the structure.

[0013] Furthermore, the method for identifying fields within a structure includes: sorting all instructions that access the same structure address according to the access address offset; identifying instructions whose access granularity covers the range of subsequent memory accesses and determining them as structure access instructions; arranging the covered smaller granularity access instructions according to the address offset order to determine the position and size of each field within the structure.

[0014] Furthermore, it also includes: repeatedly executing steps S1 to S5 on multiple data packets during the communication process; summarizing the field boundary information of all data packets to generate a complete private protocol format specification.

[0015] This invention also provides a proprietary protocol analysis device based on memory data read / write monitoring, characterized in that it includes: The data packet location module is used to monitor the data receiving function of the communication software and obtain the storage address and size of the received data packets in memory. A memory marking module is used to set the memory region corresponding to the data packet to a restricted access state. The instruction extraction module is used to extract the instructions for accessing the memory region when an exception is triggered by the program accessing the memory region, and form an instruction set. The instruction analysis module is used to analyze the address offset and access granularity of memory access for each instruction in the instruction set; The format parsing module is used to determine the field boundaries of the data packet based on the address offset and access granularity, and obtain the protocol format of the data packet.

[0016] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above-described private protocol analysis method based on memory data read / write monitoring.

[0017] Compared with the prior art, the present invention has the following advantages: I. This invention marks the memory region where the data packet is located and sets access restrictions, and uses a memory access exception mechanism to automatically capture all memory access instructions when the communication software processes the data packet. This eliminates the need for analysts to perform complex disassembly code tracing and manual debugging, significantly reducing the requirements for analysts' reverse analysis capabilities and greatly improving the efficiency of protocol analysis.

[0018] Second, this invention determines the boundaries of data packet fields by analyzing the access granularity (e.g., byte, word, double word, etc.) of each memory access instruction. This analysis method, based on the actual processing logic of the program, can accurately identify the boundary positions in the protocol where processing is done in units of meaningful fields. Since communication software inevitably performs read and write operations according to the field structure specified by the protocol when processing data packets, monitoring the granularity of these access operations can directly reflect the actual field division of the protocol. Even for dynamic fields with variable content, their boundaries and sizes can be accurately determined through the actual access behavior of the program, thus effectively solving the problem of low accuracy in identifying dynamic fields in existing sequence alignment methods.

[0019] Third, this invention addresses the situation where multiple instructions access the same memory address. By analyzing the differences in access granularity, it can automatically identify the structure fields and their internal subfields in the protocol, further improving the parsing capability and accuracy for complex protocol formats. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a private protocol analysis method based on memory data read / write monitoring in one embodiment of the present invention. Figure 2 This is a schematic diagram of the framework of a private protocol analysis device based on memory data read / write monitoring in another embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The proprietary protocol analysis method based on memory data read / write monitoring provided by this invention is based on the core idea of ​​monitoring the read / write operations of communication software on data packets in memory, and inferring the field structure of the protocol based on the granularity and location of the read / write operations. Since communication software inevitably operates on data packets in units of the fields defined in the protocol, the protocol format can be accurately reconstructed by capturing these operation instructions and analyzing their access patterns.

[0023] In one specific embodiment, taking the analysis of a proprietary TCP protocol used by an industrial control system under a Windows operating system environment as an example, the method of the present invention includes the following steps: S1: Monitor the data receiving function of the communication software to obtain the memory address and size of the received data packets. Specifically, in Windows systems, TCP communication typically uses the `recv` function to receive data. The function prototype is `int recv(SOCKETs, char *buf, int len, int flags)`, where the `buf` parameter points to the address of the receive buffer, and the `len` parameter represents the size of the buffer. By using hooking techniques to intercept the execution of the `recv` function when it is called, and extracting the values ​​of `buf` and `len` from the function parameters, the specific location and size of the received data packets in memory can be obtained. For example, when the communication software calls the `recv` function and receives a 100-byte data packet, assuming the value of the `buf` parameter is 0xA69148 and the value of the `len` parameter is 100, it indicates that the data packet is stored in a 100-byte memory area starting at address 0xA69148.

[0024] S2: Set the memory region corresponding to the data packet to a restricted access state. Specifically, after obtaining the memory location of the data packet, the memory region needs to be marked for monitoring subsequent access operations. In Windows systems, the VirtualProtect function can be called to modify the protection attributes of memory pages. The function prototype is BOOLVirtualProtect(LPVOID lpAddress, SIZE_T dwSize, DWORD flNewProtect, PDWORDlpflOldProtect). Setting the flNewProtect parameter to PAGE_NOACCESS will set the target memory region to an inaccessible state. Continuing the previous example, after calling VirtualProtect(0xA69148, 100, PAGE_NOACCESS, &oldProtect), any operation attempting to access this 100-byte memory region will trigger an access exception.

[0025] S3: When the program accesses the memory region, an exception is triggered. The instructions for accessing the memory region are extracted in the exception handling to form an instruction set.

[0026] Specifically, when subsequent code in the communication software attempts to read or process the data packet, an access exception is immediately triggered because the memory region has been set to inaccessible. The exception handler can retrieve the instruction information that caused the exception. For example, an instruction like `MOV EAX, DWORD PTR DS: [0xA69148]` attempts to read a double word (4 bytes) of data from the beginning of the data packet; this instruction will trigger an exception. In the exception handler, the information of this instruction is first recorded, including the address it accesses (0xA69148) and the access granularity (DWORD, 4 bytes). Then, the access restriction on that memory region is temporarily lifted, allowing the instruction to execute normally.

[0027] To continue monitoring subsequent memory access operations, an `int 3` breakpoint is set at the instruction following the current access instruction. `int 3` is a breakpoint instruction in the x86 architecture, with machine code 0xCC. When the program executes to the location where the `int 3` breakpoint is set, an exception will be triggered again. When handling this exception, the `int 3` breakpoint is first removed, the original instruction is restored, and then the memory region containing the data packet is reset to an inaccessible state to capture the next memory access operation. By using this combination of memory breakpoints and `int 3` breakpoints, all memory access instructions during the entire data packet processing process of the communication software can be completely captured.

[0028] In actual operation, communication software may access different parts of data packets at different granularities. For example, when processing a data packet containing a message header and a message body, the following access sequence may occur: The first instruction MOVEAX, DWORD PTR DS: [0xA69148] reads 4 bytes from offset 0, which may be the message type field; the second instruction MOV AX, WORD PTR DS: [0xA6914C] reads 2 bytes from offset 4, which may be the message length field; the third instruction MOV AL, BYTE PTR DS: [0xA6914E] reads 1 byte from offset 6, which may be the flag field. By collecting these instructions, a complete instruction set is formed.

[0029] S4: Analyze the address offset and access granularity of memory access for each instruction in the instruction set.

[0030] Specifically, when analyzing an instruction set, it's necessary to extract key information from each instruction. Taking the instruction `MOV EAX, DWORD PTR DS: [0xA69148]` as an example, by parsing the instruction's opcode and operands, we can determine that the memory address accessed by this instruction is 0xA69148, with an access granularity of DWORD (4 bytes). Subtracting the starting address of the data packet from the access addresses of all instructions yields the relative offset. For example, if the data packet's starting address is 0xA69148, then the relative offset of access address 0xA6914C is 4, and the relative offset of access address 0xA6914E is 6.

[0031] S5: Determine the field boundaries of the data packet based on the address offset and access granularity to obtain the protocol format of the data packet.

[0032] Specifically, based on the access offsets and granularity information of all collected instructions, the field structure of the data packet can be inferred. In the simplest case, if a certain offset position is accessed only by instructions of a single granularity, then the size of the field corresponding to that position is the size of the access granularity. For example, if offset 0 is only accessed by instructions of DWORD granularity, then offset 0 to offset 3 can be inferred to be a 4-byte field; if offset 4 is only accessed by instructions of WORD granularity, then offset 4 to offset 5 can be inferred to be a 2-byte field.

[0033] In more complex scenarios, multiple instructions might access the same memory address with different granularities. For example, the instruction `MOV EAX, DWORD PTR DS: [0xA69150]` accesses the address at offset 8 with a double-word granularity, reading 4 bytes; while other instructions `MOV AL, BYTE PTR DS: [0xA69150]`, `MOV BL, BYTE PTR DS: [0xA69151]`, and `MOV CX, WORD PTR DS: [0xA69152]` access the addresses at offsets 8, 9, and 10 with byte and word granularities, respectively. This typically indicates that offset 8 represents a structure with a total size of 4 bytes, containing a 1-byte field at offset 0, a 1-byte field at offset 1, and a 2-byte field at offset 2. Instructions accessing the entire structure reflect its overall size, while instructions accessing specific internal locations reveal the field divisions within the structure.

[0034] To accurately identify this structure, all instructions accessing the same starting address need to be analyzed. First, identify the instruction with the largest access granularity that covers the range of other accesses; the range accessed by this instruction represents the complete size of the structure. Then, sort the other smaller-granularity accesses within the covered range by address offset; these accesses correspond to the individual fields within the structure. In this way, not only can the existence of the structure be identified, but its internal field layout can also be accurately determined.

[0035] In practical applications, communication software may process multiple different types of data packets. For example, an industrial control protocol may contain various types such as control command packets, status query packets, and data transmission packets. By performing the aforementioned monitoring and analysis process on each type of data packet, the field structure of each data packet can be obtained. By comparing the analysis results of different data packets, common characteristics of the protocol can also be discovered, such as the first four bytes of all data packets being message type fields, and the fifth and sixth bytes being message length fields.

[0036] To verify the accuracy of the analysis results, cross-validation can be performed using multiple data packet samples. When multiple instructions access the same memory address at the same granularity, these instructions can corroborate each other, increasing the confidence level of field boundary determination. For example, if the position at offset 4 is consistently accessed by instructions at the WORD granularity during the processing of 10 data packets, then it can be highly confident that this position is a 2-byte field.

[0037] By summarizing and organizing the analysis results of all types of data packets, a complete private protocol format specification can be generated. This specification contains detailed field definitions for each message type in the protocol, including field offset positions, field lengths, and possible value ranges. For example, the analysis results might show that: the message type field is located at offset 0, with a length of 4 bytes, and values ​​of 0x01 indicating control commands, 0x02 indicating status queries, and 0x03 indicating data transmission; the message length field is located at offset 4, with a length of 2 bytes, indicating the total number of bytes in the message; and the data field starts at offset 6, with its length determined by the message length field.

[0038] In another embodiment, the present invention also provides a private protocol analysis device based on memory data read / write monitoring, characterized in that it includes: The data packet location module is used to monitor the data receiving function of the communication software and obtain the storage address and size of the received data packets in memory. A memory marking module is used to set the memory region corresponding to the data packet to a restricted access state. The instruction extraction module is used to extract the instructions for accessing the memory region when an exception is triggered by the program accessing the memory region, and form an instruction set. The instruction analysis module is used to analyze the address offset and access granularity of memory access for each instruction in the instruction set; The format parsing module is used to determine the field boundaries of the data packet based on the address offset and access granularity, and obtain the protocol format of the data packet.

[0039] In another embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above-described private protocol analysis method based on memory data read / write monitoring.

[0040] The method of this invention has demonstrated excellent results in actual testing. Taking a proprietary protocol of an industrial control system as an example, this protocol contains 15 different message types, with each message having between 5 and 20 fields. Using traditional disassembly analysis methods, it would take an experienced reverse engineer about a week to complete the analysis of the protocol format. However, using the method of this invention, the entire analysis process can be completed automatically within a few hours, and the accuracy rate of field boundary identification reaches over 95%. In particular, for dynamic fields in the protocol, such as variable-length data payload fields and optional fields that change according to message type, this method can accurately identify their boundaries, while traditional sequence alignment methods typically have an accuracy rate of less than 70% in identifying these dynamic fields.

[0041] It should be noted that although the above embodiments are illustrated using Windows systems and the TCP protocol as examples, the method of the present invention is not limited to a specific operating system or transmission protocol. In Linux systems, the ptrace system call can be used to implement similar memory monitoring functions; for the UDP protocol, the recvfrom function can be monitored instead of the recv function. Furthermore, this method is not only applicable to network protocol analysis, but can also be extended to other scenarios requiring understanding of data organization methods, such as file format analysis and data structure reverse engineering.

[0042] This invention cleverly utilizes memory access monitoring technology to transform the problem of protocol reverse engineering into a problem of memory access pattern recognition. This not only significantly reduces the difficulty and time cost of analysis, but also significantly improves the accuracy of the analysis results, providing strong technical support for the research and application of proprietary protocols.

[0043] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for analyzing proprietary protocols based on memory data read / write monitoring, characterized in that, include: S1: Monitor the data receiving function of the communication software to obtain the storage address and size of the received data packets in memory; S2: Set the memory region corresponding to the data packet to a restricted access state; S3: When the program accesses the memory region, an exception is triggered. The instructions for accessing the memory region are extracted in the exception handling and formed into an instruction set. S4: Analyze the address offset and access granularity of memory access for each instruction in the instruction set; S5: Determine the field boundaries of the data packet based on the address offset and access granularity to obtain the protocol format of the data packet.

2. The private protocol analysis method based on memory data read / write monitoring according to claim 1, characterized in that, The monitoring of the data receiving function of the communication software in step S1 specifically includes: intercepting the network data receiving function of the communication software using HOOK technology, and extracting the memory address parameter and length parameter of the data packet from the parameters of the network data receiving function, wherein the memory address parameter indicates the storage location of the data packet in memory, and the length parameter indicates the size of the memory area.

3. The method for analyzing private protocols based on memory data read / write monitoring according to claim 1, characterized in that, In step S2, setting the memory region corresponding to the data packet to a restricted access state specifically includes: calling the memory protection function provided by the system to set the access attribute of the memory region where the data packet is located to be inaccessible, so that any read, write or execution operation on the memory region will trigger an access exception.

4. The private protocol analysis method based on memory data read / write monitoring according to claim 1, characterized in that, Step S3 further includes: After extracting the instruction to access memory, temporarily remove the access restrictions on the memory region and set a breakpoint at the next instruction immediately following that instruction. When the program reaches the breakpoint, the memory region is reset to a restricted access state, and subsequent memory access operations are monitored.

5. The method for analyzing private protocols based on memory data read / write monitoring according to claim 1, characterized in that, The access granularity in step S4 includes at least one of byte, word, and double word. The granularity of memory access for each instruction is determined by parsing the opcode and operands of the instruction.

6. The method for analyzing private protocols based on memory data read / write monitoring according to claim 1, characterized in that, Step S5 further includes: When multiple instructions access the same memory address with the same granularity, the granularity size corresponding to that memory address is determined as the size of the protocol field; When multiple instructions access the same memory address at different granularities, the memory address is identified as the starting position of the structure. The instruction with the largest access granularity accesses the entire size of the structure, while the other smaller granularities access the fields inside the structure.

7. The method for analyzing private protocols based on memory data read / write monitoring according to claim 6, characterized in that, Methods for identifying fields within a structure include: sorting all instructions that access the same structure address according to the access address offset; identifying instructions whose access granularity covers the range of subsequent memory accesses and determining them as structure access instructions; arranging the covered smaller granularity access instructions according to their address offsets to determine the position and size of each field within the structure.

8. The method for analyzing private protocols based on memory data read / write monitoring according to claim 1, characterized in that, It also includes: repeatedly executing steps S1 to S5 on multiple data packets during the communication process; summarizing the field boundary information of all data packets to generate a complete private protocol format specification.

9. A proprietary protocol analysis device based on memory data read / write monitoring, characterized in that, include: The data packet location module is used to monitor the data receiving function of the communication software and obtain the storage address and size of the received data packets in memory. A memory marking module is used to set the memory region corresponding to the data packet to a restricted access state. The instruction extraction module is used to extract the instructions for accessing the memory region when an exception is triggered by the program accessing the memory region, and form an instruction set. The instruction analysis module is used to analyze the address offset and access granularity of memory access for each instruction in the instruction set; The format parsing module is used to determine the field boundaries of the data packet based on the address offset and access granularity, and obtain the protocol format of the data packet.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the private protocol analysis method based on memory data read / write monitoring as described in any one of claims 1 to 8.

Images