A method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding

Abstract

This invention discloses a method for detecting buffer out-of-bounds access errors in C and C++ based on memory-mapped encoding. During software testing, an efficient memory error detection tool is needed to detect memory errors present in the software, such as buffer overflows and buffer reuse after release. However, currently used C / C++ memory error detection tools cannot detect memory errors involving non-contiguous out-of-bounds access to buffers. This invention, based on memory-mapped encoding and instruction instrumentation, designs a buffer out-of-bounds access error detection method with broader coverage.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, and in particular relates to a method for detecting out-of-bounds access errors in C and C++ buffers based on memory mapping encoding. Background Technology

[0002] In 2011, Google engineers first proposed Address Sanitizer (Asan) for rapid memory error detection. Its working principle involves inserting extra code segments during compilation to track memory allocation and access, thereby detecting various memory errors, including buffer overflow, use-after-free, and wild pointer errors. According to Google's public information, Asan has detected over 300 previously undiscovered vulnerabilities on the Chromium project, demonstrating its powerful detection capabilities. Compared to previous tools, Asan is characterized by high efficiency, wide detection range, and ease of use. It supports mainstream C / C++ compilers such as Clang, GCC, and MSVC, and can be integrated with platforms such as Linux, Windows, macOS, and Android. In error detection, Asan can precisely locate the anomaly and provide detailed error reports. After more than a decade of development, Asan's functionality has continuously improved, and it has now become a commonly used memory debugging and code defect detection tool.

[0003] However, Asan is not perfect in its design and may miss detections. For example, when a buffer is accessed out of bounds without contiguous access (i.e., accessing another buffer across red zones), Asan cannot detect the error. Because Asan can only determine whether the accessed area is valid, but not whether the access method is reasonable, Asan may miss detections. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding.

[0005] The objective of this invention is achieved through the following technical solution: a method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding, comprising the following steps:

[0006] (1) Hijack all memory allocation functions and, after each memory allocation, set each shadow memory byte in the memory block to information that identifies the size of the memory block;

[0007] (2) Use LLVM to instrument all getelementptr instructions in the program to detect whether all memory accesses are out of bounds;

[0008] (3) For statements that access the program out of bounds, forcibly terminate the program and print an error message.

[0009] Furthermore, the mapping relationship between the address of the shadow memory (shadow_memory) and the address of the actual memory (Addr) is as follows: the value of the address of the actual memory (Addr) is shifted to the right by Scale bits and then a fixed offset value (Offset) is added to obtain the address of the shadow memory (shadow_memory).

[0010] Furthermore, in step (1), the method for setting each shadow memory byte specifically includes the following sub-steps:

[0011] (a1) The second to fourth bits of each shadow memory byte store the offset from the original memory address to the end of the memory block, the offset being the size of the remaining accessible bytes of the memory address;

[0012] (a2) The first bit of each shadow memory byte indicates whether it needs to be concatenated with the accessible size information of the previous shadow memory byte when calculating the offset: when the first bit of the shadow memory byte is 0, it indicates that it does not need to be concatenated with the accessible size information stored in the previous shadow memory byte; when the first bit of the shadow memory byte is 1, it indicates that it needs to be concatenated with the accessible size information stored in the previous shadow memory byte.

[0013] (a3) The fifth bit of each shadow memory byte indicates whether it needs to be concatenated with the accessible size information of the next shadow memory byte when calculating the offset: when the fifth bit of the shadow memory byte is 0, it indicates that it does not need to be concatenated with the accessible size information of the next shadow memory byte; when the fifth bit of the shadow memory byte is 1, it indicates that it needs to be concatenated with the accessible size information of the next shadow memory byte.

[0014] (a4) If the size of the memory block is a multiple of 8, then the sixth to eighth bits of each shadow memory byte corresponding to the memory block are set to 0; if the size of the memory block is not a multiple of 8, then the memory block is divided into two parts. The first part is from the starting address Head_Addr of the memory block to the address of the first memory block, which is Head_Addr + Size - Size%8, where Size%8 represents the remainder obtained by dividing Size by 8; the second part is from the address of the first memory block to the end address Tail_Addr of the memory block. The sixth to eighth bits of each shadow memory byte corresponding to the first part are set to 0, and the sixth to eighth bits of each shadow memory byte corresponding to the second part are set to the last three bits of the binary representation of the decimal number obtained by converting Size%8.

[0015] Furthermore, step (2) specifically includes the following sub-steps:

[0016] (2.1) Obtain the base address sourceAddr and the access offset Offset of the memory access according to the getelementptr instruction;

[0017] (2.2) Calculate the shadow memory address ShadowAddr based on the base address sourceAddr, and calculate the remaining accessible size information Offset_To_End stored in the shadow memory;

[0018] (2.3) Compare the access offset Offset with the remaining accessible size information Offset_To_End. When the access offset Offset is greater than the remaining accessible size information Offset_To_End, an out-of-bounds memory access is detected, and the program under test then performs error handling.

[0019] Furthermore, the calculation of the remaining accessible size information Offset_To_End stored in the shadow memory specifically includes the following sub-steps:

[0020] (c1) Take the first bit of the byte stored at the shadow memory address ShadowAddr. When the bit is 1, decrease the shadow memory address ShadowAddr by one byte until the bit is calculated to be 0.

[0021] (c2) Set the initial value of the remaining accessible size information Offset_To_End to 0; take the value of the second to fourth bits of the byte stored at the shadow memory address ShadowAddr, shift the current remaining accessible size information Offset_To_End three bits to the left and add the value to it, and assign the result to Offset_To_End.

[0022] Take the fifth bit of the byte stored at the shadow memory address ShadowAddr. If the value of this bit is 1, increment the shadow memory address shadowAddr by one byte.

[0023] (c3) Repeat step (c2) until the fifth bit of the byte stored at the shadow memory address ShadowAddr is 0. At this point, the remaining accessible size information obtained at the end is taken as the final result.

[0024] (c4) If the bytes stored at the shadow memory address ShadowAddr are {0,0,0,0,1,0,0,0} or {0,0,0,0,1,a,b,c}, where a,b,c are 0 or 1, assign the current shadow memory address to Begin_ShadowAddr, then increment the shadow memory address ShadowAddr by one until the original memory corresponding to the shadow memory address is inaccessible. Subtract the actual memory address corresponding to the initial shadow memory address from the actual memory address corresponding to the shadow memory address ShadowAddr, and use the difference as the remaining accessible size information Offset_To_End.

[0025] Furthermore, in step (3), the error information includes the location of the error, stack information, and shadow memory information.

[0026] The beneficial effects of this invention are:

[0027] 1) Existing memory detection technologies cannot detect memory errors caused by non-contiguous out-of-bounds access. This invention uses memory mapping encoding and instruction instrumentation to detect all memory errors caused by non-contiguous out-of-bounds access.

[0028] 2) In addition to detecting memory errors caused by non-continuous out-of-bounds access, this invention can also detect other types of memory errors that can be detected by existing technologies. Attached Figure Description

[0029] Figure 1 This is an architecture diagram of a C and C++ buffer out-of-bounds access error detection method based on memory-mapped encoding.

[0030] Figure 2 A diagram showing the data information stored in the shadow memory corresponding to a 16-byte memory block;

[0031] Figure 3 This is a diagram showing the error message when accessing non-continuous boundaries. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0033] Example 1

[0034] like Figure 1 As shown, this invention provides a method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding, comprising the following steps:

[0035] (1) Hijack all memory allocation functions and, after each memory allocation, set each shadow memory byte in the memory block to information that identifies the size of the memory block.

[0036] For example, for a 64-byte memory block, in a 64-bit Linux system, the mapped shadow memory is eight bytes long, and its shadow memory information is set as follows: Figure 2 As shown.

[0037] The mapping relationship between the address of the shadow memory (shadow_memory) and the address of the actual memory (Addr) is as follows: the address of the shadow memory (shadow_memory) is obtained by shifting the value of the address of the actual memory (Addr) to the right by Scale bits and then adding a fixed offset value (Offset).

[0038] In step (1), the method for setting each shadow memory byte specifically includes the following sub-steps:

[0039] (a1) The second to fourth bits of each shadow memory byte store the offset from the original memory address to the end of the memory block, the offset being the size of the remaining accessible bytes of the memory address.

[0040] (a2) The first bit of each shadow memory byte indicates whether it needs to be concatenated with the accessible size information of the previous shadow memory byte when calculating the offset: when the first bit of the shadow memory byte is 0, it indicates that it does not need to be concatenated with the accessible size information stored in the previous shadow memory byte; when the first bit of the shadow memory byte is 1, it indicates that it needs to be concatenated with the accessible size information stored in the previous shadow memory byte.

[0041] (a3) The fifth bit of each shadow memory byte indicates whether it needs to be concatenated with the accessible size information of the next shadow memory byte when calculating the offset: when the fifth bit of the shadow memory byte is 0, it indicates that it does not need to be concatenated with the accessible size information of the next shadow memory byte; when the fifth bit of the shadow memory byte is 1, it indicates that it needs to be concatenated with the accessible size information of the next shadow memory byte.

[0042] (a4) If the size of the memory block is a multiple of 8, then the sixth to eighth bits of each shadow memory byte corresponding to the memory block are set to 0; if the size of the memory block is not a multiple of 8, then the memory block is divided into two parts. The first part is from the starting address Head_Addr of the memory block to the address of the first memory block, which is Head_Addr + Size - Size%8, where Size%8 represents the remainder obtained by dividing Size by 8; the second part is from the address of the first memory block to the end address Tail_Addr of the memory block. The sixth to eighth bits of each shadow memory byte corresponding to the first part are set to 0, and the sixth to eighth bits of each shadow memory byte corresponding to the second part are set to the last three bits of the binary representation of the decimal number obtained by converting Size%8.

[0043] When Size%8 is 1, the decimal number 1 is converted to binary 001, and then the sixth to eighth bits of the shadow memory byte are set to 0,0,1 sequentially; when Size%8 is 2, the decimal number 2 is converted to binary 010, and then the sixth to eighth bits of the shadow memory byte are set to 0,1,0 sequentially; when Size%8 is 3, the decimal number 3 is converted to binary 011, and then the sixth to eighth bits of the shadow memory byte are set to 0,1,1 sequentially; when Size%8 is 4, the decimal number 4 is converted to binary 100. Subsequently, the sixth to eighth bits of the shadow memory byte are set to 1, 0, 0 in sequence; when Size%8 is 5, the decimal number 5 is converted to binary 101, and the sixth to eighth bits of the shadow memory byte are set to 1, 0, 1 in sequence; when Size%8 is 6, the decimal number 6 is converted to binary 110, and the sixth to eighth bits of the shadow memory byte are set to 1, 1, 0 in sequence; when Size%8 is 7, the decimal number 7 is converted to binary 111, and the sixth to eighth bits of the shadow memory byte are set to 1, 1, 1 in sequence.

[0044] (2) Use LLVM to instrument all getelementptr instructions in the program to detect whether all memory accesses are out of bounds.

[0045] Step (2) specifically includes the following sub-steps:

[0046] (2.1) Obtain the base address sourceAddr and the access offset Offset of the memory access according to the getelementptr instruction.

[0047] (2.2) Calculate the shadow memory address ShadowAddr based on the base address sourceAddr, and calculate the remaining accessible size information Offset_To_End stored in the shadow memory.

[0048] The calculation of the remaining accessible size information Offset_To_End stored in the shadow memory specifically includes the following sub-steps:

[0049] (c1) Take the first bit of the byte stored at the shadow memory address ShadowAddr. When the bit is 1, decrease the shadow memory address ShadowAddr by one byte until the bit is calculated to be 0.

[0050] (c2) Set the initial value of the remaining accessible size information Offset_To_End to 0; take the value of the second to fourth bits of the byte stored at the shadow memory address ShadowAddr, shift the current remaining accessible size information Offset_To_End three bits to the left and add the value to it, and assign the result to Offset_To_End.

[0051] Take the fifth bit of the byte stored at the shadow memory address ShadowAddr. If the value of this bit is 1, increment the shadow memory address shadowAddr by one byte.

[0052] (c3) Repeat step (c2) until the fifth bit of the byte stored at the shadow memory address ShadowAddr is 0. At this point, the remaining accessible size information obtained is taken as the final result.

[0053] (c4) If the bytes stored at the shadow memory address ShadowAddr are {0,0,0,0,1,0,0,0} or {0,0,0,0,1,a,b,c}, where a,b,c are 0 or 1, assign the current shadow memory address to Begin_ShadowAddr, then increment the shadow memory address ShadowAddr by one until the original memory corresponding to the shadow memory address is inaccessible. Subtract the actual memory address corresponding to the initial shadow memory address from the actual memory address corresponding to the shadow memory address ShadowAddr, and use the difference as the remaining accessible size information Offset_To_End.

[0054] (2.3) Compare the access offset Offset with the remaining accessible size information Offset_To_End. When the access offset Offset is greater than the remaining accessible size information Offset_To_End, an out-of-bounds memory access is detected, and the program under test then performs error handling.

[0055] For example, when compiling the C language statement a[0x30] = 0, a getelementptr statement will be generated, and out-of-bounds access detection code will be added after the getelementptr statement.

[0056] (3) For statements that access the program out of bounds, forcibly terminate the program and print an error message.

[0057] In step (3), the error information includes the location of the error, stack information, and shadow memory information.

[0058] For example, if a program attempts to access a memory location at an offset of 32 bytes from a 16-byte memory block, it will throw an error. The specific error message would be as follows: Figure 3 As shown.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding, characterized in that, Includes the following steps: (1) Hijack all memory allocation functions and, after each memory allocation, set each shadow memory byte in the memory block to information that identifies the size of the memory block; (2) Use LLVM to instrument all getelementptr instructions in the program to detect whether all memory accesses are out of bounds; Step (2) specifically includes the following sub-steps: (2.1) Obtain the base address sourceAddr and the access offset Offset of the memory access according to the getelementptr instruction; (2.2) Calculate the shadow memory address ShadowAddr based on the base address sourceAddr, and calculate the remaining accessible size information Offset_To_End stored in the shadow memory; (2.3) Compare the access offset Offset with the remaining accessible size information Offset_To_End. When the access offset Offset is greater than the remaining accessible size information Offset_To_End, an out-of-bounds memory access is detected, and the program under test then performs error handling. The calculation of the remaining accessible size information Offset_To_End stored in the shadow memory specifically includes the following sub-steps: (c1) Take the first bit of the byte stored at the shadow memory address ShadowAddr. When the bit is 1, decrease the shadow memory address ShadowAddr by one byte until the bit is calculated to be 0. (c2) Set the initial value of the remaining accessible size information Offset_To_End to 0; take the value of the second to fourth bits of the byte stored at the shadow memory address ShadowAddr, shift the current remaining accessible size information Offset_To_End three bits to the left and add the value to it, and assign the result to Offset_To_End. Take the fifth bit of the byte stored at the shadow memory address ShadowAddr. If the value of this bit is 1, increment the shadow memory address shadowAddr by one byte. (c3) Repeat step (c2) until the fifth bit of the byte stored at the shadow memory address ShadowAddr is 0. At this point, the remaining accessible size information obtained at the end is taken as the final result. (c4) If the bytes stored in the shadow memory address ShadowAddr are {0,0,0,0,1,0,0,0} or {0,0,0,0,1,a,b,c}, where a,b,c are 0 or 1, assign the current shadow memory address to Begin_ShadowAddr, then increment the shadow memory address ShadowAddr by one until the original memory corresponding to the shadow memory address is inaccessible. Subtract the actual memory address corresponding to the initial shadow memory address from the actual memory address corresponding to the shadow memory address ShadowAddr, and use the difference as the remaining accessible size information Offset_To_End; (3) For statements that access the program out of bounds, forcibly terminate the program and print error messages.

2. The method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding according to claim 1, characterized in that, The mapping relationship between the address of the shadow memory (shadow_memory) and the address of the actual memory (Addr) is as follows: the address of the shadow memory (shadow_memory) is obtained by shifting the value of the address of the actual memory (Addr) to the right by Scale bits and then adding a fixed offset value (Offset).

3. The method for detecting out-of-bounds access errors in C and C++ buffers based on memory-mapped encoding according to claim 1, characterized in that, In step (1), the method for setting each shadow memory byte specifically includes the following sub-steps: (a1) The second to fourth bits of each shadow memory byte store the offset from the original memory address to the end of the memory block, the offset being the size of the remaining accessible bytes of the memory address; (a2) The first bit of each shadow memory byte indicates whether it needs to be concatenated with the accessible size information of the previous shadow memory byte when calculating the offset: when the first bit of the shadow memory byte is 0, it indicates that it does not need to be concatenated with the accessible size information stored in the previous shadow memory byte; when the first bit of the shadow memory byte is 1, it indicates that it needs to be concatenated with the accessible size information stored in the previous shadow memory byte. (a3) The fifth bit of each shadow memory byte indicates whether it needs to be concatenated with the accessible size information of the next shadow memory byte when calculating the offset: when the fifth bit of the shadow memory byte is 0, it indicates that it does not need to be concatenated with the accessible size information of the next shadow memory byte; when the fifth bit of the shadow memory byte is 1, it indicates that it needs to be concatenated with the accessible size information of the next shadow memory byte. (a4) If the size of the memory block is a multiple of 8, then the sixth to eighth bits of each shadow memory byte corresponding to the memory block are set to 0; if the size of the memory block is not a multiple of 8, then the memory block is divided into two parts. The first part is from the starting address Head_Addr of the memory block to the address of the first memory block, which is Head_Addr + Size - Size%8, where Size%8 represents the remainder obtained by dividing Size by 8; the second part is from the address of the first memory block to the end address Tail_Addr of the memory block. The sixth to eighth bits of each shadow memory byte corresponding to the first part are set to 0, and the sixth to eighth bits of each shadow memory byte corresponding to the second part are set to the last three bits of the binary representation of the decimal number obtained by converting Size%8.

4. The method for detecting out-of-bounds access errors of C and C++ buffers based on memory-mapped encoding according to claim 1, characterized in that, In step (3), the error information includes the location of the error, stack information, and shadow memory information.

Images