Real-time memory recovery method and system with fusion of process memory and instruction sequence

Abstract

The application belongs to the technical field of program analysis, and relates to a real-time memory recovery method and system for process memory and instruction sequence fusion. The method comprises the following steps: establishing an index for memory pages in a process memory dump file to form a memory page index; analyzing an instruction record file to establish a mapping relationship between an instruction ID and a file position and form an instruction index; scanning memory patch files in a specified directory, generating incremental memory checkpoints at specified intervals based on the memory page index, the instruction index and the memory patch files; according to a target instruction ID and a memory address, using the memory checkpoints and incremental data to quickly restore and query the memory state at a target time, and using a multi-level cache mechanism to improve the memory query performance. The application realizes efficient processing and fast query of large process memory dump data, and solves the problems of large memory dump file size, low query efficiency and excessive storage space occupation in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of program analysis technology, specifically relating to a real-time memory recovery method and system that integrates process memory and instruction sequence. Background Technology

[0002] In existing technologies, memory recovery methods primarily focus on two directions: static memory dumping and instruction-level tracing. These methods attempt to restore the memory state for subsequent analysis in scenarios involving process runtime anomalies, crashes, or debugging. Several typical existing implementation schemes are briefly described below: 1) Recovery methods based on memory dumps. These methods restore and analyze the runtime state of a process by generating a memory dump file. For example, common core dump tools can completely save the memory space at a specific moment. This method treats process memory as an independent snapshot, directly exporting the memory image through system calls, and loading the file to reconstruct the state during recovery. This method can quickly capture the memory layout, supports offline analysis, and basic process restart. In its implementation, the system generates a dump file when a process encounters an anomaly and uses a loader to restore the memory. This method provides memory recovery capabilities with minimal intervention.

[0003] 2) Instruction Sequence Replay-Based Recovery Method. This method recovers memory by recording or replaying the sequence of instructions executed by the program. It employs dynamic binary instrumentation tools or hardware tracing mechanisms to capture the program's instruction flow and attempt to deduce the memory write process. This method recovers memory based on instruction timing, enabling the simulation of partial execution paths and improving the context-dependent nature of the recovery.

[0004] 3) Checkpoint-based recovery method. This method saves checkpoints periodically during runtime, and rolls back to the most recent checkpoint and continues execution in case of errors or recovery. This method supports memory recovery across the entire environment, and by defining checkpoints to simulate the process memory layout, it can improve the completeness of memory recovery.

[0005] The main drawbacks of existing process memory recovery methods include: 1) Recovery methods based on memory dumps. Recovery methods based on memory dumps primarily rely on generating a complete memory snapshot of the process to reconstruct its state. However, this approach has the following drawbacks: First, it depends on the completeness and accuracy of the dump file. Different system environments or process states may lead to untimely dumps or missing content, resulting in incomplete restored memory information, or even information that does not match the actual execution state. Second, the dump process is usually statically captured, lacking integration of dynamic instruction sequences, which may lead to fuzzy recovery and lost context. This makes it difficult to accurately restore memory dependencies using automated methods. Furthermore, under high resource consumption, the dump file size is enormous, making it inefficient for handling real-time recovery needs.

[0006] 2) Instruction Sequence Replay-Based Recovery Method. This method records the program's instruction execution log and replays the log sequentially to attempt to reconstruct memory changes during debugging. However, this method can only theoretically simulate the execution path and still has the following drawbacks: First, it processes instruction sequences individually, lacking real-time fusion of memory states. The generated instruction sequences are often isolated instruction operations, resulting in a large number of instruction records being ineffective for memory recovery, leading to significant waste of time and resources. Second, it is only suitable for scenarios with simple instruction sequences and has weak processing capabilities for complex multi-threaded scenarios, making it difficult to recover the complete memory context. Furthermore, this method cannot directly integrate memory page changes, ignoring initial memory caches or system calls occurring during instruction execution. Lacking higher-level fusion information, it heavily relies on the completeness of the instruction sequence. When the instruction sequence is very large, this method struggles to cover all memory states within a reasonable timeframe, leading to incomplete recovery results.

[0007] 3) Checkpoint-based recovery method. This method proposes a checkpoint-based recovery approach. The core idea is to periodically save memory snapshots of the process and roll back to a specified point to rebuild the state when needed. When a process anomaly is detected, a checkpoint file is generated; when recovery is required, the checkpoint is loaded and the operation is performed; if no complex dependencies are detected, recovery is treated as an independent operation by default. While this method is very simple to implement, its memory recovery capabilities have significant shortcomings: the recovery scope is too limited, only considering memory recovery and failing to recognize more complex instruction sequence fusion, resulting in overly coarse memory recovery results. Secondly, this method relies entirely on static checkpoint generation for state reconstruction. When the process uses dynamic memory allocation or multi-threaded interaction, this method cannot accurately capture the actual execution semantics, leading to recovery errors or omissions. Furthermore, this method lacks context sensitivity; the recovery is mainly based on a single checkpoint, without considering the program's interactions with memory and instructions over a larger scope. This method cannot combine runtime instruction details to achieve real-time recovery, and cannot guarantee the completeness and accuracy of the recovery.

[0008] Existing technologies also disclose methods that reconstruct process states by recording process memory access and instruction event logs at the system level and replaying the operations sequentially during debugging. However, the method of this invention is suitable for recovery scenarios that require real-time fusion of process memory and instruction sequences. Existing methods rely solely on information from the recorded instruction sequences for recovery, resulting in slow recovery speed and low recovery efficiency, and making it difficult to recover memory data that was not processed in the instruction sequence.

[0009] In summary, current methods for process memory recovery, whether based on static memory dumps, instruction sequence replay, or virtualization snapshots, can achieve memory capture and reconstruction to some extent. However, the closest similar technical solutions to this invention often only utilize memory dumps or instruction sequence information individually, lacking the integration of both. Therefore, they have significant limitations in terms of recovery accuracy, real-time performance, and consistency. Summary of the Invention

[0010] To address the above-mentioned problems, this invention provides a real-time memory recovery method and system that integrates process memory and instruction sequence.

[0011] The technical solution adopted in this invention is as follows: A real-time memory recovery method that integrates process memory and instruction sequence includes the following steps: Create an index for the memory pages in the process memory dump file, forming a memory page index; Parse the instruction log file, establish a mapping relationship between instruction IDs and file locations, and form an instruction index; Scan memory patch files in the specified directory and generate incremental memory checkpoints at specified intervals based on memory page index, instruction index, and memory patch files; Based on the target instruction ID and memory address, the memory state at the target time is quickly restored and queried using memory checkpoints and incremental data, and a multi-level caching mechanism is adopted to improve memory query performance.

[0012] Furthermore, the construction of the memory page index includes: extracting all memory page data from the input process memory dump file and constructing an index from the memory page address to the file offset.

[0013] Furthermore, for the memory patch file, the corresponding instruction ID is extracted based on the file name, and a mapping is established between the instruction ID and the memory patch file path.

[0014] Furthermore, the step of generating incremental memory checkpoints at specified intervals includes: starting from a base checkpoint, applying all memory patch files and memory changes caused by instruction execution in chronological order, saving only the modified memory pages, and forming incremental memory checkpoints.

[0015] Furthermore, the method of quickly restoring and querying the memory state at the target time using memory checkpoints and incremental data includes: first, finding the memory checkpoint that is closest to but does not exceed the target instruction ID, loading the memory checkpoint as the base state, and then applying all memory changes from the memory checkpoint to the target instruction ID to restore the memory state at the target time, thereby achieving accurate memory query based on the instruction execution context.

[0016] Furthermore, the method of quickly restoring and querying the memory state at the target time using memory checkpoints and incremental data yields memory query results that include the memory data itself and the corresponding state information, wherein the state information indicates whether the data at each memory location is known.

[0017] Furthermore, the multi-level caching mechanism includes: using on-demand loading and LRU caching strategies for initial memory pages to avoid loading all memory data at once; and using a caching mechanism for checkpoint data and intermediate calculation results to reduce redundant calculations and file I / O operations.

[0018] A real-time memory recovery system that integrates process memory and instruction sequence, comprising: The memory page index building module is responsible for building an index for memory pages in the process memory dump file, forming a memory page index. The instruction index building module is responsible for parsing the instruction record file, establishing a mapping relationship between instruction IDs and file locations, and forming an instruction index; The patch file scanning module is responsible for scanning all memory patch files in the specified directory; The incremental checkpoint generation module is responsible for generating incremental memory checkpoints at specified intervals based on memory page indexes, instruction indexes, and memory patch files. The memory query processing module is responsible for quickly restoring and querying the memory state at the target time based on the target instruction ID and memory address, using memory checkpoints and incremental data. The cache optimization module is responsible for using a multi-level caching mechanism to reduce redundant loading and improve memory query performance.

[0019] This invention achieves efficient processing and fast querying of large process memory dump data through memory paging organization, incremental checkpoint generation, and index optimization mechanisms, solving the problems of large memory dump file size, low query efficiency, and excessive storage space consumption in existing technologies. This method significantly improves the ability to analyze the memory state during program runtime.

[0020] The following section elaborates on the beneficial effects of the present invention, starting from the key aspects of the technical solution: Key point 1: A set of efficient data organization strategies based on memory pages was designed and implemented. By dividing memory data into pages and creating indexes, combined with a caching mechanism, the response speed of memory queries was significantly improved. Key Point 2: An incremental checkpoint generation mechanism was designed and implemented. By analyzing and recording memory changes, only the differences from the previous state are saved, which effectively solves the problem of excessive storage space occupied by complete memory snapshots. Key Point 3: An innovative memory query method was designed and implemented to find the checkpoint that is closest to but does not exceed the target instruction ID, load the checkpoint as the base state, and then apply all memory changes from the checkpoint to the target instruction ID, thereby achieving accurate memory query based on instruction execution context. Attached Figure Description

[0021] Figure 1 This is a flowchart of the main steps of a real-time memory recovery method that integrates process memory and instruction sequence.

[0022] Figure 2 This is a flowchart illustrating the specific implementation of a real-time memory recovery method that integrates process memory and instruction sequences.

[0023] Figure 3 This is a diagram showing the module composition of a real-time memory recovery system that integrates process memory and instruction sequences.

[0024] Figure 4 This is a schematic diagram of the specific architecture of a real-time memory recovery system that integrates process memory and instruction sequence. Detailed Implementation

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] The technical problem to be solved by this invention is as follows: 1. Real-time and low-overhead memory recovery: (1) How to build a mechanism for real-time memory recovery based on the fusion of instruction sequence and memory state, avoiding the problems of traditional static memory transfer; (2) How to control the computation and I / O overhead in the memory recovery process within an acceptable range to support high-frequency memory queries.

[0027] 2. Consistency Integration of Instruction Sequence and Memory State: (1) How to accurately extract the impact of memory writes from instruction records and integrate it consistently with the initial memory / patch data to avoid memory recovery deviations caused by missing context; (2) How to handle the consistency between memory dumps and instruction sequences in multi-threaded scenarios to ensure that the memory reconstruction results at the specified instruction ID are repeatable and deterministic. The method in this invention solves the problems that may exist in multi-threaded scenarios because it records the initial memory, paging status, and instruction sequence simultaneously.

[0028] 3. Incremental recovery capability at any instruction time: (1) How to support fast location and memory recovery at any instruction number position to avoid performance bottlenecks caused during the recovery process; (2) How to efficiently calculate byte-level increments between checkpoints to ensure that the recovery granularity meets the accuracy requirements without introducing excessive time overhead.

[0029] This invention discloses a real-time memory recovery method that integrates process memory and instruction sequences. The scheme follows a technical approach of "instruction-driven byte-level memory extraction + incremental checkpoint storage + on-demand paging reconstruction," achieving fast, low-overhead memory recovery at any instruction number. The scheme first extracts memory changes from instruction records and constructs incremental checkpoints. Then, based on a hierarchical indexing and on-demand loading mechanism, it performs instantaneous memory recovery and querying, supporting 32 / 64-bit instructions and large-scale instruction data. The innovation of this scheme lies in: 1) The design implements byte-level fusion modeling of "instruction-memory": the actual impact of writing to memory is directly extracted from the instruction record structure to form a change sequence of "address-new value-instruction number". The operands of each instruction are compared byte by byte before and after instruction execution, and only the changed bytes are retained to ensure minimization and verifiability.

[0030] 2) An "incremental checkpointing" system was designed and implemented: by analyzing and recording the differences in memory states at different points in time, only the changed memory pages and their contents are saved. The system first generates an initial memory checkpoint, and then in the subsequent checkpoint generation process, it uses the previous checkpoint as a basis to record and store only the changed memory pages.

[0031] 3) The design implements a "three-step synthesis" consistent page reconstruction: the page value is taken as the previous checkpoint and is superimposed in the order of "range instruction change - patch page - boundary instruction change" to ensure that the memory state at the target instruction number is consistent with the execution semantics.

[0032] 4) The design implements an efficient I / O path with on-demand loading and multi-level indexing: the initial memory file builds a page offset index and caches it, and queries only load the necessary pages, reducing the I / O overhead of large-scale data and improving throughput and response efficiency.

[0033] 5) The design implements fast deterministic recovery of arbitrary instruction numbers: when querying, the nearest checkpoint is selected as the baseline, and only the increment from the checkpoint to the target number interval is synthesized to avoid full replay.

[0034] This invention proposes a real-time memory recovery method that integrates process memory and instruction sequence. By incrementally combining instruction execution records with memory snapshots, this method can recover the historical state of process memory at a specified instruction location and supports querying memory values ​​within any address range.

[0035] In one embodiment of the present invention, a real-time memory recovery method that fuses process memory and instruction sequences is provided, such as... Figure 1 As shown, it includes the following steps: Create an index for the memory pages in the process memory dump file, forming a memory page index; Parse the instruction log file, establish a mapping relationship between instruction IDs and file locations, and form an instruction index; Scan memory patch files in the specified directory and generate incremental memory checkpoints at specified intervals based on memory page index, instruction index, and memory patch files; Based on the target instruction ID and memory address, the memory state at the target time is quickly restored and queried using memory checkpoints and incremental data, and a multi-level caching mechanism is adopted to improve memory query performance.

[0036] In one embodiment of the present invention, the specific implementation process of the above-mentioned real-time memory recovery method that integrates process memory and instruction sequence is as follows: Figure 2 As shown, it includes a data input layer, an index and checkpoint generation layer, a query processing layer, and an optimization and output layer. The specific steps of this method are as follows: Step 1: Memory Data Organization and Index Building. Extract all memory page data from the input process memory dump file and build an index mapping memory page addresses to file offsets. Specifically, the system scans the initial memory dump file, creates an index for each memory page, forming a memory page index, and supports persistent storage and loading of the index to accelerate subsequent queries. Simultaneously, the system parses the instruction log file, determines the largest instruction ID, and establishes a direct mapping from instruction IDs to file locations, forming an instruction index.

[0037] Step 2: Patch File Scanning and Processing. The system scans all memory patch files in the specified directory, extracts the corresponding instruction IDs based on the filenames, and establishes a mapping between instruction IDs and patch file paths. These patch files record snapshots of the memory state before the execution of specific instructions, providing basic data for incremental checkpoint generation.

[0038] Step 3: Incremental Checkpoint Generation. Based on the user-specified segment size (the interval between checkpoints), the system generates memory checkpoints at fixed intervals. Specifically, starting from a base checkpoint (initially empty), the system applies all patch files and memory changes caused by instruction execution in chronological order, saving only the modified memory pages to form incremental checkpoints. Each checkpoint file contains only the changed memory pages (8 bytes of address + 4096 bytes of data), significantly reducing storage space requirements. This method directly extracts the actual write impact on memory from the instruction record structure, forming a change sequence of "address-new value-instruction number." For each instruction, the operands are compared byte-by-byte to see the changes before and after instruction execution, retaining only the changed bytes to ensure minimization and verifiability. Here, the address refers to the operand address, the new value refers to the changed value of the operand, and the instruction number is the instruction ID. When an instruction at a certain instruction number is modified, the memory at the corresponding memory address is modified to the new value.

[0039] Step 4: Memory Query Processing. When querying the memory state at a specific instruction ID, the system first finds the checkpoint closest to, but not exceeding, the target instruction ID, loads this checkpoint as the base state, and then applies all memory changes from this checkpoint to the target instruction ID (including memory modifications caused by patch files and instruction execution), ultimately restoring the memory state at the target time. Specifically, page values ​​are stacked starting from the previous checkpoint, following the order of "range instruction changes - patch page - boundary instruction changes," ensuring that the memory state at the target instruction number is consistent with the execution semantics. Here, "range instructions" refers to all instructions from the previous checkpoint to the current position, and "boundary instructions" refers to the instruction following a page swap. Each instruction may modify the value of memory at some memory addresses; stacking refers to accumulating these memory modifications.

[0040] Step 5: Memory Data Access and Caching Optimization. To improve query performance, the system employs a multi-level caching mechanism. For initial memory pages, on-demand loading and an LRU (Least Recently Used) caching strategy are used to avoid loading all memory data at once. Caching mechanisms are also used for checkpoint data and intermediate calculation results to reduce redundant calculations and file I / O operations. The LRU caching strategy is a cache eviction algorithm based on access time, prioritizing the eviction of the least recently used data to free up space for new data.

[0041] Step 6: Output Results. The memory query results include the memory data itself and the corresponding status information (indicating whether the data at each memory location is known). The system supports saving the query results to a file in binary format, and can also generate a text file containing status information for easy subsequent analysis and use.

[0042] Through the above steps, this method achieves efficient checkpoint generation and memory query when processing large-scale process memory dump data, which is significantly better than the traditional full memory snapshot method, greatly reducing storage space usage and improving query performance.

[0043] This invention provides efficient processing and querying of process memory dump data. The corresponding system supports incremental memory checkpoint generation, accurate memory state restoration, and fast memory querying.

[0044] In one embodiment of the present invention, a real-time memory recovery system that integrates process memory and instruction sequence is provided, the module composition of which is as follows: Figure 3 As shown, the specific architecture of the system is as follows: Figure 4 As shown. The system includes the following modules: The memory page index building module is responsible for extracting and organizing initial memory data from process memory dump files and building memory page indexes to support efficient access.

[0045] Command Index Building Module: Parses the command record file, establishes a mapping relationship between command IDs and file locations, and supports quick location of specific commands.

[0046] Patch file scanning module: Scans all memory patch files in the specified directory, extracts the corresponding instruction IDs based on the file names, and establishes a mapping from instruction IDs to patch file paths.

[0047] Incremental checkpoint generation module: Based on memory changes and patch files, it generates incremental memory checkpoints at specified intervals, significantly reducing storage space usage.

[0048] Memory query processing module: Based on the target instruction ID and memory address, starting from the appropriate checkpoint, apply incremental changes to restore the memory state at the target time.

[0049] Cache optimization module: Employs a multi-level caching mechanism to reduce redundant loading and improve memory query performance, including initial memory page caching and checkpoint caching.

[0050] The overall architecture of the above system adopts a layered design. The input is a process memory dump file and an instruction log file. After processing by the memory page index building module and the instruction index building module, a data foundation that can be accessed efficiently is formed. The incremental checkpoint generation module generates periodic checkpoints on the basic data. The memory query processing module uses these checkpoints and incremental data to quickly restore and query the memory state at any instruction moment.

[0051] Other embodiments of the present invention: 1) Regarding the organization of memory page data, in addition to segmenting and indexing memory pages of fixed size, variable-sized memory blocks or dynamic partitioning strategies based on memory region attributes can also be used. These alternative methods, while maintaining the structured representation of memory data, may provide more efficient space utilization and access performance in specific scenarios (such as processing large contiguous memory regions).

[0052] 2) For multi-level caching and on-demand loading strategies, this can be achieved not only through memory page caching management, but also through content similarity-based preloading mechanisms or distributed caching architectures. These alternatives maintain high memory access performance while adapting to different scales of memory data processing needs, achieving optimized resource utilization.

[0053] It should be understood that the methods and systems disclosed in the above embodiments of this invention can be implemented in other ways. For example, the above module division can be implemented in other ways, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. The various steps and modules in this invention can be implemented in the form of software functional units and can be stored in a computer-readable storage medium, including several instructions to cause a computer device to execute some or all of the steps of the method described in this invention.

[0054] For example, one embodiment of the present invention provides a computer device (computer, server, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0055] For example, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk, etc.) that stores a computer program, which, when executed by a computer, implements the various steps of the method of the present invention.

[0056] For example, another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0057] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A real-time memory recovery method that integrates process memory and instruction sequence, characterized in that, Includes the following steps: Create an index for the memory pages in the process memory dump file, forming a memory page index; Parse the instruction log file, establish a mapping relationship between instruction IDs and file locations, and form an instruction index; Scan memory patch files in the specified directory and generate incremental memory checkpoints at specified intervals based on memory page index, instruction index, and memory patch files; Based on the target instruction ID and memory address, the memory state at the target time is quickly restored and queried using memory checkpoints and incremental data, and a multi-level caching mechanism is adopted to improve memory query performance.

2. The method according to claim 1, characterized in that, The construction of the memory page index includes: extracting all memory page data from the input process memory dump file and constructing an index from the memory page address to the file offset.

3. The method according to claim 1, characterized in that, For the memory patch file, extract the corresponding instruction ID based on the file name, and establish a mapping from the instruction ID to the memory patch file path.

4. The method according to claim 1, characterized in that, The process of generating incremental memory checkpoints at specified intervals includes: starting from a base checkpoint, applying all memory patch files and memory changes caused by instruction execution in chronological order, saving only the modified memory pages, and forming incremental memory checkpoints.

5. The method according to claim 1, characterized in that, The method of quickly restoring and querying the memory state at the target time using memory checkpoints and incremental data includes: first, finding the memory checkpoint that is closest to but does not exceed the target instruction ID, loading the memory checkpoint as the base state, and then applying all memory changes from the memory checkpoint to the target instruction ID to restore the memory state at the target time, thereby achieving accurate memory query based on the instruction execution context.

6. The method according to claim 5, characterized in that, The method utilizes memory checkpoints and incremental data to quickly restore and query the memory state at the target time. The resulting memory query results include the memory data itself and the corresponding state information, whereby the state information indicates whether the data at each memory location is known.

7. The method according to claim 1, characterized in that, The multi-level caching mechanism includes: using on-demand loading and LRU caching strategies for initial memory pages to avoid loading all memory data at once; and using caching mechanisms for checkpoint data and intermediate calculation results to reduce redundant calculations and file I / O operations.

8. A real-time memory recovery system that integrates process memory and instruction sequence, characterized in that, include: The memory page index building module is responsible for building an index for memory pages in the process memory dump file, forming a memory page index. The instruction index building module is responsible for parsing the instruction record file, establishing a mapping relationship between instruction IDs and file locations, and forming an instruction index; The patch file scanning module is responsible for scanning all memory patch files in the specified directory; The incremental checkpoint generation module is responsible for generating incremental memory checkpoints at specified intervals based on memory page indexes, instruction indexes, and memory patch files. The memory query processing module is responsible for quickly restoring and querying the memory state at the target time based on the target instruction ID and memory address, using memory checkpoints and incremental data. The cache optimization module is responsible for using a multi-level caching mechanism to reduce redundant loading and improve memory query performance.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images