Memory-friendly log storage method in embedded multi-threading scenario

Abstract

The application discloses a memory-friendly log storage method in an embedded multi-thread scene and belongs to the technical field of computer log processing. The method comprises two stages of device-side log recording and storage and development-side log analysis and restoration. After the device side receives a log call, an encoded log package containing an address sequence and dynamic data values is generated through coding processing, and then is stored in a ring queue connected at the head and tail through a two-layer matching mechanism folding and deduplication, and is asynchronously dumped to a FLASH to form an encoded log file after reaching a threshold. The development side obtains the encoded log file, loads a mapping dictionary, replaces the address sequence with original symbol information and fills dynamic data during decoding, and generates readable text logs. The application realizes efficient memory sharing, real-time log redundancy elimination and source deep compression, greatly reduces memory occupation and storage overhead, improves multi-thread concurrent performance, has high decoding fidelity, and is suitable for resource-limited embedded multi-thread systems.

Description

Technical Field

[0001] This invention belongs to the field of computer log processing technology, specifically relating to a memory-friendly log storage method in embedded multithreaded scenarios. Background Technology

[0002] In embedded systems, especially resource-constrained real-time systems, logging is a critical function for fault diagnosis, status monitoring, and system debugging. However, traditional log storage mechanisms face many challenges: I / O performance bottlenecks and system reliability issues: Traditional solutions rely on the file system to write directly to external storage. Frequent I / O operations consume CPU resources and increase response latency, which does not meet real-time requirements.

[0003] Inefficient memory usage and concurrency issues: Existing memory log solutions often use independent double buffers or thread-independent buffer slots, which leads to memory waste, severe fragmentation, and multi-threaded concurrent writing can easily cause data overwriting or race conditions.

[0004] Storage space waste and information redundancy issues: Raw text logs contain a large amount of repetitive string information, existing compression schemes are lagging and coarse-grained, structured logs do not solve content redundancy, lack real-time deduplication and merging capabilities, and store a large amount of invalid and duplicate information. Summary of the Invention

[0005] To address the problems mentioned in the background section, this invention provides a memory-friendly log storage method for embedded multi-threaded scenarios, thereby solving the problems of low memory utilization, high log redundancy, and large storage overhead in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A memory-friendly log storage method for embedded multithreaded scenarios includes the following steps: S1: The device receives a log call request, calls the original ELF program file which is from the same source as the program running on the device, is compiled by the development environment and retains a complete symbol table, loads the symbol table in the ELF program file to generate an address-symbol name mapping dictionary, encodes the log, identifies static data and dynamic data, replaces the static data with the corresponding virtual address through the mapping dictionary to form an address sequence, and obtains an encoded log packet composed of address sequence and dynamic data. S2: Perform log folding processing on the encoded log packets, and filter out duplicate or identical logs through a two-layer matching mechanism to reduce storage usage; S3: Store the folded encoded log packets into a circular queue. When the number of encoded log packets in the circular queue reaches the threshold, trigger a dump task asynchronously. Persist the encoded log packets containing dynamic data to FLASH without blocking cache operations. S4: The development side obtains the encoded log packet stored in the FLASH memory on the device side; S5: Decodes the encoded log packet based on the mapping dictionary, replaces the address sequence with the original symbol information, calls dynamic data and fills it in the original format, and generates a readable text log file.

[0007] Compared with the prior art, the beneficial effects of the present invention are: 1. Significantly improved memory utilization: By using a circular queue to enable log writing and asynchronous dumping to share the same physical memory, compared with the traditional independent dual-buffer scheme, fixed memory overhead can be saved under the same buffering capacity, making it suitable for resource-constrained embedded systems.

[0008] 2. Concurrency processing performance optimization: By using a circular queue, concurrent access to multi-threaded writes and background dumps is coordinated, improving the efficiency of log recording in a multi-threaded environment.

[0009] 3. Significantly reduced log storage redundancy: Through two-layer matching log folding processing, duplicate or identical pattern logs are filtered in real time to avoid the repeated storage of completely redundant content. Logs with the same pattern are merged, which significantly improves the effective information density in memory and reduces the occupation of invalid storage.

[0010] 4. Significantly reduced space usage per log entry: Encoding is performed at the source of log generation through address mapping, converting lengthy strings into short address values, thus reducing the storage space usage per log entry. Furthermore, the decoding process is performed offline, without affecting the device's runtime performance.

[0011] 5. High-fidelity log restoration: The development end decodes the original ELF program file based on its symbol table, which can accurately restore the original log content and generate a readable text log that is consistent with the plaintext log, ensuring the accuracy of fault diagnosis and system debugging. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the process of this application; Figure 2 This is a flowchart of a fast matching process based on an LRU linked list. Figure 3 A flowchart for fuzzy matching based on a log fingerprint database; Figure 4 This is a flowchart of the encoding process. Detailed Implementation

[0013] To facilitate understanding of the technical content of this invention by those skilled in the art, the invention will be further described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the scope of the invention.

[0014] Memory-friendly log storage methods in embedded multithreaded scenarios, such as Figure 1 As shown, it includes the following steps: Phase 1: Recording and storing data during equipment operation; This phase is executed on resource-constrained devices, with the goal of capturing all logs losslessly with minimal memory and CPU overhead (all code below is pseudocode and does not involve specific implementation; it is for illustrative purposes only).

[0015] S1: Log encoding (instant compression); Triggered by: The application calling the logging interface (e.g., LOG_INFO(“Sensor%s error:%d”, name, code)); Operation: Intercept the call, parse the parameters, and identify the static part (function name "LOG_INFO", string constant "Sensor", "error:") and the dynamic part (variable name, code). Obtain the encoded log file and the original ELF program file generated during compilation, which retains the complete symbol table; use tools (nm, objdump) to parse the ELF file and generate a dictionary of address-to-symbol name mappings; replace each static string with its specific virtual address (a 4 / 8-byte number) in the program binary file. Output: A single encoded log packet consisting of an address sequence and dynamic data values. The file size is reduced by tens of times compared to the original.

[0016] S2: Log folding (content-aware compression); Trigger: The encoded log packet enters the folding process; Operation: Calculate the hash fingerprint of the core content of the log packet (or directly compare the address sequence); Query the hash table, which maps content fingerprints to a node in the LRU linked list; If a match is found: if the node corresponds to a cached log entry, no new data is written; instead, the number of times the log entry appears and the last timestamp are updated in the node, and the node is moved to the head of the LRU list. If a match is missed: a new node is created at the head of the LRU list. If the list is full, the tail node (the least repeated log entry) is evicted. Next, the static pattern part of the log is extracted (removing time, dynamic parameters such as name and code fields), and matched in the "fingerprint table". If a match is found, it will be merged with the similar log group; otherwise, the fingerprint table is updated, and finally a new log is inserted into the cache queue. Output: For duplicate logs, no new storage space will be used; for new logs, they will be stored in the cache.

[0017] S3: Caching and dumping (concurrent pipeline); Cache (multi-threaded writes): The thread acquires the cache lock and checks the remaining space in the circular queue managed by the atomic variable; Write the folded log data (or index) to the location pointed to by the "cache pointer"; move the "cache pointer" forward, decrement the atomic space variable by 1, and release the lock; this process is extremely fast, has small lock granularity, and is friendly to multi-threaded contention; Dump (Asynchronous Persistence): When the amount of data in the queue reaches a threshold, an independent dump task is triggered asynchronously; The dump task briefly locks the data, reads the "dump pointer" and "cache pointer", calculates the size N of the consecutive log blocks to be saved, and then immediately releases the lock. In a lock-free state, this block of N consecutive log data is written from the circular queue to the Flash. During the writing process, cache operations can be performed in parallel without blocking each other. After writing is complete, lock again briefly, move the "dump pointer" forward by N positions, increment the atomic space variable by N (to free up space), and finally release the lock; Output: After caching, the logs are temporarily stored in memory and await dumping. The dumped logs are then persistently stored in the flash memory and can be exported for viewing.

[0018] Phase Two: Offline parsing and restoration on the development side; This phase is executed on a well-resourced development PC, with the goal of restoring the highly coded files generated on the device to fully readable text logs.

[0019] S4: Preparation: The development team retrieves the stored logs; S5: Read the mapping dictionary generated from the symbol table; The decoding program reads each record in the encoded log file; for each address value, it searches in the loaded symbol table dictionary, replaces it with the original function name and string constant, and fills in the dynamic data (variable values, timestamps, etc.) in the original format.

[0020] Output: Generates a text log file that is exactly the same as the plaintext log content, but sorted by time, for developers to analyze.

[0021] The following sections will elaborate on the three core functions: I. Log caching and dumping; One of the core functions of this application is log caching and dumping. In a multi-threaded scenario, logs from other threads are recorded into memory (this process will be referred to as "log caching" or "caching" below). After the logs in memory accumulate to a certain level, the operation of writing the logs to FLASH is completed without affecting the continued caching of logs (this process will be referred to as "dumping" below).

[0022] Key data structures: The "caching" and "dumping" operations rely on atomic operations and mutexes provided by the system. It also relies on a critical data structure, a circular queue, for caching. The total size of the circular queue is the total size of the buffer available for the log, which is also the total memory consumed by the entire log module described in the application (other temporary variables are negligible). Two pointers are used: a "cache pointer" pointing to the storage location of the next log entry; and a "dump pointer" pointing to the last cached location. A mutex is needed to synchronize the "caching operations" of different threads (hereinafter referred to as the "cache lock"), and an atomic variable is needed to record the currently available space.

[0023] Cache logs: The thread attempts to acquire the "cache lock," and waits if it cannot. Upon successful acquisition, it checks the available space and waits until the space is not empty.

[0024] If the space is not empty, insert a log entry at the position pointed to by the "cache pointer", then modify the pointer, the remaining space size, and release the lock.

[0025] Check if a dump has been triggered. If a dump has been triggered, proceed with the dump process; otherwise, end the caching process.

[0026] Dump Log: The dump process is a singleton, meaning there will only be one dump instance and no multi-threaded concurrent dumping.

[0027] Acquire the cache lock, calculate the number of logs after successful acquisition, and then release the cache lock. This step is to confirm the number of logs that the cache needs to process in this operation, ensuring that read / write synchronization will not occur.

[0028] Starting from the position of the "dump pointer", the logs obtained in the previous step are transcribed into the FLASH for permanent storage.

[0029] Block again, waiting for the cache lock, and modify the remaining storage space. Release the lock after modification is complete.

[0030] II. Layered intelligent log compression; This feature primarily optimizes and filters log content. The first layer is based on LRU window-based fast matching, which can quickly filter out completely duplicate logs that appear within a short period. The second layer is based on a "keyword hash table" for fuzzy matching, mainly used to filter out logs that appear periodically but over a long time span, or logs with the same pattern but different parameters.

[0031] First layer: Fast matching based on LRU window; This layer of matching only applies to character-type log data segments. If two log segments have identical content, they are considered suitable for folding. For example, two logs with different timestamps but identical content. This effectively avoids wasting space by displaying error messages everywhere. The key data structure for log folding is a doubly linked list (hereinafter referred to as the LRU list) that stores a certain number of cached logs.

[0032] When caching logs, the log entries in the linked list are traversed sequentially, and each entry is checked for duplicates. If a duplicate is found, the count and duration of that log entry in the log cache are updated. Simultaneously, the position of the corresponding log entry in the LRU linked list is moved to the head of the list.

[0033] If no duplicate logs are found, the log at the end of the LRU list is deleted, and a new log is inserted at the head of the list. This is because, after the list is maintained according to the above operation, the tail of the list contains the logs that have not been duplicated for the longest time, so the probability of them being duplicated again is the lowest, while the head of the list contains the most recently appeared logs, so the probability of them reappearing is higher.

[0034] The hash table used to locate log positions is updated synchronously. To save space, log contents are not stored repeatedly in the LRU list; instead, a hash table maps the log's index in the cache queue to a pointer in the LRU list.

[0035] The second layer: a fingerprint database based on content hashing and an adaptive time window; A separate "log fingerprinting" module is introduced, working in parallel with the existing LRU window, specifically for capturing and compressing non-contiguous but repetitive log entries. If a newly added log entry is not optimized away by the previous LRU layer, it will attempt to match it at the current layer. This module relies on two core data structures: 1. Template table; This is a fixed-capacity array or linked list used to store the "prototype" of the log pattern. Each template entry, LogTemplate, contains: struct LogTemplate { uint32_t id; / / Template unique ID char normalized_pattern[MAX_PATTERN_LEN]; / / The normalized pattern string, such as "Error: open file %s failed." uint64_t first_seen_time; / / First appearance timestamp uint64_t last_seen_time; / / Timestamp of the most recent occurrence uint32_t total_count; / / Total number of occurrences uint32_t recent_count; / / Number of recent occurrences (used in the elimination algorithm) uint32_t sample_param_index; / / Index of the sample parameter in the parameter pool / / ...This can be expanded to include other statistical fields, such as maximum / minimum parameter values. }; 2. Fingerprint index; This is a hash table that enables fast lookup of "fingerprint -> template ID".

[0036] Key: The hash value calculated from the normalized pattern string (such as 32-bit CRC or MurmurHash).

[0037] Value: The index or ID of the corresponding template in the template table.

[0038] The entire system operates as follows: Step 1: Log normalization (preprocessing); This is a prerequisite for pattern matching. The original log "Task

[1234] Error:open file'config.ini' failed at 10:00:00" is broken down as follows: Static pattern part: Extract fixed strings and semantic positions, and turn them into normalized strings: "Task[%d]Error:open file'%s' failed at %s".

[0039] Dynamic parameter list: Extracted variable value array: [1234, "config.ini", "10:00:00"].

[0040] Calculate fingerprint: Calculate the hash value of the normalized pattern string and use it as the key for this query.

[0041] Step 2: Fingerprint matching and merging; Use the calculated fingerprint to query the fingerprint index.

[0042] If a match is found, the corresponding LogTemplate is located and the merge is performed: Update last_seen_time.

[0043] Increment total_count and recent_count by 1.

[0044] (Optional) Update sample_param_index with the current dynamic parameters with a certain probability (e.g., 1%) and save an example.

[0045] If no match is found, proceed to the template creation process.

[0046] Step 3: Template creation and deprecation; Create a new template: Assign a new LogTemplate entry to the new log pattern, fill in the normalization pattern, timestamp, count of 1, and store an example parameter. At the same time, establish a mapping from the new fingerprint to this template ID in the fingerprint index.

[0047] Capacity Management and Eviction: When the template table is full, an older template needs to be evicted to free up space. The eviction strategy is crucial to algorithm performance; a hybrid strategy of "recent frequency + time decay" is recommended. Maintain a recent_count for each template, and increment it each time a match is successful.

[0048] Periodically (or before each insertion), decay the recent_count of all templates (e.g., recent_count = recent_count / 2).

[0049] When eviction occurs, the template with the smallest (recent_count) and the oldest (last_seen_time) is selected. This adaptively retains the most recently occurring patterns and evicts patterns that have not been used for a long time.

[0050] III. Log encoding and decoding based on program compilation and linking; Program logs typically contain information such as program functions and commonly used strings. Storing function names, parameter lists, and constant strings as strings is very memory-intensive. This application utilizes the relative addresses of all functions (including member functions) stored in the program's binary file within the code segment (.text section) and the addresses of string constants within the read-only data segment (.rodata section) to compress and encode the logs. Its advantages are: significantly saving log space; and the log's confidentiality depends on the program's encryption level, eliminating the need for independent encoding / decoding and encryption methods.

[0051] The "log encoding" procedure flow, such as Figure 4 As shown; 1. Parameter Analysis and Classification: Identify and extract static parts from the logs: function names (e.g., "FunctionA"), log level labels (e.g., "ERROR"), and fixed message templates (e.g., ":error=").

[0052] Identify dynamic components: variable values ​​(such as the value of err_code), timestamps, thread IDs, etc. Decompose the log into a structured form and identify the parts that can be compressed and encoded. In the method described in this application, "function names" and "character constants" can be encoded; the remaining parts can be defined as character constant macros for encoding using "character constants".

[0053] 2. Address mapping lookup and replace: For each coded part, its corresponding memory address is looked up using a pre-generated "symbol-address mapping table" (generated by the tool after compilation and built into the program).

[0054] The original complete string that needed to be stored is replaced with this address value (usually a 4-byte or 8-byte integer). Each log entry is also required to include a "module ID," which identifies the name of the compiled module. This ID is used to identify the compiled module during parsing.

[0055] 3. Data Packaging: The converted address sequence, dynamic data values, and a small header describing the structure of this log (such as the type and length of each part) are serialized into the memory log buffer according to a predetermined format.

[0056] "Log Decoding" Program Flow; Materials needed: Obtain the encoding log file and its complete corresponding raw program ELF file without the symbol table removed. These files can be generated synchronously during the program packaging stage.

[0057] Loading symbol tables: Using tools such as nm and objdump, extract the address-symbol name mapping table from the ELF file and load it into the decoding program to form an efficient lookup data structure (such as a hash table).

[0058] Read encoded logs: Read binary log blocks one by one according to the encoding format, and parse their headers and individual data segments.

[0059] Address reverse lookup: For each address value in the log data, look it up in the loaded symbol table.

[0060] If found, the address is replaced with the corresponding function name or string constant. If not found, the original address is preserved or marked with a special flag (e.g., ...). <unknown:0x12345>)show.

[0061] Recombination and formatting: The static string and dynamic data values ​​obtained after deparse are recombined according to the format of the original log template to generate a complete and readable log string.

Claims

1. A memory-friendly log storage method for embedded multi-threaded scenarios, characterized in that, Includes the following steps: S1: The device receives a log call request, calls the original ELF program file which is from the same source as the program running on the device, is compiled by the development environment and retains a complete symbol table, loads the symbol table in the ELF program file to generate an address-symbol name mapping dictionary, encodes the log, identifies static data and dynamic data, replaces the static data with the corresponding virtual address through the mapping dictionary to form an address sequence, and obtains an encoded log packet composed of address sequence and dynamic data. S2: Perform log folding processing on the encoded log packets, and filter out duplicate or identical logs through a two-layer matching mechanism to reduce storage usage; S3: Store the folded encoded log packets into a circular queue. When the number of encoded log packets in the circular queue reaches the threshold, trigger a dump task asynchronously. Persist the encoded log packets containing dynamic data to FLASH without blocking cache operations. S4: The development side obtains the encoded log packet stored in the FLASH memory on the device side; S5: Decodes the encoded log packet based on the mapping dictionary, replaces the address sequence with the original symbol information, calls dynamic data and fills it in the original format, and generates a readable text log file.

2. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 1, characterized in that, The circular queue consists of multiple log storage units connected end-to-end, and is configured with a cache pointer, a dump pointer, a cache lock, and an atomic variable. Specifically, the cache pointer points to the target storage location of the next encoded log packet to be stored in the circular queue, initially pointing to the beginning of the queue; the dump pointer points to the end of the encoded log packet that was last dumped in the circular queue, initially pointing to the same location as the cache pointer; the cache lock is a mutex lock used to synchronize the cache operations of multiple threads on the circular queue, avoiding data conflicts caused by concurrent writes from multiple threads. Atomic variables are used to record the current available storage space size of the circular queue in real time. Their initial value is equal to the maximum number of coded log packets that the circular queue can store. The value of the atomic variable is decremented by 1 for each coded log packet stored and incremented by 1 for each coded log packet dumped.

3. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 2, characterized in that, The specific execution flow of the multi-threaded caching operation described in S3 is as follows: S3.11: When a thread initiates a request to acquire a cache lock, if the cache lock is already held by another thread, the current thread enters a waiting state until it successfully acquires the cache lock. S3.12: After a thread acquires a cache lock, it immediately reads the available space size recorded in the atomic variable. If the value of the atomic variable is 0, the thread holds the cache lock and waits until the value of the atomic variable is not 0 before proceeding to the next step. S3.13: The thread writes the folded encoded log packet or index completely to the storage location pointed to by the cache pointer in the circular queue; S3.14: After writing the encoded log packet, the thread moves the cache pointer forward by one log storage unit towards the tail of the circular queue, so that the cache pointer always points to the target position of the subsequent encoded log packets to be stored. S3.15: Simultaneously decrement atomic variables by 1; S3.16: The thread releases the cache lock, allowing other waiting threads to acquire the cache lock and perform cache operations; S3.17: After the thread releases the cache lock, it checks whether the amount of encoded log packets stored in the current circular queue has reached the preset dump threshold. If the threshold is reached, the dump task is triggered; otherwise, the current cache operation ends.

4. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 3, characterized in that, In S3, dump tasks are executed in a singleton pattern, meaning that only one dump instance exists at a time. The specific execution flow of a dump task is as follows: S3.21: The dump task initiates a cache lock acquisition request. After successfully acquiring the cache lock, it calculates the number N of the log packets to be dumped by comparing the positions of the cache pointer and the dump pointer in the circular queue. The value of N is equal to the number of log storage units between the cache pointer and the dump pointer. S3.22: After calculating the number N of the encoded log packets to be dumped, the dump task immediately releases the cache lock to avoid holding the cache lock for a long time and blocking multi-threaded cache operations; S3.23: The dump task enters a lock-free state, starting from the starting position pointed to by the dump pointer in the circular queue, continuously reading N encoded log packets, and writing the encoded log packets to FLASH in batches according to the preset writing format to achieve persistent storage; during this process, multi-threaded cache operations can be executed in parallel, and the dump operation and cache operation do not block each other; S3.24: After all the encoded log packets are written to FLASH and the storage is confirmed to be successful, the dump task initiates another cache lock acquisition request. After successful acquisition, the dump pointer is moved forward by N log storage units towards the tail of the circular queue, so that the dump pointer points to the position after the last encoded log packet of this dump. S3.25: The dump task performs an increment-N operation on the atomic variable, releasing N log storage units of available space in the circular queue. The updated atomic variable value accurately reflects the current available space. S3.26: The dump task releases the cache lock, completes the dump operation, and waits for the next dump threshold to be triggered.

5. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 4, characterized in that, During the execution of the dump task, if the number of encoded log packets in the circular queue continues to increase, causing the cache pointer to exceed the tail boundary of the circular queue and wrap back to the beginning of the queue, i.e. the circular queue is in a loop, the dump task will take advantage of the looping characteristic of the circular queue to read the encoded log packets to be dumped in two segments in the order of "from the dump pointer position to the tail of the queue + from the beginning of the queue to the cache pointer position", ensuring that N encoded log packets to be dumped are read completely without omission or duplicate reading.

6. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 1, characterized in that, The encoding process described in S1 includes: S1.1: Identify static and dynamic data in the logs. Static data includes function names, string constants, and log level labels. Dynamic data includes variable values, timestamps, thread IDs, and module IDs. The module ID is used to identify the compilation module corresponding to the log data. S1.2: The memory address corresponding to each static part is found by using a mapping dictionary. The mapping dictionary records the one-to-one correspondence between the static part and the virtual address in the program binary file. The virtual address is an integer of 4 bytes or 8 bytes. S1.3: Replace the static part with the corresponding virtual address to form an address sequence. Then, combine it with the dynamic data value to generate an encoded log package containing the address sequence, dynamic data, and a small header. The small header is used to describe the log structure, including the type and length information of each part of the data.

7. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 6, characterized in that, The log folding process described in S2 includes a two-layer matching mechanism, specifically: The first layer is based on fast matching using an LRU linked list. It calculates the hash fingerprint of the core content of the encoded log packet, queries the hash table to determine if there is a duplicate encoded log packet, and updates the occurrence count and last timestamp of the corresponding encoded log packet if the hash table matches, and moves the corresponding node in the LRU linked list to the head. If no match is found, a new node is created at the head of the LRU linked list. If the LRU linked list is full, the node that has not been repeated for the longest time at the tail is removed, and the hash table is modified synchronously. The second layer: Based on fuzzy matching of the log fingerprint database, the encoded log packets that were not filtered by the first layer are normalized to obtain the normalized pattern string and dynamic parameter list. The hash value of the normalized pattern string is calculated and the fingerprint index is queried. If a match is found, it is merged into the corresponding log template. If no match is found, a new template is created and the fingerprint index is updated.

8. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 7, characterized in that, The log template eviction strategy in the second-level matching is a hybrid strategy of "recent frequency + time decay": Maintain the recent occurrence count (recent_count) and the last occurrence timestamp (last_seen_time) fields for each log template; Each time a log template is successfully matched, the value of the recent_count field of that template is incremented by 1; Periodically or before each new template is inserted, the value of the recent_count field of all log templates is decayed, and the decay method is recent_count = recent_count / 2; When the template table is full, the log template with the smallest recent_count value is selected. If there are multiple templates with the same recent_count value, the template with the earliest last_seen_time is further selected and removed from the template table to make room for the new template.

9. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 6, characterized in that, In S1, the ELF program file is parsed using the nm tool or objdump tool to extract the mapping relationship between addresses and symbol names. The symbol names include function names and string constant names. A mapping dictionary is constructed based on the extracted mapping relationship, and the mapping dictionary adopts a hash table structure.

10. The memory-friendly log storage method for embedded multi-threaded scenarios according to claim 9, characterized in that, The specific execution flow of the decoding process described in S5 is as follows: S5.1: Read the binary log data stored in the FLASH encoded log packet, and parse the small header, address sequence, dynamic data value and module ID of each encoded log packet according to the encoding format; S5.2: Confirm the compiled module corresponding to the module ID log; S5.3: For each address value in the address sequence, search in the loaded mapping dictionary; if a corresponding symbol name is found, replace the address value with that symbol name; If no corresponding symbol name is found, the original address value is retained or displayed with a preset special mark; S5.4: Recombine the replaced static and dynamic data according to the format of the original log template to fill in the complete log content; S5.5: Sort all reconstructed logs by timestamp and generate a readable text log file that is completely identical to the plaintext log content.

Images