Log management method and device, equipment and storage medium

Abstract

The application relates to a log management method, device, system, equipment and storage medium, wherein a current driving scene label of an automatic driving system is first determined; then, in response to the current driving scene label, a target scene template corresponding to the current driving scene label is matched from a pre-defined scene template library; then, adaptive layout reconstruction is performed on a memory pool of a log according to the target scene template; in a log recording process, an operating system kernel IO stack is bypassed, an NVMe command is constructed through a user state drive, and log data is written into an NVMe solid state memory through a high-speed bus. The application can realize deterministic log disk landing and scene adaptive memory management.

Description

Technical Field

[0001] This application relates to the field of log management technology, specifically to a log management method, apparatus, device, and storage medium. Background Technology

[0002] To enable autonomous driving in smart cars, designers need to use log recording as a troubleshooting tool during the software development process of the onboard computer system.

[0003] However, existing vehicle logging systems mainly suffer from the following technical shortcomings: First, storage I / O paths suffer from non-deterministic latency. Traditional solutions write to persistent storage via file interfaces provided by the operating system (such as fwrite). The kernel's dirty page flushing mechanism triggers forced disk flushing under memory pressure, generating unpredictable I / O storms. This results in latency jitter of up to tens of milliseconds for the autonomous driving master control thread, severely impacting real-time performance. Moreover, related log management technologies (such as buddy systems and slab allocators) are designed for general computing scenarios, treating memory as a homogeneous resource and failing to differentiate management based on the time-varying nature of autonomous driving logs.

[0004] Second, runtime memory management is static and rigid. The memory pool of the existing log system uses a fixed configuration: parameters such as block size distribution, defragmentation trigger conditions, and allocation priority remain unchanged after system startup, which cannot adapt to the dynamic changes in autonomous driving scenarios. Summary of the Invention

[0005] One objective of this application is to provide a log management method that enables deterministic log writing to disk and scenario-adaptive memory management; another objective of this application is to provide a log management device; a third objective of this application is to provide a vehicle; and a fourth objective of this application is to provide a storage medium.

[0006] To achieve the above objectives, firstly, this application provides a log management method, the log management method comprising: Determine the current driving scenario label for the autonomous driving system; In response to the current driving scenario tag, a target scenario template corresponding to the current driving scenario tag is matched from a predefined scenario template library; Based on the target scenario template, perform adaptive layout reconstruction on the log memory pool; During the logging process, the system bypasses the operating system kernel I / O stack, constructs NVMe commands through the user-space driver, and writes the log data to the NVMe solid-state storage via a high-speed bus.

[0007] Furthermore, The method of bypassing the operating system kernel I / O stack, constructing NVMe commands through a user-space driver, and writing log data to the NVMe solid-state storage via a high-speed bus includes: Allocate a memory pool available for direct memory access in physical memory as a runtime log buffer; The log data is constructed as self-describing entries aligned with the physical sectors of the NVMe solid-state memory; In the user-space driver, an NVMe write command is constructed, wherein the data buffer pointer in the NVMe write command points to a physical address in the memory pool; The NVMe write command is submitted to the submission queue to write log data to the NAND flash physical page of the NVMe solid-state storage via the DMA engine; A disk write thread runs on a dedicated CPU core and uses a polling mode to monitor the command completion status.

[0008] Furthermore, The driving scenario tags configured in the log management method include tags for at least one of the following scenarios: High-speed cruising scenarios, urban congestion scenarios, intersection traffic scenarios, parking scenarios, and emergency safety scenarios.

[0009] Furthermore, The process of determining the current driving scenario label for the autonomous driving system includes: Receive status signals from the autonomous driving system at a fixed frequency; Based on the status signal, the current driving scenario label is determined.

[0010] Furthermore, The target scenario template is used to define memory management strategy parameters, which include at least one of the following: Target block distribution, fragmentation tolerance threshold, defragmentation trigger frequency, large block retention ratio, and number of emergency reserved blocks; The target block distribution is used to define the target proportion of each order of blocks in the memory pool.

[0011] Furthermore, The step of performing adaptive layout reconstruction of the log memory pool according to the target scenario template includes: In response to the detection of a driving scenario switch, a refactoring is triggered during a system idle window to adjust the current block distribution of the memory pool to the target block distribution, including: Calculate the deviation between the actual block distribution of the current memory pool and the target block distribution; Perform targeted defragmentation on memory regions where the fragmentation density exceeds the fragmentation tolerance threshold; Perform down migration on blocks with excessive order and up migration on blocks with insufficient order; Update the fragmentation rate limit, large block water level, and compaction timer period of the memory pool according to the target scenario template.

[0012] Furthermore, The log management method includes: During the driving scenario maintenance, the fragmentation status of the memory pool is monitored, and dynamic fragmentation management of the memory pool is performed according to the target scenario template and the fragmentation status. The dynamic fragment management includes: When the fragmentation rate is below 50% of the fragmentation tolerance threshold, the sorting process is completely paused. When the fragmentation rate is in the range of 50% to 100% of the fragmentation tolerance threshold, light compaction is triggered at the compaction trigger frequency. When the fragmentation rate exceeds the fragmentation tolerance threshold, heavy defragmentation is immediately triggered until the fragmentation rate drops to a safe range.

[0013] Furthermore, The log management method includes: The fragmentation heatmap of the memory pool is updated at a fixed frequency; wherein the fragmentation heatmap is used to divide the address space of the memory pool into multiple regions and calculate the fragmentation density, block order dispersion and live data locality of each region; When performing targeted defragmentation in adaptive layout reconstruction or dynamic fragmentation management, regions where both the fragment density and the block order dispersion are higher than the corresponding thresholds are preferentially selected as defragmentation targets.

[0014] Furthermore, The log management method includes: Under the same driving scenario tag, monitor and count the number of times the heavy cleaning is triggered; When the number of triggers reaches a set threshold within a set time window, the self-evolution correction process of the scene template library is triggered. The self-correction process includes: Extract fragment heatmap snapshots during heavy cleanup to analyze local characteristics of surviving data to identify the direction of parameter deviation; The target block distribution parameters in the target scene template are corrected based on the deviation direction using a moving exponential weighted average algorithm. The corrected target block distribution parameters are persistently updated to the scene template library.

[0015] To achieve the above objectives, in a second aspect, this application also provides a log management device, the log management device comprising: The scene perception module is used to determine the current driving scene label of the autonomous driving system; and is also used to match the target scene template corresponding to the current driving scene label from a predefined scene template library in response to the current driving scene label. The memory pool management module is used to perform adaptive layout reconstruction of the log memory pool according to the target scenario template; The user-space deterministic I / O module is used to bypass the operating system kernel I / O stack during the logging process, construct NVMe commands through the user-space driver, and write log data to NVMe solid-state storage via a high-speed bus.

[0016] To achieve the above objectives, in a third aspect, this application also provides a vehicle, including: a processor and a memory, the processor being configured to execute a control program stored in the memory to implement the log management method as described in any of the first aspects.

[0017] To achieve the above objectives, in a fourth aspect, this application also provides a storage medium storing one or more programs that can be executed by one or more processors to implement the log management method as described in any of the first aspects.

[0018] The beneficial effects of this application are: In this application, firstly, by bypassing the operating system kernel I / O stack, user-mode drivers directly construct NVMe commands and write log data to NVMe solid-state storage via a high-speed bus, completely eliminating the uncertainties caused by kernel page caching and interrupt handling. This reduces the log write latency from unpredictable tens of milliseconds to deterministic microseconds, meeting the real-time requirements of autonomous driving systems. Secondly, by determining the current driving scenario label, matching the corresponding target scenario template, and performing adaptive layout reconstruction of the log memory pool based on the template, the log memory pool can dynamically adjust with changes in the driving scenario, overcoming the static and rigid defects of existing memory management technologies and improving memory resource utilization efficiency and overall system performance. Moreover, through scene template-driven adaptive layout reconstruction, the memory pool layout is actively adjusted when switching driving scenarios, shifting the timing of memory fragmentation elimination from "passively triggered during business runtime" to "actively executed during scene switching in the idle window." This effectively reduces the frequency of runtime defragmentation and interference with the autonomous driving main control thread, ensuring the stable operation of the onboard computing system. In short, this application can achieve deterministic log write to disk and scene-adaptive memory management. Attached Figure Description

[0019] Figure 1 This diagram illustrates a log management method provided in an embodiment of this application. Figure 2This invention provides a system architecture diagram of a log management method according to another embodiment of the present application. Figure 3 This diagram illustrates the memory fragmentation state distribution and targeted reorganization of a log management method provided in an embodiment of this application. Figure 4 This diagram illustrates a block diagram of a log management device according to another embodiment of this application; Figure 5 This diagram illustrates a vehicle according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0021] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0022] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0023] refer to Figures 1 to 3 As shown, this embodiment provides a log management method that can be applied to vehicles. The method may include: S110. Determine the current driving scenario label of the autonomous driving system; S120. In response to the current driving scenario label, match the target scenario template corresponding to the current driving scenario label from the predefined scenario template library; S130. Based on the target scenario template, perform adaptive layout reconstruction on the log memory pool; S140. During the log recording process, bypass the operating system kernel I / O stack, construct NVMe commands through the user-mode driver, and write the log data to the NVMe solid-state storage via the high-speed bus.

[0024] It should be noted that S110, S120, etc. are merely step identifiers, and the execution order of the method does not necessarily follow the numerical order from smallest to largest. For example, step S120 can be executed first and then step S110 can be executed. This application does not impose any restrictions.

[0025] In step S110, the log management method may configure at least one driving scenario label. For example, the driving scenario labels configured in the log management method may include labels for at least one scenario such as highway cruising scenario, urban congestion scenario, intersection crossing scenario, parking scenario, and emergency safety scenario.

[0026] The determination of the scene label does not rely on the future trajectory prediction of the planning algorithm, but is based on the currently perceived environmental state and the changes in vehicle state that have occurred. To determine the current driving scene label, state signals from the autonomous driving system are first received at a fixed frequency, and then the current driving scene label is determined based on these state signals.

[0027] For example, the vehicle's scene recognition module can operate at a fixed frequency (such as 20 Hz), comprehensively receive status signals from multiple modules of the autonomous driving system, and then determine the current driving scene label based on the obtained status signals.

[0028] The criteria for determining high-speed cruise scenarios include: the vehicle's current speed is greater than a set threshold (e.g., 60 km / h) and remains so for a certain period of time (e.g., 5 seconds); the current road type is a highway or urban expressway, output by a high-precision map or visual recognition; the number of surrounding obstacles is less than a set threshold (e.g., 3) and their relative speeds are stable; and the curvature of the planned trajectory is less than a set threshold (e.g., 0.01).

[0029] The criteria for determining urban congestion scenarios include: the vehicle's current speed is less than a set threshold (e.g., 30 km / h) and this continues for a certain period of time (e.g., 10 seconds); the number of surrounding obstacles is greater than a set threshold (e.g., 5) and there are frequent entry and exit behaviors; the confidence level of traffic light or stop line detection is greater than a set threshold (e.g., 0.8); and the planned trajectory has multiple directional changes within a short distance.

[0030] The conditions for determining the intersection traffic scenario include: the high-precision map shows that the current location is less than a set distance (e.g., 50 meters) from the center of the intersection; visual detection identifies intersection signs, zebra crossings, or multi-directional lane lines; and the planned trajectory has a heading angle change greater than a set angle (e.g., 45 degrees).

[0031] The criteria for determining a parking scenario include: the vehicle speed is less than a set threshold (e.g., 10 kilometers per hour); parking space markings or parking area signs are detected; and the planned trajectory ends at the center of the parking space.

[0032] In step S120, a scene template library may be configured in the vehicle, and the scene template library can be loaded when the vehicle's system starts. The scene template library may include a correspondence between multiple scene tags and multiple scene templates. The scene tags included in the scene template library are the same as the scene tags configured in the vehicle.

[0033] For example, the driving scenario tags configured in a vehicle may include scenario tags for highway cruising, urban congestion, intersection crossing, parking, and emergency safety. Then, the scenario template library contains multiple scenario tags for the aforementioned highway cruising, urban congestion, intersection crossing, parking, and emergency safety scenarios, each corresponding to a scenario template.

[0034] In this step, after determining the current scene tag, the scene tag that is the same as the current scene tag can be found in the scene template library, and the corresponding scene template can be determined as the target scene template.

[0035] The target scenario template is used to define memory management strategy parameters, which include at least one of the following: target block distribution, fragmentation tolerance threshold, compaction trigger frequency, large block retention ratio, and number of emergency reserved blocks. The target block distribution is used to define the target proportion of blocks of each order in the memory pool.

[0036] In some implementations, memory management strategy parameters can be obtained through offline calibration: in a simulation environment or closed site, the autonomous driving system is run in various scenarios to collect statistics on log type distribution, generation frequency, and size distribution, and the optimal block distribution ratio and fragmentation threshold are calculated accordingly.

[0037] The correspondence between scene labels and memory management strategy parameters can be found in the table below.

[0038]

[0039] In step S130, after the target scenario template is determined, the log memory pool can be adaptively reconstructed based on the determined target scenario template so that it corresponds to the target scenario template, thereby adapting it to the current driving scenario label, that is, adapting it to the current driving scenario.

[0040] Specifically, in response to the detection of a driving scenario switch, a reconstruction can be triggered in the system's idle window to adjust the current block distribution of the memory pool to the target block distribution.

[0041] The adaptive layout reconstruction may include the following: based on the target block distribution of the target scene template, prioritize the targeted reorganization of areas in the fragment heatmap where the fragment density is higher than the threshold and the block order dispersion is high; migrate the order blocks with high idle rates in the current scene to the order blocks required by the next scene; reset the fragment rate upper limit and the large block reserved water level line according to the target scene template; and activate the emergency reserved block pool corresponding to the target scene.

[0042] In some implementations, scene switching detection involves the scene recognition module outputting scene labels at a fixed frequency. Multi-frame de-jitter is performed (e.g., three consecutive frames, corresponding to 300 milliseconds, are used to determine a scene switch). Reconstruction triggering conditions include: the new scene label differs from the current scene label; the system CPU (central processing unit) load is below a set threshold (e.g., 40%) to ensure that reconstruction does not preempt business resources; and there are currently no urgent security scenarios to handle.

[0043] The refactoring process includes the following steps: Step 1: Load the target template. Read the strategy parameters corresponding to the new scene from the scene template library, including target block distribution, fragmentation tolerance threshold, consolidation trigger frequency, large block retention ratio, and number of emergency reserved blocks.

[0044] Step 2: Calculate the deviation between the actual block distribution of the current memory pool and the target block distribution. Compare the actual block distribution of the current memory pool with the target block distribution, and calculate the deviation for each order of blocks. A positive deviation indicates an excess of blocks of that order, while a negative deviation indicates an underabundance.

[0045] Step 3: Perform targeted defragmentation on memory regions with fragmentation density exceeding the fragmentation tolerance threshold. Based on the fragmentation heatmap, select regions with fragmentation density and block order dispersion exceeding the threshold as defragmentation targets. Perform live data migration and free block merging on these regions.

[0046] Step 4: Block Migration. For blocks of excess order, perform downward migration; for blocks of insufficient order, perform upward migration. That is, for blocks of order with positive deviation (excess blocks), perform downward migration: split the excess large-order blocks into smaller-order blocks needed for the next scene. During splitting, prioritize blocks located in areas with high fragment density, achieving both organization and migration in one step. For blocks of order with negative deviation (insufficient blocks), perform upward migration: find adjacent free blocks of the same order and merge them; if there are insufficient adjacent blocks, select adjacent blocks from lower-order blocks and merge them level by level to improve the order.

[0047] Step 5: Update the fragmentation rate cap, large block waterline, and compaction timer period of the memory pool according to the target scenario template. Update the global parameters of the memory pool: update the fragmentation rate cap to the fragmentation tolerance threshold of the target template; update the waterline of the largest block order to the large block retention ratio of the target template; update the compaction timer period to the compaction trigger frequency of the target template.

[0048] Step 6: Isolation Zone Switching. Return the blocks in the old scenario's emergency reserved block pool to the general pool (if the new scenario has fewer reserved blocks); allocate blocks from the general pool to build the new scenario's emergency reserved block pool (if the new scenario has more reserved blocks); mark the new reserved blocks as isolated and not participate in daily allocation.

[0049] Reconstruction completion determination: Reconstruction is determined to be complete when the deviation of each order block is less than the set proportion (e.g., 10%) and the fragmentation rate is lower than the tolerance threshold of the target template. If the reconstruction execution time exceeds the set upper limit (e.g., 500 milliseconds) and is still not completed, the reconstruction is paused, the current progress is recorded, and execution will continue in the next idle window.

[0050] In step S140, the operating system kernel I / O stack refers to the collective term for a series of software layers and subsystems in the operating system kernel used to handle input / output (I / O) requests. It is responsible for processing read and write operations issued by applications (user mode) layer by layer through the file system, block layer, general block device driver, etc., and finally passing them to the hardware device driver (such as NVMe (non-volatile memory) driver) to perform the actual data transmission, and returning the result along the original path.

[0051] Specifically, a user-space log I / O path can be constructed on the application processing cluster of the in-vehicle computing platform. A 4KB aligned memory pool available for DMA (Direct Memory Access) is allocated in physical memory as a runtime buffer for logs. Log data is directly used by the user-space driver to construct NVMe commands, which are then submitted to the in-vehicle NVMe solid-state storage via the PCIe interface. The NVMe controller directly reads the log data from physical memory via DMA and writes it to the NAND flash physical pages. The disk write thread runs on a dedicated CPU core and monitors the command completion status using a polling mode, without going through the operating system page cache or triggering kernel interrupt handling.

[0052] It should be noted that PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard used to connect core components such as CPU and memory with external devices (such as graphics cards, solid-state drives, and network cards) for high-bandwidth, low-latency data transmission.

[0053] In some implementations... User-space deterministic I / O paths can be implemented based on an in-vehicle heterogeneous computing platform, which includes an in-vehicle application processing computing cluster, large-capacity physical memory, a high-speed PCIe interface, and an in-vehicle NVMe solid-state storage.

[0054] The application processing cluster consists of multi-core ARM processors running a Linux operating system, and hosts the autonomous driving perception, planning, control, and logging business threads. Through a CPU affinity binding mechanism, the application processing cluster fixes the log writing thread to run on a dedicated physical core, avoiding CPU resource contention with the autonomous driving main control thread.

[0055] The physical memory uses large-capacity, low-power memory as the physical carrier for the log runtime memory pool. The memory pool requests physically contiguous DMA available space through the alignment allocation interface provided by the platform, ensuring that the memory address and length meet the block device alignment requirements.

[0056] The PCIe high-speed interface connects application processing clusters to automotive NVMe solid-state storage, supporting high-bandwidth, low-latency data transmission. NVMe solid-state storage typically has 4KB physical sectors and supports direct memory access via PCIe transaction layer packets.

[0057] In this implementation, the user-space deterministic I / O path is constructed as follows: In user-space addresses, a 4KB aligned physical contiguous memory pool is requested through the platform-provided aligned memory allocation interface. This memory pool serves as a runtime buffer for log data and is directly managed by user-space code, without going through the operating system's virtual memory allocator.

[0058] After log data is constructed into 4KB-aligned self-describing entries in the memory pool, the user-space driver directly constructs NVMe write commands. The command structure contains information such as opcode, namespace identifier, starting logical block address, and data length. The data buffer pointer points to a 4KB-aligned physical address in the memory pool. If the write data length exceeds the limit of a single NVMe command, multiple commands are constructed, and the physical page distribution of the data buffer is described by a physical region page list.

[0059] User-space drivers map the PCIe configuration space to the user-space address space via memory mapping, bypassing the operating system's NVMe kernel driver. The driver writes the constructed NVMe commands to the circular buffer of the commit queue, and through the DMA engine inside the NVMe controller, directly reads log data from physical memory via the PCIe bus, bypassing the operating system's page cache and CPU data copying, and writes the data to the NAND flash physical pages.

[0060] The disk write thread runs on a dedicated CPU core and continuously monitors the status of the completion queue using a polling mode. When a command completion flag is detected, the result is read from the completion queue, the corresponding background buffer is released, and the disk write progress is updated. The entire process does not trigger operating system interrupt handlers, and does not involve system calls or context switching.

[0061] It's important to note that traditional logging solutions rely on the operating system kernel I / O stack (file system, page cache, block layer, interrupt handling, etc.). The kernel's dirty page flushing mechanism triggers unpredictable forced disk flushing under memory pressure, resulting in I / O latency jitter of tens of milliseconds, severely impacting the real-time performance of the autonomous driving master control thread. This invention constructs a user-space deterministic I / O path, allocating an aligned memory pool available for DMA in physical memory. The user-space driver directly constructs NVMe write commands, submitting them to the onboard NVMe solid-state storage via the PCIe interface. The DMA engine within the NVMe controller directly reads log data from the memory pool via the PCIe bus and writes it to NAND flash physical pages. The entire process completely bypasses the operating system page cache, kernel NVMe driver, interrupt handlers, system calls, and context switching. The disk write thread runs on a dedicated CPU core, using a polling mode to monitor command completion status, avoiding the unpredictability of interrupt responses. As a result, the log write latency is reduced from tens of milliseconds in traditional solutions to a deterministic microsecond level, providing reliable time determinism for fault diagnosis and real-time debugging of autonomous driving systems.

[0062] Furthermore, related memory management technologies (buddy systems, slab allocators) treat memory as a homogeneous resource, with memory pool parameters remaining fixed after system startup. This fails to adapt to the drastic changes in log generation frequency, size distribution, and real-time requirements under different autonomous driving scenarios. This invention uses a scene recognition module to determine the current driving scenario label (highway cruising, urban congestion, intersection crossing, parking, emergency safety, etc.) in real time and matches the corresponding target scene template from a predefined scene template library. This template defines memory management strategy parameters such as target block distribution, fragmentation tolerance threshold, defragmentation trigger frequency, large block retention ratio, and emergency reserved block count. When a scene switch is detected, the system triggers adaptive layout reconstruction in the idle window: calculating the deviation between the actual block distribution of the current memory pool and the target block distribution, performing targeted defragmentation in areas with high fragmentation density and block order dispersion, migrating blocks with excessive order downwards and blocks with insufficient order upwards, and updating the fragmentation rate upper limit, large block waterline, and defragmentation timer period. Through the above mechanism, the block distribution and fragmentation management strategy of the memory pool can be dynamically adjusted according to the driving scenario, so that the memory resource configuration is matched with the log generation pattern of the current scenario in real time, which significantly improves memory utilization efficiency and overall system performance.

[0063] Compared with existing technologies that wait for fragmentation to accumulate to a certain level before passively triggering global defragmentation, this invention shifts the timing of memory fragmentation elimination from "passively triggered during business operation" to "actively executed during scene switching in the idle window" and "mildly fine-tuned during scene maintenance," thereby effectively reducing the interference of runtime defragmentation on the autonomous driving main control thread and ensuring the stable operation of the vehicle computing system.

[0064] In this embodiment, when the business thread records logs, it directly retrieves aligned buffer blocks from the pre-allocated memory pool. The allocation operation takes O(1) constant time, completely eliminating the millisecond-level uncertain latency caused by real-time lookup and splitting of general memory allocators (such as buddy systems). Combined with user-space deterministic I / O paths, the entire process from log generation to persistence completion has predictable low latency, meeting the stringent requirements of autonomous driving systems for real-time data recording.

[0065] In summary, this invention couples upper-layer driving semantics (scene tags, scene templates) with lower-layer hardware characteristics (dedicated CPU cores, PCIe DMA, NVMe) across layers, and introduces an online self-evolution closed loop for scene templates (dynamically adjusting target block distribution parameters based on heavy compaction frequency). This constructs a deterministic guarantee system covering the entire chain from scene perception and memory layout to storage disk persistence, significantly improving the real-time performance, stability, and resource utilization efficiency of autonomous driving vehicle computing systems.

[0066] In one exemplary embodiment, a log management method is provided, applicable to a vehicle. (Reference) Figure 2 and Figure 3 As shown, in this log management method, during the maintenance of the driving scenario, the fragmentation status of the memory pool is monitored, and dynamic fragmentation management of the memory pool is performed according to the target scenario template and the fragmentation status. The dynamic fragmentation management may include: When the fragmentation rate is below 50% of the fragmentation tolerance threshold, the sorting process is completely paused. When the fragmentation rate is between 50% and 100% of the fragmentation tolerance threshold, light compaction is triggered according to the compaction trigger frequency. When the fragmentation rate exceeds the fragmentation tolerance threshold, heavy defragmentation is immediately triggered until the fragmentation rate drops to a safe range.

[0067] The scene maintenance period refers to a state where the scene is under the same label for multiple consecutive frames (e.g., 300 milliseconds). During this period, fragment management enters a dynamic fine-tuning mode, that is, the memory pool is dynamically managed for fragmentation based on the target scene template and fragmentation status.

[0068] The global fragmentation rate can be calculated at a fixed frequency (e.g., every 100 milliseconds), defined as 1 minus the ratio of the maximum contiguous free memory to the total free memory.

[0069] In some implementations... Dynamic fragment management can be configured with a three-level response strategy: Level 1, Green Zone: The fragmentation rate is below 50% of the fragmentation tolerance threshold (referred to as the fragmentation tolerance threshold). In this state, the memory pool is healthy, defragmentation is completely paused, and only the fragmentation heatmap is updated, saving CPU resources.

[0070] Level 2, Yellow Zone: Fragmentation rate is between 50% and 100% of the tolerance threshold. Lightweight defragmentation is triggered at the frequency set by the current scenario template. Lightweight defragmentation only processes the 1 to 2 units with the highest fragmentation density in the fragmentation heatmap, and the defragmentation time for a single operation does not exceed the set upper limit (e.g., 10 milliseconds) to avoid affecting business threads.

[0071] Level 3, Red Zone: Fragmentation rate exceeds the tolerance threshold. Immediately trigger heavy defragmentation, processing all cells with fragmentation density greater than the set threshold until the fragmentation rate drops to the yellow zone.

[0072] In addition, in this embodiment, when the allocation failure rate of a certain type of log increases, the target proportion of the corresponding order block for that type of log is temporarily adjusted. For example, when a certain type of log (such as point cloud data) fails to allocate multiple times consecutively (e.g., 3 times consecutively), a temporary adjustment is triggered: the target proportion of the corresponding order block for that type of log is temporarily increased by a set percentage (e.g., 20%); idle blocks are allocated from adjacent order blocks to supplement it; if the original order blocks are insufficient after allocation, they are restored first in the next reorganization cycle.

[0073] In this embodiment, the fragmentation heatmap of the memory pool can also be updated at a fixed frequency. The fragmentation heatmap is updated at a fixed frequency using a snapshot copy mechanism, and the memory pool is not locked during the update process.

[0074] The fragmentation heatmap is used to divide the address space of the memory pool into multiple regions and calculate the fragmentation density, block order dispersion, and live data locality for each region. When performing targeted defragmentation in adaptive layout refactoring or dynamic fragmentation management, regions with both fragmentation density and block order dispersion higher than the corresponding thresholds can be preferentially selected as defragmentation targets.

[0075] In some implementations... A general heatmap data structure is used to characterize the degree of fragmentation in memory address regions.

[0076] Fragmentation heatmaps divide the memory pool address space into fixed-size regions (e.g., each heatmap unit is 64KB). For each unit, the following three metrics are calculated: Fragmentation density: Defined as the ratio of the total size of free blocks within a cell to the total size of the cell. A fragmentation density close to 1 indicates that the region is almost entirely empty, close to 0 indicates that it is almost entirely full, and an intermediate value indicates severe fragmentation.

[0077] Block order dispersion: defined as the ratio of the number of order types of free blocks within a cell to the maximum possible number of order types. A dispersion close to 1 indicates that the free blocks in that region are of mixed sizes, making sorting difficult; a dispersion close to 0 indicates that the blocks are of uniform size, making sorting easy.

[0078] Live data locality: defined as the ratio of the number of allocated blocks of the same type of log within a unit to the total number of allocated blocks within the unit. High locality indicates that the data types in that area are concentrated, resulting in high migration efficiency during cleanup; low locality indicates that the data is mixed, requiring frequent context switching during cleanup.

[0079] The fragmented heatmap is updated at a fixed frequency (e.g., every 100 milliseconds). During an update, the memory pool metadata index is traversed, and each heatmap unit is assigned an address, with the three metrics recalculated. The update process does not lock the memory pool; a snapshot copy mechanism is used: first, a snapshot of the current metadata index is copied; then, the three metrics are calculated on the snapshot; finally, the heatmap pointer is atomically replaced. This mechanism ensures that business threads can still access the memory pool without locks during heatmap updates.

[0080] Targeted consolidation selects the consolidation target area based on the fragment heatmap. Regions with fragment density greater than a set threshold (e.g., 0.5) and block order dispersion greater than a set threshold (e.g., 0.6) are prioritized as consolidation targets. For these regions, live data migration is performed: grouping by log type, and arranging live data of the same type consecutively towards the beginning of the region; freeing up tail space and merging into the largest possible contiguous blocks; if the merged block order matches the target deviation of the target scene template, it is retained; otherwise, further splitting or merging is performed.

[0081] Compared to global defragmentation, targeted defragmentation only performs the defragmentation on areas with high fragmentation density, reducing data migration and lowering defragmentation overhead and CPU usage.

[0082] In this embodiment, based on the fragmentation rate's location within the green zone (below the threshold of 50%), yellow zone (50%~100%), and red zone (above the threshold), complete pause, lightweight defragmentation, and heavy defragmentation are performed respectively. Defragmentation is triggered only when necessary, significantly reducing interference with autonomous driving business threads. The memory pool is divided into multiple regions, and fragmentation density, block order dispersion, and locality of live data are calculated in real time. Defragmentation of severely fragmented local regions is prioritized, avoiding global scanning and significantly reducing CPU usage and defragmentation time. During updates, metadata snapshots are copied and pointers are atomically replaced, eliminating the need for business threads to wait during updates and removing latency caused by memory pool locking. Furthermore, in this embodiment, when a certain type of log fails to allocate continuously, the target proportion of the corresponding order block is temporarily increased, and free blocks from adjacent orders are allocated to quickly alleviate memory shortages and prevent log loss. In short, this invention, through a three-level response strategy, targeted defragmentation, and adaptive adjustment, brings the runtime memory fragmentation rate close to zero, ensuring the long-term stable operation of the autonomous driving system.

[0083] In one exemplary embodiment, a log management method is provided, applicable to a vehicle. (Reference) Figure 2 and Figure 3As shown, in this log management method, the number of times heavy maintenance is triggered is monitored and counted under the same driving scenario tag. When the number of triggers reaches a set threshold within a set time window, the self-evolution correction process of the scenario template library is triggered.

[0084] It should be noted that both the duration window and the threshold can be set according to actual needs, and their specific values ​​are not limited. For example, the duration window can be set to 600 seconds, and the threshold can be set to 3 times.

[0085] In some implementations... During the scenario maintenance period, the fragmentation management fine-tuning module maintains a "heavy defragmentation cumulative counter," categorized and statistically analyzed by scenario tag. Whenever the dynamic fine-tuning level 3 response strategy triggers heavy defragmentation (i.e., the fragmentation rate exceeds the tolerance threshold), the counter for the corresponding scenario tag is incremented by 1. When the counter for the same scenario tag accumulates to a set threshold (e.g., 3 times) within a set time window (e.g., consecutive 600 seconds), it is determined that the scenario template has deviated from its parameters, triggering a self-evolutionary correction process. The first step is to extract fragment heatmap snapshots during heavy cleanup to analyze the locality characteristics of surviving data and identify the direction of parameter deviation, that is, to identify the direction of deviation between the target block distribution parameters in the current scenario template and the actual log generation pattern.

[0086] First, a snapshot of the fragment heatmap can be extracted each time a heavy defragmentation is triggered, and the distribution characteristics of the locality index of the survival data can be analyzed in detail—that is, the concentration of the types and the distribution of the number of logs allocated on each block of each order.

[0087] If the number of allocated blocks of a certain order (such as 16KB medium-order blocks) consistently exceeds the target proportion of that order in the target block distribution, and the allocation failure rate statistics show that the number of allocation failures for that order block is significantly higher than that for other orders, then it is determined that the proportion of that order in the target block distribution is too small. If the idle rate (number of free blocks / total number of blocks) of a certain order of block is consistently higher than a set threshold (e.g., 40%), and the allocation frequency of that order of block is significantly lower than that of other orders, then it is determined that the proportion of that order in the target block distribution is too large.

[0088] The second step involves using a moving exponential weighted average algorithm to correct the target block distribution parameters in the target scene template based on the deviation direction, so that the corrected parameters gradually converge toward the direction of the actually observed high-frequency allocation order.

[0089] The corrected target block distribution parameters can be calculated using a moving exponential weighted average algorithm. For the order k, which has a relatively small proportion of the judgment, the correction formula is: New target percentage = α × current actual occupancy rate + (1-α) × old target percentage Where α is the smoothing coefficient (e.g., 0.3), and the current actual occupancy rate is the proportion of the number of blocks of this order that have been allocated at the time of heavy compaction triggering to the total number of blocks of this order.

[0090] After correction, the proportions of other orders are normalized to ensure that the sum of the proportions of all orders remains at 100%.

[0091] The third step is to persistently update the corrected target block distribution parameters to the scene template library.

[0092] The corrected target block distribution parameters can be written to the persistent storage area of ​​the scene template library (such as a template configuration file partition on NVMe solid-state storage). A copy-on-write strategy is used during updates: the parameters are first written to a temporary file, their integrity is verified, and then the original template file is atomically replaced. After the update is complete, the scene template cache is synchronously updated in memory so that subsequent scene matching uses the new parameters. For example, when entering the scene later, the updated scene template is used to perform adaptive layout reconstruction.

[0093] In this implementation, after correction, when entering the scene again, the frequency of heavy cleanup triggering during the previous N (e.g., 3) scene maintenance periods is monitored. If the frequency of heavy cleanup triggering decreases, the correction is confirmed to be effective; if the triggering frequency does not decrease or even increases, the template parameters before correction are rolled back, and the α smoothing coefficient of the scene is reduced to weaken the subsequent correction magnitude.

[0094] It should be noted that in this implementation, the scene template library loads the initial parameters calibrated offline when the system is first started, serving as the initial values ​​for each scene template. The online self-evolutionary closed loop continuously corrects the initial values ​​during system operation. The corrected parameters remain valid after a system restart (they are persistently stored), but the system provides an interface to "restore factory calibration," allowing the template parameters for a specified scene or all scenes to be restored to their offline calibration initial values.

[0095] This embodiment bypasses the operating system page cache and kernel scheduling, reducing disk write latency from unpredictable tens of milliseconds of jitter to deterministic microseconds, meeting the real-time requirements of autonomous driving. Simultaneously, through proactive pre-allocation and adaptive layout reconstruction driven by scene templates, fragmentation shifts from "generated during business operations" to "eliminated and reorganized during system idle time," resulting in a near-zero runtime fragmentation rate. Furthermore, this embodiment utilizes an online self-evolution closed-loop mechanism for scene templates, upgrading target block distribution parameters from one-time offline calibration to continuous online correction, eliminating the problem of templates deviating from actual log generation patterns after long-term operation, and ensuring that policy parameters maintain optimal matching throughout the system's entire lifecycle. Additionally, log allocation during business operations preferentially retrieves data directly from the pre-allocated buffer block pool, achieving O(1) constant-time allocation and eliminating millisecond-level latency caused by real-time split lookup in the partner system. This embodiment couples the hardware characteristics of the in-vehicle computing platform (dedicated CPU core, PCIe DMA, NVMe) with upper-layer driving semantics (scene tags, log type timing), and achieves continuous optimization through the self-evolution of scene templates, forming a deterministic guarantee across the entire chain from perception to storage.

[0096] In one exemplary embodiment, a log management apparatus is provided, applicable to a vehicle. This apparatus can be used to implement the log management methods of the above embodiments. For example, refer to... Figure 4 As shown, the device may include: Scene perception module 10 is used to determine the current driving scene label of the autonomous driving system; and is also used to match the target scene template corresponding to the current driving scene label from a predefined scene template library in response to the current driving scene label. The memory pool management module 20 is used to perform adaptive layout reconstruction of the log memory pool according to the target scenario template. User-space deterministic I / O module 30 is used to bypass the operating system kernel I / O stack during the log recording process, construct NVMe commands through user-space drivers, and write log data to NVMe solid-state storage via a high-speed bus.

[0097] In one exemplary embodiment, a log management device is provided, applicable to a vehicle. This device can be used to implement the log management methods of the above embodiments. In this embodiment, reference is made to... Figure 4 As shown, the user-mode deterministic I / O module 30 can be used for: Allocate a memory pool available for direct memory access in physical memory as a runtime log buffer; The log data is constructed as self-describing entries aligned with the physical sectors of the NVMe solid-state memory; In the user-space driver, an NVMe write command is constructed, wherein the data buffer pointer in the NVMe write command points to a physical address in the memory pool; The NVMe write command is submitted to the submission queue to write log data to the NAND flash physical page of the NVMe solid-state storage via the DMA engine; A disk write thread runs on a dedicated CPU core and uses a polling mode to monitor the command completion status.

[0098] In one exemplary embodiment, a log management device is provided, applicable to a vehicle. This device can be used to implement the log management methods of the above embodiments. In this embodiment, reference is made to... Figure 4 As shown, the scene perception module 10 can be used for: Receive status signals from the autonomous driving system at a fixed frequency; Based on the status signal, the current driving scenario label is determined.

[0099] In one exemplary embodiment, a log management device is provided, applicable to a vehicle. This device can be used to implement the log management methods of the above embodiments. In this embodiment, reference is made to... Figure 4 As shown, the memory pool management module 20 can be used for: In response to the detection of a driving scenario switch, a refactoring is triggered during a system idle window to adjust the current block distribution of the memory pool to the target block distribution, including: Calculate the deviation between the actual block distribution of the current memory pool and the target block distribution; Perform targeted defragmentation on memory regions where the fragmentation density exceeds the fragmentation tolerance threshold; Perform down migration on blocks with excessive order and up migration on blocks with insufficient order; Update the fragmentation rate limit, large block water level, and compaction timer period of the memory pool according to the target scenario template.

[0100] In one exemplary embodiment, a log management device is provided, applicable to a vehicle. This device can be used to implement the log management methods of the above embodiments. In this embodiment, reference is made to... Figure 4 As shown, the memory pool management module 20 can be used for: During the driving scenario maintenance, the fragmentation status of the memory pool is monitored, and dynamic fragmentation management of the memory pool is performed according to the target scenario template and the fragmentation status. The dynamic fragment management includes: When the fragmentation rate is below 50% of the fragmentation tolerance threshold, the sorting process is completely paused. When the fragmentation rate is in the range of 50% to 100% of the fragmentation tolerance threshold, light compaction is triggered at the compaction trigger frequency. When the fragmentation rate exceeds the fragmentation tolerance threshold, heavy defragmentation is immediately triggered until the fragmentation rate drops to a safe range.

[0101] In one exemplary embodiment, a log management device is provided, applicable to a vehicle. This device can be used to implement the log management methods of the above embodiments. In this embodiment, reference is made to... Figure 4 As shown, the memory pool management module 20 can be used for: The fragmentation heatmap of the memory pool is updated at a fixed frequency; wherein the fragmentation heatmap is used to divide the address space of the memory pool into multiple regions and calculate the fragmentation density, block order dispersion and live data locality of each region; When performing targeted defragmentation in adaptive layout reconstruction or dynamic fragmentation management, regions where both the fragment density and the block order dispersion are higher than the corresponding thresholds are preferentially selected as defragmentation targets.

[0102] In one exemplary embodiment, a log management device is provided, applicable to a vehicle. This device can be used to implement the log management methods of the above embodiments. In this embodiment, reference is made to... Figure 4 As shown, the scene perception module 10 can be used for: Under the same driving scenario tag, monitor and count the number of times the heavy cleaning is triggered; When the number of triggers reaches a set threshold within a set time window, the self-evolution correction process of the scene template library is triggered. The self-correction process includes: Extract fragment heatmap snapshots during heavy cleanup to analyze local characteristics of surviving data to identify the direction of parameter deviation; The target block distribution parameters in the target scene template are corrected based on the deviation direction using a moving exponential weighted average algorithm. The corrected target block distribution parameters are persistently updated to the scene template library.

[0103] This embodiment provides a vehicle for implementing the above-described log management method. For example... Figure 5 As shown, the vehicle may include a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any of the foregoing method embodiments.

[0104] The communication bus 1140 mentioned in the above vehicle can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0105] Communication interface 1120 is used for communication between the aforementioned vehicle and other devices.

[0106] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0107] The processor 1110 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0108] In another embodiment of this application, a computer-readable storage medium is provided, on which a program for the methods described in the above embodiments is stored, wherein when the program is executed by a processor, it implements the steps of the methods described in any of the foregoing method embodiments.

[0109] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A log management method, characterized in that, The log management method includes: Determine the current driving scenario label for the autonomous driving system; In response to the current driving scenario tag, a target scenario template corresponding to the current driving scenario tag is matched from a predefined scenario template library; Based on the target scenario template, perform adaptive layout reconstruction on the log memory pool; During the logging process, the system bypasses the operating system kernel I / O stack, constructs NVMe commands through the user-space driver, and writes the log data to the NVMe solid-state storage via a high-speed bus.

2. The log management method according to claim 1, characterized in that, The method of bypassing the operating system kernel I / O stack, constructing NVMe commands through a user-space driver, and writing log data to the NVMe solid-state storage via a high-speed bus includes: Allocate a memory pool available for direct memory access in physical memory as a runtime log buffer; The log data is constructed as self-describing entries aligned with the physical sectors of the NVMe solid-state memory; In the user-space driver, an NVMe write command is constructed, wherein the data buffer pointer in the NVMe write command points to a physical address in the memory pool; The NVMe write command is submitted to the submission queue to write log data to the NAND flash physical page of the NVMe solid-state storage via the DMA engine; A disk write thread runs on a dedicated CPU core and uses a polling mode to monitor the command completion status.

3. The log management method according to claim 1, characterized in that, The driving scenario tags configured in the log management method include tags for at least one of the following scenarios: High-speed cruising scenarios, urban congestion scenarios, intersection traffic scenarios, parking scenarios, and emergency safety scenarios.

4. The log management method according to claim 1, characterized in that, The process of determining the current driving scenario label for the autonomous driving system includes: Receive status signals from the autonomous driving system at a fixed frequency; Based on the status signal, the current driving scenario label is determined.

5. The log management method according to claim 1, characterized in that, The target scenario template is used to define memory management strategy parameters, which include at least one of the following: Target block distribution, fragmentation tolerance threshold, defragmentation trigger frequency, large block retention ratio, and number of emergency reserved blocks; The target block distribution is used to define the target proportion of each order of blocks in the memory pool.

6. The log management method according to claim 5, characterized in that, The step of performing adaptive layout reconstruction of the log memory pool according to the target scenario template includes: In response to the detection of a driving scenario switch, a refactoring is triggered during a system idle window to adjust the current block distribution of the memory pool to the target block distribution, including: Calculate the deviation between the actual block distribution of the current memory pool and the target block distribution; Perform targeted defragmentation on memory regions where the fragmentation density exceeds the fragmentation tolerance threshold; Perform down migration on blocks with excessive order and up migration on blocks with insufficient order; Update the fragmentation rate limit, large block water level, and compaction timer period of the memory pool according to the target scenario template.

7. The log management method according to claim 5, characterized in that, The log management method includes: During the driving scenario maintenance, the fragmentation status of the memory pool is monitored, and dynamic fragmentation management of the memory pool is performed according to the target scenario template and the fragmentation status. The dynamic fragment management includes: When the fragmentation rate is below 50% of the fragmentation tolerance threshold, the sorting process is completely paused. When the fragmentation rate is in the range of 50% to 100% of the fragmentation tolerance threshold, light compaction is triggered at the compaction trigger frequency. When the fragmentation rate exceeds the fragmentation tolerance threshold, heavy defragmentation is immediately triggered until the fragmentation rate drops to a safe range.

8. The log management method according to claim 7, characterized in that, The log management method includes: The fragmentation heatmap of the memory pool is updated at a fixed frequency; wherein the fragmentation heatmap is used to divide the address space of the memory pool into multiple regions and calculate the fragmentation density, block order dispersion and live data locality of each region; When performing targeted defragmentation in adaptive layout reconstruction or dynamic fragmentation management, regions where both the fragment density and the block order dispersion are higher than the corresponding thresholds are preferentially selected as defragmentation targets.

9. The log management method according to claim 7, characterized in that, The log management method includes: Under the same driving scenario tag, monitor and count the number of times the heavy cleaning is triggered; When the number of triggers reaches a set threshold within a set time window, the self-evolution correction process of the scene template library is triggered. The self-correction process includes: Extract fragment heatmap snapshots during heavy cleanup to analyze local characteristics of surviving data to identify the direction of parameter deviation; The target block distribution parameters in the target scene template are corrected based on the deviation direction using a moving exponential weighted average algorithm. The corrected target block distribution parameters are persistently updated to the scene template library.

10. A log management device, characterized in that, The log management device includes: The scene perception module is used to determine the current driving scene label of the autonomous driving system; and is also used to match the target scene template corresponding to the current driving scene label from a predefined scene template library in response to the current driving scene label. The memory pool management module is used to perform adaptive layout reconstruction of the log memory pool according to the target scenario template; The user-space deterministic I / O module is used to bypass the operating system kernel I / O stack during the logging process, construct NVMe commands through the user-space driver, and write log data to NVMe solid-state storage via a high-speed bus.

11. A vehicle, characterized in that, include: A processor and a memory, the processor being configured to execute a control program stored in the memory to implement the log management method as described in any one of claims 1-9.

12. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the log management method as described in any one of claims 1-9.

Images