Vehicle Log Subsystem Architecture for Performance and Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log subsystems face inefficiencies that hinder system performance and resource utilization, requiring improvements in flexibility and resource management to enhance log data applications.
Innovation Solution
A log subsystem architecture with a log generation module, processing module, and output module, featuring a two-stage buffer design for reduced disk IO operations and intra-process communication to improve performance without additional resource costs, along with flexible output modes and cross-machine log fusion capabilities using an open source framework.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log data is written directly to disk frequently, then log data is preserved reliably, but disk IO operations increase and system performance decreases
Solution Approach 1:
The log output unit is segmented into a front-end unit and a back-end unit that operate independently. The front-end unit handles log data collection and buffering, while the back-end unit handles disk IO operations. This segmentation allows the front-end to continue collecting logs without being blocked by slow disk writes, thus improving system performance while ensuring log data is preserved through the back-end's reliable writing.
Solution Approach 2:
The front-end unit performs preliminary actions by collecting and buffering log data in memory before the back-end unit writes it to disk. This preliminary buffering action allows log data to be ready for immediate writing without frequent disk IO operations, maintaining both reliability of log preservation and performance by reducing the frequency of slow disk accesses.
2Productivity
If multiple threads are used for log processing, then log output performance improves, but system resource consumption increases
Solution Approach 1:
The front-end unit and back-end unit are merged within the same program process, sharing the same memory space and process resources. This merging allows efficient inter-unit communication through shared memory without the overhead of inter-process communication, improving log output performance while minimizing additional system resource consumption compared to using completely separate processes.
Solution Approach 2:
A synchronization mechanism acts as an intermediary between the front-end unit and back-end unit to coordinate their operations. This intermediary manages the handoff of log data from the front-end buffer to the back-end processing, enabling parallel operation of both units without requiring excessive system resources for complex thread management or communication protocols.
3Stability of the object's composition
If log output mode is fixed, then system stability is maintained, but flexibility to adapt to different debugging needs decreases
Solution Approach 1:
The log output unit is designed with dynamic configurability, allowing the output mode (file, console, or other formats) to be changed during system operation. The front-end unit can dynamically switch between different output destinations based on debugging needs, while the back-end unit adapts its processing accordingly. This dynamic design maintains system stability through controlled changes while providing the flexibility to adapt to different debugging scenarios.
Solution Approach 2:
The log output unit is designed with multi-functionality to support multiple output modes (file output, console output, and other formats) within a single unified architecture. This universal design allows the same log subsystem to serve different debugging needs without requiring separate systems, maintaining stability through a consistent core architecture while providing adaptability through configurable output options.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosure relates to the field of computer technologies, and specifically provides a log subsystem, a log system, a vehicle log system, and a vehicle, to solve a problem of how to improve performance and flexibility of the log subsystem, reduce system resource occupation, and extend log data applications. To this end, the log subsystem of the disclosure includes a log generation module, a log processing module, and a log output module. Each module runs in a program process, so that system resource occupation is reduced, and performance of the log subsystem is improved. A log output unit designed based on two stages of buffers, two threads, and two modes of wake-up and wait can not only improve log output performance but also avoid causing additional system resource costs. In addition, cross-machine log fusion of log data in a plurality of log subsystems is implemented using another mature open source logging framework, so that a developer can obtain stronger data support, which is conducive to acceleration of system development.