Embedded system black flower card flash root cause positioning method
By deploying a data agent in the embedded system to collect performance data in real time, and combining it with AI models and SO dynamic libraries to automatically analyze stack data, the problem of difficult problem localization in embedded systems is solved, enabling fast and accurate root cause localization and reducing the analysis difficulty and solution cycle for developers.
Patent Information
- Application Number
- CN202510866977.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-11-14
AI Technical Summary
Existing monitoring systems cannot monitor embedded systems in real time, lack root cause analysis capabilities, making it difficult to locate problems, time-consuming, and reliant on post-event log analysis, resulting in incomplete data and high communication costs for developers.
By deploying a data agent in the embedded system to collect performance data in real time and monitoring based on preset alarm rules, the system can automatically analyze stack data using AI models and SO dynamic libraries to pinpoint the root cause of problems in real time.
It enables rapid and accurate location of embedded system problems, reduces the difficulty of analysis for developers, shortens the problem-solving cycle, and reduces the investment of human resources.
Smart Images
Figure CN120950280A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method for locating the root cause of black card flashing in embedded systems. Background Technology
[0002] Most current monitoring systems only provide cloud service monitoring and do not cover embedded monitoring. They also only have the ability to monitor and alert, but lack the ability to analyze root causes and automatically identify the root causes of problems. Furthermore, most analysis tools are only used in development environments and are difficult to analyze problems in production environments, such as when a vehicle infotainment system suddenly freezes. The traditional method of locating the problem is to collect logs and then analyze them. However, because it is impossible to predict when the problem will occur, there are too many types of logs after the problem occurs, or the data for analysis is incomplete, making analysis difficult. Incomplete data also requires re-instrumenting and verification, and it is difficult for new packages to replicate the operating environment and process of the previous problem. This makes the problem location and resolution cycle very long. Moreover, after the problem occurs, notifying the various developers is also time-consuming. Often, a software version is iterated by multiple developers, so multiple issue tickets and communications are required to close the loop on the problem. This results in the time from the occurrence of a problem to its final resolution being measured in days or hours. Summary of the Invention
[0003] The purpose of this invention is to provide a method for locating the root cause of black spots, flashing, and scratching in embedded systems, so as to quickly and accurately locate the root cause of these issues.
[0004] This invention provides a root cause localization method for black card flashing in an embedded system, comprising: a data acquisition agent and a server, wherein the data acquisition agent is deployed in an embedded system environment;
[0005] The data collection agent collects performance data in real time and sends the performance data to the server. The performance data includes: CPU utilization, CPU load, CPU frequency, memory usage, and disk usage.
[0006] The server receives the performance data and monitors the performance data based on pre-set alarm rules. When an alarm rule is triggered, the server sends a stack capture command to the data collection agent.
[0007] The data acquisition agent receives and executes the stack capture command to obtain stack data, and then sends the stack data to the server.
[0008] The server sends the stack data to the AI model and the SO dynamic library respectively. The AI model parses the stack data to obtain the root cause response 1 that triggers the alarm rule; the SO dynamic library parses the stack data to obtain the root cause response 2 that triggers the alarm rule.
[0009] Furthermore, performance data is collected in real time by the data collection agent and sent to the server, including:
[0010] The data acquisition agent includes multiple data collectors, a buf assembler component, and a buf uploader component;
[0011] Ops data and buf data are collected from the system and applications through a data collector; wherein the performance data is included in the ops data;
[0012] The data is assembled using the buf assembler component and then passed to the buf uploader.
[0013] The assembled data is transmitted to the server via the buf uploader component.
[0014] Furthermore, the assembled data is transmitted to the server via the buf uploader component, including:
[0015] The buf uploader component serializes the assembled data into protobuf and transmits the serialized data to the Gateway using TCP and RPC protocols.
[0016] The received serialized data is routed to the server via the Gateway.
[0017] Furthermore, the server receives the performance data and monitors it based on pre-configured alarm rules. When an alarm rule is triggered, the server sends a stack trace capture instruction to the data collection agent, including:
[0018] The server component includes the parse component, the prometheus component, the alertmanager component, and the instructionoperation component.
[0019] The performance data is parsed using the parse component, and the performance data is detected using the pre-configured alarm rules in the prometheus component. When an abnormality in the performance data triggers an alarm rule, an alarm is issued through the alertmanager component, and a stack trace capture instruction is sent to the data collection agent through the instructionoperation component.
[0020] Furthermore, the data acquisition agent receives and executes the stack capture command to obtain stack data, and then sends the stack data to the server, including:
[0021] Stack data is collected by the data collection agent and continuously reported until the size of the captured data or the capture duration meets the pre-set configuration rules.
[0022] Furthermore, the server sends the stack data to the AI model and the SO dynamic library respectively, including:
[0023] The server also includes an analysis component;
[0024] The stack data is parsed using the parse component to obtain trace data, and the trace data is sent to the AI model and SO dynamic library for parsing using the analysis component.
[0025] Furthermore, the root cause response 1 and root cause response 2 are displayed through a visual interface.
[0026] The present invention has at least the following beneficial effects:
[0027] This invention uses a data acquisition agent to collect performance data of various processes on an embedded system in real time. Based on pre-set alarm rules, it monitors this performance data in real time. When the performance data meets the alarm rules, an alarm is triggered, automatically triggering the capture of exception stack information. The invention then combines AI models and SO dynamic libraries to analyze the anomalies in the stack data, thereby determining the root cause that triggered the alarm rules. This invention, through real-time monitoring and automated analysis using AI models and SO dynamic libraries, can quickly locate the root cause of problems (blackouts, flashing lights, etc.), reducing the difficulty for developers in locating the performance analysis of embedded system applications, shortening the problem-solving cycle, and reducing the human resource investment in problem location. It also solves the problems of traditional methods, such as reliance on post-event log analysis, incomplete data, difficulty in reproduction, and high communication costs for developers.
[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description
[0029] Figure 1 A schematic diagram of a root cause localization method for black card flashing in an embedded system provided by the present invention;
[0030] Figure 2 This is an architecture diagram of the data acquisition agent provided by the present invention;
[0031] Figure 3 This is an architecture diagram of the server provided by the present invention. Detailed Implementation
[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0033] Example 1: Combination Figures 1-3 This embodiment is described below.
[0034] This embodiment is a root cause localization method for black card flashing in an embedded system, including: a data acquisition agent and a server. The data acquisition agent is deployed in the embedded system environment and the acquisition frequency and target process are specified through the Yami config configuration file.
[0035] The data collection agent collects performance data in real time and sends the performance data to the server. The performance data includes, but is not limited to: CPU utilization, CPU load, CPU frequency, memory usage, and disk usage.
[0036] The server receives the performance data and monitors the performance data based on pre-set alarm rules. When an alarm rule is triggered, the server sends a stack capture command to the data collection agent.
[0037] The data acquisition agent receives and executes the stack capture command to obtain stack data, and then sends the stack data to the server.
[0038] The server sends the stack data to the AI model and the SO dynamic library respectively. The AI model parses the stack data to obtain the root cause response 1 that triggers the alarm rule; the SO dynamic library parses the stack data to obtain the root cause response 2 that triggers the alarm rule.
[0039] Black indicates a black screen, where the embedded system screen suddenly goes black and becomes unresponsive; "Ghosting" indicates a distorted screen display, with garbled characters or images appearing; "Stuttering" indicates a lag, with system operation being delayed and slow to respond; "Flickering" indicates a crash, where the application or system suddenly crashes and exits. These are common performance issues in embedded devices (such as in-vehicle systems and smart hardware). Traditional methods require manual log collection and analysis, which is time-consuming and inefficient.
[0040] This invention uses a data acquisition agent to collect performance data of various processes on an embedded system in real time. Based on pre-set alarm rules, it monitors the performance data in real time and triggers an alarm when the performance data meets the alarm rules (e.g., CPU utilization > 80% for 5 seconds). It also automatically triggers the capture of exception stack information and then analyzes the stack data using an AI model and SO dynamic library to determine the root cause of the alarm, such as "kernel driver deadlock" or "memory leak." This invention, through real-time monitoring and automated analysis using AI models and SO dynamic libraries, can quickly locate the root cause of problems (blackouts, flashing lights, etc.), reducing the difficulty for developers in locating the performance analysis of embedded system applications, shortening the problem-solving cycle, and reducing the human resource investment in problem location. It also solves the problems of traditional methods, such as reliance on post-event log analysis, incomplete data, difficulty in reproduction, and high communication costs for developers.
[0041] Furthermore, performance data is collected in real time by the data collection agent and sent to the server, including:
[0042] The data acquisition agent includes multiple data collectors, a buf assembler component, and a buf uploader component; the data collectors include: eBPF Tracer, Ftrace, and NormalCollectors;
[0043] Ops data and buf data are collected from the system and applications through a data collector; wherein the performance data is included in the ops data;
[0044] The data is assembled using the buf assembler component and then passed to the buf uploader.
[0045] The assembled data is transmitted to the server via the buf uploader component.
[0046] Data collectors gather different types of data (ops and buf) and pass them to internal components. For example, eBPF Tracer and FTrace collect system tracing-related data, while Normal Collectors collect general system and application-level data, such as basic system information and application stack information. Internal components (such as ops receiver, opsdispatcher, buf uploader, and buf assembler) receive, distribute, and assemble the data. For instance, the ops receiver component receives data, the ops dispatcher component distributes and processes the data, the buf assembler component assembles the data, and the buf uploader transmits the assembled data to the server.
[0047] Furthermore, the assembled data is transmitted to the server via the buf uploader component, including:
[0048] The buf uploader component serializes the assembled data into protobuf and transmits the serialized data to the Gateway using TCP and RPC protocols.
[0049] The received serialized data is routed to the server via the Gateway.
[0050] The Gateway performs preliminary processing on the received data, such as protocol conversion and load balancing, and then forwards the processed data to the Server.
[0051] Furthermore, the server receives the performance data and monitors it based on pre-configured alarm rules. When an alarm rule is triggered, the server sends a stack trace capture instruction to the data collection agent, including:
[0052] The server component includes the parse component, the prometheus component, the alertmanager component, and the instructionoperation component.
[0053] The performance data is parsed using the `parse` component, and pre-configured alert rules in the `prometheus` component are used to monitor the performance data. When performance data anomalies trigger alert rules, the `alertmanager` component sends an alert notification, notifies developers via email, and automatically adds relevant personnel to WeChat / DingTalk group chats to share the issue and its root cause, ensuring developers are promptly informed and can address it. The `instructionoperation` component then sends a stack trace capture command to the data collection agent.
[0054] Furthermore, the data acquisition agent receives and executes the stack capture command to obtain stack data, and then sends the stack data to the server, including:
[0055] Stack data is collected by the data collection agent and continuously reported until the size of the captured data or the capture duration meets the pre-set configuration rules.
[0056] Furthermore, the server sends the stack data to the AI model and the SO dynamic library respectively, including:
[0057] The server also includes an analysis component;
[0058] The stack data is parsed using the parse component to obtain trace data, and the trace data is sent to the AI model and SO dynamic library for parsing using the analysis component.
[0059] Furthermore, the root cause response 1 and root cause response 2 are displayed through a visual interface.
[0060] Grafana, as a web UI tool, retrieves data from the REST API provided by the server and visualizes it in the form of charts, making it easier for developers to monitor system status. The visualization interface includes a dashboard, an alarm configuration interface, a log interface, and a performance analysis interface. The dashboard displays data information for various metrics, such as CPU utilization, CPU load, CPU frequency, sensor temperature, memory information, network information, and process information. The alarm configuration interface is used to configure thresholds for various monitoring metrics, i.e., setting alarm rules. Once an metric triggers an alarm condition, an alarm message is generated, and the subsequent action logic is executed, such as capturing stack traces. The visualization interface also displays the root cause of the problem and the metric and log data at the time the problem occurred. The log interface analyzes the output information generated during program execution, displaying multi-dimensional log information, such as application logs, kernel logs, and network logs. The performance analysis interface is used for detailed analysis of trace data.
[0061] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0062] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for root cause localization of black-patterned flashing cards in an embedded system, characterized in that, include: The data acquisition agent and the server are deployed in an embedded system environment. The data collection agent collects performance data in real time and sends the performance data to the server. The performance data includes: CPU utilization, CPU load, CPU frequency, memory usage, and disk usage. The server receives the performance data and monitors the performance data based on pre-set alarm rules. When an alarm rule is triggered, the server sends a stack capture command to the data collection agent. The data acquisition agent receives and executes the stack capture command to obtain stack data, and then sends the stack data to the server. The server sends the stack data to the AI model and the SO dynamic library respectively. The AI model parses the stack data to obtain the root cause response 1 that triggers the alarm rule; the SO dynamic library parses the stack data to obtain the root cause response 2 that triggers the alarm rule.
2. The method for root cause localization of black-patterned flashing cards in an embedded system according to claim 1, characterized in that, The performance data is collected in real time by the data collection agent and sent to the server, including: The data acquisition agent includes multiple data collectors, a buf assembler component, and a buf uploader component; Ops data and buf data are collected from the system and applications through a data collector; wherein the performance data is included in the ops data; The data is assembled using the buf assembler component and then passed to the buf uploader. The assembled data is transmitted to the server via the buf uploader component.
3. The method for root cause localization of black-patterned flashing cards in an embedded system according to claim 2, characterized in that, The assembled data is transmitted to the server via the buf uploader component, including: The buf uploader component serializes the assembled data into protobuf and transmits the serialized data to the Gateway using TCP and RPC protocols. The received serialized data is routed to the server via the Gateway.
4. The root cause localization method for black spot flashing in an embedded system according to claim 3, characterized in that, The server receives the performance data and monitors it based on pre-configured alarm rules. When an alarm rule is triggered, the server sends a stack trace capture command to the data collection agent, including: The server component includes the parse component, the prometheus component, the alertmanager component, and the instructionoperation component. The performance data is parsed using the parse component, and the performance data is detected using the pre-configured alarm rules in the prometheus component. When an abnormality in the performance data triggers an alarm rule, an alarm is issued through the alertmanager component, and a stack trace capture instruction is sent to the data collection agent through the instructionoperation component.
5. The root cause localization method for black-patterned flashing cards in an embedded system according to claim 4, characterized in that, The data acquisition agent receives and executes the stack capture command to obtain stack data, and then sends the stack data to the server, including: Stack data is collected by the data collection agent and continuously reported until the size of the captured data or the capture duration meets the pre-set configuration rules.
6. The method for root cause localization of black-patterned flashing cards in an embedded system according to claim 5, characterized in that, The server sends the stack data to the AI model and the SO dynamic library respectively, including: The server also includes an analysis component; The stack data is parsed using the parse component to obtain trace data, and the trace data is sent to the AI model and SO dynamic library for parsing using the analysis component.
7. The root cause localization method for black card flashing in an embedded system according to claim 6, characterized in that, The root cause response 1 and root cause response 2 are displayed through a visual interface.