Thread jitter monitoring method and computing device
Patent Information
- Application Number
- CN202610533437.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-21
- Publication Date
- 2026-09-01
AI Technical Summary
[0004]然而,相关技术中的线程抖动监测方式,需要针对性地修改用户态代码或容器镜像,无法适应多种多样的抖动监测场景
[0089]第五方面,本申请实施例提供一种计算机程序产品,包括计算机程序,该计算机程序被电子设备执行时实现第一方面以及任一可能实现方式中所涉及的线程抖动监测方法。
Smart Images

Figure CN122672884A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a thread jitter monitoring method and computing device. Background Technology
[0002] In applications with extremely high real-time requirements, such as industrial control, autonomous driving, and cloud computing, systems typically employ containerization technology to deploy real-time tasks. These real-time tasks often run as periodic threads, and the determinism of their execution time directly impacts the stability and reliability of the system. Jitter, the deviation between the actual execution time and the expected execution time of a task, is a key indicator for measuring the performance of real-time systems. Accurate monitoring of real-time thread jitter provides a foundation for high system performance.
[0003] In related technologies, users typically employ an intrusive approach by modifying the system's real-time program to monitor thread jitter. Specifically, technicians, within a user-space monitoring agent, periodically collect and report thread execution time data by calling callback functions that complete a cycle. A second platform then calculates the jitter value based on the reported data.
[0004] However, the thread jitter detection methods in related technologies require specific modifications to user-space code or container images, and cannot adapt to a wide variety of jitter detection scenarios. Summary of the Invention
[0005] This application provides a thread jitter monitoring method and computing device that does not require intrusive modification of user-space code or container images and can adapt to jitter monitoring in multiple scenarios.
[0006] In a first aspect, embodiments of this application provide a thread jitter monitoring method, including:
[0007] Based on a preset dynamic probe running in the kernel of the operating system, obtain the thread identification information and time tracking point information of the real-time thread in the operating system;
[0008] Based on the thread identifier information, determine whether the real-time thread is the target thread;
[0009] If the real-time thread is determined to be the target thread, then the jitter value of the target thread is calculated based on the time tracking point information of the target thread;
[0010] The target thread is subjected to jitter processing based on the jitter value.
[0011] This application provides a non-intrusive thread jitter monitoring method. For the operating system to be monitored, a dynamic probe is pre-configured in the operating system kernel. This dynamic probe can directly acquire kernel-level real-time thread identification information and time tracking point information. The thread identification information can identify whether the currently running real-time thread is the target thread. If it is determined to be the target thread, the jitter value of the target thread can be directly calculated through the time tracking point information collected by the dynamic probe, thereby achieving efficient and accurate jitter processing. Since the data collected for threads in this application is all implemented through the kernel's dynamic probe, kernel-level dynamic data tracking is achieved. This eliminates the need for customized modifications to user-space code or container images for different operating systems, reducing the complexity of jitter monitoring and making it applicable to jitter monitoring in multiple scenarios.
[0012] In one possible implementation, the time tracking point information includes scheduling switching information and verification event information;
[0013] Accordingly, calculating the jitter value of the target thread based on the time tracking point information of the target thread includes:
[0014] Based on the verification event information, the scheduling switching information is checked for false alarms to determine whether the scheduling switching event corresponding to the scheduling switching information is a false alarm event.
[0015] If the scheduling switch event is determined to be a false alarm event, then the scheduling switch information is filtered.
[0016] If it is determined that the scheduling switch event is not a false alarm event, then the jitter value of the target thread is calculated based on the scheduling switch information.
[0017] In this embodiment, a dynamic probe can collect two types of time tracking point information: scheduling switching information for jitter value calculation and verification event information for false alarm event verification. By verifying the event information, it is possible to determine whether the current scheduling switching event is a false alarm event, effectively identify and filter out false alarm events caused by external interference, and ensure that only real and valid periodic scheduling is used for jitter calculation, thereby improving the accuracy of monitoring results and avoiding the overhead of processing invalid events, thus reducing computing power costs.
[0018] In one possible implementation, the verification event information includes timer event information;
[0019] Accordingly, the step of performing false alarm verification on the scheduling handover information based on the verification event information to determine whether the scheduling handover event corresponding to the scheduling handover information is a false alarm event includes:
[0020] Based on the timer event information, determine whether there is a timer event associated with the scheduling switch event;
[0021] If an associated timer event exists, then the scheduling switch event is determined not to be a false alarm event;
[0022] If no associated timer event exists, the scheduling switch event is determined to be a false alarm event.
[0023] This application further refines the verification event information into timer event information. By determining whether the scheduling switching event is associated with a timer event, a direct and efficient basis for false alarm verification is provided. Since the core characteristic of periodic threads is that they are driven by timers, using timers as the judgment benchmark for periodic scheduling, it is only considered valid when the scheduling event is associated with a timer event. This is the core implementation method of the double confirmation mechanism, which can accurately eliminate false scheduling caused by non-timer triggers, ensuring that the jitter value calculation only originates from the expected periodic task, making the false alarm filtering logic more accurate and targeted, further improving the accuracy of monitoring results, and further saving computing power.
[0024] In one possible implementation, the timer event information includes tracking the timer assignment status;
[0025] Accordingly, determining whether there is a timer event associated with the scheduling switch event based on the timer event information includes:
[0026] Obtain the tracking timer assignment status corresponding to the scheduling switching event;
[0027] If the tracking timer assignment status is valid, then it is determined that there is a timer event associated with the scheduling switch event;
[0028] If the tracking timer assignment state is invalid, it is determined that there is no timer event associated with the scheduling switch event.
[0029] Here, in this embodiment, timer event information is expressed by tracking the timer assignment status. By reading this status value, it is directly determined whether the current scheduling switch event is associated with a timer event. Specifically, when the target thread is scheduled, if the tracked timer status is valid, it is determined that there is an associated timer event; if it is invalid, it is determined that there is no association. This simplifies the complex timing association judgment into a single status variable, allowing the dynamic probe to complete the core double confirmation verification with only one status read at the scheduling point, reducing the complexity of the verification and further ensuring the stability of the operating system.
[0030] In one possible implementation, before obtaining the tracking timer assignment status corresponding to the scheduling switching event, the method further includes:
[0031] When the preset dynamic probe captures a timer event and the execution interval between two schedulings of the target thread is greater than a preset time threshold, the tracking timer is updated to the valid state;
[0032] When a scheduling switch occurs and the thread being switched out is the target thread, the tracking timer is updated to an invalid state;
[0033] When the execution interval between two scheduling operations of the target thread is less than a preset time threshold, the tracking timer is updated to an invalid state.
[0034] This application specifies the update rules for the tracking timer's assignment state: it is set to valid when a timer event is triggered, invalid when the target thread is switched out of the Central Processing Unit (CPU), and invalid when the interval between two scheduling events is too short. These explicit update rules ensure that the tracking timer state accurately reflects the real-time correlation between the current scheduling event and the timer event. Invalidating the state when the target thread switches out of the CPU is used to filter interrupted or voluntarily yielding scenarios. Invalidating the state when the execution interval is less than a threshold further enhances the ability to eliminate false alarms with short intervals, thereby improving the accuracy of the double confirmation mechanism, further saving computational resources for jitter monitoring, and improving the stability and security of the operating system.
[0035] In one possible implementation, the scheduling switching information includes a scheduling timestamp;
[0036] Accordingly, calculating the jitter value of the target thread based on the scheduling switching information includes:
[0037] Obtain the current scheduling timestamp and the previous scheduling timestamp of the scheduling switch event;
[0038] The jitter value of the target thread is calculated based on the current scheduling timestamp, the previous scheduling timestamp, and the preset period.
[0039] This application provides a specific implementation method for calculating jitter value based on scheduling timestamp. By obtaining the precise timestamps of two adjacent valid schedulings and comparing them with a preset period time, the jitter value of each scheduling can be accurately calculated. Through a simple and accurate calculation method, accurate jitter monitoring is achieved, ensuring the stability and security of the operating system.
[0040] In one possible implementation, the thread identification information includes a parent process identifier and / or a network namespace pointer of the container to which it belongs; determining whether the real-time thread is the target thread based on the thread identification information includes:
[0041] Obtain the monitoring identifier corresponding to the thread identifier information; wherein, the monitoring identifier corresponding to the parent process identifier is the process identifier of the main process to be monitored, and the monitoring identifier corresponding to the network namespace pointer of the container to which it belongs is the network namespace pointer of the target container to be monitored;
[0042] The thread identifier information is matched with the corresponding identifier to be monitored;
[0043] If a match is found, the real-time thread is determined to be the target thread.
[0044] This application's embodiments achieve accurate identification and filtering of target threads by pre-setting a monitoring identifier and matching it with real-time thread identifier information. Specifically, by utilizing container network namespace pointers, it is possible to accurately identify and monitor all threads running within a specific container, adapting to cloud-native and containerized deployment environments. This solves the problem of traditional host process identifier-based monitoring methods failing in the context of widespread container technology adoption, ensuring the accuracy of the monitoring system in containerized deployments. Utilizing parent process identifiers allows for the associated monitoring of a specific process and all its child threads, significantly reducing the number of processes that need to be compared and analyzed, thereby significantly reducing performance overhead during the monitoring process.
[0045] In one possible implementation, the step of jittering the target thread based on the jitter value includes:
[0046] If the jitter value is greater than the preset jitter threshold, then the target thread is determined to have experienced a jitter anomaly.
[0047] Based on the time tracking point information, jitter anomaly information is output; wherein, the jitter anomaly information includes at least one of the current maximum jitter error value, function call stack information, and the longest time-consuming function call point.
[0048] In this embodiment, jitter handling is specifically implemented as threshold comparison and anomaly information output. The anomaly information includes the maximum jitter error value, function call stack, and the longest-running function call point. This achieves real-time jitter anomaly detection and outputs rich diagnostic data. The function call stack and the longest-running point enable precise problem localization, significantly reducing the time developers spend troubleshooting the root cause of jitter and improving the maintainability of the operating system.
[0049] Secondly, embodiments of this application provide a thread jitter monitoring device, comprising: an acquisition module, a determination module, a calculation module, and a processing module, wherein,
[0050] The acquisition module is used to acquire thread identification information and time tracking point information of real-time threads in the operating system based on a preset dynamic probe running in the kernel of the operating system.
[0051] The determination module is used to determine whether the real-time thread is the target thread based on the thread identification information;
[0052] The calculation module is used to calculate the jitter value of the target thread based on the time tracking point information of the target thread if the real-time thread is determined to be the target thread.
[0053] The processing module is used to perform jitter processing on the target thread based on the jitter value.
[0054] In one possible implementation, the time tracking point information includes scheduling switching information and verification event information;
[0055] Accordingly, the computing module is used for:
[0056] Based on the verification event information, the scheduling switching information is checked for false alarms to determine whether the scheduling switching event corresponding to the scheduling switching information is a false alarm event.
[0057] If the scheduling switch event is determined to be a false alarm event, then the scheduling switch information is filtered.
[0058] If it is determined that the scheduling switch event is not a false alarm event, then the jitter value of the target thread is calculated based on the scheduling switch information.
[0059] In one possible implementation, the verification event information includes timer event information;
[0060] Accordingly, the computing module is used for:
[0061] Based on the timer event information, determine whether there is a timer event associated with the scheduling switch event;
[0062] If an associated timer event exists, then the scheduling switch event is determined not to be a false alarm event;
[0063] If no associated timer event exists, the scheduling switch event is determined to be a false alarm event.
[0064] In one possible implementation, the timer event information includes tracking the timer assignment status;
[0065] Accordingly, the computing module is specifically used for:
[0066] Obtain the tracking timer assignment status corresponding to the scheduling switching event;
[0067] If the tracking timer assignment status is valid, then it is determined that there is a timer event associated with the scheduling switch event;
[0068] If the tracking timer assignment state is invalid, it is determined that there is no timer event associated with the scheduling switch event.
[0069] In one possible implementation, before obtaining the tracking timer assignment status corresponding to the scheduling switching event, the calculation module is further configured to:
[0070] When the preset dynamic probe captures a timer event and the execution interval between two schedulings of the target thread is greater than a preset time threshold, the tracking timer is updated to the valid state;
[0071] When a scheduling switch occurs and the thread being switched out is the target thread, the tracking timer is updated to an invalid state;
[0072] When the execution interval between two scheduling operations of the target thread is less than a preset time threshold, the tracking timer is updated to an invalid state.
[0073] In one possible implementation, the scheduling switching information includes a scheduling timestamp;
[0074] Accordingly, the computing module is also specifically used for:
[0075] Obtain the current scheduling timestamp and the previous scheduling timestamp of the scheduling switch event;
[0076] The jitter value of the target thread is calculated based on the current scheduling timestamp, the previous scheduling timestamp, and the preset period.
[0077] In one possible implementation, the thread identification information includes the parent process identifier and / or the network namespace pointer of the container to which it belongs; the determining module is specifically used for:
[0078] Obtain the monitoring identifier corresponding to the thread identifier information; wherein, the monitoring identifier corresponding to the parent process identifier is the process identifier of the main process to be monitored, and the monitoring identifier corresponding to the network namespace pointer of the container to which it belongs is the network namespace pointer of the target container to be monitored;
[0079] The thread identifier information is matched with the corresponding identifier to be monitored;
[0080] If a match is found, the real-time thread is determined to be the target thread.
[0081] In one possible implementation, the processing module is used to:
[0082] If the jitter value is greater than the preset jitter threshold, then the target thread is determined to have experienced a jitter anomaly.
[0083] Based on the time tracking point information, jitter anomaly information is output; wherein, the jitter anomaly information includes at least one of the current maximum jitter error value, function call stack information, and the longest time-consuming function call point.
[0084] Thirdly, embodiments of this application provide a computing device, which includes a memory and a processor;
[0085] Memory and processor are coupled;
[0086] Memory is used to store program instructions;
[0087] The processor is used to invoke program instructions to cause the computing device to execute the thread jitter monitoring method as described in the first aspect and any possible implementation.
[0088] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by an electronic device, are used to implement the thread jitter monitoring method as described in the first aspect and any possible implementation.
[0089] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by an electronic device, implements the thread jitter monitoring method involved in the first aspect and any possible implementation.
[0090] The thread jitter monitoring method and computing device provided in this application pre-configure a dynamic probe in the kernel of the operating system to be monitored. This dynamic probe can directly acquire the thread identification information and time tracking point information of the real-time thread at the kernel level. The thread identification information can identify whether the currently running real-time thread is the target thread. If it is determined to be the target thread, the jitter value of the target thread can be directly calculated through the time tracking point information collected by the dynamic probe, thereby achieving efficient and accurate jitter processing. Since the data collected for threads in this application is all implemented through the kernel's dynamic probe, kernel-level dynamic data tracking is achieved. There is no need to customize and modify user-space code or container images for different operating systems, which reduces the complexity of jitter monitoring and can be applied to jitter monitoring in multiple scenarios. Attached Figure Description
[0091] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0092] Figure 1 A schematic diagram of the structure of an operating system provided in an embodiment of this application;
[0093] Figure 2 A flowchart illustrating the thread jitter monitoring method provided in this application embodiment. Figure 1 ;
[0094] Figure 3 A flowchart illustrating the thread jitter monitoring method provided in this application embodiment. Figure 2 ;
[0095] Figure 4 A schematic diagram of the event process of a dual confirmation mechanism provided in an embodiment of this application;
[0096] Figure 5 A flowchart illustrating the method for calculating jitter values provided in an embodiment of this application;
[0097] Figure 6 This is a schematic diagram of the structure of a thread jitter monitoring device provided in an embodiment of this application;
[0098] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation
[0099] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.
[0100] To facilitate understanding, the technical terms involved in the embodiments of this application will be explained first.
[0101] Containers are a lightweight virtualization technology used to package applications and their runtime environment (including dependencies, configuration files, etc.) together, ensuring consistent application performance across different computing environments. Containers provide process-level resource isolation and limitations. In this embodiment, the container serves as an environment for running real-time tasks, and fine-grained monitoring of real-time threads within the container is achieved through kernel dynamic tracing technology.
[0102] Real-Time Thread (RT Thread): This refers to a thread with strict time constraints, whose tasks need to be completed within a specific time to ensure the normal operation of the system. Real-Time Threads are commonly used in latency-sensitive scenarios such as industrial control and autonomous driving. This application's embodiments calculate the jitter value during the execution of real-time threads by tracking their scheduling points, interrupt events, and timer trigger points, thereby evaluating whether the system meets real-time requirements.
[0103] Jitter refers to the deviation between the actual execution time and the expected execution time of a task. It reflects the uncertainty of system performance and is an important indicator of real-time performance. Excessive jitter can cause real-time tasks to fail to complete on time, leading to system performance degradation or even functional failure. This application's embodiments improve system stability by accurately calculating jitter values and detecting anomalies.
[0104] Kernel probe (kprobe): This refers to the dynamic probe in this embodiment. It is a dynamic tracing mechanism provided by the operating system kernel, used to insert probes into kernel code to capture information about specific functions or events without modifying the kernel code. kprobe can be used to monitor kernel function calls, variable changes, and other kernel events. This embodiment utilizes kprobe technology to set key tracing points, achieving non-intrusive monitoring of real-time thread behavior in a containerized environment.
[0105] A scheduling point is a point in time in which the operating system kernel decides to switch between currently running processes or threads. Common scheduling points include scheduler switches (sched_switch), where a process is switched off the CPU while another process is switched onto the CPU. Scheduling points record the time information for each process from suspension to wake-up, and from running to pause. This application's embodiments analyze the scheduling behavior of real-time threads and calculate their execution time jitter by tracking these scheduling points.
[0106] A timer is a hardware or software mechanism used to trigger an operation or event after a specified time. The High Resolution Timer (hrtimer) in the operating system kernel is a commonly used tool for achieving microsecond-level precision timing. In this embodiment, the timer is used to trigger periodic tasks, such as periodic data acquisition and transmission tasks in industrial control. This embodiment calculates jitter values by tracking timer events, such as High Resolution Timer Start (hrtimer_start) and High Resolution Timer Expire Entry (hrtimer_expire_entry), combined with scheduling point data.
[0107] Non-intrusive monitoring is a data acquisition method that does not require modification of the target system code and does not significantly affect the target system's performance. This application's embodiments use kprobe technology to implement non-intrusive monitoring, thereby avoiding the additional overhead and adverse effects on the original system logic caused by traditional log instrumentation methods.
[0108] The parent-child relationship filtering mechanism is a method that filters target threads by checking the parent-child relationship between the main process and its child processes. Specifically, after the main process is awakened, its process identifier (PID) is recorded. Then, related child threads are matched and monitored based on this PID, filtering out irrelevant data. This avoids the high-overhead operation of frequently comparing all thread identifiers (IDs) in traditional methods and improves data processing efficiency in multi-threaded concurrent scenarios.
[0109] Double verification mechanism: This refers to a technical means of ensuring data accuracy by cross-validating data through two independent but related data sources. In this embodiment, these two data sources are: the real-time thread scheduling switching point and the timer trigger and clear event. This eliminates false alarms caused by external interference or non-target tasks, ensuring that the jitter calculation results are reliable and accurate.
[0110] Anomaly Detection: When the jitter value of a real-time task exceeds a set threshold range, an alarm mechanism is triggered and anomaly information is recorded. This application embodiment supports automatic output of the maximum jitter error, function call stack information, and the location of the longest execution time, providing developers with a basis for troubleshooting.
[0111] Problem localization: By analyzing key information in the anomaly detection log, such as function call stacks and execution times, the location or cause of the jitter problem can be quickly identified (e.g., a function taking too long to execute). In this embodiment, this function greatly shortens the problem investigation cycle and improves fault repair efficiency.
[0112] Traditional thread jitter monitoring methods require specific modifications to user-space code or container images, making them unsuitable for diverse jitter monitoring scenarios. To address these issues, the thread jitter monitoring method and computing device provided in this application pre-configure a dynamic probe within the operating system kernel. This dynamic probe directly acquires kernel-level real-time thread identification information and time tracking point information. The thread identification information identifies whether the currently running real-time thread is the target thread. If it is determined to be the target thread, the jitter value of the target thread can be directly calculated using the time tracking point information collected by the dynamic probe, thereby achieving efficient and accurate jitter processing.
[0113] Optionally, embodiments of this application provide a method and apparatus for monitoring jitter in a multi-channel real-time container system. By combining kernel dynamic tracing technology with container isolation characteristics, it achieves accurate monitoring and problem localization of real-time thread jitter, realizing a non-intrusive monitoring method and system for container process jitter in a real-time system. It is suitable for scenarios with extremely high real-time requirements, such as industrial control and autonomous driving.
[0114] The application scenarios involved in the embodiments of this application are described below.
[0115] The specific application environment architecture or hardware architecture on which the thread jitter monitoring method depends is described herein. The thread jitter monitoring system can be a computer device. Optionally, the computing device may include a processor.
[0116] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the architecture of the thread jitter monitoring system. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. The components of the computer device can be implemented in hardware, software, or a combination of software and hardware. In some embodiments, the embodiments of this application may also be implemented using multiple separate terminal devices.
[0117] In practice, the processor can acquire data through the internal bus, input / output interface and / or communication interface.
[0118] The processor can pre-configure dynamic probes in the operating system kernel. These dynamic probes can directly obtain the thread identification information and time tracking point information of real-time threads at the kernel level. The thread identification information can identify whether the currently running real-time thread is the target thread. If it is determined to be the target thread, the jitter value of the target thread can be directly calculated through the time tracking point information collected by the dynamic probe, thereby achieving efficient and accurate jitter processing.
[0119] In the specific implementation process, the processor can also be connected to a touch screen or the screen of a terminal device to receive user commands while displaying the above content, so as to realize interaction with the user.
[0120] It should be understood that the aforementioned processor can be implemented by reading instructions from memory and executing those instructions, or it can be implemented through chip circuitry.
[0121] Optionally, Figure 1 A schematic diagram of the structure of an operating system provided in this application embodiment, such as... Figure 1 As shown, the operating system 200 provided in this embodiment includes multiple containers to be monitored (in... Figure 1 (The first container 21, the second container 22, and the third container 23 are used as examples) and kernel 24.
[0122] Understandably, the number of containers to be monitored can be determined based on the actual situation. Figure 1 This is for illustrative purposes only.
[0123] Kernel 24 establishes communication connections with multiple containers to be monitored and performs kernel-level collection and monitoring of real-time threads in the kernel space of operating system 200.
[0124] like Figure 1 As shown, the first container 21, the second container 22, and the third container 23 each run different real-time task instances, such as Ethernet for Control Automation Technology (EtherCAT) real-time tasks, industrial control tasks, or high real-time application threads. Each container is isolated from the others and runs independently, while the kernel 24, as the core of the operating system, is responsible for uniformly capturing the scheduling events, timer events, and running status information of all real-time threads within the containers.
[0125] This application supports container isolation environments. By leveraging the isolation features of containers, it enables real-time independent monitoring of threads in different container instances within a containerized environment, effectively solving the problem that traditional tools struggle to perceive fine-grained behaviors within containers.
[0126] Specifically, kernel 24 is pre-configured with dynamic probes, which can directly obtain kernel-level thread identification information and time tracking point information without deploying additional user-level agents inside the container or modifying the application code of the target thread, thereby achieving non-intrusive, highly reliable, and low-overhead real-time thread monitoring.
[0127] Specifically, in the first container 21, the kernel 24 collects process data of the first container 21, the second container 22, and the third container 23 through dynamic probes. By penetrating the isolation boundaries of each container, the kernel 24 directly collects process data of real-time threads in multiple containers (such as real-time thread running time points, scheduling switching timestamps, etc.), thereby achieving non-intrusive unified jitter monitoring of real-time threads in multiple containers.
[0128] The above architecture supports unified monitoring, efficient computation, and centralized management of real-time threads in multiple containers simultaneously, providing stable, accurate, and low-latency thread jitter monitoring capabilities for high real-time systems.
[0129] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.
[0130] Figure 2 A flowchart illustrating the thread jitter monitoring method provided in this application embodiment. Figure 1 ,like Figure 2 As shown, embodiments of this application provide a thread jitter monitoring method, which is described in detail below:
[0131] S301: Based on the preset dynamic probe running in the kernel of the operating system, obtain the thread identification information and time tracking point information of the real-time thread in the operating system.
[0132] It should be noted that preset dynamic probes refer to probe points that are pre-set after kernel startup or before monitoring starts, rather than being created in real time. The purpose is to ensure that the probes are ready before the real-time thread begins running, and can fully capture all relevant events. Specifically, thread identification information is used in subsequent steps to identify which threads belong to the target real-time task to be monitored, and time tracking point information is used for calculating jitter values and jitter handling.
[0133] In one possible implementation, before obtaining the thread identification information and time tracking point information of real-time threads in the operating system based on a preset dynamic probe running in the operating system kernel, the method further includes:
[0134] Configure a preset dynamic probe in the operating system kernel.
[0135] Among them, the preset dynamic probe is used to obtain the thread identification information and time tracking point information of real-time threads in the operating system.
[0136] In one possible implementation, during the operating system initialization phase, the kprobe dynamic tracing function is enabled, and multiple key trace points are set in the kernel. Optionally, the main process ID to be monitored and related parameters, such as the maximum allowable jitter threshold, can also be configured.
[0137] Optionally, the dynamic probe here can be a dynamic probe based on kprobe technology, a static probe based on tracepoints, or a dynamic probe based on the Extended Berkeley Packet Filter (eBPF). This application embodiment does not impose specific limitations. This application embodiment will specifically describe a dynamic probe based on kprobe technology. Using kprobe technology enables non-interfering tracing of kernel-mode behavior without modifying application code or introducing additional load, thus ensuring the original system performance and stability.
[0138] Optionally, configuring preset dynamic probes can be done by writing a kernel module, calling the kprobe registration interface in the module's initialization function, and setting a probe structure for each kernel function that needs to be monitored. Alternatively, the probe logic can be compiled directly into the kernel, and the relevant tracepoints can be enabled through kernel configuration options. This is suitable for customized kernel environments.
[0139] Optionally, the thread identification information may include at least one of the following: Parent Process Identifier (PPID), process identifier, process name, and network namespace pointer (net_ns) of the container to which it belongs.
[0140] The parent process identifier is used to identify the identity of the parent process that created the current thread, while the process identifier is used to identify the identity of the current thread itself.
[0141] In some embodiments, the parent process identifier is primarily used for filtering target threads. Specifically, the process identifier of the main process to be monitored is pre-recorded. When a scheduling switch occurs, the parent process identifier of the current thread is obtained and compared with the pre-recorded main process identifier. If they match, it indicates that the current thread is a child thread created by the main process and should be included in the monitoring scope; if they do not match, it is directly filtered without further processing.
[0142] In some embodiments, the process identifier is primarily used to continuously track and identify the target thread after the filtering process has passed. For example, after confirming that the current thread is the target thread, the monitoring module records its process identifier to quickly identify the thread in subsequent scheduling events, so as to collect scheduling timestamps, calculate jitter values, and perform other operations.
[0143] Optionally, the time tracking point information may include at least one of the following: scheduling switching information (scheduling timestamp and / or process context), timer event information (timer pointer and / or trigger timestamp), and interrupt event information (interrupt entry and / or exit timestamp).
[0144] In this embodiment, before implementing jitter monitoring, a preset dynamic probe needs to be configured in the operating system kernel for subsequent data capture. Ensuring the dynamic probe functions correctly is a prerequisite for non-intrusive monitoring. This configuration allows the entire monitoring method to run independently of user-space programs without modifying application code or container images, achieving non-intrusive jitter monitoring. It is applicable to a wide variety of operating systems, improving the practicality of jitter monitoring and ensuring the stability and security of the operating system.
[0145] Optionally, embodiments of this application utilize kprobe to implement kernel-level dynamic tracing. By leveraging the kprobe technology provided by the operating system kernel, real-time thread behavior can be dynamically traced without modifying the kernel code or affecting system stability.
[0146] Specifically, the following key trace points are set in the kernel using dynamic probes:
[0147] Interrupt entry / exit: Records the time points during the interrupt handling process, used to analyze the impact of interrupts on real-time tasks.
[0148] Timer Entry / Exit: Records the timer trigger and end times to confirm whether there is jitter caused by a timed task.
[0149] Process wake-up point: Tracks the time point at which a real-time process is woken up, used to analyze the reasons for task delays.
[0150] Process scheduling point: Records the time when a real-time process switches from the scheduling queue to the running or suspended state, and is used to calculate task execution time and scheduling delay.
[0151] S302: Determine whether the real-time thread is the target thread based on the thread identification information.
[0152] S303: If the real-time thread is determined to be the target thread, then calculate the jitter value of the target thread based on the time tracking point information of the target thread.
[0153] S304: Perform jitter processing on the target thread based on the jitter value.
[0154] In one possible implementation, jitter processing is applied to the target thread based on the jitter value, including:
[0155] If the jitter value is greater than the preset jitter threshold, it is determined that the target thread has a jitter abnormality; based on the time tracking point information, the jitter abnormality information is output.
[0156] The jitter anomaly information includes at least one of the following: the current maximum jitter error value, function call stack information, and the longest-running function call point.
[0157] It is understood that the preset jitter threshold here can be determined according to the actual situation, and the embodiments of this application do not impose any restrictions on it.
[0158] Optionally, the preset jitter threshold can be obtained from user input received from input / output ports, or it can be flexibly configured based on different application scenarios.
[0159] Alternatively, the specific method for outputting jitter exception information can be kernel log output, ring buffer reporting, or transmission by user-space programs.
[0160] Optionally, jitter abnormality information can be displayed on a preset display interface or sent to a user-preset terminal device for the user to view.
[0161] This application's embodiments specify jitter handling as threshold comparison and anomaly information output. The anomaly information includes the maximum jitter error value, function call stack, and the longest-running function call point. Real-time jitter anomaly detection is achieved, and rich diagnostic data is output. The function call stack and the longest-running point enable precise problem localization, greatly reducing the time developers spend troubleshooting the root cause of jitter and improving the maintainability of the operating system.
[0162] Optionally, embodiments of this application support the persistence and aggregation of anomaly information. Jitter anomaly information can be written to persistent storage, such as a file system or remote log service, and aggregated statistically by dimensions such as time, container identifier, or thread name to generate a jitter distribution histogram. This facilitates operations and maintenance personnel in analyzing the real-time health of the system from a macroscopic perspective and identifying periodic or sporadic jitter patterns.
[0163] Optionally, embodiments of this application support adaptive alarms based on historical thresholds. The operating system can automatically learn the statistical distribution of historical jitter values to determine historical thresholds. If the current jitter value exceeds the historical threshold, an alarm is triggered. This adapts to the inherent minor fluctuations of different tasks, avoiding overly strict or lenient fixed thresholds. This improves the stability and security of the operating system.
[0164] Optionally, embodiments of this application can achieve adaptive optimization in conjunction with scheduling strategies. For example, when jitter anomalies are detected and the longest time-consuming point is a specific function, the system can automatically recommend adjustments to scheduling parameters to the user space (such as increasing the priority of real-time tasks or reducing the granularity of shared locks).
[0165] In one possible implementation, the time tracking point information also includes interruption event information; correspondingly, after outputting jitter anomaly information based on the time tracking point information, the method further includes: outputting problem location results based on the interruption event information and the jitter anomaly information.
[0166] In one possible implementation, the time tracking point information also includes interrupt event information. This interrupt event information is independent of the aforementioned timer event information: the timer event information is used to verify the effectiveness of the scheduling in real time and determine whether jitter should be calculated; the interrupt event information is used for root cause analysis after a jitter anomaly occurs. When a jitter anomaly occurs, the system queries the interrupt event closest to the jitter occurrence time, calculates its interrupt processing time, and if the time exceeds a threshold, outputs the interrupt event as a potential cause to assist in locating the problem. For example, the output might be "IRQ 123 took 45 microseconds, possibly due to jitter."
[0167] This application's embodiments add interruption event information to the time tracking point information, and after outputting jitter anomaly information, combine the interruption information to output problem localization results. This provides interrupt-level auxiliary analysis capabilities, helping developers determine whether jitter is caused by a specific interrupt, thereby further narrowing down the problem scope and enhancing the accuracy and comprehensiveness of jitter root cause analysis.
[0168] Optionally, the problem location results include at least one of the following: interrupt type (such as hardware interrupt number or software interrupt name); total interrupt handling time (time difference from interrupt entry to exit); whether the target thread was preempted during interrupt handling; and the sub-function with the longest processing time in the interrupt handling.
[0169] In one possible implementation, when a jitter error exceeding a set threshold is detected in a real-time thread, the system triggers at least one of the following exception reporting mechanisms: outputting the current maximum jitter error value; printing the most recent function call stack information; and marking and outputting the longest-running function call point. Based on this information, the problem localization scope can be quickly narrowed down, providing developers with clear clues about the issue. Providing detailed problem diagnostic information, including the maximum error value, function call stack, and execution time, helps users quickly identify performance bottlenecks and resolve problems.
[0170] In one possible implementation, the current maximum jitter error value refers to the maximum value among all valid jitter values recorded since the start of monitoring or since the last statistical reset. This value is an actual measurement and reflects the most severe jitter of the system during the monitoring period.
[0171] Specifically, whenever a scheduling is determined to be a valid scheduling and the jitter value is calculated through the double confirmation mechanism, the jitter value is compared with the currently recorded maximum jitter error value: if the current jitter value is greater than the recorded maximum value, the current maximum jitter error value is updated to the current jitter value; if the current jitter value is less than or equal to the recorded maximum value, the current maximum jitter error value remains unchanged.
[0172] When outputting jitter abnormality information, the system can simultaneously output the current maximum jitter error value and the preset jitter threshold, so that users can intuitively understand the extent to which the actual jitter deviates from the allowable range.
[0173] Furthermore, based on the maximum time elapsed and function call stack information in the output logs, the scope of the problem can be narrowed down, and corresponding measures can be taken to optimize system performance.
[0174] This application provides a non-intrusive thread jitter monitoring method. For the operating system to be monitored, a dynamic probe is pre-configured in the operating system kernel. This dynamic probe can directly acquire kernel-level real-time thread identification information and time tracking point information. The thread identification information can identify whether the currently running real-time thread is the target thread. If it is determined to be the target thread, the jitter value of the target thread can be directly calculated through the time tracking point information collected by the dynamic probe, thereby achieving efficient and accurate jitter processing. Since the data collected for threads in this application is all implemented through the kernel's dynamic probe, kernel-level dynamic data tracking is achieved. This eliminates the need for customized modifications to user-space code or container images for different operating systems, reducing the complexity of jitter monitoring and making it applicable to jitter monitoring in multiple scenarios.
[0175] The following detailed explanation of step S302 is provided in conjunction with specific embodiments.
[0176] In one possible implementation, determining whether a real-time thread is a target thread based on thread identification information includes: obtaining a monitor identifier corresponding to the thread identification information; matching the thread identification information with the corresponding monitor identifier; and if the match is successful, determining that the real-time thread is the target thread.
[0177] This application embodiment achieves accurate identification and filtering of target threads by pre-setting a monitoring identifier and matching it with real-time thread identifier information. This mechanism enables the monitoring system to flexibly configure specific threads, processes, or containers that need to be monitored, thereby avoiding indiscriminate tracking of all threads in the system, significantly reducing the overhead of data collection, processing, and storage, improving monitoring efficiency, accurately reducing unnecessary computing power, and improving the stability of the operating system.
[0178] In one possible implementation, the thread identification information includes the parent process identifier and / or the network namespace pointer of the container to which it belongs; accordingly, obtaining the identifier to be monitored corresponding to the thread identification information includes: if the thread identification information includes the parent process identifier, then obtaining the process identifier of the main process to be monitored; if the thread identification information includes the network namespace pointer of the container to which it belongs, then obtaining the network namespace pointer of the target container to be monitored.
[0179] Here, the embodiments of this application can identify the target process through a container network namespace pointer or a parent process identifier. Specifically, by using a container network namespace pointer, all threads running within a specific container can be accurately identified and monitored, adapting to cloud-native and containerized deployment environments. This solves the problem of traditional host process identifier-based monitoring methods failing in the context of widespread container technology adoption, ensuring the accuracy of the monitoring system in containerized deployments. By using a parent process identifier, associated monitoring of a specific process and all its child threads can be achieved, significantly reducing the number of processes that need to be compared and analyzed, thereby significantly reducing the performance overhead during the monitoring process.
[0180] Optionally, if the real-time process is determined to be the target process, the thread's scheduling timestamp, timer event information, etc., are recorded for jitter calculation. If the real-time process is determined not to be the target process, the process returns directly without any data collection or processing.
[0181] Taking thread identification information including the parent process identifier as an example, this paper introduces a method for determining whether a real-time thread is a target thread. To reduce unnecessary monitoring overhead, this application proposes a process filtering mechanism based on parent-child relationships: when the main process is awakened, the parent process ID is recorded using kprobe. During subsequent monitoring of real-time threads, irrelevant threads are filtered by checking whether their parent process ID matches the main process ID. This method significantly reduces the number of processes that need to be compared and analyzed, thereby significantly reducing the performance overhead during monitoring.
[0182] In one possible implementation, when the main process is awakened, its PID is recorded and used as the basis for subsequent filtering; when a context switch or other specified event occurs, the target thread is quickly filtered out by checking the parent-child relationship; then, key indicators such as scheduling switching time, interrupt handling time, and timer event time of the target thread are collected to achieve jitter calculation.
[0183] Optionally, embodiments of this application support simultaneous monitoring of multiple main processes. For example, a list of main process identifiers can be maintained to support simultaneous monitoring of multiple independent process families, with each family performing PPID matching and jitter calculation independently.
[0184] Optionally, embodiments of this application support combined use with container network namespaces. In a multi-container environment, parent process identifier matching is combined with container network namespace pointer matching. First, all threads within the target container are filtered out using the container network namespace pointer, and then child threads of a specific main process are filtered out using the parent process identifier, achieving two-level precise filtering.
[0185] Optionally, when the PPID of the target thread is found to be the same as that of the main process during the matching process, but the process name of the thread is not as expected, a warning log can be output to indicate that there may be process name spoofing or configuration error.
[0186] Figure 3 A flowchart illustrating the thread jitter monitoring method provided in this application embodiment. Figure 2 ,like Figure 3 As shown, embodiments of this application provide a thread jitter monitoring method, which is described in detail below:
[0187] S401: Based on the preset dynamic probe running in the kernel of the operating system, obtain the thread identification information and time tracking point information of the real-time thread in the operating system.
[0188] The time tracking point information includes scheduling switching information and verification event information.
[0189] S402: Determine whether the real-time thread is the target thread based on the thread identification information.
[0190] S403: If the real-time thread is determined to be the target thread, then the scheduling switching information is checked for false alarms based on the verification event information to determine whether the scheduling switching event corresponding to the scheduling switching information is a false alarm event.
[0191] In one possible implementation, the verification event information includes timer event information; accordingly, based on the verification event information, false alarm verification is performed on the scheduling switching information to determine whether the scheduling switching event corresponding to the scheduling switching information is a false alarm event, including: based on the timer event information, determining whether there is a timer event associated with the scheduling switching event; if there is an associated timer event, determining that the scheduling switching event is not a false alarm event; if there is no associated timer event, determining that the scheduling switching event is a false alarm event.
[0192] Optionally, the verification method is to compare the scheduling switching point data with the timer trigger event data. If the data is consistent, the jitter is confirmed to be real; if they are inconsistent, it may be caused by external interference factors, and such false alarms should be eliminated.
[0193] This application further refines the verification event information into timer event information. By determining whether the scheduling switching event is associated with a timer event, a direct and efficient basis for false alarm verification is provided. Since the core characteristic of periodic threads is that they are driven by timers, using timers as the judgment benchmark for periodic scheduling, it is only considered valid when the scheduling event is associated with a timer event. This is the core implementation method of the double confirmation mechanism, which can accurately eliminate false scheduling caused by non-timer triggers, ensuring that the jitter value calculation only originates from the expected periodic task, making the false alarm filtering logic more accurate and targeted, further improving the accuracy of monitoring results, and further saving computing power.
[0194] It is understandable that associated timer events refer to timer events that are closely adjacent to the scheduling switch event in time and logically constitute a triggering relationship. Specifically, this can be achieved using a global tracking timer variable: whenever a timer is triggered, this variable is assigned a pointer to the current timer; when a scheduling event occurs, if this variable is not empty, then an association is considered to exist.
[0195] In one possible implementation, the timer event information includes the tracking timer assignment status; accordingly, based on the timer event information, determining whether there is a timer event associated with the scheduling switch event includes: obtaining the tracking timer assignment status corresponding to the scheduling switch event; if the tracking timer assignment status is valid, then determining that there is a timer event associated with the scheduling switch event; if the tracking timer assignment status is invalid, then determining that there is no timer event associated with the scheduling switch event.
[0196] Here, in this embodiment, timer event information is expressed by tracking the timer assignment status. By reading this status value, it is directly determined whether the current scheduling switch event is associated with a timer event. Specifically, when the target thread is scheduled, if the tracked timer status is valid, it is determined that there is an associated timer event; if it is invalid, it is determined that there is no association. This simplifies the complex timing association judgment into a single status variable, allowing the dynamic probe to complete the core double confirmation verification with only one status read at the scheduling point, reducing the complexity of the verification and further ensuring the stability of the operating system.
[0197] In one possible implementation, before obtaining the tracking timer assignment state corresponding to the scheduling switch event, the method further includes: when the preset dynamic probe captures a timer event and the execution interval between two scheduling events of the target thread is greater than a preset time threshold, updating the tracking timer to a valid state; when a scheduling switch occurs and the thread being switched out is the target thread, updating the tracking timer to an invalid state; and when the execution interval between two scheduling events of the target thread is less than the preset time threshold, updating the tracking timer to an invalid state.
[0198] It should be noted that the preset time threshold can be set according to the periodic characteristics of the real-time task, such as 10% of the expected periodic time or a fixed value (e.g., 100 microseconds). This threshold is used to identify short-interval scheduling that clearly does not conform to the normal periodic pattern, such as scenarios where a task voluntarily yields the CPU and is quickly awakened again.
[0199] This application specifies the update rules for the tracking timer's assignment state: it is set to valid when a timer event is triggered, invalid when the target thread is switched out of the Central Processing Unit (CPU), and invalid when the interval between two scheduling events is too short. These explicit update rules ensure that the tracking timer state accurately reflects the real-time correlation between the current scheduling event and the timer event. Invalidating the state when the target thread switches out of the CPU is used to filter interrupted or voluntarily yielding scenarios. Invalidating the state when the execution interval is less than a threshold further enhances the ability to eliminate false alarms with short intervals, thereby improving the accuracy of the double confirmation mechanism, further saving computational resources for jitter monitoring, and improving the stability and security of the operating system.
[0200] S404: If the scheduling switch event is determined to be a false alarm event, then filter the scheduling switch information.
[0201] S405: If it is determined that the scheduling switch event is not a false alarm event, then calculate the jitter value of the target thread based on the scheduling switch information.
[0202] In one possible implementation, the scheduling switching information includes a scheduling timestamp; accordingly, the jitter value of the target thread is calculated based on the scheduling switching information, including: obtaining the current scheduling timestamp and the previous scheduling timestamp of the scheduling switching event; and calculating the jitter value of the target thread based on the current scheduling timestamp, the previous scheduling timestamp, and a preset period.
[0203] Here, the difference between the actual running time and the expected running time is calculated to obtain the jitter value: based on the scheduling switch timestamp, the delay experienced by each target thread from suspension to being woken up, and the time taken from being woken up to completing the task are calculated.
[0204] The calculation formula is:
[0205] Jitter value = |Actual runtime - Expected runtime|
[0206] =|T_actual_run-T_expected_run|=
[0207] |(T_end_sched_switch-T_start_sched_switch)-T_expected_run|
[0208] Where T_actual_run represents the actual running time, T_expected_run represents the expected running time, T_end_sched_switch represents the end timestamp of the scheduling switch, and T_start_sched_switch represents the start timestamp of the scheduling switch.
[0209] This application provides a specific implementation method for calculating jitter value based on scheduling timestamp. By obtaining the precise timestamps of two adjacent valid schedulings and comparing them with a preset period time, the jitter value of each scheduling can be accurately calculated. Through a simple and accurate calculation method, accurate jitter monitoring is achieved, ensuring the stability and security of the operating system.
[0210] In one possible implementation, based on the collected data, the jitter value corresponding to each target thread is calculated by comparing the actual running time with the expected running time; and a dual confirmation mechanism of scheduling points and timers is used to eliminate false alarms caused by external factors, thereby ensuring the accuracy of jitter data.
[0211] S406: Perform jitter processing on the target thread based on the jitter value.
[0212] The implementation methods of steps S401 and S402 are the same as those of steps S301 and S302, and the implementation method of step S406 is the same as that of step S304, which will not be described in detail here.
[0213] This application embodiment can use dynamic probes to collect two types of time tracking point information, including scheduling switching information for jitter value calculation and verification event information for false alarm event verification. By verifying the event information, it is possible to determine whether the current scheduling switching event is a false alarm event, effectively identify and filter out false alarm events caused by external interference, and ensure that only real and valid periodic scheduling is used for jitter calculation, thereby improving the accuracy of monitoring results and avoiding the overhead of processing invalid events, reducing computing power costs.
[0214] This application embodiment achieves non-intrusive thread jitter monitoring based on kernel dynamic probes through the above steps. Specifically, target threads are first filtered out using thread identification information, then timer event information is used to verify false positives of scheduled events, jitter values are calculated only for non-false positive events, and finally jitter processing is performed based on the jitter values. The entire process does not require modification of user-space code or container images and can adapt to diverse real-time scenarios.
[0215] To more clearly illustrate the technical solution of this application, the following is combined with... Figure 4 The specific working process of the dual confirmation mechanism will be further explained. Figure 4 A schematic diagram of the event process of a dual confirmation mechanism provided in this application embodiment is shown below. Figure 4 As shown, it includes four typical scenarios (ideal state, interrupted by other timers, missed timer, and voluntary relinquishment).
[0216] It needs to be explained that, Figure 4 The green background boxes represent normal, valid scheduling events / tasks that need to participate in jitter calculation; these are ideal operating conditions or valid data. The red background boxes represent abnormal, disruptive scheduling events / tasks that need filtering; these are sources of false jitter.
[0217] Scenario 1 (Ideal State): The target thread is scheduled by the expected timer (timer0). When the timer is triggered, the tracking timer state is set to valid. When the target thread is scheduled, it reads the valid state, determines it as a non-false alarm, and calculates the jitter value normally. After the task is completed, it is switched off the CPU, and the state is cleared to invalid.
[0218] Ideally, the real-time thread (Ethercat) that triggers real-time task scheduling is scheduled by timer0. The normal calculation of process scheduling jitter value is (this scheduling time minus the last scheduling time) - the set period time. The tracing timer is cleared every time.
[0219] Scenario 2 (Interrupted by other timers): The target thread is preempted by a higher-priority task triggered by another timer (timer1). When the target thread is switched off the CPU, its state is cleared to invalid. When the preempted task completes and the target thread resumes scheduling, the read state is invalid, which is determined to be a false alarm event. This scheduling is filtered out, and jitter is not calculated.
[0220] If a real-time task is interrupted by another timer1 and returns to the real-time task, the time between the two schedulings is short and needs to be filtered out. The filtering condition is that if the previous process is the target process, the tracing_timer will be cleared.
[0221] In the diagram, `prev==EtherCAT` indicates that the previous process was a real-time thread (Ethercat). `tracing_timer=NULL` indicates that the tracing timer is empty (invalid).
[0222] Scenario 3 (Missed Timer): The target thread fails to respond to the first timer (timer0) trigger in time, but is scheduled when the second timer (timer1) triggers. When timer1 triggers, it updates the status to valid. When the target thread is scheduled, it reads the valid status, determines it is not a false alarm, and calculates the jitter value normally.
[0223] After two consecutive timers trigger the real-time thread (Ethercat) task, Ethercat is not scheduled in the first timer0. The tracing_timer will be recorded as timer1, but the jitter value will be calculated normally.
[0224] In the diagram, tracing_timer=T1 indicates that the previous process assigned the tracing timer a value of T1 (valid state).
[0225] Specifically, T1 here refers to Timer1. In scenario 3, tracing_timer not only records a valid state, but also records which specific timer it is, providing extended capabilities for debugging, more granular verification, and multi-task monitoring.
[0226] Scenario 4 (Voluntary Yield): The target thread voluntarily yields the CPU after being scheduled (e.g., by calling `sched_yield`). Upon yielding, its state is cleared to invalid. Subsequently, other timers trigger and set the state to valid. However, when the target thread is scheduled again, because the previous scheduling interval was too short and the state was cleared upon yielding, the system, based on the interval threshold, determines this scheduling as a false alarm (or filters it using other auxiliary conditions) and does not calculate jitter.
[0227] The real-time thread process was scheduled, but for a very short time. The real-time thread actively yielded the scheduling. At this time, the sending of periodic data was not actually executed, but the program had already completed its execution. It was triggered by timer1 for the second time, resulting in a very short interval between the two executions, less than 100us. This scenario was filtered out.
[0228] In other words, this scheme only calculates thread jitter when the target thread is scheduled, the timer trigger state is valid, and the scheduling interval meets the periodic constraint. For scheduling events caused by voluntarily yielding the CPU or abnormally short intervals, since the target thread clears the tracking timer when it switches out of the CPU, these scheduling events will not be included in the jitter calculation, thus filtering out false alarms. In addition, optionally, a time threshold mechanism can also be used for filtering.
[0229] It should be noted that, Figure 4 The bottom box is a magnified view of the timeline of Scene 4, where t0 is the timer trigger time base, real-time thread: 360us (EtherCAT: 360us) is the actual execution time of the target thread, and <500us is the preset interval threshold, which is used to intuitively show the filtering logic of invalid jitter data in this embodiment of the application when the thread execution time is too short and the interval between two scheduling is less than the threshold.
[0230] As can be seen from the above scenario analysis, the dual confirmation mechanism of this application can effectively distinguish between real periodic scheduling and various interference scheduling, ensuring the accuracy of jitter detection.
[0231] In the diagram, the scheduler (sched) represents the kernel scheduling action and is the key node that triggers the dynamic probe to collect scheduling information and executes the tracing_timer status judgment. The swapper process (swapper) represents the CPU idle state and is used to identify the context switch before and after EtherCAT task scheduling, reflecting the complete timeline of task scheduling.
[0232] Optionally, in conjunction with the above embodiments, the jitter calculation method during task scheduling is as follows: `tracing_timer` is assigned a value at the timer entry point. Then, at the `kprobe` entry point during process switching, it first determines whether the parent ID of the process is the target tracing task. Then, it compares the process name to determine if the process is the target process. If not, it returns directly. Next, it determines whether the previous process is the target process. If it is, the `tracing_timer` is cleared. If it is and the tracing timer is not empty, the jitter value is calculated as: current timestamp - previous timestamp - set period time. Accordingly, Figure 5 This is a flowchart illustrating the method for calculating jitter values provided in the embodiments of this application. Figure 5 The method shown can obtain accurate jitter data.
[0233] Optionally, Figure 6 This is a flowchart illustrating a jitter processing method provided in an embodiment of this application, as shown below. Figure 6 As shown, the process nodes of this application embodiment include:
[0234] The nodes are: "Monitoring process initialization", "Tracking container process wake-up interface" (wake_up_new_task), "Capturing real-time thread parent process identifier, container information and container core binding level information for subsequent jitter value calculation", "Calculating process jitter data", and "Outputting the time-consuming call point if jitter exceeds the maximum set value".
[0235] The node “Capture real-time thread parent process identifier, container information, and container core binding level information for subsequent jitter value calculation” is used to capture at least one of the following information after locking the target tracking task: interrupt handling entry (irq_handler_entry), interrupt handling exit (irq_handler_exit), high-precision timer expiration entry (hrtimer_expire_entry), high-precision timer expiration exit (hrtimer_expire_exit), scheduling wake-up (sched_waking), scheduling wake-up completion (sched_wakeup), and scheduling switch (sched_switch).
[0236] In this embodiment, initialization and environment configuration are performed beforehand, including: installing the real-time jitter monitoring module ko program, and passing in the user's real-time process name and the name of the monitored real-time process. Then, the target container is started: a target container running a real-time task, such as an application for industrial control or autonomous driving, is started in the experimental environment. The monitoring module will wake up the kporbe interface in the kernel to obtain the real-time main process PID of the target container and the container network information marker pointer net_ns. At this time, a monitoring node node is established for the container space.
[0237] Next, interrupt events are captured: each time an interrupt occurs, the interrupt entry and exit times are recorded to analyze the impact of interrupts on the execution time of real-time tasks.
[0238] Secondly, capture timer events: when the timer event entry and exit are triggered, record the timestamp and timer pointer information for subsequent verification with scheduling point data and for locating and analyzing problems of excessive time consumption.
[0239] Capturing scheduling switch events: When a `sched_switch` event occurs, first determine if the process belongs to the monitored container `net_ns`, and whether the parent process of the target process scheduled to enter the running state is the traced process. Compare the process name with the target process name. If both are true, record the current timestamp. If the traced timer is not empty, calculate the process jitter value: Jitter value = current recorded timestamp - previous timestamp - set loop period time. If the timer is empty, no calculation is performed.
[0240] Optionally, embodiments of this application may be configured with a kprobe tracking module, a data filtering module, a data processing module, an anomaly detection module, and a problem localization module.
[0241] The kprobe tracing module is used to set up kernel-level dynamic probes using kprobe, enabling the tracking of real-time thread behavior (such as interrupts and scheduling).
[0242] The data filtering module is used to achieve efficient filtering of monitoring targets (main process and its child threads) based on parent-child relationships.
[0243] The data processing module is used to process the collected data, including calculating jitter error and eliminating false alarms.
[0244] The anomaly detection module is used to detect whether there is jitter exceeding the threshold range and trigger subsequent problem location and log output functions.
[0245] The problem localization module is used to analyze abnormal situations. It outputs the position with the longest execution time through function call stack information to help developers quickly locate problems.
[0246] In conjunction with the above embodiments, this application supports data acquisition and analysis under conditions of multiple real-time tasks running in parallel, and can be widely applied to high-concurrency, high-real-time scenarios such as industrial control and autonomous driving. The multi-channel real-time container system jitter monitoring method based on kernel dynamic tracing technology combined with container isolation characteristics has significant advantages such as non-intrusiveness, high precision, low overhead, and multi-channel support. This gives it broad application potential in many fields requiring high real-time performance, high stability, and complex concurrent task scheduling. The following are several typical divergent application scenarios of this application's embodiments:
[0247] Industrial Automation and Industrial Control Systems: Industrial control systems typically handle highly real-time tasks, such as equipment coordination, motion control, and robot operation in production lines. These tasks are extremely sensitive to time jitter; any delay can lead to decreased production efficiency or equipment damage. The application method involves using the technology described in this application to monitor containerized tasks in industrial control systems in real time. It accurately tracks the scheduling points, timer trigger events, and execution times of each device in the production line, calculates potential jitter, and quickly identifies the root cause of problems. This improves the stability of industrial equipment operation, reduces errors caused by jitter, optimizes production line scheduling algorithms, and improves overall production efficiency.
[0248] Autonomous Driving and In-Vehicle Systems: Autonomous vehicles require real-time processing of massive amounts of sensor data (such as cameras, radar, and lidar) and rapid responses to environmental changes. Since autonomous driving systems typically operate in containerized environments, a method is needed to deeply monitor real-time tasks within the container. This application involves using the technology described in this application to monitor real-time tasks (such as path planning, obstacle detection, and emergency braking) responsible for critical decision-making and action control within the in-vehicle computing unit. It detects and analyzes jitter caused by hardware interruptions or other task interference, ensuring that real-time tasks execute according to the expected cycle. This improves the response speed of the autonomous driving system to changes in the external environment, ensuring driving safety; and reduces unnecessary delays caused by jitter, thereby optimizing vehicle performance.
[0249] 5G (Fifth Generation Mobile Communication Technology) communication networks and edge computing: 5G networks need to process a large number of data packets and require extremely low latency and high reliability. Edge computing nodes typically use containerization technology to deploy multiple services, which have complex dependencies and resource contention. The application method is to use the technology in the embodiments of this application to monitor the containerized network functions (such as virtual base station management, data packet forwarding, etc.) in edge computing nodes in real time. This detects data packet processing delays caused by resource contention or hardware interruptions and quickly locates the source of the problem. This can improve the data transmission quality of 5G networks, reduce end-to-end latency, optimize the resource allocation strategy of edge computing nodes, and improve overall network performance.
[0250] Financial trading systems are extremely sensitive to latency; even microsecond-level delays can lead to significant economic losses. Modern financial trading systems increasingly employ distributed architectures and containerized deployments, necessitating a method for high-precision jitter monitoring in distributed environments. This method involves deploying the technology described in this application to monitor the real-time threads of critical trading services (such as order matching engines). It detects thread scheduling jitter caused by resource contention or uncertainties, providing detailed diagnostic information to help optimize trading performance. This reduces trading latency, improves trading execution speed and reliability, helps quickly pinpoint performance bottlenecks, and minimizes downtime or economic losses caused by abnormal fluctuations.
[0251] Cloud computing and large-scale distributed systems: Cloud computing platforms run numerous containerized microservices, which often have complex data interactions and dependencies. Under high load, process scheduling and resource contention can easily lead to service performance degradation or even crashes. The application method is as follows: deploy the embodiments of this application on the cloud platform to achieve independent monitoring of thread behavior in each microservice instance; detect scheduling jitter in different microservice instances, as well as problems caused by context switching or timer events; output anomaly diagnostic information to optimize inter-microservice communication mechanisms and resource allocation strategies. This improves the stability of the cloud platform under high load conditions; optimizes inter-microservice collaboration efficiency; and reduces overall response time.
[0252] Aerospace and Defense: The aerospace and defense sectors involve numerous real-time control tasks, such as UAV navigation, flight control, and missile guidance. These tasks require extremely high precision and cannot tolerate any delays or errors, otherwise, serious consequences may result. The application involves using the technology described in this application to continuously monitor the core real-time threads in the flight control system; capturing scheduling points and timer trigger events during the execution of key sensor signal processing modules and navigation algorithms during flight; and providing detailed log information when anomalies occur to assist engineers in quickly troubleshooting and repairing faults. This enhances the safety and reliability of the flight control system and ensures the stable operation of aerospace and defense equipment in complex environments.
[0253] Smart Manufacturing and the Internet of Things (IoT): Smart manufacturing and IoT devices typically rely on embedded operating systems and distributed architectures to achieve data interaction and collaboration between devices. In such scenarios, efficient and accurate monitoring of equipment operating status is crucial. Its application methods include:
[0254] Deploying embodiments of this application on IoT gateways or smart manufacturing equipment allows for the collection of key thread behavior data through kernel dynamic tracing technology; jitter detection is performed on the data transmission and synchronization process between multiple devices in the IoT environment; and diagnostic information is output when anomalies are detected to optimize the operating efficiency and stability of the IoT system. This ensures that each step in the smart manufacturing process is executed as planned, thereby improving product quality and production efficiency; enhances the robustness of the IoT system in complex environments; and reduces the probability of communication failures.
[0255] Figure 6 This is a schematic diagram of a thread jitter monitoring device provided in an embodiment of this application. The thread jitter monitoring device 70 includes: an acquisition module 701, a determination module 702, a calculation module 703, and a processing module 704, wherein...
[0256] The acquisition module is used to acquire thread identification information and time tracking point information of real-time threads in the operating system based on preset dynamic probes running in the kernel of the operating system.
[0257] The determination module is used to determine whether a real-time thread is the target thread based on the thread identification information.
[0258] The calculation module is used to calculate the jitter value of the target thread based on the time tracking point information of the target thread if the real-time thread is determined to be the target thread.
[0259] The processing module is used to perform jitter processing on the target thread based on the jitter value.
[0260] In one possible implementation, the time tracking point information includes scheduling switching information and verification event information;
[0261] Accordingly, the calculation module is used for:
[0262] Based on the verification event information, false alarm verification is performed on the scheduling switching information to determine whether the scheduling switching event corresponding to the scheduling switching information is a false alarm event.
[0263] If the scheduling switch event is determined to be a false alarm, then the scheduling switch information is filtered out;
[0264] If it is determined that the scheduling switch event is not a false alarm event, then the jitter value of the target thread is calculated based on the scheduling switch information.
[0265] In one possible implementation, the verification event information includes timer event information;
[0266] Accordingly, the calculation module is used for:
[0267] Based on the timer event information, determine whether there are any timer events associated with the scheduling switch event;
[0268] If an associated timer event exists, then the scheduling switch event is determined not to be a false alarm event;
[0269] If no associated timer event exists, the scheduling switch event is determined to be a false alarm event.
[0270] In one possible implementation, the timer event information includes tracking the timer assignment status;
[0271] Accordingly, the calculation module is specifically used for:
[0272] Get the assignment status of the tracking timer corresponding to the scheduling switch event;
[0273] If the tracking timer assignment status is valid, then it is determined that there is a timer event associated with the scheduling switch event;
[0274] If the tracking timer assignment status is invalid, it is determined that there is no timer event associated with the scheduling switch event.
[0275] In one possible implementation, before obtaining the tracking timer assignment status corresponding to the scheduling switching event, the calculation module is further configured to:
[0276] When the preset dynamic probe captures a timer event and the execution interval between two schedulings of the target thread is greater than the preset time threshold, the tracking timer will be updated to a valid state.
[0277] When a scheduling switch occurs and the thread being switched out is the target thread, the tracking timer will be updated to an invalid state;
[0278] When the execution interval between two scheduled executions of the target thread is less than a preset time threshold, the tracking timer will be updated to an invalid state.
[0279] In one possible implementation, the scheduling switching information includes a scheduling timestamp;
[0280] Accordingly, the calculation module is also specifically used for:
[0281] Get the timestamp of the current scheduling and the timestamp of the previous scheduling of the scheduling switch event;
[0282] The jitter value of the target thread is calculated based on the current scheduling timestamp, the previous scheduling timestamp, and the preset period.
[0283] In one possible implementation, the thread identification information includes the parent process identifier and / or the network namespace pointer of the container to which it belongs; the determining module is specifically used for:
[0284] Obtain the monitoring identifier corresponding to the thread identifier information; where the monitoring identifier corresponding to the parent process identifier is the process identifier of the main process to be monitored, and the monitoring identifier corresponding to the network namespace pointer of the container to which it belongs is the network namespace pointer of the target container to be monitored.
[0285] Match the thread identifier information with the corresponding identifier to be monitored;
[0286] If a match is found, the real-time thread is determined to be the target thread.
[0287] In one possible implementation, the processing module is used for:
[0288] If the jitter value is greater than the preset jitter threshold, it is determined that the target thread has a jitter anomaly;
[0289] Based on the time tracking point information, output jitter anomaly information; among which, jitter anomaly information includes at least one of the current maximum jitter error value, function call stack information, and the longest time-consuming function call point.
[0290] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application.
[0291] like Figure 7 As shown, the computing device may include a processor 71 and a memory 72, wherein the processor 71 and the memory 72 are coupled and can communicate with each other; for example, the processor 71 and the memory 72 communicate through a communication bus 73, the memory 72 is used to store computer execution instructions, and the processor 71 is used to call the computer execution instructions in the memory to execute the thread jitter monitoring method shown in the above method embodiment.
[0292] Optionally, the computing device may also include a communication interface, which may include a transmitter and / or a receiver.
[0293] Optionally, the aforementioned processor can be a central processing unit (CPU), a graphics processing unit (GPU), other general-purpose processors, a digital signal processor (DSP), or an application-specific integrated circuit (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0294] This application provides a computer-readable storage medium storing computer program instructions; when the program instructions are executed by an electronic device or a computing device, the computing electronic device or computing device performs the above-described thread jitter monitoring method.
[0295] This application provides a computer program product, which includes computer program instructions. When the computer program instructions are executed, they cause a computing electronic device or computing device to perform the above-described thread jitter monitoring method.
[0296] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.
[0297] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable terminal device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0298] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0299] These computer program instructions can also be loaded onto a computer or other programmable terminal device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0300] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, the embodiments of this application are also intended to include these modifications and variations.
[0301] In the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. The term "or" and its variations can mean "and / or." In the embodiments of this application, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In the embodiments of this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0302] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the invention disclosed in the specification and in practice. The embodiments of this application are intended to cover any variations, uses, or adaptations of the embodiments of this application that follow the general principles of the embodiments of this application and include common knowledge or customary technical means in the art not disclosed in the embodiments of this application.
Claims
1. A method for detecting thread jitter, characterized in that, include: Based on a preset dynamic probe running in the kernel of the operating system, obtain the thread identification information and time tracking point information of the real-time thread in the operating system; Based on the thread identifier information, determine whether the real-time thread is the target thread; If the real-time thread is determined to be the target thread, then the jitter value of the target thread is calculated based on the time tracking point information of the target thread; The target thread is subjected to jitter processing based on the jitter value.
2. The method according to claim 1, characterized in that, The time tracking point information includes scheduling switching information and verification event information; Accordingly, calculating the jitter value of the target thread based on the time tracking point information of the target thread includes: Based on the verification event information, the scheduling switching information is checked for false alarms to determine whether the scheduling switching event corresponding to the scheduling switching information is a false alarm event. If the scheduling switch event is determined to be a false alarm event, then the scheduling switch information is filtered. If it is determined that the scheduling switch event is not a false alarm event, then the jitter value of the target thread is calculated based on the scheduling switch information.
3. The method according to claim 2, characterized in that, The verification event information includes timer event information; Accordingly, the step of performing false alarm verification on the scheduling handover information based on the verification event information to determine whether the scheduling handover event corresponding to the scheduling handover information is a false alarm event includes: Based on the timer event information, determine whether there is a timer event associated with the scheduling switch event; If an associated timer event exists, then the scheduling switch event is determined not to be a false alarm event; If no associated timer event exists, the scheduling switch event is determined to be a false alarm event.
4. The method according to claim 3, characterized in that, The timer event information includes tracking the timer assignment status; Accordingly, determining whether there is a timer event associated with the scheduling switch event based on the timer event information includes: Obtain the tracking timer assignment status corresponding to the scheduling switching event; If the tracking timer assignment status is valid, then it is determined that there is a timer event associated with the scheduling switch event; If the tracking timer assignment state is invalid, it is determined that there is no timer event associated with the scheduling switch event.
5. The method according to claim 4, characterized in that, Before obtaining the tracking timer assignment status corresponding to the scheduling switching event, the method further includes: When the preset dynamic probe captures a timer event and the execution interval between two schedulings of the target thread is greater than a preset time threshold, the tracking timer is updated to the valid state; When a scheduling switch occurs and the thread being switched out is the target thread, the tracking timer is updated to an invalid state.
6. The method according to claim 2, characterized in that, The scheduling switching information includes a scheduling timestamp; Accordingly, calculating the jitter value of the target thread based on the scheduling switching information includes: Obtain the current scheduling timestamp and the previous scheduling timestamp of the scheduling switch event; The jitter value of the target thread is calculated based on the current scheduling timestamp, the previous scheduling timestamp, and the preset period.
7. The method according to any one of claims 1 to 6, characterized in that, The thread identification information includes the parent process identifier and / or the network namespace pointer of the container to which it belongs; The step of determining whether the real-time thread is the target thread based on the thread identifier information includes: Obtain the monitoring identifier corresponding to the thread identifier information; wherein, the monitoring identifier corresponding to the parent process identifier is the process identifier of the main process to be monitored, and the monitoring identifier corresponding to the network namespace pointer of the container to which it belongs is the network namespace pointer of the target container to be monitored; The thread identifier information is matched with the corresponding identifier to be monitored; If a match is found, the real-time thread is determined to be the target thread.
8. The method according to any one of claims 1 to 6, characterized in that, The step of jittering the target thread based on the jitter value includes: If the jitter value is greater than the preset jitter threshold, then the target thread is determined to have experienced a jitter anomaly. Based on the time tracking point information, jitter anomaly information is output; wherein, the jitter anomaly information includes at least one of the current maximum jitter error value, function call stack information, and the longest time-consuming function call point.
9. A computing device, characterized in that, The computing device includes a memory and a processor; The memory and the processor are coupled; The memory is used to store program instructions; The processor is used to invoke the program instructions to cause the computing device to perform the method as described in any one of claims 1-8.
10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.