A Linux-based real-time scheduling system for HiFi audio playback

Abstract

This invention relates to the fields of computer operating systems and audio processing technology, specifically to a real-time scheduling system for Hi-Fi audio playback based on Linux. The system includes a real-time core that handles only real-time tasks, a buffer management module that maintains a receive buffer and a playback buffer, a multi-core task allocation module that distributes real-time audio tasks to multiple real-time cores, an RTOS preemptive task scheduling module that preempts real-time cores, a decoding module that performs decoding tasks based on data in the receive buffer, and a playback module that performs playback tasks based on data in the playback buffer. This invention, through its unique core isolation architecture, preemptive scheduling mechanism, and dual-buffer design, ensures timely response to real-time audio tasks, solving the problem that existing Linux-based audio systems struggle to meet the real-time requirements of Hi-Fi audio.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer operating systems and audio processing, and particularly relates to a real-time scheduling system for Linux-based HIFI audio playback. BACKGROUND

[0002] Audio solutions based on Linux operating systems are widely used in consumer and professional HIFI audio systems due to their open source nature and strong customizability. Typical application scenarios include high-fidelity music playback, lossless audio decoding, and multi-channel surround sound processing.

[0003] However, in the field of HIFI audio, high-quality audio playback has very high real-time requirements for the system. However, the general scheduling strategy of the traditional Linux kernel (such as the Completely Fair Scheduler, CFS) focuses more on the overall throughput and fairness of the system in design, rather than optimizing for real-time tasks. This leads to unpredictable delays and jitter in the collection, decoding, and output of audio data during HIFI audio playback, and in severe cases, even causes audio discontinuity or distortion, making it difficult to meet the stringent requirements of high-end HIFI devices for millisecond-level response accuracy and long-term stable operation.

[0004] Therefore, there is a need for a scheduling solution optimized for Linux and HIFI to address the deficiencies of existing Linux-based audio systems in real-time scheduling, which makes it difficult to meet the real-time requirements of HIFI audio. SUMMARY

[0005] The present application relates to the technical field of computer operating systems and audio processing, and particularly relates to a real-time scheduling system for Linux-based HIFI audio playback.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A real-time scheduling system for Linux-based HIFI audio playback, including a real-time core that only processes real-time tasks, an audio processing layer, and a real-time scheduling layer. The audio processing layer includes a buffer management module, a decoding module, and a playback module. The real-time scheduling layer includes an RTOS preemption task scheduling module and a multi-core task allocation module, wherein:

[0008] The buffer management module is used to maintain a receiving buffer and a playback buffer, wherein the receiving buffer is used to store raw audio data, and the playback buffer is used to store audio data to be played back;

[0009] The multi-core task allocation module is configured to obtain audio real-time tasks, and allocate the audio real-time tasks to a plurality of real-time cores, wherein the audio real-time tasks comprise decoding tasks and playing tasks;

[0010] The RTOS preemption task scheduling module is further configured to preempt the real-time cores, and perform the decoding tasks based on data in the receiving buffer by the decoding module;

[0011] The RTOS preemption task scheduling module is further configured to preempt the real-time cores, and perform the playing tasks based on data in the playing buffer by the playing module.

[0012] As a further scheme of the present application, the receiving buffer and the playing buffer are maintained, comprising:

[0013] The audio stream parameters, delay constraints and system load of audio data in the current receiving buffer and the playing buffer are obtained, wherein the delay constraints represent a maximum allowed end-to-end delay customized by a user;

[0014] The receiving buffer theoretical minimum value and the playing buffer safety margin value are obtained according to the audio parameters and the delay constraints;

[0015] The adjustment coefficient is obtained according to the system load;

[0016] The receiving buffer theoretical minimum value and the playing buffer safety margin value are adjusted according to the adjustment coefficient to obtain the capacity of the receiving buffer and the playing buffer.

[0017] As a further scheme of the present application, the audio real-time tasks are obtained, and the audio real-time tasks are allocated to a plurality of real-time cores, comprising:

[0018] The audio real-time tasks are obtained;

[0019] The user behavior characteristics are obtained, and the user concentration is obtained according to the user behavior characteristics;

[0020] The complexity index of the audio data is obtained according to the audio data corresponding to the audio real-time tasks;

[0021] The running parameters of each real-time core are obtained, and the comprehensive load index of each real-time core is obtained according to the running parameters;

[0022] The audio real-time tasks are allocated to the plurality of real-time cores based on the user concentration, the complexity index and the comprehensive load index.

[0023] As a further scheme of the present application, the complexity index of the audio data is obtained according to the audio data corresponding to the audio real-time tasks, comprising:

[0024] The spectral entropy value of the audio data at each frequency is counted;

[0025] The spectral entropy value of the audio data at each frequency is normalized, and the normalized values ​​are squared and summed to obtain the complexity index.

[0026] As a further aspect of the present invention: obtaining the operating parameters of each real-time core, and obtaining the comprehensive load index of each real-time core based on the operating parameters, including:

[0027] Obtain real-time core instruction cycle utilization, cache miss count, context switch count, and core temperature over a preset time period.

[0028] The overall load index is obtained based on instruction cycle utilization, cache misses, context switches, and core temperature.

[0029] As a further aspect of this invention: based on user focus, complexity index, and overall load index, real-time audio tasks are allocated to multiple real-time cores, including:

[0030] If the user focus index indicates that the user is focused and the complexity index exceeds the preset threshold, then a real-time core will be designated as an exclusive core that only processes real-time audio tasks, and the real-time audio tasks will be assigned to that core.

[0031] If the user focus index indicates that the user is focused and the complexity index does not exceed the preset threshold, then all real-time audio tasks are assigned to multiple real-time cores based on the comprehensive load index.

[0032] If user focus indicates a user's unfocused state and the complexity index exceeds a preset threshold, then designate a real-time core as the exclusive core for processing real-time audio tasks, assign the real-time audio tasks to that core, and allow other low-priority real-time tasks to be assigned to that core.

[0033] If user focus indicates a user's unfocused state and the complexity index does not exceed a preset threshold, then based on the comprehensive load index, audio real-time tasks and other real-time tasks are mixed and allocated to multiple real-time cores according to a preset ratio.

[0034] As a further aspect of the present invention, it also includes an RTOS initialization module, used for:

[0035] When the Linux system boots, the RTOS kernel is built in a pre-defined contiguous region of memory;

[0036] Establish a communication channel between the RTOS kernel and the Linux kernel;

[0037] Create a dedicated API that encapsulates the underlying task control operations of the RTOS. This dedicated API is used to create real-time tasks, with audio real-time tasks having the highest priority.

[0038] As a further aspect of the present invention: it also includes an application layer, a kernel layer, and a hardware abstraction layer; the real-time scheduling layer further includes a task synchronization module; the playback module includes an audio equalization module and a sampling rate conversion module, wherein:

[0039] The application layer is used to parse user requests and store audio source data into the receive buffer;

[0040] The kernel layer provides system call and interface support for the application layer, audio processing layer, and real-time scheduling layer;

[0041] The hardware abstraction layer is used to provide a unified hardware access interface;

[0042] The task synchronization module is used to synchronize tasks from different real-time cores;

[0043] The audio equalizer module is used to adjust the frequency response of audio data;

[0044] The sampling rate conversion module is used to convert the sampling rate of audio data.

[0045] The beneficial effects of this invention are:

[0046] This invention provides a real-time scheduling system for HiFi audio playback based on Linux. It includes a real-time core that handles only real-time tasks, a buffer management module that maintains a receive buffer and a playback buffer, a multi-core task allocation module that distributes real-time audio tasks to multiple real-time cores, an RTOS preemptive task scheduling module that preempts real-time cores, a decoding module that performs decoding tasks based on data in the receive buffer, and a playback module that performs playback tasks based on data in the playback buffer. The core advantage of this invention lies in its unique core isolation architecture, preemptive scheduling mechanism, and dual-buffer design, which significantly improves the real-time performance and stability of audio processing. The system physically isolates real-time tasks from non-real-time tasks by setting up real-time cores, utilizing dedicated real-time cores to handle high-priority audio tasks. The RTOS preemptive scheduling mechanism ensures timely response to real-time audio tasks, and the multi-core task allocation module fully utilizes the parallel computing capabilities of multi-core processors, avoiding performance bottlenecks caused by resource contention. Furthermore, the dual-buffer design effectively suppresses the impact of audio stream fluctuations on playback continuity, realizing an audio processing pipeline optimized for HiFi, reducing sound quality loss and playback anomalies, thus maintaining high-fidelity sound quality and low jitter output even in complex multi-tasking scenarios. This solves the problem that existing Linux-based audio systems are insufficient in real-time scheduling and cannot meet the real-time requirements of HiFi audio. Attached Figure Description

[0047] The invention will now be further described with reference to the accompanying drawings.

[0048] Figure 1 This is a system architecture diagram of the real-time scheduling system for HIFI audio playback based on Linux, as described in this invention.

[0049] Figure 2 yes Figure 1 A diagram showing the specific steps performed by the buffer management module.

[0050] Figure 3 yes Figure 1 A diagram showing the specific steps executed by the multi-core task allocation module. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 As shown, this invention is a real-time scheduling system for HiFi audio playback based on Linux, comprising a real-time core that handles only real-time tasks, an audio processing layer, and a real-time scheduling layer. The audio processing layer includes a buffer management module, a decoding module, and a playback module. The real-time scheduling layer includes an RTOS preemptive task scheduling module and a multi-core task allocation module, wherein:

[0053] The buffer management module is used to maintain the receive buffer and the playback buffer, where the receive buffer is used to store the raw audio data and the playback buffer is used to store the audio data to be played.

[0054] The multi-core task allocation module is used to acquire real-time audio tasks and allocate them to multiple real-time cores. The real-time audio tasks include decoding tasks and playback tasks.

[0055] The RTOS preemptive task scheduling module is also used to preempt the real-time core and execute decoding tasks based on the data in the receive buffer through the decoding module;

[0056] The RTOS preemptive task scheduling module is also used to preempt the real-time core and execute playback tasks based on the data in the playback buffer through the playback module.

[0057] It is understood that the above modules can run in parallel; therefore, the order of processes described above does not represent the actual operating order of this system. Except for the real-time core, all of the above modules can be implemented using computer program products on existing hardware.

[0058] In this invention, the system divides each core of the processor into independent task processing units, with different types of tasks assigned to different cores for execution. For example, real-time audio processing tasks are assigned to cores 0 and 1, which only handle real-time tasks and do not participate in the scheduling of non-real-time tasks, thus avoiding interference between non-real-time and real-time tasks. Other non-real-time tasks are assigned to cores 2 and 3, achieving physical isolation between real-time and non-real-time tasks. Specifically, this invention achieves more efficient multi-core utilization through core binding and dynamic migration, improving processing efficiency and better handling high-load audio processing.

[0059] The receive buffer stores the raw audio data received from the audio source, while the playback buffer stores the audio data to be played (such as data after decoding and processing). This dual-buffer mechanism, combined with high-precision clock synchronization and other technologies, enables a Hi-Fi optimized audio processing pipeline. Compared to general audio systems, this invention's dual-buffer mechanism prioritizes sound quality and real-time performance in audio processing, while dynamic buffer adjustment and clock synchronization mechanisms reduce sound quality loss and playback anomalies.

[0060] The decoding module is used to decode audio data, supporting various HiFi audio formats such as FLAC, DSD, WAV, and ALAC. For each format, the module integrates and optimizes the corresponding decoding library to improve decoding efficiency. For example, for DSD format, a native decoding method is used to avoid audio quality loss during transcoding; for high-bitrate FLAC files, multi-core parallel decoding is utilized to improve decoding speed. The playback module can be implemented using any existing technology capable of playing audio data.

[0061] The RTOS preemptive task scheduling module utilizes the core scheduling mechanism of an RTOS (such as FreeRTOS) and works in conjunction with the Linux kernel scheduler. The system categorizes tasks into real-time and non-real-time tasks. Tasks with high real-time requirements, such as audio decoding and audio output, are marked as real-time tasks, while other tasks are non-real-time tasks. The RTOS preemptive task scheduling module has a higher priority than the Linux kernel scheduler. When a real-time task becomes ready, the RTOS scheduler immediately preempts the Linux kernel, taking over processor resources and scheduling tasks according to their priority and deadline. After a real-time task completes, processor resources are returned to the Linux kernel, and non-real-time tasks continue execution.

[0062] RTOS is an existing technology. However, existing Linux systems that integrate RTOS, such as RT-Linux and Xenomai, while having strong real-time performance, have high configuration and development thresholds. Developers need to be familiar with the underlying principles of real-time kernels and Linux kernels, and perform complex kernel trimming, compilation and debugging. This is not conducive to the rapid construction and deployment of HiFi audio systems. Users need to do a lot of secondary development themselves, and there is a lack of special optimization for HiFi audio scenarios. The utilization efficiency of multi-core processors is also not high.

[0063] The present invention implements the RTOS preemptive task scheduling module through the following steps:

[0064] In one embodiment, the Linux-based real-time scheduling system for HiFi audio playback further includes an RTOS initialization module, which specifically performs the following steps:

[0065] When the Linux system boots, the RTOS kernel is built in a pre-defined contiguous region of memory;

[0066] Establish a communication channel between the RTOS kernel and the Linux kernel;

[0067] Create a dedicated API that encapsulates the underlying task control operations of the RTOS. This dedicated API is used to create real-time tasks, with audio real-time tasks having the highest priority.

[0068] In this embodiment, the system loads a simplified RTOS kernel (such as FreeRTOS) into a pre-defined contiguous region of memory during the startup phase. A communication bridge between the two kernels is established through shared memory and interrupt channels, avoiding the complex low-level trimming and compilation processes of traditional dual-kernel solutions. By encapsulating low-level RTOS task control operations (such as task creation, priority setting, and context switching), this invention encapsulates real-time tasks such as audio decoding and output into independent modules, assigning them a higher priority than the Linux kernel scheduler.

[0069] A pseudocode example for a dedicated API is as follows:

[0070] rt_task_t *audio_task = rt_create_task(

[0071] "audio_output", / / Task name

[0072] AUDIO_TASK_PRIORITY, / / Highest priority

[0073] audio_output_entry, / / Task entry function

[0074] AUDIO_STACK_SIZE, / / Stack size (bytes)

[0075] AUDIO_TASK_PERIOD / / Periodic task period (ms) );

[0077] The rt_create_task() function encapsulates the task parameters into an RTOS Task Control Block (TCB), which is then passed to the RTOS kernel via a shared memory queue. The RTOS then performs stack allocation and context initialization.

[0078] When audio data arrives at the receive or playback buffer, the RTOS scheduler immediately preempts Linux kernel resources and dynamically allocates core resources based on priority, ensuring low end-to-end latency for highly complex audio streams. Most importantly, this invention eliminates the need for user intervention in low-level kernel debugging, resulting in a simpler mechanism. Furthermore, it features priority optimization for Hi-Fi audio tasks, leading to faster response times.

[0079] In addition, combined Figure 1 As shown, in one embodiment, the Linux-based real-time scheduling system for HiFi audio playback in this invention further includes an application layer, a kernel layer, and a hardware abstraction layer; the real-time scheduling layer further includes a task synchronization module; the playback module includes an audio equalization module and a sampling rate conversion module, wherein:

[0080] The application layer is used to parse user requests and store audio source data into the receive buffer;

[0081] The kernel layer provides system call and interface support for the application layer, audio processing layer, and real-time scheduling layer;

[0082] The hardware abstraction layer is used to provide a unified hardware access interface;

[0083] The task synchronization module is used to synchronize tasks from different real-time cores;

[0084] The audio equalizer module is used to adjust the frequency response of audio data;

[0085] The sampling rate conversion module is used to convert the sampling rate of audio data.

[0086] Specifically, the application layer includes modules such as a user interface, audio source management, and playback control. The user interface provides a graphical user interface, allowing users to perform operations such as system settings, audio source selection, and playback parameter adjustment; the audio source management module is responsible for managing various audio sources such as local audio files and network streams; and the playback control module implements basic control functions such as playing, pausing, stopping, fast forwarding, and rewinding audio.

[0087] The kernel layer is custom-developed based on the Arch Linux kernel, integrating real-time patching and optimized configurations. It provides basic functions such as process management, memory management, and device drivers, and offers necessary system calls and interface support to the upper real-time scheduling layer.

[0088] The Hardware Abstraction Layer (HAL) is responsible for shielding the differences between various hardware devices and providing a unified hardware access interface, including audio interfaces, processors, memory, and network interfaces. For different Hi-Fi hardware devices, such as sound cards and DACs (digital-to-analog converters), the HAL provides dedicated drivers and configuration files to ensure efficient communication between the system and the hardware devices.

[0089] The task synchronization module is responsible for coordinating data interaction and synchronization between tasks on different cores to avoid data conflicts and inconsistencies.

[0090] The audio equalizer module allows users to adjust the frequency response of audio according to personal preferences and the characteristics of their audio equipment. The sample rate conversion module can convert audio data with different sample rates into a uniform sample rate, ensuring consistent audio playback.

[0091] Furthermore, in a preferred embodiment, the size of the buffer can be dynamically adjusted according to the audio sampling rate, number of channels, and latency requirements to ensure a continuous supply of audio data and avoid buffer overflow or underflow.

[0092] Combination Figure 2 As shown, in this embodiment, the buffer management module maintains the receive buffer and playback buffer using the following specific steps:

[0093] S201. Obtain the audio stream parameters, delay constraints, and system load of the audio data in the current receive buffer and playback buffer, where the delay constraints represent the user-defined maximum allowable end-to-end delay.

[0094] S202. Based on the audio parameters and delay constraints, obtain the theoretical minimum value of the receiving buffer and the safe margin value of the playback buffer;

[0095] S203. Obtain the adjustment coefficient based on the system load;

[0096] S204. Adjust the theoretical minimum value of the receiving buffer and the safety margin value of the playback buffer according to the adjustment coefficient to obtain the capacity of the receiving buffer and the playback buffer.

[0097] The dynamic buffer adjustment scheme proposed in this invention achieves an optimal balance between continuous audio data supply and system resource consumption by sensing the characteristics of the audio stream and the system status in real time. First, by dynamically acquiring key parameters such as audio sampling rate and number of channels, and combining them with user-defined end-to-end latency constraints (e.g., 5ms or lower), the system can accurately calculate the theoretical minimum requirements for the receiving and playback buffers, ensuring a stable data stream even in high bitrate (e.g., 24bit / 192kHz) or complex multi-channel (e.g., 5.1 surround sound) scenarios. Second, a system load sensing mechanism is introduced. By monitoring key indicators such as CPU utilization and memory bandwidth pressure in real time, the buffer capacity is dynamically adjusted—when the system load is high, the buffer is increased (up to 150% of the theoretical value) to effectively suppress audio interruptions or stuttering caused by resource contention; while when the load is low, the buffer is appropriately reduced to decrease memory usage and latency. In particular, this embodiment maximizes the utilization of buffer resources while ensuring absolute real-time performance through a two-layer design of "theoretical minimum value + dynamic adjustment coefficient". Compared with the traditional fixed-size buffer scheme, it eliminates the risk of underflow (playback interruption) and avoids overflow waste (inefficient memory use). At the same time, through personalized configuration of latency constraints, it perfectly adapts to different application scenarios from professional recording studios (ultra-low latency requirements) to home HiFi (comfortable latency range), significantly improving the robustness of the system in complex environments and the consistency of user experience.

[0098] The present invention also provides a more preferred embodiment to illustrate the above process more clearly:

[0099] The system collects the following key parameters in real time:

[0100] Audio stream parameters: current audio sampling rate (S), number of channels (C), bit depth (B), which can be taken as the average value;

[0101] System load: Average CPU utilization (CPU_load);

[0102] Latency constraint: The maximum allowable end-to-end latency set by the user (Latency_max, in milliseconds).

[0103] Calculation of the theoretical minimum value of the receive buffer (min_receive):

[0104] min_receive=(S×C×B / 8)×(Latency_max / 1000);

[0105] Calculation of the playback buffer safety margin (min_play):

[0106] min_play=(S×C×B / 8)×Safety_Margin;

[0107] Safety_Margin is a preset fixed safety factor, such as 50ms.

[0108] Calculation of adjustment factor:

[0109] First, calculate the load correction factor (load_factor):

[0110] load_factor=1+α×(CPU_load-Target_Load);

[0111] Where α is the preset adjustment sensitivity coefficient, and Target_Load is the ideal CPU load threshold, such as 60%.

[0112] Then, the final adjustment factor (adjust_ratio) is calculated:

[0113] adjust_ratio=max(load_factor,Min_Ratio);

[0114] Min_Ratio is the minimum adjustment limit, such as 0.5.

[0115] Finally, update the buffer capacity:

[0116] The new capacity of the receive buffer (new_receive) is:

[0117] new_receive=min_receive×adjust_ratio;

[0118] The new size of the playback buffer (new_play) is:

[0119] new_play=min_play×adjust_ratio.

[0120] Furthermore, in a preferred embodiment, the Linux-based real-time scheduling system for HiFi audio playback also includes a load balancing mechanism. This mechanism monitors the task load of each core in real time, and when the load on a core is too high, some tasks are migrated to cores of the same type with lower loads. For example, if the real-time task load on core 0 is too high, some real-time tasks can be migrated to core 1. Task migration is implemented through the kernel's process migration interface, ensuring data consistency and task continuity during the migration process.

[0121] Specifically, combined Figure 3As shown, the specific steps of the multi-core task allocation module in acquiring real-time audio tasks and allocating them to multiple real-time cores include:

[0122] S301, Obtain real-time audio tasks;

[0123] S302. Obtain user behavior characteristics and determine user focus based on these characteristics;

[0124] S303. Obtain the complexity index of the audio data based on the audio data corresponding to the real-time audio task;

[0125] S304. Obtain the operating parameters of each real-time core, and obtain the comprehensive load index of each real-time core based on the operating parameters;

[0126] S305 distributes real-time audio tasks to multiple real-time cores based on user focus, complexity index, and overall load index.

[0127] This embodiment constructs a comprehensive load index by real-time monitoring of the operating parameters of each physical core (such as CPU utilization, cache hit rate, and temperature), accurately identifying high-load cores. Then, it utilizes the Linux kernel's native process migration interface to achieve seamless task transfer, ensuring data consistency and timing continuity for critical tasks such as audio decoding and output during the migration process. This avoids audio stuttering or latency spikes caused by core overload in traditional solutions. Furthermore, considering the complexity index of the audio data itself, it automatically increases core resource allocation for content with high transient changes (such as densely packed drum beats) or high sampling rates, ensuring that the deadlines for real-time decoding and output tasks are strictly met.

[0128] It's worth noting that, unlike other operations, a user's experience listening to music is closely related to their attention span. When a user is fully focused on listening to music, they can notice more sonic details and are more sensitive to the system's audio data processing. Conversely, when a user is not focused, such as while working, they are less likely to notice sound flaws. Therefore, this invention innovatively introduces a user behavior analysis module. By monitoring various factors such as the type of front-end application (e.g., a music player focused on or a browser running in the background), the frequency of input events (e.g., keyboard / touch operation intervals), and the screen interaction area, the system quantifies user focus (e.g., focus mode = 10, multitasking mode = 1). When a user is fully focused on listening to complex music, the system prioritizes real-time tasks on low-noise, high-performance dedicated cores (e.g., Core0 / Core1) and dynamically increases their resource allocation. In multitasking scenarios (e.g., listening to music while processing documents), the system intelligently reduces the priority of audio tasks and rationally allocates flexible core resources, balancing efficiency and user experience.

[0129] In summary, this embodiment, through the three-dimensional collaboration of "load perception - user intent recognition - content feature analysis", not only solves the performance bottleneck caused by resource contention between real-time and non-real-time tasks in traditional multi-core systems, but also greatly improves the overall utilization of multi-core processors while ensuring HiFi sound quality and low latency.

[0130] Specifically, in one embodiment, step S303, obtaining the complexity index of the audio data based on the audio data corresponding to the real-time audio task, specifically includes:

[0131] Statistical analysis of the spectral entropy value of audio data at each frequency;

[0132] The spectral entropy value of the audio data at each frequency is normalized, and the normalized values ​​are squared and summed to obtain the complexity index.

[0133] This embodiment accurately captures the degree of energy distribution disorder caused by transient changes (such as drum beats and string glissando) or complex textures (such as the interweaving of multiple voices in a symphony) in music by calculating the spectral entropy value of the audio signal at each frequency component in real time. The higher the spectral entropy value, the more complex the audio content in that frequency band. Furthermore, the spectral entropy is normalized in the frequency dimension and summed by squares. This balances the characteristic that the human ear is more sensitive to mid-high frequencies (suppressing low-frequency interference through normalization) and amplifies the complexity differences of key frequency bands (strengthening the weight of high-frequency transients through square operations). The resulting exponent can intuitively reflect the overall real-time processing difficulty of the audio.

[0134] To illustrate the above process more clearly, the present invention also provides a more specific embodiment:

[0135] The complexity index is calculated using the following formula:

[0136] ;

[0137] In this implementation, DCI stands for Complexity Index, f represents the frequency variable spanning the range of human hearing, and SpectralEntropy(f) represents the spectral entropy value of the audio signal at frequency f, which measures the degree of disorder in the energy distribution of the audio spectrum. A higher spectral entropy indicates more complex and drastic frequency components in that frequency band (e.g., transient percussion, complex chords); conversely, a lower spectral entropy indicates simpler audio in that frequency band (e.g., pure tones, stable beats). Because the human ear is more sensitive to mid-to-high frequencies, small changes in the high-frequency band have a greater impact on hearing. Therefore, this embodiment divides the spectral entropy value by the frequency f to balance the weights of different frequency bands. Simultaneously, this embodiment also amplifies the differences through squaring operations, enhancing the distinction between high-frequency and low-frequency bands and ensuring that the high-complexity frequency band dominates the total.

[0138] Furthermore, in a preferred embodiment, step S304, obtaining the operating parameters of each real-time core and obtaining the comprehensive load index of each real-time core based on the operating parameters, specifically includes:

[0139] Obtain real-time core instruction cycle utilization, cache miss count, context switch count, and core temperature over a preset time period.

[0140] The overall load index is obtained based on instruction cycle utilization, cache misses, context switches, and core temperature.

[0141] The instruction cycle utilization over a preset time period can be interpreted as the instruction cycle utilization over the past 100ms, reflecting the actual proportion of CPU computing resources used. The cache miss count refers to the number of L3 cache misses, reflecting memory access efficiency. A surge in L3 miss rate may indicate decreased access efficiency, leading to a sharp drop in performance for audio tasks. The context switch count measures task scheduling overhead. Core temperature characterizes the hardware thermal load status.

[0142] This embodiment synchronously collects key operating parameters of the real-time cores over a preset time period. These heterogeneous data are then fused into a single value using weighted summation or other methods, accurately quantifying the true load pressure on each core. In actual scheduling, the system can dynamically identify "hidden overload" cores (such as cores with only 70% CPU utilization but excessively high cache miss rates or temperatures approaching thresholds) based on the comprehensive load index. It intelligently avoids assigning real-time tasks such as audio decoding and output to these cores, instead prioritizing cores with balanced overall load. This ensures that high-priority audio tasks always run on hardware with ample computing resources and stable response latency. This mechanism not only solves the problem of misjudging resources that appear idle but are actually strained in multi-core systems, but also improves the overall utilization of multi-core processors while ensuring low-latency, high-quality HiFi audio output.

[0143] After obtaining user focus, complexity index, and overall load index, core allocation can be performed using any strategy based on the actual situation. This invention provides a preferred approach:

[0144] In this embodiment, step S305, which assigns real-time audio tasks to multiple real-time cores based on user focus, complexity index, and comprehensive load index, specifically includes:

[0145] If the user focus index indicates that the user is focused and the complexity index exceeds the preset threshold, then a real-time core will be designated as an exclusive core that only processes real-time audio tasks, and the real-time audio tasks will be assigned to that core.

[0146] If the user focus index indicates that the user is focused and the complexity index does not exceed the preset threshold, then all real-time audio tasks are assigned to multiple real-time cores based on the comprehensive load index.

[0147] If user focus indicates a user's unfocused state and the complexity index exceeds a preset threshold, then designate a real-time core as the exclusive core for processing real-time audio tasks, assign the real-time audio tasks to that core, and allow other low-priority real-time tasks to be assigned to that core.

[0148] If user focus indicates a user's unfocused state and the complexity index does not exceed a preset threshold, then based on the comprehensive load index, audio real-time tasks and other real-time tasks are mixed and allocated to multiple real-time cores according to a preset ratio.

[0149] In this embodiment, when the user is focused and the audio complexity is extremely high, the system automatically designates a single real-time core (such as Core0) as the "dedicated core" for audio tasks, completely isolating non-real-time task interference and ensuring that the deadlines for decoding and output tasks are strictly met, while maintaining the lowest latency and highest sound quality. For regular audio in a focused state, tasks are dynamically allocated based on the comprehensive load index of each core, making full use of multi-core parallel capabilities to improve efficiency. When the user is not focused (such as listening to music while working) and encounters highly complex audio, the system still retains a dedicated core to ensure audio real-time performance, but allows low-priority tasks (such as logging) to share the core resources, balancing performance and functional requirements. In non-focused and low-complexity scenarios (such as podcasts or light music), tasks are mixed and allocated to multiple cores according to a preset ratio (such as 50% for real-time audio tasks and 50% for other real-time tasks), maximizing hardware utilization. This embodiment perfectly adapts to diverse needs from professional recording studios to home HiFi scenarios, truly realizing end-to-end resource management of intelligent perception, precise decision-making, and dynamic optimization.

[0150] This invention provides a real-time scheduling system for HiFi audio playback based on Linux. It includes a real-time core that handles only real-time tasks, a buffer management module that maintains a receive buffer and a playback buffer, a multi-core task allocation module that distributes real-time audio tasks to multiple real-time cores, an RTOS preemptive task scheduling module that preempts real-time cores, a decoding module that performs decoding tasks based on data in the receive buffer, and a playback module that performs playback tasks based on data in the playback buffer. The core advantage of this invention lies in its unique core isolation architecture, preemptive scheduling mechanism, and dual-buffer design, which significantly improves the real-time performance and stability of audio processing. The system physically isolates real-time tasks from non-real-time tasks by setting up real-time cores, utilizing dedicated real-time cores to handle high-priority audio tasks. The RTOS preemptive scheduling mechanism ensures timely response to real-time audio tasks, and the multi-core task allocation module fully utilizes the parallel computing capabilities of multi-core processors, avoiding performance bottlenecks caused by resource contention. Furthermore, the dual-buffer design effectively suppresses the impact of audio stream fluctuations on playback continuity, realizing an audio processing pipeline optimized for HiFi, reducing sound quality loss and playback anomalies, thus maintaining high-fidelity sound quality and low jitter output even in complex multi-tasking scenarios. This solves the problem that existing Linux-based audio systems are insufficient in real-time scheduling and cannot meet the real-time requirements of HiFi audio.

[0151] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A real-time scheduling system for Linux-based HIFI audio playback, characterized in that, The real-time core only processes real-time tasks, the audio processing layer includes a buffer management module, a decoding module and a playing module, and the real-time scheduling layer includes an RTOS preemption task scheduling module and a multi-core task allocation module, wherein: The buffer management module is used for maintaining a receiving buffer and a playing buffer, wherein the receiving buffer is used for storing original audio data, and the playing buffer is used for storing audio data to be played; The multi-core task allocation module is used for obtaining audio real-time tasks and allocating the audio real-time tasks to the plurality of real-time cores, wherein the audio real-time tasks include decoding tasks and playing tasks; The RTOS preemption task scheduling module is also used for preempting the real-time cores and performing the decoding tasks based on data in the receiving buffer through the decoding module; The RTOS preemption task scheduling module is also used for preempting the real-time cores and performing the playing tasks based on data in the playing buffer through the playing module; The method for obtaining the audio real-time tasks and allocating the audio real-time tasks to the plurality of real-time cores comprises: obtaining the audio real-time tasks; obtaining a user behavior feature and obtaining a user concentration degree according to the user behavior feature; obtaining a complexity index of audio data according to the audio data corresponding to the audio real-time tasks; obtaining a comprehensive load index of each real-time core according to running parameters of each real-time core; allocating the audio real-time tasks to the plurality of real-time cores based on the user concentration degree, the complexity index and the comprehensive load index; The method for allocating the audio real-time tasks to the plurality of real-time cores based on the user concentration degree, the complexity index and the comprehensive load index comprises: if the user concentration degree represents that the user is in a concentrated state and the complexity index exceeds a preset threshold, designating one real-time core as an exclusive core for processing only the audio real-time tasks and allocating the audio real-time tasks to the core; if the user concentration degree represents that the user is in a concentrated state and the complexity index does not exceed the preset threshold, allocating all the audio real-time tasks to the plurality of real-time cores based on the comprehensive load index; if the user concentration degree represents that the user is not in a concentrated state and the complexity index exceeds the preset threshold, designating one real-time core as an exclusive core for processing the audio real-time tasks and allocating the audio real-time tasks to the core while allowing other low-priority real-time tasks to be allocated to the core; if the user concentration degree represents that the user is not in a concentrated state and the complexity index does not exceed the preset threshold, allocating the audio real-time tasks and other real-time tasks to the plurality of real-time cores based on a preset proportion based on the comprehensive load index. 2.The Linux-based real-time scheduling system for HIFI audio playback of claim 1, wherein, The method for maintaining the receiving buffer and the playing buffer comprises: obtaining an audio stream parameter, a delay constraint and a system load of audio data in the current receiving buffer and the playing buffer, wherein the delay constraint represents a maximum allowed end-to-end delay customized by a user; obtaining a theoretical minimum value of the receiving buffer and a safety margin value of the playing buffer according to the audio stream parameter and the delay constraint; obtaining an adjustment coefficient according to the system load; adjusting the theoretical minimum value of the receiving buffer and the safety margin value of the playing buffer according to the adjustment coefficient to obtain the capacity of the receiving buffer and the playing buffer. 3.The Linux-based real-time scheduling system for HIFI audio playback of claim 1, wherein, The method for obtaining the complexity index of the audio data according to the audio data corresponding to the audio real-time tasks comprises: statistical audio data at each frequency spectrum entropy value; normalizing the spectrum entropy value of the audio data at each frequency, and squaring and summing the normalized value to obtain a complexity index.

4. The real-time scheduling system for Linux-based HIFI audio playback of claim 1, wherein, obtaining the running parameters of each real-time core, and obtaining the comprehensive load index of each real-time core according to the running parameters, including: obtaining the instruction cycle utilization, cache miss times, context switch times and core temperature of the real-time core in the past preset time period; obtaining the comprehensive load index according to the instruction cycle utilization, cache miss times, context switch times and core temperature.

5. The real-time scheduling system for Linux-based HIFI audio playback of claim 1, wherein, It also includes an RTOS initialization module for: establishing an RTOS kernel in a preset continuous area of memory when the Linux system starts; establishing a communication channel between the RTOS kernel and the Linux kernel; creating a dedicated API, which encapsulates the underlying task control operation of the RTOS, and the dedicated API is used to create real-time tasks, of which the audio real-time task has the highest priority.

6. The real-time scheduling system for Linux-based HIFI audio playback of claim 1, wherein, It also includes an application layer, a kernel layer and a hardware abstraction layer; the real-time scheduling layer also includes a task synchronization module; the playing module includes an audio equalization module and a sampling rate conversion module, wherein: the application layer is used to analyze user requests and store audio source data into a receiving buffer; the kernel layer is used to provide system calls and interface support for the application layer, the audio processing layer and the real-time scheduling layer; the hardware abstraction layer is used to provide a unified hardware access interface; the task synchronization module is used to synchronize tasks of different real-time cores; the audio equalization module is used to adjust the frequency response of the audio data; the sampling rate conversion module is used to convert the sampling rate of the audio data.

Images