Intelligent methods and systems for preventing and optimizing audio and video playback stuttering.

Abstract

This application relates to the field of audio and video processing technology, and provides an intelligent method and system for preventing and optimizing audio and video playback stuttering. In this application, firstly, a playback status data set is collected in real time, including bitstream parameters, hardware resource usage data, and network transmission status parameters. Then, a playback load correlation graph is constructed based on this data set, describing the mutual influence relationships between parameters. Next, a dynamic resource allocation strategy is generated based on the playback load correlation graph, adjusting the hardware resource allocation ratio and network transmission priority. Finally, the strategy is executed, and data changes are continuously monitored, dynamically adjusting the strategy to maintain playback smoothness. Thus, through multi-parameter correlation modeling and dynamic optimization, precise stuttering prevention is achieved, improving the audio and video playback experience.

Description

Technical Field

[0001] This application relates to the field of audio and video processing technology, and in particular to an intelligent method and system for preventing and optimizing audio and video playback stuttering. Background Technology

[0002] With the rapid development of audio and video technology, users' demands for smooth audio and video playback are increasing. Currently, audio and video playback stuttering remains one of the main factors affecting user experience, especially in multi-scenario playback environments such as mobile devices and smart TVs, where stuttering occurs frequently. Existing optimization methods for audio and video stuttering mostly focus on single-dimensional adjustments, such as reducing network transmission pressure through dynamic bitrate switching or releasing hardware resources through process priority adjustments. However, these methods fail to fully consider the complex interrelationships between multiple parameters during playback. For example, when network packet loss rate increases, simply reducing the bitrate may lead to image quality loss, but ignores the synergistic effect between CPU decoding capabilities and memory caching; similarly, simply increasing GPU rendering priority may crowd out other process resources, triggering new stuttering risks.

[0003] Furthermore, resource allocation strategies lack dynamic and correlation analysis. Traditional methods typically trigger optimization actions based on preset thresholds, such as automatically terminating background processes when CPU utilization exceeds a set value. However, this static threshold mechanism struggles to adapt to the diverse needs of different device hardware configurations, network environments, and audio / video content. For instance, during high-resolution video playback, GPU rendering pressure and memory bandwidth usage are strongly correlated, but current technologies fail to quantitatively model this relationship, leading to delayed or inaccurate resource allocation decisions. Moreover, the collection and analysis of playback status data are often processed in isolation, failing to fully consider the inter-parameter influence chains, making it difficult for stuttering prevention measures to fundamentally address playback anomalies caused by multi-factor coupling.

[0004] With the widespread adoption of 5G networks and the development of ultra-high-definition video technology, the amount of audio and video data is growing exponentially, further intensifying the hardware resource and network transmission pressures on playback devices. Existing single-dimensional optimization and static decision-making mechanisms are no longer sufficient to meet the demands for smooth playback in complex scenarios. Summary of the Invention

[0005] In view of the above, and aiming to at least partially address the shortcomings of existing technologies and bring new solutions to the field of audio transmission, this application provides, in a first aspect, an intelligent stuttering prevention and optimization method for audio and video playback, the method comprising:

[0006] A set of playback status data collected in real time during the audio and video playback process of the playback device, the set of playback status data including the bitstream parameters of the current playback progress, device hardware resource usage data and network transmission status parameters;

[0007] A playback load correlation graph is constructed based on the playback status data set. The playback load correlation graph is used to describe the mutual influence relationship between the bitstream parameters, the hardware resource usage data and the network transmission status parameters. It includes nodes, edges and edge weights. The nodes correspond to parameter types, the edges correspond to the correlation relationships between parameters, and the edge weights correspond to the correlation strength.

[0008] A dynamic resource allocation strategy is generated based on the playback load correlation graph. The dynamic resource allocation strategy is used to adjust the hardware resource allocation ratio and network transmission priority.

[0009] The dynamic resource allocation strategy is executed, and the changes in the playback status data set are continuously monitored. The dynamic resource allocation strategy is adjusted according to the changes.

[0010] Secondly, embodiments of this application also provide an intelligent stuttering prevention and optimization system for audio and video playback, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or code, and the processor is used to execute the programs, instructions or code in the machine-readable storage medium to implement the intelligent stuttering prevention and optimization method for audio and video playback.

[0011] In summary, the intelligent stuttering prevention and optimization method and system for audio and video playback provided in this application constructs a playback load correlation graph by real-time collection of bitstream parameters, hardware resource usage data, and network transmission status parameters during audio and video playback. This graph quantifies the interrelationships between multiple parameters, and dynamically generates resource allocation strategies based on the graph. This effectively solves the problems of lag and inaccuracy in stuttering prevention caused by single-dimensional optimization and static threshold decisions. Specifically, the playback load correlation graph abstracts the three types of parameters—bitstream parameters, hardware resource usage data, and network transmission status parameters—into nodes and characterizes the correlation strength through edge weights. This overcomes the limitations of traditional isolated parameter analysis, allowing the indirect influence of cross-dimensional parameters such as network latency and CPU decoding time, bitrate and bandwidth utilization to be visualized through path analysis, thus providing a global decision-making basis for resource allocation. Furthermore, the dynamic resource allocation strategy generation and continuous optimization mechanism, compared to static threshold triggering, can identify the impact links of abnormal parameters before stuttering occurs based on the real-time updated graph node status and path weights. For example, it can predict GPU rendering pressure through strong correlation paths between bitrate spikes and bandwidth utilization, and adjust memory caching strategies and network transmission priorities in advance, achieving a shift from passive response to proactive prevention. Simultaneously, through time-series correlation analysis and dynamic graph updates, it ensures the adaptability of the resource allocation strategy to different device hardware configurations, network environments, and audio / video content, avoiding the shortcomings of fixed optimization rules in handling complex scenarios, and significantly improving the smoothness and stability of audio and video playback.

[0012] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the above drawings without creative effort.

[0014] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0015] Figure 1 This is a flowchart illustrating an intelligent stuttering prevention and optimization method for audio and video playback provided in an embodiment of this application.

[0016] Figure 2 This is a schematic diagram of the architecture of an audio and video playback application scenario provided in an embodiment of this application.

[0017] Figure 3 This is a schematic diagram of an intelligent stuttering prevention and optimization system for audio and video playback provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0019] Please see Figure 1 and Figure 2 As shown, Figure 1 This is a flowchart illustrating the intelligent audio-visual scene switching optimization method combining pattern recognition provided in this application embodiment. Figure 2 This is a schematic diagram of an audio / video playback scenario. The audio / video playback scenario may include an audio / video content providing platform 100 and multiple playback devices 200 for playing the audio / video content provided by the audio / video content providing platform 100. In this embodiment, the method can be implemented by the playback devices 200, and the specific implementation may vary depending on the actual application scenario. For example, Figure 2 As shown, the playback device can be a computer terminal, mobile phone, tablet computer or other device with data transmission, analysis and processing capabilities. The audio and video content providing platform 100 can be a server, server cluster, computer equipment or the like. For example, it can be a backend server for providing live streaming services. This embodiment does not specifically limit this.

[0020] like Figure 1 As shown, the method includes steps S110-S140, which will be described in detail below.

[0021] Step S110: Collect playback status data set of the playback device in real time during the audio and video playback process. The playback status data set includes the bitstream parameters of the current playback progress, device hardware resource usage data, and network transmission status parameters.

[0022] In this embodiment, a playback device such as a smartphone, tablet, or laptop playing streaming video in a mobile network environment is used as an example to continuously collect playback status data. As an example, the playback device is configured with an octa-core CPU, a hexa-core GPU, and 6GB of RAM. The currently playing streaming video is an adaptive bitrate video with an initial resolution of 1080P and an encoding format of H264 / AVC.

[0023] During video playback, the three types of parameters mentioned above can be collected at fixed time intervals (e.g., every t1 time unit) through a preset data acquisition interface. Bitstream parameters are obtained by parsing the video stream's encapsulation format, including the encoding format, resolution, and bitrate of the current playback frame. Hardware resource usage data is read by calling the performance monitoring interface provided by the playback device's operating system kernel, including CPU process utilization, GPU rendering thread utilization, and application memory usage. Network transmission status parameters are collected through the network layer's traffic monitoring module, including the packet loss rate, latency, and bandwidth utilization of the current transmission link. After collection, the three types of parameters are aligned according to the collection timestamp to form a playback status data set containing multiple sets of time-series data.

[0024] In this embodiment, step S110 may include the following sub-steps S111-S114, which will be described in detail below.

[0025] Step S111: Obtain the bitstream parameters of the current playback progress output by the audio and video playback software in real time. The bitstream parameters include the encoding format, resolution and bitrate of the current frame.

[0026] In this embodiment, the audio and video playback software built into the playback device integrates a media information extraction module. This module obtains the timestamp of the current playback position by listening to the playback progress callback function, and locates the corresponding media data block from the video file's index structure based on the timestamp. For example, when the playback progress reaches timestamp T1, the media information extraction module parses the video frame header information corresponding to that timestamp, extracts the encoding format field, and determines that the current frame's encoding standard is H264 / AVC, the frame type is I-frame, and the compression mode is intra-frame predictive coding. Simultaneously, it reads the resolution information, including the horizontal pixel count W1, the vertical pixel count H1, and the pixel bit depth B1, where the pixel bit depth B1 determines the color representation range of each pixel. Bitrate calculation is achieved by statistically analyzing the amount of data in the current frame per unit time, specifically the ratio of the compressed data size of the current frame to the playback duration. This value is obtained by parsing the bitrate descriptor field of the video stream.

[0027] In this embodiment, step S111 may include the following sub-steps S1111-S1115, which will be described in detail below.

[0028] Step S1111: Monitor the playback progress callback interface of the audio / video playback software in real time to obtain the current frame number and corresponding timestamp.

[0029] In this embodiment, the audio / video playback software registers a playback progress callback interface. This interface is triggered frame by frame during video playback, returning the sequence number Fseq and the corresponding timestamp Tseq of the current playback frame each time it is triggered. This interface can be monitored using a multi-threaded mechanism, storing the acquired frame sequence number Fseq and timestamp Tseq in a circular buffer. The buffer size is set to accommodate information from N1 consecutive frames to avoid data overflow. For example, when the video plays to frame F1, the callback interface returns Fseq=F1 and Tseq=T1; when playing to frame F2, it returns Fseq=F2 and Tseq=T2, and so on, forming a one-to-one correspondence between frame sequence numbers and timestamps.

[0030] Step S1112: Locate the encoding metadata of the current frame from the index segment of the audio / video file according to the frame number, and extract the encoding format information, which includes the encoding standard and the intra-frame and inter-frame compression mode.

[0031] In this embodiment, the audio and video files can use the MP4 container format. The index segment is located in the moov atom of the file and contains the offset address and length information of each video frame. The system queries the index segment based on the frame sequence number Fseq to obtain the starting offset address and data length of the current frame in the file. Then, it reads the frame header data at that offset address and parses the encoding format information. The encoding standard is determined by the encoding type field in the frame header. For example, a field value of 0x01 indicates the H264 / AVC standard, and a field value of 0x02 indicates the H265 / HEVC standard. The intra-frame and inter-frame compression modes are determined by the frame type flag bit. A flag bit of 0 indicates intra-frame compression (I-frame), a flag bit of 1 indicates inter-frame compression (P-frame), and a flag bit of 2 indicates bidirectional predictive compression (B-frame).

[0032] Step S1113: Read the resolution information of the current frame, which includes the number of horizontal pixels, the number of vertical pixels, and the pixel depth.

[0033] In this embodiment, the resolution information of the current frame is stored in the Sequence Parameter Set (SPS) of the video stream, and the relevant parameters can be obtained by parsing the SPSnal unit. The number of horizontal pixels is calculated by parsing the width_in_mbs_minus1 field in the SPS, specifically (width_in_mbs_minus1+1)×16; the number of vertical pixels is calculated by parsing the height_in_map_units_minus1 field and the frame field coding mode. If it is frame coding mode, the number of vertical pixels is (height_in_map_units_minus1+1)×16; the pixel bit depth is determined by parsing the bit_depth_luma_minus8 field, specifically bit_depth_luma_minus8+8. This value determines the number of bits for the luminance component of each pixel. The bit depth of the chrominance component is usually the same as that of the luminance component.

[0034] Step S1114: Calculate the bitrate of the current frame, where the bitrate is the ratio of the original data size to the compressed data size of the current frame multiplied by the playback frame rate.

[0035] In this embodiment, the compressed data size of the current frame is obtained by reading the frame length information in the index segment, denoted as S1 bytes. The original data size is calculated based on the resolution and pixel bit depth using the formula: horizontal pixel count × vertical pixel count × pixel bit depth ÷ 8 (unit: bytes), denoted as S0 bytes. The playback frame rate is obtained by parsing the ftyp atoms of the video stream, denoted as Fps frames / time unit. Therefore, the bit rate of the current frame is (S0 / S1) × Fps × 8 (unit: bits / time unit), where multiplying by 8 converts bytes to bits.

[0036] Step S1115: Combine the timestamp of the current playback progress with the encoding format, the resolution and the bitrate as the bitstream parameters of the current playback progress.

[0037] In this embodiment, a bitstream parameter structure can be created, including a timestamp field, an encoding standard field, a compression mode field, horizontal pixel number segments, vertical pixel number segments, pixel bit depth field, and a bitrate field. The timestamp Tseq obtained in step S1111 is filled into the timestamp field, the encoding standard and compression mode extracted in step S1112 are filled into the corresponding fields, the horizontal pixel count, vertical pixel count, and pixel bit depth read in step S1113 are filled into the resolution-related fields, and the bitrate calculated in step S1114 is filled into the bitrate field, forming a complete bitstream parameter structure, which is then stored in the bitstream parameter queue in timestamp order.

[0038] Step S112: Read the hardware resource usage data provided by the device operating system in real time. The hardware resource usage data includes the CPU process usage rate, the GPU rendering thread usage rate, and the application memory usage.

[0039] In this embodiment, hardware resource usage data can be collected by calling the operating system's system call interface. For CPU process utilization, the system call process management interface obtains the process identifier (PID) corresponding to the audio / video playback software. Then, it iterates through the CPU time slice allocation table, calculating the ratio of the number of CPU time slices occupied by the PID in the most recent sampling period to the total number of time slices, thus obtaining the CPU process utilization. For GPU rendering thread utilization, the performance counter interface provided by the GPU driver is called to obtain the total number of threads related to video rendering and the number of currently active threads. The ratio of active threads to the total number of threads is calculated as the GPU rendering thread utilization. For application memory usage, the memory management interface is called to read the size of the audio / video playback software's memory-mapped file, heap memory usage, and stack memory usage. The sum of these three is the application memory usage, which is then compared with the device's total physical memory to obtain the memory usage ratio.

[0040] In this embodiment, step S112 may include the following sub-steps S1121-S1126, which will be described in detail below.

[0041] Step S1121: Call the device operating system's performance monitoring interface to obtain the total CPU utilization, CPU time slice allocation for each process, and process identifier.

[0042] In this embodiment, the operating system provides a proc file system interface to obtain the total CPU utilization by reading the / proc / stat file. This file contains the time slice counts of the CPU in user mode, system mode, idle mode, etc., since system startup. The total utilization is calculated as (user mode time slice + system mode time slice) ÷ (total time slice) × 100%. Simultaneously, the system reads the / proc / [pid] / stat file to obtain the CPU time slice allocation for each process, where [pid] is the process identifier. This file records the user mode CPU time and system mode CPU time of the corresponding process. The system iterates through the PIDs of all processes, collects the process identifiers and corresponding CPU time slice data, and stores them in the process CPU information table.

[0043] Step S1122: Filter the target process identifier corresponding to the audio and video playback software, and calculate the proportion of the target process's CPU time slice to the total CPU time slice as the CPU process utilization rate.

[0044] In this embodiment, the target process identifier of the audio / video playback software can be obtained through application package name mapping. For example, if the package name of the playback software is com.example.player, the list of PIDs corresponding to this package name can be obtained by querying the ActivityManagerService of the Android system, denoted as PidList. Processes in PidList are filtered from the process CPU information table, and the user-mode CPU time and system-mode CPU time of the above processes are accumulated to obtain the total CPU time Tc of the target process. The total CPU time slice Ttotal is the total CPU time slice in the current sampling period (calculated from the latest data in the / proc / stat file), then the CPU process utilization rate is (Tc / Ttotal) × 100%.

[0045] Step S1123: Call the performance counter interface of the GPU driver to obtain the total number of rendering threads of the GPU, the number of idle rendering threads, and the identifier of the rendering threads occupied by the audio and video playback software.

[0046] In this embodiment, the GPU driver provides an OpenGLES extension interface. The total number of GPU rendering threads, Ntotal, is queried by calling the glGetIntegeri_v function. The parameter of this function is GL_NUM_COMPUTE_WORK_GROUPS_X, and the return value is the total number of rendering threads. The number of idle rendering threads, Nidle, is obtained by calling the glGetIntegeri_v function with the GL_NUM_IDLE_COMPUTE_WORK_GROUPS parameter. The rendering thread identifiers used by the audio / video playback software are obtained through a GPU debugging tool. This tool can track the thread IDs corresponding to each rendering task and filter out a list of thread IDs, TidList, belonging to the playback software.

[0047] Step S1124: Calculate the ratio of the number of rendering threads occupied by the audio and video playback software to the total number of rendering threads as the GPU rendering thread utilization rate.

[0048] In this embodiment, the number of rendering threads Nused occupied by the audio and video playback software is the number of threads in TidList, and the total number of rendering threads of the GPU is Ntotal. Then, the rendering thread utilization rate of the GPU is (Nused / Ntotal) × 100%. For example, if Ntotal is N2 and Nused is N3, then the rendering thread utilization rate is (N3 / N2) × 100%. This value reflects the resource consumption of the GPU in the video rendering task.

[0049] Step S1125: Call the memory management interface to obtain the total physical memory of the playback device, the amount of memory used, and the size of the memory-mapped file of the audio / video playback software.

[0050] In this embodiment, the memory management interface is the operating system's sysinfo system call. This call retrieves the total physical memory (MemTotal) and the used memory (MemUsed - MemFree-Buffers-Cached). The size of the memory-mapped file for the audio / video playback software is obtained by reading the / proc / [pid] / maps file, which records the process's memory-mapped region information. Mapped regions of type "anon_inode:mmap" with permissions containing "rw-" are selected, and their sizes are summed to obtain the memory-mapped file size MmapSize.

[0051] Step S1126: Calculate the ratio of the memory-mapped file size of the audio / video playback software to the total physical memory as the application's memory usage.

[0052] In this embodiment, the application memory usage is (MmapSize / MemTotal) × 100%. For example, if MmapSize is M1 bytes and MemTotal is M2 bytes, then the memory usage is (M1 / M2) × 100%, which is used to evaluate the proportion of physical memory used by the playback software.

[0053] Step S113: Collect network transmission status parameters of the network transmission layer in real time. The network transmission status parameters include the packet loss rate, latency, and bandwidth utilization of the current transmission link.

[0054] In this embodiment, network transmission status parameters are collected through hook functions in the network protocol stack. Packet loss rate is calculated by statistically analyzing the difference between the number of sent data packets and the number of received acknowledgment data packets; specifically, it is the ratio of the number of data packets for which no acknowledgment has been received to the total number of sent data packets within a preset time window. Latency is obtained by recording the sending timestamp and receiving acknowledgment timestamp of each data packet, calculating the difference between the two, and averaging the results. Bandwidth utilization is determined by measuring the ratio of the actual transmission rate of the current network interface to the interface's maximum bandwidth. The actual transmission rate is calculated by statistically analyzing the total number of bytes of data packets passing through the network interface per unit time, and the maximum bandwidth is obtained by querying the hardware specifications of the network interface.

[0055] In this embodiment, step S113 may include the following sub-steps S1131-S1136, which will be described in detail below.

[0056] Step S1131: Obtain network data packets for audio and video data transmission, and record the source address, destination address, sending timestamp, and sequence number of each data packet. The network data packets include at least one of UDP data packets and TCP data packets.

[0057] In this embodiment, as an example, the network transport layer uses the UDP protocol to transmit audio and video data. All UDP packets passing through the network interface can be captured by setting a raw socket to listen on that interface. For each captured packet, its IP header and UDP header are parsed to extract the source address (SrcIP), destination address (DstIP), sending timestamp (Tsnd) (added by the kernel's network protocol stack), and sequence number (Seq) (set by the application layer for packet sorting). This information is stored in a packet information structure, with each structure corresponding to a network packet, forming a packet information list.

[0058] Step S1132: Receive the acknowledgment data packet returned by the other end, match the sequence numbers of the sent data packet and the acknowledgment data packet, and count the number of data packets that have not received acknowledgment.

[0059] In this embodiment, the audio / video server employs a Selective Acknowledgment (SACK) mechanism, returning an acknowledgment packet for each received data packet. This acknowledgment packet contains a range of sequence numbers for the received data packets. The system parses the received acknowledgment packets, extracts the acknowledgment sequence number range [SeqStart, SeqEnd], and then iterates through the list of sent data packet information, checking whether the sequence number Seq for each data packet falls within the range [SeqStart, SeqEnd]. If it does not fall within the range, it is marked as an unacknowledged data packet, and the number of unacknowledged data packets, Nlost, is accumulated.

[0060] Step S1133: Calculate the proportion of the number of data packets that have not received acknowledgments to the total number of data packets sent as the packet loss rate of the current transmission link.

[0061] In this embodiment, the total number of data packets sent, Ntotal, is the total number of UDP data packets captured within the current sampling period, and the number of data packets without acknowledgment is Nlost. Therefore, the packet loss rate is (Nlost / Ntotal) × 100%. For example, if Ntotal data packets are sent within time unit t4, and Nlost data packets are without acknowledgment, then the packet loss rate is (Nlost / Ntotal) × 100%, which reflects the reliability of network transmission.

[0062] Step S1134: Calculate the difference between the receiving timestamp of each acknowledgment data packet and the sending timestamp of the corresponding sending data packet to obtain the current transmission link delay time, wherein the median of the difference between the receiving timestamp of each acknowledgment data packet and the sending timestamp of the corresponding sending data packet is taken as the delay time.

[0063] In this embodiment, the receive timestamp Trcv of the confirmed data packet is recorded through the kernel's network interface receive queue, and the send timestamp Tsnd of the sent data packet is stored in the data packet information structure. For each confirmed data packet, the delay time ΔT = Trcv - Tsnd is calculated, resulting in a set of delay time data ΔT1, ΔT2, ..., ΔTn. This set of data is sorted in ascending order, and the value at the middle position is taken as the delay time Tdelay. If the number of data is even, the average of the two middle values ​​is taken.

[0064] Step S1135: Obtain the maximum bandwidth capacity of the current transmission link and the actual transmission rate of the current audio and video data. The maximum bandwidth capacity is obtained through the link negotiation protocol.

[0065] In this embodiment, the current transmission link is a 4G LTE network. The maximum bandwidth capacity Bmax is obtained by querying the mobile network modem using AT commands, which return the theoretical maximum downlink rate of the current cell. The actual transmission rate Rcurrent is calculated by counting the total number of bytes of audio and video data packets received per unit time. Specifically, within a time unit t5, the total number of bytes of data packets received is S bytes (including UDP header and IP header), so Rcurrent = S bytes × 8 / t5 (unit: bits / time unit).

[0066] Step S1136: Calculate the ratio of the actual transmission rate to the maximum bandwidth capacity as the bandwidth utilization rate of the current transmission link.

[0067] In this embodiment, the bandwidth utilization rate is (Rcurrent / Bmax) × 100%. For example, if Rcurrent is R1 bits per time unit and Bmax is B1 bits per time unit, then the bandwidth utilization rate is (R1 / B1) × 100%, which reflects the current network bandwidth usage.

[0068] Step S114: Align the bitstream parameters, the hardware resource usage data, and the network transmission status parameters with the same timestamp to form the playback status data set.

[0069] In this embodiment, the system assigns an independent timestamp generator to the acquisition process of each type of parameter, ensuring that the acquisition timestamps of all parameters are based on the same clock source. During the data alignment stage, the hardware resource usage data and network transmission status parameters are interpolated or extracted based on the acquisition timestamp of the bitstream parameters to keep the three types of parameters synchronized in the time dimension. For example, if the acquisition frequency of hardware resource usage data is once every t2 time unit, while the acquisition frequency of bitstream parameters is once every t3 time unit (t2≠t3), then the time series of hardware resource usage data is adjusted to the same time interval as the bitstream parameters through a linear interpolation algorithm, ultimately forming a structured playback status data set containing timestamp fields, bitstream parameter fields, hardware resource usage data fields, and network transmission status parameter fields.

[0070] Step S120: Construct a playback load correlation graph based on the playback status data set. The playback load correlation graph is used to describe the mutual influence relationship between the bitstream parameters, the hardware resource usage data and the network transmission status parameters, including nodes, edges and edge weights. The node corresponds to the parameter type, the edge corresponds to the correlation relationship between parameters, and the edge weight corresponds to the correlation strength.

[0071] In this embodiment, as an example only, a playback load correlation graph is constructed based on the playback status data set collected in step S110. First, all parameter types are extracted from the playback status data set, including bitstream parameters (encoding format, resolution, bitrate), hardware resource usage data (CPU process utilization, GPU rendering thread utilization, memory application usage), and network transmission status parameters (packet loss rate, latency, bandwidth utilization), for example, a total of nine parameter types. Each parameter type is treated as a node in the graph, and the unique identifier of the node is the parameter name.

[0072] Next, the relationships between the above parameters are analyzed. For example, an increase in bitrate may lead to longer CPU decoding time, thereby increasing CPU process utilization; an increase in bandwidth utilization may be positively correlated with bitrate; an increase in latency may lead to the accumulation of unplayed video frames in the memory cache, thereby increasing memory application usage. These relationships are represented by edges, where the direction of the edge indicates the direction of influence (e.g., bitrate → CPU process utilization), and the edge weight indicates the strength of the relationship. Finally, the nodes, edges, and edge weights are integrated to form a directed weighted graph, i.e., the playback load relationship graph.

[0073] In this embodiment, step S120 may include the following sub-steps S121-S127, which will be described in detail below.

[0074] Step S121: Combine the bitstream parameters, hardware resource usage data and network transmission status parameters in the playback status data set in pairs to form a parameter pair set, which includes multiple parameter pairs.

[0075] In this embodiment, as an example only, the parameter types include bitstream parameters (C1: encoding format, C2: resolution, C3: bitrate), hardware resource usage data (H1: CPU process utilization, H2: GPU rendering thread utilization, H3: memory application usage), and network transmission status parameters (N1: packet loss rate, N2: latency, N3: bandwidth utilization), totaling nine parameter types. These parameter types are combined in pairs, regardless of the combination order (i.e., (A,B) and (B,A) are considered the same parameter pair), resulting in C(9,2)=36 parameter pairs. For example, parameter pair (C3,H1) represents the combination of bitrate and CPU process utilization, parameter pair (N2,H3) represents the combination of latency and memory application usage, and all parameter pairs constitute a parameter pair set.

[0076] In this embodiment, step S121 may include the following sub-steps S1211-S1212, which will be described in detail below.

[0077] Step S1211: Traverse all parameter types of bitstream parameters, hardware resource usage data, and network transmission status parameters to generate a list of parameter types without duplicates.

[0078] In this embodiment, the parameter type list is divided into three sublists: a bitstream parameter sublist [C1,C2,C3], a hardware resource usage data sublist [H1,H2,H3], and a network transmission status parameter sublist [N1,N2,N3]. These are then merged to obtain the total parameter type list ParamList = [C1,C2,C3,H1,H2,H3,N1,N2,N3]. ParamList is iterated through to ensure that each parameter type is unique within the list, avoiding duplicate combinations.

[0079] Step S1212: Use a double loop to traverse the parameter type list, generate all possible pairwise combinations, and exclude combinations of parameter type and itself.

[0080] In this embodiment, the outer loop iterates through the i-th parameter type in ParamList (i ranges from 0 to 8), and the inner loop iterates through the j-th parameter type (j ranges from i+1 to 8), generating parameter pairs (ParamList[i], ParamList[j]). For example, when i=0 (C1), j=1 (C2) generates parameter pairs (C1, C2), j=2 (C3) generates (C1, C3), and so on, until j=8 (N3) generates (C1, N3); when i=1 (C2), j=2 (C3) generates (C2, C3), and so on, ultimately generating 36 parameter pairs, which are stored in the parameter pair set PairSet.

[0081] Step S122: Perform time series correlation analysis on each parameter pair and calculate the correlation coefficient of each parameter pair at consecutive timestamps. The correlation coefficient reflects the linear correlation strength between the parameters, wherein the correlation coefficient is the Pearson correlation coefficient.

[0082] In this embodiment, bitstream parameters, hardware resource usage data, and network transmission status parameters can all be understood as a type of parameter. Taking the parameter pair (C3, H1) (bitrate and CPU process utilization) as an example, time series correlation analysis is performed. First, the time series data of bitrate C3 and CPU process utilization H1 are extracted from the playback status data set. Each data point corresponds to a timestamp, forming two sets of time series data C3(t) and H1(t), where t is the timestamp index. Then, the Pearson correlation coefficient is calculated for these two sets of time series data. The Pearson correlation coefficient r is used to measure the degree of linear correlation between two variables, and its value ranges from [-1, 1]. The larger the absolute value of r, the stronger the linear correlation.

[0083] In this embodiment, step S122 may include the following sub-steps S1221-S1226, which will be described in detail below.

[0084] Step S1221: Set a sliding time window for each parameter pair, the length of which is adjusted according to the frame rate of the audio and video.

[0085] In this embodiment, as an example, the frame rate of audio and video is Fps (frames per time unit). The length L of the sliding time window is defined as the time span containing M consecutive frames, i.e., L = M / Fps (time unit). The value of M is determined according to the frame rate. For example, when Fps = 30 frames per time unit, M = 30, then L = 1 time unit, indicating that the sliding time window covers the parameter data corresponding to all frames within 1 time unit. For each parameter pair, a sliding time window of the same length is set to ensure that the time span of the time series data is consistent.

[0086] Step S1222: Fill the parameter pair values ​​in the playback status data set into the sliding time window in timestamp order to form time series data.

[0087] In this embodiment, the parameter pairs in the playback status data set are arranged in ascending order of timestamps. The sliding time window starts from the first timestamp and slides sequentially, with each slide step being 1 time unit. For each sliding position, the window contains the parameter pair values ​​within L time units. For example, when the parameter pair (C3, H1) is at the sliding window position t, it contains the bitrate C3 value from timestamp tL to t and the CPU process utilization H1 value, forming two sets of time series data: C3_win=[C3(tL),C3(t-L+1),...,C3(t)] and H1_win=[H1(tL),H1(t-L+1),...,H1(t)].

[0088] Step S1223: Perform detrending processing on the time series data within the sliding time window to eliminate the linear trend component in the data.

[0089] In this embodiment, linear regression can be used to detrend the time series data. For the time series data X_win=[x1,x2,...,xn] (e.g., C3_win), with the timestamp index as the independent variable t (t=1,2,...,n) and X_win as the dependent variable, linear regression is performed to obtain the regression equation x=at+b, where a is the slope and b is the intercept. The detrended data X_detrend=[x1-(a×1+b),x2-(a×2+b),...,xn-(a×n+b)] eliminates the linear trend component while retaining the volatility component.

[0090] Step S1224: Calculate the covariance of the detrended time series data and the standard deviation of each time series data.

[0091] In this embodiment, as an example, for two sets of detrended time series data X_detrend and Y_detrend (such as detrended data C3_detrend for bitrate and detrended data H1_detrend for CPU process utilization), the covariance is first calculated. The covariance calculation process is as follows: First, calculate the deviation between all data points in X_detrend and the mean of X_detrend, and the deviation between all data points in Y_detrend and the mean of Y_detrend; then multiply the two deviations of the corresponding data points to obtain a product value; then sum these product values, and finally divide the sum by the time series length minus one (i.e., the number of data points minus one) to obtain the covariance. Here, the mean of X_detrend is the arithmetic mean of all data points in X_detrend, the mean of Y_detrend is the arithmetic mean of all data points in Y_detrend, and the time series length refers to the number of data points in X_detrend or Y_detrend.

[0092] Next, the standard deviation of each time series data is calculated. For the standard deviation of X_detrend, the calculation process is as follows: First, calculate the deviation between all data points in X_detrend and the mean of X_detrend. Then, square each deviation to obtain a set of squared deviation values. Sum these squared deviation values, divide the sum by the time series length minus one, and finally take the square root of the result to obtain the standard deviation of X_detrend. The calculation process for the standard deviation of Y_detrend is the same as that for X_detrend: first calculate the deviation, then the squared deviation, sum the squared deviations, divide by the time series length minus one, and finally take the square root.

[0093] Step S1226: Repeat the above steps according to the sliding time window order to obtain the correlation coefficient sequence of each parameter pair under all consecutive timestamps.

[0094] In one example of this embodiment, the sliding time window starts from the timestamp t_start, with each sliding step being 1 time unit, until the timestamp t_end, resulting in a total of K = (t_end - t_start - L + 1) sliding window positions. For each position, the correlation coefficient r_k (k = 1, 2, ..., K) of the parameter pair is calculated, forming a correlation coefficient sequence [r_1, r_2, ..., r_K], which reflects the change in the correlation strength of the parameter pair within different time windows.

[0095] Step S123: Parameter pairs whose absolute values ​​of correlation coefficients exceed a set correlation strength threshold are identified as target parameter pairs, wherein the target parameter pairs are parameter pairs with significant correlation.

[0096] In this embodiment, for example, the correlation strength threshold is set as Thr, which is determined according to the statistical analysis of historical data of audio and video playback. For example, Thr is set to x% (x is a percentage determined according to empirical values). For the correlation coefficient r of each parameter pair, its absolute value |r| is calculated. If |r| > Thr, then this parameter pair is determined as the target parameter pair. For example, for the parameter pair (C3, H1), the correlation coefficient r1 = 0.75, |r1| = 0.75 > Thr = 0.6, so (C3, H1) is the target parameter pair; for the parameter pair (C1, N1), the correlation coefficient r2 = 0.3, |r2| = 0.3 < Thr, so it is not the target parameter pair. All target parameter pairs form the target parameter pair set.

[0097] Step S124: Take each of the bitstream parameters, the hardware resource occupancy data, and the network transmission status parameters as nodes of the playback load association map, and set attributes for each node, including the parameter name, unit, and the value range during normal playback.

[0098] In this embodiment, each parameter type corresponds to a node, and the unique identifier of the node is the parameter name. For example, the bit rate C3 corresponds to the node Node_C3, and the CPU process occupancy rate H1 corresponds to the node Node_H1. Set attributes for each node: the parameter name is the name of the parameter type (such as "bit rate"); the unit is determined according to the parameter type (for example, the unit of the bit rate is kbps, and the unit of the CPU process occupancy rate is a percentage); the value range during normal playback is determined through statistical analysis of historical data. For example, the normal value range of the bit rate C3 is [C3_min, C3_max], and the normal value range of the CPU process occupancy rate H1 is [H1_min, H1_max].

[0099] In this embodiment, step S124 may include the following sub-steps S1241 - S1245, which will be described in detail below.

[0100] Step S1241: Collect historical data of the bitstream parameters, the hardware resource occupancy data, and the network transmission status parameters of multiple playback devices during normal audio and video playback.

[0101] In this embodiment, the historical data is collected through a cloud server. For example, it may include the parameter values of playback devices of different brands and models when playing audio and video of different coding formats and resolutions. For example, collect the parameter data such as the bit rate, CPU process occupancy rate, and packet loss rate of 1000 playback devices when playing 720P, 1080P, and 4K resolution videos. The data sampling period is 1 time unit, and the total amount of data is N records.

[0102] Step S1242: Perform statistical analysis on the historical data and calculate the mean, standard deviation and statistical confidence interval for each parameter.

[0103] In this embodiment, historical data for each parameter type is statistically analyzed, and the arithmetic mean μ and standard deviation σ are calculated. Then, a statistical confidence interval [μ-1.96σ, μ+1.96σ] is calculated based on the confidence level (e.g., 95%). For example, if the historical data mean for bitrate C3 is μC3 and the standard deviation is σC3, then the confidence interval is [μC3-1.96σC3, μC3+1.96σC3].

[0104] Step S1243: Use the upper and lower limits of the statistical confidence interval as the calibration range of the parameter during normal playback.

[0105] In this embodiment, the calibration range of the parameters is the statistical confidence interval [μ-1.96σ, μ+1.96σ], which includes the parameter values ​​in 95% of normal playback scenarios. For example, the calibration range of the bitrate C3 is [C3_min, C3_max] = [μC3-1.96σC3, μC3+1.96σC3].

[0106] Step S1244: Set attributes for each node, including parameter name, parameter unit and calibration value range, such as bit rate in kbps and CPU utilization rate as percentage.

[0107] In this embodiment, node attributes can be stored in key-value pairs. For example, the attributes of node Node_C3 are {"name":"bitrate","unit":"kbps","range":[C3_min,C3_max]}, and the attributes of node Node_H1 are {"name":"CPU process utilization","unit":"%","range":[H1_min,H1_max]}.

[0108] Step S1245: Continuously monitor the values ​​of the bitstream parameters, hardware resource usage data, and network transmission status parameters during audio and video playback, periodically update the historical data, and adjust the upper and lower limits of the calibration value range based on the updated historical data.

[0109] In this embodiment, the current playback status data set can be uploaded to the cloud server every t6 time units. The cloud server merges the new data into the historical database, recalculates the average value μ and standard deviation σ, and then updates the calibration value range. For example, when 100 bars of historical data with a bitrate C3 are added, μC3_new and σC3_new are recalculated, and the calibration value range is updated to [μC3_new-1.96σC3_new, μC3_new+1.96σC3_new].

[0110] Step S125: Use the target parameter pair as the edge of the playback load correlation graph, and set attributes for each edge, including the names of the two associated nodes and the corresponding correlation coefficient.

[0111] In this embodiment, each target parameter pair corresponds to an edge, with the two ends of the edge being the nodes corresponding to the two parameter types in the target parameter pair. For example, the target parameter pair (C3, H1) corresponds to the edge Edge_C3_H1, which connects nodes Node_C3 and Node_H1. The edge attributes include: the names of the two associated nodes (such as "bitrate" and "CPU process utilization"), and the corresponding association coefficient r (such as 0.75). If parameter pairs (A, B) and (B, A) are both target parameter pairs, then two edges Edge_A_B and Edge_B_A with opposite directions are created, corresponding to different association coefficients.

[0112] Step S126: Calculate the edge weight based on the correlation coefficient of the edge. The edge weight is the absolute value of the correlation coefficient multiplied by a preset scaling factor, which is preset according to the smoothness requirements of audio and video playback.

[0113] In this embodiment, the preset scaling factor is K, which is determined based on the smoothness level of audio and video playback. For example, the higher the smoothness requirement, the larger the value of K (e.g., K=1.5). The formula for calculating the edge weight W is W=|r|×K, where r is the correlation coefficient corresponding to the edge. For example, if the correlation coefficient of the edge Edge_C3_H1 is r=0.75 and the scaling factor K=1.5, then the edge weight W=0.75×1.5=1.125. The edge weight is used to measure the strength of the correlation between parameters; the larger the weight, the stronger the correlation.

[0114] Step S127: Construct a playback load association graph containing a set of nodes, a set of edges, and edge weights. The paths in the playback load association graph represent the indirect influence relationships between parameters.

[0115] In this embodiment, the node set Nodes contains nodes corresponding to all parameter types (e.g., Node_C1, Node_C2, ..., Node_N3); the edge set Edges contains edges corresponding to all target parameter pairs (e.g., Edge_C3_H1, Edge_H1_H2, ...); and the edge weight set Weights contains the weight value corresponding to each edge. These three sets are integrated to form a playback load correlation graph G=(Nodes, Edges, Weights). Paths in the graph are formed by connecting multiple edges end-to-end, representing indirect influence relationships between parameters. For example, the path Node_C3→Node_H1→Node_H2 indicates that the bitrate indirectly affects the GPU rendering thread utilization by influencing the CPU process utilization.

[0116] Step S130: Generate a dynamic resource allocation strategy based on the playback load correlation graph. The dynamic resource allocation strategy is used to adjust the hardware resource allocation ratio and network transmission priority.

[0117] In this embodiment, the dynamic resource allocation strategy is generated based on node anomaly detection and path analysis of the playback load correlation graph (hereinafter referred to as the "graph"). First, all nodes in the graph are traversed to identify abnormal nodes whose values ​​exceed the calibrated range. For example, when the current value of bitrate C3, C3_current, is greater than C3_max, Node_C3 is marked as an abnormal node. Then, starting from the abnormal node, the edges in the graph are traversed to extract all paths associated with the abnormal node. The path weight of each path (the product of the weights of all edges in the path) is calculated, and target impact paths whose path weights exceed a preset path threshold are selected.

[0118] For each endpoint node (target adjustment object) in the target impact path, a hardware resource allocation ratio adjustment scheme and a network transmission priority adjustment scheme are generated separately. For example, if the target adjustment object is CPU process utilization H1 (a hardware resource node), the required reduction in resource ratio is calculated based on the degree of abnormality of H1 (the difference between the current value and the upper limit of the normal range); if the target adjustment object is latency N2 (a network transmission node), the required increase in transmission priority is calculated based on the degree of abnormality of N2. Finally, the two adjustment schemes are merged to form a dynamic resource allocation strategy.

[0119] In this embodiment, step S130 may include the following sub-steps S131-S137, which will be described in detail below.

[0120] Step S131: Traverse all nodes in the playback load correlation graph and identify abnormal nodes whose values ​​exceed the calibration range.

[0121] In this embodiment, each node in the node set Nodes is traversed, and the current value Vcurrent and the calibrated value range [Vmin, Vmax] of the node are read. If Vcurrent < Vmin or Vcurrent > Vmax, the node is marked as an abnormal node, and the abnormal type (below the lower limit or above the upper limit) and the degree of abnormality D = |Vcurrent - (Vmin + Vmax) / 2| / (Vmax - Vmin) are recorded. The value range of the degree of abnormality D is [0, 1], and the larger D is, the more serious the abnormality is.

[0122] Step S132: Starting from the abnormal node, traverse the edges in the load association graph, extract all paths associated with the abnormal node, and form an associated path set.

[0123] In this embodiment, the depth-first search (DFS) algorithm can be used to traverse the graph. Starting from the abnormal node, all reachable nodes are explored along the direction of the edges to form a path list. For example, the associated paths of the abnormal node Node_C3 (abnormal bit rate) include Node_C3 → Node_H1 (bit rate → CPU process occupancy rate), Node_C3 → Node_H1 → Node_H2 (bit rate → CPU process occupancy rate → GPU rendering thread occupancy rate), etc. All paths constitute the associated path set.

[0124] Step S133: Perform a product operation on the edge weights in each associated path to obtain a path weight, which reflects the degree of influence of the abnormal node on other nodes through this associated path.

[0125] In this embodiment, the path weight Pw is the product of all edge weights in the path. For example, if the edge weights of the path Node_C3 → Node_H1 → Node_H2 are W1 and W2 respectively, then the path weight Pw = W1 × W2. The larger the path weight, the more significant the indirect influence of the abnormal node on the end node through this path.

[0126] Step S134: Determine the associated paths with path weights exceeding the preset path threshold as the target influence paths, and extract the end nodes in the target influence paths as the target adjustment objects.

[0127] In this embodiment, the preset path threshold is Pthr, which is determined according to the statistics of path weights in historical lag cases. For the path weight Pw of each associated path, if Pw > Pthr, then this path is determined as the target influence path, and its end node is the target adjustment object that needs to be adjusted. For example, if Pw = 1.2 > Pthr = 1.0 for the path Node_C3 → Node_H1, then the end node Node_H1 is the target adjustment object.

[0128] Step S135: For hardware resource nodes in the target adjustment object, calculate the required resource ratio based on the degree of abnormality of the hardware resource nodes, and generate a hardware resource allocation ratio adjustment plan.

[0129] In this embodiment, hardware resource nodes include CPU process utilization H1, GPU rendering thread utilization H2, etc. For a hardware resource node Node_H, its abnormality level is Dh, and the resource ratio to be adjusted is Rh = Dh × Rh_base, where Rh_base is the base adjustment ratio (preset according to the hardware type). For example, if Node_H1 has Dh = 0.3 and Rh_base = 20%, then Rh = 0.3 × 20% = 6%, generating the adjustment plan "reduce CPU process utilization by 6%".

[0130] Step S136: For network transmission nodes in the target adjustment object, calculate the transmission priority that needs to be adjusted based on the degree of abnormality of the network transmission nodes, and generate a network transmission priority adjustment scheme.

[0131] In this embodiment, network transmission nodes include packet loss rate N1, latency N2, etc. For a network transmission node Node_N, its anomaly level is Dn, and the required transmission priority level is Rn=ceil(Dn×Rn_base), where Rn_base is the base priority level (preset according to the network type), and ceil is the floor function. For example, if Node_N2 has Dn=0.4 and Rn_base=level 3, then Rn=ceil(0.4×3)=level 2, generating the adjustment scheme "increase the transmission priority corresponding to the latency by 2 levels".

[0132] Step S137: Integrate the hardware resource allocation ratio adjustment scheme and the network transmission priority adjustment scheme to form a dynamic resource allocation strategy.

[0133] In this embodiment, the fusion process includes pairing adjustment items, calculating the synergy coefficient, handling conflicts, and sorting the execution order. For example, the hardware adjustment item "reduce CPU process utilization by 6%" is paired with the network adjustment item "increase transmission priority by 2 levels," the synergy coefficient (edge ​​weight × hardware adjustment range × network adjustment range) is calculated, adjustment combinations with significant synergistic effects are selected, conflicting adjustment items are eliminated, and the execution order is sorted according to node centrality to finally form a dynamic resource allocation strategy.

[0134] In this embodiment, step S137 may include the following sub-steps S13701-S13711, which will be described in detail below.

[0135] Step S13701: Extract all hardware adjustment items in the hardware resource allocation ratio adjustment scheme. Each hardware adjustment item includes the node name, adjustment direction, and adjustment magnitude of the target hardware resource class node. The adjustment direction is to increase or decrease the resource allocation ratio, and the adjustment magnitude is the amount of change in the resource allocation ratio.

[0136] In this embodiment, the hardware resource allocation ratio adjustment scheme is an adjustment plan generated for hardware resource nodes, including CPU process utilization H1, GPU rendering thread utilization H2, etc. When extracting hardware adjustment items from this scheme, each hardware adjustment item needs to specify three elements: the node name of the target hardware resource node, such as "CPU process utilization" or "GPU rendering thread utilization"; the adjustment direction, i.e., whether to increase or decrease the resource allocation ratio, for example, when the CPU process utilization is too high, the adjustment direction is "decrease"; and the adjustment magnitude, i.e., the specific amount of change in the resource allocation ratio, such as "decrease the resource allocation ratio by x%". For example, if the adjustment content for CPU process utilization H1 in the hardware resource allocation ratio adjustment scheme is "reduce CPU process utilization by y%", then the corresponding hardware adjustment item is (node ​​name: CPU process utilization, adjustment direction: decrease, adjustment magnitude: y%).

[0137] Step S13702: Extract all network adjustment items in the network transmission priority adjustment scheme. Each network adjustment item includes the node name, adjustment direction, and adjustment magnitude of the target network transmission class node. The adjustment direction is to increase or decrease the transmission priority, and the adjustment magnitude is the change level of the transmission priority.

[0138] In this embodiment, the network transmission priority adjustment scheme is an adjustment plan generated for network transmission type nodes, including packet loss rate N1, latency N2, etc. When extracting network adjustment items from this scheme, each network adjustment item also specifies three elements: the node name of the target network transmission type node, such as "latency" or "packet loss rate"; the adjustment direction, i.e., whether the transmission priority is increased or decreased, for example, when the latency is too long and affects the smoothness of playback, the adjustment direction is "increase"; and the adjustment magnitude, i.e., the specific level of change in transmission priority, such as "increase the transmission priority by level z". For example, if the adjustment content for latency N2 in the network transmission priority adjustment scheme is "increase the transmission priority corresponding to the latency by level w", then the corresponding network adjustment item is (node ​​name: latency, adjustment direction: increase, adjustment magnitude: level w).

[0139] Step S13703: Pair the target hardware resource class node of each hardware adjustment item with the target network transmission class node of each network adjustment item to form an adjustment item pair set.

[0140] In this embodiment, all extracted hardware and network adjustment items are traversed. The target hardware resource node of each hardware adjustment item is combined with the target network transmission node of each network adjustment item to form an adjustment item pair. For example, if the target hardware resource node of hardware adjustment item A is "CPU process utilization" and the target network transmission node of network adjustment item B is "latency", then they are paired to form an adjustment item pair (CPU process utilization, latency). In this way, a set of adjustment item pairs is formed through pairing and combination.

[0141] Step S1374: For each pair of adjustment items, obtain the edge weights connecting the two target nodes from the playback load correlation graph.

[0142] In this embodiment, the playback load correlation graph stores the relationships between nodes and their corresponding edge weights. For each adjustment item pair in the adjustment item pair set, the playback load correlation graph is queried to find the edge connecting the two target nodes in the adjustment item pair, and then the edge weight of that edge is obtained. For example, if the adjustment item pair is (CPU process utilization, latency), the playback load correlation graph is searched for an edge connecting the "CPU process utilization" node and the "latency" node. If such an edge exists, its edge weight W is obtained.

[0143] Step S13705: Calculate the coordination coefficient of the adjustment term pair based on the obtained edge weights. The coordination coefficient is the product of the edge weight multiplied by the adjustment range of the hardware adjustment term and the adjustment range of the network adjustment term. The positive or negative sign of the coordination coefficient indicates the coordination relationship or antagonistic relationship of the adjustment term pair.

[0144] In this embodiment, for each pair of adjustment items, the edge weight obtained in step S13704 is multiplied by the adjustment magnitude of the hardware adjustment item and the adjustment magnitude of the network adjustment item in that pair to obtain a synergy coefficient. For example, in the pair of adjustment items (CPU process utilization, latency), if the edge weight is W, the adjustment magnitude of the hardware adjustment item is y%, and the adjustment magnitude of the network adjustment item is w, then the synergy coefficient = W × y% × w. When the synergy coefficient is positive, it indicates that the hardware adjustment item and the network adjustment item have a synergistic relationship in improving playback smoothness, that is, their adjustment directions are consistent and mutually reinforcing; when the synergy coefficient is negative, it indicates that they have an antagonistic relationship, that is, their adjustment directions may conflict or weaken each other.

[0145] Step S13706: Select adjustment item pairs with positive synergy coefficients and absolute values ​​exceeding the set synergy threshold as synergistic adjustment combinations with synergistic effects.

[0146] In this embodiment, a collaborative threshold Thr_co is set, which is preset based on the smoothness requirements of audio and video playback and historical adjustment effects. The collaborative coefficients of all adjustment pairs are iterated. If the collaborative coefficient is positive and its absolute value is greater than Thr_co, then the adjustment pair is determined to be a collaborative adjustment combination with a collaborative effect. For example, if the collaborative coefficient of an adjustment pair is C1, and the collaborative threshold Thr_co is set to C0, if C1>0 and |C1|>C0, then the adjustment pair belongs to a collaborative adjustment combination.

[0147] Step S13707: For the hardware adjustment item and network adjustment item in each collaborative adjustment combination, retain the adjustment direction and adjustment range of the hardware adjustment item and network adjustment item.

[0148] In this embodiment, for the selected collaborative adjustment combinations, the adjustment direction and magnitude of the hardware adjustment items and the network adjustment items are retained. For example, if the hardware adjustment item in the collaborative adjustment combination is (node ​​name: CPU process utilization, adjustment direction: decrease, adjustment magnitude: y%), and the network adjustment item is (node ​​name: latency, adjustment direction: increase, adjustment magnitude: w level), then the adjustment direction and magnitude information such as "decrease y%" and "increase w level" are retained for subsequent integration into the dynamic resource allocation strategy.

[0149] Step S13708: Traverse all adjustment items (including adjustment items in coordinated adjustment combinations and adjustment items that do not form coordinated combinations, the adjustment items including hardware adjustment items and network adjustment items), check for conflicts between adjustment items, and perform conflict handling. Conflict handling includes: if increasing the resource allocation ratio of a hardware adjustment item leads to a greater degree of abnormality in the corresponding network transmission node, or increasing the transmission priority of a network adjustment item leads to a greater degree of abnormality in the corresponding hardware resource node, then reduce the adjustment magnitude of that adjustment item until the conflict is eliminated.

[0150] In this embodiment, all adjustment items are checked one by one to check for conflicts. Conflicts mainly manifest as follows: increasing the resource allocation ratio of a hardware adjustment item will worsen the anomaly of the corresponding network transmission node; or increasing the transmission priority of a network adjustment item will exacerbate the anomaly of the corresponding hardware resource node. For example, if a hardware adjustment item is "increase the resource allocation ratio of GPU rendering thread utilization by m%", and the corresponding network transmission node "bandwidth utilization" is currently in an abnormal state, increasing the GPU rendering thread utilization may further increase the data transmission volume, thereby exacerbating the anomaly of bandwidth utilization, which constitutes a conflict. For conflicting adjustment items, the adjustment magnitude is reduced gradually while monitoring the changes in the anomaly of the corresponding node until the conflict is eliminated, i.e., the hardware adjustment no longer exacerbates the anomaly of the network transmission node, or the network adjustment no longer exacerbates the anomaly of the hardware resource node.

[0151] Step S13709: Integrate the synergy adjustment combinations and the adjustment items after conflict resolution into an adjustment item list in descending order of synergy coefficient.

[0152] In this embodiment, the collaborative adjustment combinations are first sorted from high to low according to their collaborative coefficients. The higher the collaborative coefficient, the better the collaborative effect on improving playback smoothness, and the higher the priority should be given to the collaborative adjustment combination. Then, the adjustment items that have not formed a collaborative combination but have undergone conflict resolution are also sorted according to certain rules (such as the degree of impact on playback smoothness). Finally, the sorted collaborative adjustment combinations and the conflict-resolved adjustment items are integrated together to form an adjustment item list.

[0153] Step S13710: Add an execution order marker to each adjustment item in the adjustment item list. The execution order is determined based on the centrality of the target node corresponding to the adjustment item in the playback load correlation graph. Adjustment items corresponding to nodes with higher centrality are executed first.

[0154] In this embodiment, node centrality is an indicator that measures the importance of a node in the playback load correlation graph. Higher centrality indicates a greater influence of the node on other nodes and a more critical role in resource allocation. By calculating the centrality of the target node corresponding to each adjustment item in the adjustment item list within the playback load correlation graph, execution order markers are added to each adjustment item according to its centrality from highest to lowest. For example, the adjustment item corresponding to the target node with the highest centrality is marked as "1," indicating it will be executed first; the next highest centrality is marked as "2," and so on.

[0155] Step S13711: Based on the list of adjustment items and the execution order marker, generate a dynamic resource allocation strategy that includes hardware resource allocation ratio adjustment content, network transmission priority adjustment content, and execution order. Each adjustment item in the dynamic resource allocation strategy corresponds to a determined target node, adjustment direction, adjustment magnitude, and execution order.

[0156] In this embodiment, each adjustment item in the adjustment item list is combined with its corresponding execution order marker to clarify the target node, adjustment direction, adjustment magnitude, and execution order of each adjustment item. For example, the first adjustment item in the adjustment item list is (target node: CPU process utilization, adjustment direction: decrease, adjustment magnitude: y%, execution order: 1), and the second adjustment item is (target node: latency, adjustment direction: increase, adjustment magnitude: w, execution order: 2), etc. Integrating this information forms the final dynamic resource allocation strategy, which can clearly guide the playback device to adjust the hardware resource allocation ratio and network transmission priority.

[0157] Step S140: Execute the dynamic resource allocation strategy, continuously monitor the changes in the playback status data set, and adjust the dynamic resource allocation strategy according to the changes to maintain the smoothness of audio and video playback.

[0158] In this embodiment, the execution of the dynamic resource allocation strategy is achieved by calling the device's system interface. The hardware resource allocation ratio adjustment scheme is sent to the device's resource management module, which modifies the CPU's process priority (via the `setpriority` system call), the GPU's rendering queue scheduling rules (by modifying OpenGLES context parameters), and the memory page cache allocation strategy (by adjusting the ` / proc / sys / vm / dirty_ratio` parameter). The network transmission priority adjustment scheme is sent to the network management module, which modifies the Differential Service Code Point (DSCP) value of audio and video data packets (by setting the `IP_TOS` option via the `setsockopt` system call) and the queue scheduling algorithm (switching from FIFO to RED).

[0159] After implementing the strategy, the system continuously monitors the playback status data set at t7 time units, obtains new parameter values, and updates the playback load correlation graph. If abnormal nodes still exist, the target impact path and adjustment plan are recalculated, resource allocation ratios and transmission priorities are adjusted, and the process is repeated until the values ​​of all nodes return to the normal range. For example, if the CPU process utilization H1 is still higher than the normal range after implementing the strategy, its resource allocation ratio is further reduced by 5%, and the associated nodes are checked to see if they have returned to normal.

[0160] In this embodiment, step S140 may include the following sub-steps S141-S146, which will be described in detail below.

[0161] Step S141: Send the hardware resource allocation ratio adjustment plan in the dynamic resource allocation strategy to the resource management module of the playback device. The resource management module modifies the CPU process priority, GPU rendering queue scheduling rules and memory page cache allocation strategy according to the adjustment plan.

[0162] In this embodiment, the resource management module can receive the adjustment scheme and parse the hardware adjustment items via inter-process communication (IPC). For CPU process priority adjustment, the `setpriority(PRIO_PROCESS, pid, priority)` system call is invoked to lower the priority value of the audio / video playback process (the smaller the value, the lower the priority). For GPU rendering queue scheduling rules, the `EGL_RENDER_BUFFER` parameter of `EGLContext` is modified to limit the maximum number of rendering threads. For memory page cache allocation strategy, the kernel parameter `vm.dirty_background_ratio` is adjusted to reduce the proportion of page cache allocated to the playback software.

[0163] Step S142: Send the network transmission priority adjustment scheme in the dynamic resource allocation strategy to the network management module so as to modify the differential service code point value and queue scheduling algorithm of audio and video data packets according to the network transmission priority adjustment scheme.

[0164] In this embodiment, the network management module can receive the adjustment scheme through the Netlink socket and parse the network adjustment items. For the Differential Service Code Point (DSCP) value adjustment, the `setsockopt(sockfd,IPPROTO_IP,IP_TOS,&dscp_value,sizeof(dscp_value))` system call is invoked to increase the DSCP value of audio and video data packets from 0x00 (default) to 0x28 (high priority). For the queue scheduling algorithm, the scheduling algorithm of the network interface is switched from `pfifo_fast` to `htb` (HierarchicalTokenBucket) using the `tc` command to allocate a higher bandwidth quota to the audio and video data streams.

[0165] Step S143: After executing the dynamic resource allocation strategy, continuously monitor the playback status data set of the playback device to obtain new bitstream parameters, hardware resource usage data, and network transmission status parameters.

[0166] In this embodiment, the monitoring process is basically the same as the data acquisition process in step S110, and will not be described again here.

[0167] Step S144: Input the new playback status data set into the playback load association graph, and update the node values ​​and edge association coefficients.

[0168] In this embodiment, the current value Vcurrent of the node is updated according to the new parameter value, the correlation coefficient r of the parameter pair is recalculated (using the method in step S122), and the correlation coefficient and weight of the edge are updated. For example, if the correlation coefficient between the bitrate C3 and the CPU process utilization rate H1 changes from 0.75 to 0.65 after the bitrate C3 decreases, then the correlation coefficient of the edge Edge_C3_H1 is updated to 0.65, and the edge weight is updated to 0.65×K.

[0169] Step S145: Re-identify abnormal nodes. If abnormal nodes still exist, recalculate the target influence path and target adjustment object based on the updated map.

[0170] In this embodiment, steps S131-S134 are repeated to identify anomalous nodes and target adjustment objects based on the updated graph. If the number of anomalous nodes decreases or the anomalousness level decreases, the current strategy continues to be executed; if the anomalous nodes remain unchanged or worsen, an adjustment plan is regenerated.

[0171] Step S146: Adjust the resource allocation ratio and transmission priority in the dynamic resource allocation strategy, and repeat steps S141-S145 until the value of the abnormal node returns to the normal range.

[0172] In this embodiment, the hardware resource allocation ratio and network transmission priority are adjusted according to the new target adjustment object and the degree of anomaly. For example, the reduction ratio of CPU process utilization is increased from 6% to 8%. The dynamic resource allocation strategy is regenerated and executed. The monitoring and adjustment process is repeated until the values ​​of all nodes are within the calibration range.

[0173] like Figure 3 The diagram shown is a schematic of an intelligent stuttering prevention and optimization system for audio and video playback provided in an embodiment of this application. The intelligent stuttering prevention and optimization system includes components such as a processor, a machine-readable storage medium, and input / output devices. The machine-readable storage medium is connected to the processor and is used to store programs, instructions, or code. The processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the aforementioned intelligent stuttering prevention and optimization method for audio and video playback. The intelligent switching optimization system for audio and video scenes can be... Figure 2 The audio and video content provided in this embodiment may be a platform or a component thereof, but this embodiment does not impose specific limitations.

[0174] The machine-readable storage medium may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium is used to store a program, which the processor executes upon receiving an execution instruction.

[0175] The processor may be an integrated circuit chip with signal processing capabilities. The processor mentioned above can be, but is not limited to, a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc.

[0176] In summary, the intelligent stuttering prevention and optimization method and system for audio and video playback provided in this application achieves quantitative modeling of the multi-parameter correlation relationships during audio and video playback by constructing a playback load correlation graph, avoiding the limitations of single-dimensional optimization in existing technologies. This graph abstracts bitstream parameters, hardware resource usage data, and network transmission status parameters into nodes, and characterizes the correlation strength between parameters through edge weights, presenting the direct and indirect influence relationships between parameters in a visual path format. For example, when network latency increases, the graph can quickly locate its indirect impact on CPU decoding time through path weight calculation, thereby providing a global decision-making basis for resource allocation. This correlation modeling method enables dynamic resource allocation strategies to comprehensively consider the coupling effects of multiple factors, avoiding optimization imbalances caused by isolated parameter adjustments, and improving the accuracy and foresight of stuttering prevention. Furthermore, the generation and execution process of the dynamic resource allocation strategy realizes a paradigm shift from "passive response" to "active prevention." By merging and updating the correlation graph in real-time monitoring of playback status datasets, this application can identify abnormal nodes and their impact paths before stuttering occurs. For example, when a sudden increase in bitrate causes bandwidth utilization to exceed the normal range, the system can predict the upward trend of GPU rendering thread occupancy through graph path analysis and adjust the memory page cache allocation strategy in advance to alleviate pressure. Compared with the threshold triggering mechanism of existing technologies, this dynamic adjustment process has higher timeliness and adaptability, and can optimize resource allocation ratios and transmission priorities in real time according to different playback scenarios, device states, and network environments, significantly reducing the probability of stuttering. Furthermore, this application ensures the continuous optimization capability of resource allocation strategies through time series correlation analysis and dynamic graph update mechanisms. For example, during long-term video playback, the system can calculate the correlation coefficient sequence of parameter pairs through a sliding time window, capture the dynamic changes in parameter correlation strength, and adjust edge weights and path thresholds accordingly, so that the correlation graph always reflects the true correlation relationship of the current playback state. This dynamic adaptability enables this method to adapt to different encoding formats, resolutions, and network fluctuation scenarios, achieving cross-device and cross-environment audio and video smooth playback optimization, fundamentally improving the user experience.

[0177] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The embodiments, implementation methods, and related technical features of this application can be combined and substituted with each other without conflict. The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A smart method for preventing and optimizing audio and video playback stuttering, characterized in that, The method comprises: Real-time collection of a set of playing state data of a playing device in an audio and video playing process, the set of playing state data comprising a code stream parameter of a current playing progress, device hardware resource occupation data and network transmission state parameter; Construction of a playing load association graph based on the set of playing state data, the playing load association graph being used to describe mutual influence relationships between the code stream parameter, the device hardware resource occupation data and the network transmission state parameter, comprising nodes, edges and edge weights, the nodes corresponding to parameter types, the edges corresponding to parameter association relationships, and the edge weights corresponding to association strengths; Generation of a dynamic resource allocation strategy according to the playing load association graph, the dynamic resource allocation strategy being used to adjust a hardware resource allocation proportion and a network transmission priority; Execution of the dynamic resource allocation strategy, continuous monitoring of changes in the set of playing state data, and adjustment of the dynamic resource allocation strategy according to the changes; The construction of the playing load association graph based on the set of playing state data comprises: Two-by-two combination of the code stream parameter, the device hardware resource occupation data and the network transmission state parameter in the set of playing state data to form a set of parameter pairs, the set of parameter pairs comprising a plurality of parameter pairs; Time series correlation analysis of each parameter pair to calculate association coefficients of each parameter pair at consecutive time stamps, the association coefficients reflecting linear association strengths between parameters, specifically comprising: setting a sliding time window for each parameter pair, the length of the sliding time window being adjusted according to a frame rate of the audio and video; filling parameter pair values in the set of playing state data into the sliding time window in time stamp order to form time series data; performing detrend processing on the time series data in the sliding time window to eliminate linear trend components in the data; calculating covariance of the time series data after the detrend processing and standard deviations of each time series data; obtaining the association coefficient of the parameter pair in the sliding time window according to the quotient of the covariance divided by the product of two standard deviations; repeating the above steps in the order of the sliding time window to obtain an association coefficient sequence of each parameter pair at all consecutive time stamps; Determining a target parameter pair from parameter pairs whose absolute values of association coefficients exceed a set association strength threshold. The code stream parameter, the device hardware resource occupation data and the network transmission state parameter are taken as nodes of a playing load association graph, and attributes are set for each node, specifically including: collecting historical data of the code stream parameter, the device hardware resource occupation data and the network transmission state parameter of a plurality of playing devices during normal playing of audio and video; statistically analyzing the historical data to calculate the average value, the standard deviation and the statistical confidence interval of each parameter; taking the upper and lower limits of the statistical confidence interval as the calibrated value range during normal playing of the parameter; setting attributes for each node, including the parameter name, the parameter unit and the calibrated value range; continuously monitoring the values of the code stream parameter, the device hardware resource occupation data and the network transmission state parameter during playing of audio and video, periodically updating the historical data, and adjusting the upper and lower limits of the calibrated value range according to the updated historical data; The target parameter pair is taken as an edge of the playing load association graph, and attributes are set for each edge; An edge weight is calculated according to the association coefficient of the edge, the edge weight being the absolute value of the association coefficient multiplied by a preset scaling factor, and the scaling factor being preset according to the smoothness requirement of playing of audio and video; A playing load association graph containing a node set, an edge set and an edge weight is constructed.

2. The method for smart stall prevention optimization for audio-video playback of claim 1, wherein, The dynamic resource allocation strategy is generated according to the playing load association graph, including: All nodes in the playing load association graph are traversed to identify abnormal nodes whose values exceed the calibrated value range; Starting from the abnormal nodes, edges in the playing load association graph are traversed to extract all paths associated with the abnormal nodes, forming an associated path set; The edge weights in each associated path are multiplied to obtain a path weight, and the path weight reflects the influence degree of the abnormal node on other nodes through the associated path; Associated paths with a path weight exceeding a preset path threshold are determined as target influence paths, and end nodes in the target influence paths are extracted as target adjustment objects; For hardware resource class nodes in the target adjustment objects, a resource proportion to be adjusted is calculated according to the abnormal degree of the hardware resource class nodes, a hardware resource allocation proportion adjustment scheme is generated, the hardware resource class nodes include CPU process occupation rate and GPU rendering thread occupation rate, and the abnormal degree represents the deviation of the current value from the normal range; For network transmission class nodes in the target adjustment objects, a transmission priority to be adjusted is calculated according to the abnormal degree of the network transmission class nodes, a network transmission priority adjustment scheme is generated, and the network transmission class nodes include packet loss rate and delay time; The hardware resource allocation proportion adjustment scheme and the network transmission priority adjustment scheme are fused to form a dynamic resource allocation strategy.

3. The method for smart stall prevention optimization for audio-video playback of claim 2, wherein, The hardware resource allocation proportion adjustment scheme and the network transmission priority adjustment scheme are fused to form a dynamic resource allocation strategy, including: All hardware adjustment items in the hardware resource allocation proportion adjustment scheme are extracted, and each hardware adjustment item contains the node name of the target hardware resource class node, the adjustment direction and the adjustment amplitude; extracting all network adjustment items in the network transmission priority adjustment scheme, each network adjustment item containing a node name of a target network transmission class node, an adjustment direction and an adjustment amplitude; pairing the target hardware resource class node of each hardware adjustment item with the target network transmission class node of each network adjustment item to form an adjustment item pair set; for each adjustment item pair, obtaining an edge weight connecting the two target nodes from the playback load association graph; calculating a synergy coefficient of the adjustment item pair according to the obtained edge weight; screening adjustment item pairs with positive synergy coefficients and absolute values exceeding a set synergy threshold as synergy adjustment combinations with synergistic effects; retaining the adjustment direction and the adjustment amplitude of the hardware adjustment item and the network adjustment item in each synergy adjustment combination; traversing all adjustment items to check conflicts between the adjustment items and perform conflict processing, wherein the adjustment items include the hardware adjustment items and the network adjustment items; integrating the synergy adjustment combinations and the adjustment items after conflict processing into an adjustment item list in order of the synergy coefficients from high to low; adding an execution order label to each adjustment item in the adjustment item list, the execution order being determined according to the centrality of the target node corresponding to the adjustment item in the playback load association graph; generating a dynamic resource allocation strategy containing hardware resource allocation proportion adjustment content, network transmission priority adjustment content and execution order according to the adjustment item list and the execution order label, each adjustment item in the dynamic resource allocation strategy including a target node, an adjustment direction, an adjustment amplitude and an execution order.

4. The method for smart stall prevention optimization for audio-video playback of claim 2, wherein, The execution of the dynamic resource allocation strategy, continuous monitoring of the changes in the playback state data set, and adjustment of the dynamic resource allocation strategy according to the changes, include: sending the hardware resource allocation proportion adjustment scheme in the dynamic resource allocation strategy to the resource management module of the playback device, and the resource management module modifying the process priority of the CPU, the rendering queue scheduling rule of the GPU and the page cache allocation strategy of the memory according to the hardware resource allocation proportion adjustment scheme; sending the network transmission priority adjustment scheme in the dynamic resource allocation strategy to the network management module to modify the differentiated services code point value and the queue scheduling algorithm of the audio and video data packets according to the network transmission priority adjustment scheme; after executing the dynamic resource allocation strategy, continuously monitoring the playback state data set of the playback device to obtain new stream parameters, device hardware resource occupation data and network transmission state parameters; inputting the new playback state data set into the playback load association graph to update the values of the nodes and the association coefficients of the edges; re-identifying abnormal nodes, and if abnormal nodes still exist, recalculating the target influence path and the target adjustment object according to the updated graph; adjusting the resource allocation proportion and the transmission priority in the dynamic resource allocation strategy, and repeating the above steps until the values of the abnormal nodes return to the normal range.

5. The method for smart stall prevention optimization for audio-video playback of claim 1, wherein, The real-time collection of the playback state data set of the playback device during audio and video playback includes: real-time acquisition of stream parameters of the current playback progress output by the audio and video playback software, the stream parameters containing the encoding format, resolution and code rate of the current frame; Real-time reading of device hardware resource occupation data provided by the device operating system, the device hardware resource occupation data including CPU process occupation rate, GPU rendering thread occupation rate, and application program memory occupation amount; Real-time collection of network transmission state parameters of the network transmission layer, the network transmission state parameters including current transmission link packet loss rate, delay time, and bandwidth utilization rate; Aligning the code stream parameters, the device hardware resource occupation data, and the network transmission state parameters according to the same time stamp to form the playback state data set.

6. The intelligent stutter prevention optimization method for audio and video playback according to claim 5, characterized in that: The real-time acquisition of the current playback progress code stream parameters output by the audio and video playback software includes: Real-time monitoring of the playback progress callback interface of the audio and video playback software to acquire the current frame sequence number and the corresponding time stamp; locating the encoding metadata of the current frame from the index segment of the audio and video file according to the frame sequence number, extracting the encoding format, which includes the encoding standard and the intraframe and interframe compression mode; reading the resolution of the current frame, which includes the horizontal pixel number, the vertical pixel number, and the pixel bit depth; calculating the code rate of the current frame, which is the ratio of the original data size to the compressed data size multiplied by the playback frame rate; combining the time stamp of the current playback progress to associate the encoding format, the resolution, and the code rate as the code stream parameters of the current playback progress; wherein the intraframe and interframe compression mode is determined by the frame type flag bit, the flag bit being 0 for intraframe compression, 1 for interframe compression, and 2 for bidirectional prediction compression; The real-time reading of the device hardware resource occupation data provided by the device operating system includes: Calling the performance monitoring interface of the device operating system to acquire the total CPU occupation rate, the CPU time slice allocation of each process, and the process identifier; screening the target process identifier corresponding to the audio and video playback software to calculate the proportion of the CPU time slice of the target process in the total CPU time slice as the CPU process occupation rate; calling the performance counter interface of the GPU driver to acquire the total rendering thread number, the idle rendering thread number, and the rendering thread identifier occupied by the audio and video playback software; calculating the proportion of the rendering thread number occupied by the audio and video playback software in the total rendering thread number as the GPU rendering thread occupation rate; calling the memory management interface to acquire the total physical memory amount, the used memory amount, and the memory mapping file size of the audio and video playback software of the playback device; calculating the proportion of the memory mapping file size of the audio and video playback software in the total physical memory amount as the application program memory occupation amount; combining the CPU process occupation rate, the GPU rendering thread occupation rate, and the application program memory occupation amount into the device hardware resource occupation data; The real-time collection of the network transmission state parameters of the network transmission layer includes: The network data packet of the audio and video data transmission is acquired, the source address, the destination address, the sending time stamp and the serial number of each data packet are recorded, the network data packet includes at least one of UDP data packet and TCP data packet; the confirmation data packet returned by the opposite end is received, the serial numbers of the sending data packet and the confirmation data packet are matched, the number of the data packets not received is counted; the proportion of the number of the data packets not received in the total number of the sending data packets is calculated as the packet loss rate of the current transmission link; the difference between the receiving time stamp of each confirmation data packet and the sending time stamp of the corresponding sending data packet is calculated to obtain the delay time of the current transmission link; the maximum bandwidth capacity of the current transmission link and the actual transmission rate of the current audio and video data are acquired, the maximum bandwidth capacity is acquired through the link negotiation protocol; the proportion of the actual transmission rate in the maximum bandwidth capacity is calculated as the bandwidth utilization rate of the current transmission link; the packet loss rate, the delay time and the bandwidth utilization rate are combined into the network transmission state parameter.

7. A smart stuttering prevention and optimization system for audio and video playback, characterized in that, The machine readable storage medium is used for storing programs, instructions or codes, and the processor is used for executing the programs, instructions or codes in the machine readable storage medium to realize the intelligent stall prevention optimization method for audio and video playing according to any one of claims 1-6.

Images