Frame data processing method

Abstract

The invention belongs to the field of data processing, and relates to a frame data processing method, which comprises the following steps of: performing priority marking processing on input frame data, and binding a coherence weight tag obtained based on frame content relevance calculation for each piece of frame data so as to generate frame data with coherence weight tags; performing lock-free buffer storage processing on the frame data with the coherence weight label, recording the quantity of the frame data in the current buffer area, and dynamically adjusting the capacity of the buffer area to generate the frame data with the coherence weight label stored in a lock-free frame buffer area; and carrying out buffer area state monitoring processing on the frame data with the coherence weight tag stored in the lockless frame buffer area, and obtaining the adjusted frame data with the coherence weight tag based on a buffer area monitoring result. According to the invention, the video playing fluency can be improved, and the video playing delay can be reduced.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more particularly to a method for processing frame data. Background Technology

[0002] In Qt client applications, video players need to continuously process and render frame data to ensure smooth playback. Especially in scenarios such as real-time video surveillance and high-definition video on demand, where there are clear requirements for image continuity and low latency, efficient buffering and cross-thread transmission of frame data directly affect the user experience. How to stably process frame data when the main thread load fluctuates has become one of the core requirements of video playback technology in the Qt environment.

[0003] In existing technologies, the frame data processing for video playback in Qt clients often adopts a "static buffer + thread lock synchronization" approach: a fixed-size buffer is used to temporarily store the decoded frame data, a lock class is used to control the access permissions of the child thread and the main thread to the buffer, the discard object is determined only based on the data storage time order of the frame data, and the frame data is passed to the rendering thread through a single type of signal slot connection.

[0004] However, in the above scheme, the static buffer cannot dynamically adjust its storage capacity according to the main thread load. When the main thread load increases, the buffer is full and frame data is dropped out of order, which destroys the continuity of the picture. At the same time, the thread lock synchronization will cause the main thread to be blocked frequently while waiting for the lock to be released, which will prolong the time for frame data transmission. Furthermore, the lack of priority processing for frame data will further aggravate the rendering latency and make it difficult to meet the requirements of smooth video playback and low latency. Summary of the Invention

[0005] This application provides a method for processing frame data, which aims to improve the smoothness of video playback and reduce the latency of video playback.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions: A method for processing frame data includes: Priority marking is performed on the input frame data, and a coherence weight label calculated based on the correlation of frame content is bound to each frame data to generate frame data with coherence weight labels. The frame data with coherent weight labels is stored in a lock-free buffer. The number of frame data in the current buffer is recorded, and the buffer capacity is dynamically adjusted to generate frame data with coherent weight labels stored in the lock-free frame buffer. The frame data with coherence weight labels stored in the lock-free frame buffer is subjected to buffer state monitoring processing, and the adjusted frame data with coherence weight labels is obtained based on the buffer monitoring results.

[0007] The beneficial effects of this application are as follows: 1. Traditional solutions use static buffers with fixed capacities, which cannot adapt to fluctuations in the main thread load. When a full frame is reached, frame data is discarded only in the order of storage time, easily disrupting the continuity of the video. This application addresses this by combining "binding a continuity weight label based on content relevance to frame data" with "adjusting frame data based on buffer status monitoring results": first, it prioritizes frame data based on the relevance of frame content (rather than just storage time), and then adjusts frame data retention accordingly based on the buffer status, minimizing the drawbacks of traditional disordered discarding; simultaneously, "dynamically adjusting the buffer capacity" can flexibly adapt to storage capacity according to the main thread load, reducing the frequency of frame dropping when the buffer is full, thereby improving the continuity of the video and meeting the requirements for continuity in real-time video surveillance and high-definition video-on-demand in the background technology.

[0008] 2. Traditional solutions rely on lock mechanisms for thread synchronization. The main thread must wait for the lock to be released before accessing the buffer, which can easily lead to blocking and increase frame data transmission latency. This application, "performing lock-free buffered storage processing of frame data with coherence weight labels," abandons the traditional lock synchronization method, reducing the probability of the main thread being blocked due to waiting for the lock. At the same time, "recording the current number of frame data in the buffer" can provide real-time feedback on the buffer load, providing a basis for dynamically adjusting the capacity. This minimizes the frequent thread interactions and waiting caused by the fixed buffer capacity in traditional methods, thereby reducing the time spent transmitting frame data from the buffer to subsequent processing stages and alleviating the rendering latency problem in the background technology.

[0009] 3. Traditional solutions do not prioritize frame data and process all frame data using a single logic, making it difficult to adapt to the varying impacts of different frame data on the continuity of the scene. This application "binds a continuity weight tag based on content relevance to frame data," giving frame data a clear priority attribute (instead of the traditional indiscriminate processing). Subsequent lock-free storage and buffer state adjustments are all based on frame data with priority tags. During buffer adjustment, frame data that has a greater impact on the continuity of the scene can be prioritized and retained, reducing the latency of high-priority frame data caused by low-priority frame data occupying resources. Compared with the traditional no-priority processing method, it is more adaptable to the efficient frame data processing needs in scenarios with fluctuating main thread load in the background technology, further alleviating the rendering latency problem. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the overall process of a frame data processing method provided in an embodiment of this application; Figure 2 This is another overall flowchart illustrating a frame data processing method provided in an embodiment of this application; Detailed Implementation The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] Terminology Explanation: 1. `std::atomic`: This is a template class provided by the C++ standard library for implementing atomic operations. Atomic operations mean that the operation will not be interrupted during execution, ensuring data consistency and integrity when multiple threads access it concurrently. For example, `std::atomic` <int>Atomic operations can be performed on integer types. Compared to ordinary variables, using `std::atomic` can avoid data race issues in a multi-threaded environment. In the lock-free framebuffer mechanism of this application, `std::atomic` is used to manage the read and write pointers of the circular queue, enabling child threads and the main thread to interact with data efficiently without locks, thus improving thread synchronization efficiency. 2. `QAtomicInt`: This is a class provided by the Qt framework for performing atomic operations on integers across different platforms. It is similar to `std::atomic`. <int>Similar in function, but better suited to the Qt programming environment. Atomic operations such as atomic reference counting, test-and-set, get-and-store, and get-and-add can all be implemented using QAtomicInt. In this invention, QAtomicInt is used to record the current frame count in the buffer, and combined with std::atomic, it enables dynamic adjustment of the buffer capacity, avoiding the thread blocking problems caused by traditional lock-based mechanisms (such as QMutex), and optimizing the multi-threaded processing of frame data in the Qt environment.

[0013] 3. QMutex: Qt's QMutex class is used for inter-thread access serialization. Its purpose is to protect objects, data structures, or code segments, ensuring that only one thread can access them at a time. For example, when a thread calls the QMutex's lock() function to acquire the lock, other threads attempting to call lock() will be blocked until the thread holding the lock calls unlock() to release it. In traditional Qt multi-threaded frame processing, QMutex is commonly relied upon to ensure data access safety, but frequent lock contention increases thread switching overhead and can easily block the main thread. This application's lock-free framebuffering mechanism breaks through the dependence on lock mechanisms such as QMutex, employing a new atomic operation method to achieve thread synchronization and improve overall performance. 4. I-frame (Intra-coded Frame): This is a key frame type in video coding. An I-frame contains complete image information and can be decoded independently without relying on other frames, making it the foundation of video decoding. During video playback, if an I-frame is lost, subsequent frames that depend on it (such as P-frames) may not be correctly decoded or rendered, leading to serious errors in the image, such as "image tearing." In the priority marking and scheduling scheme for fusion frame content coherence in this invention, the "coherence weight" of the I-frame is set to the highest (10), and a "dependency chain weight bonus" is added to its subsequent associated P-frames to ensure the integrity and priority of the I-frame and its related frames during the rendering process. 5. P-frame (Predicted Frame): A predicted frame is a frame obtained through predictive coding based on previous frames (usually I-frames or other P-frames) in video coding. A P-frame only stores the difference information from the reference frame, reconstructing the complete image by referring to previous frames. Although the P-frame has been decoded, during the rendering stage, it still relies on the nearest I-frame or reference frame to generate complete pixel information for correct rendering. If the I-frame it depends on is lost, local pixel errors may occur during P-frame rendering, manifesting as "screen tearing" in fast-moving scenes. This application, through a priority marking and scheduling scheme, ensures the priority preservation of I-frames while also considering the dependency relationship between P-frames and I-frames, maintaining the rendering effectiveness of associated P-frames.

[0014] 6. Qt Meta-Object System: This is a core feature of the Qt framework, providing a signal-slot mechanism for inter-object communication, runtime type information, and a dynamic property system. In this application, the Qt Meta-Object System is used to bind a custom attribute, "coherence weight," to frame data. This ensures that frame priority information is not lost during signal-slot transmission, thereby guaranteeing the effectiveness of priority scheduling throughout the rendering process. Furthermore, the system provides a flexible adaptation scheme for the signal-slot connection type of high-priority frames, further optimizing rendering efficiency.

[0015] 7. Circular Queue: A special data structure whose storage space is used cyclically. In the lock-free framebuffering mechanism of this application, a circular queue is used to store frame data. Its read and write pointers are managed through std::atomic atomic variables, achieving lock-free read and write operations. When the write pointer reaches the end of the queue, it automatically returns to the beginning of the queue to continue writing; the read pointer does the same. This structure can efficiently utilize storage space, avoid frequent memory allocation and deallocation, and, combined with the lock-free mechanism, greatly improves the efficiency of data interaction between threads and reduces the performance bottleneck caused by traditional queue structures. 8. Dependency Chain Weighting: This is a weighting strategy proposed in this application. In a video frame sequence, P-frames following an I-frame are dependent on the I-frame. The closer a P-frame is to the I-frame, the higher its dependence on the I-frame, and the more important it is in the rendering process. Therefore, this application adds a "dependency chain weighting" to the P-frames subsequently associated with the I-frame. For example, the first P-frame after the I-frame has a 20% weighting, the second P-frame has a 15% weighting, and so on. The specific weighting percentage is determined based on actual testing and optimization. In this way, when the buffer is full and frames need to be discarded, P-frames closely associated with the I-frame can be retained first, ensuring the continuity of video rendering. 9. `Qt::QueuedConnection`: This is a connection type in Qt's signal and slot mechanism. When using `Qt::QueuedConnection` to connect signals and slots, the slot function does not execute immediately after the signal is emitted. Instead, the event is placed in the event queue of the receiving thread, waiting for that thread's event loop to process it. This connection type ensures the safety of cross-thread signal transmission. In this application, `Qt::QueuedConnection` is used by default for signal and slot connections in all frames, ensuring stable signal transmission in a multi-threaded environment.

[0016] 10. Qt::DirectConnection: This is also a connection type in Qt's signal and slot mechanism. Unlike Qt::QueuedConnection, when using Qt::DirectConnection to connect a signal and a slot, the slot function is executed immediately in the thread that sent the signal after the signal is emitted. In this application, both Qt::QueuedConnection and Qt::DirectConnection are bound to high-weight frames (weight ≥ 8). When crossing threads, Qt::QueuedConnection is used to ensure safety; when the main thread is idle, it automatically switches to Qt::DirectConnection to directly call the slot function, reducing the queue waiting time of high-priority frames and optimizing the processing efficiency of high-priority frames in the signal and slot mechanism.

[0017] Reference Figure 1 and Figure 2 This application provides a method for processing frame data, including: S101. Perform priority marking processing on the input frame data, and bind a coherence weight label calculated based on the correlation of frame content to each frame data to generate frame data with coherence weight labels. S102. Perform lock-free buffer storage processing on the frame data with coherent weight labels, record the number of frame data in the current buffer, and dynamically adjust the buffer capacity to generate frame data with coherent weight labels stored in the lock-free frame buffer. S103. Perform buffer state monitoring on the frame data with coherence weight labels stored in the lock-free frame buffer, and obtain the adjusted frame data with coherence weight labels based on the buffer monitoring results.

[0018] Specifically, firstly, taking the decoded frame data from the input Qt client video playback scenario as the processing object, frame type identification processing is performed on the frame data to distinguish between intra-coded frames (I-frames) and predicted frames. The process involves generating frame data with frame type identifiers (e.g., P-frames). Next, using this frame data, the pixel differences between consecutive frames are calculated (e.g., similarity is determined by the sum of grayscale deviations of corresponding pixels in adjacent frames). This determines whether the predicted frames belong to consecutive similar predicted frames, generating frame data with similarity judgment results. Then, using this frame data with similarity judgment results, a coherence weight calculation logic is constructed by combining the frame type identifier and the similarity judgment results. This logic sets the coherence weight label value of the intra-coded frames to the highest reference value in the scene, decreases the coherence weight label value of consecutive similar predicted frames sequentially according to the order of the predicted frames, and adds a dependency chain weight bonus to the subsequent predicted frames associated with the intra-coded frames (the dependency chain weight bonus decreases as the distance between the predicted frame and the intra-coded frame increases). This logic quantifies the content correlation of the frame data, generating frame data with coherence weight values. Finally, using this frame data with coherence weight values, the Qt meta-object system (Qt...) is used to... The Meta-Object System binds coherence weight values ​​to the structure of frame data, creating a one-to-one correspondence between coherence weight labels and frame data. This ensures that weight attributes are not lost in subsequent processing steps, generating frame data with coherence weight labels. This frame data with coherence weight labels will be used as the processing object in subsequent steps for lock-free buffer storage processing.

[0019] This solution differs from traditional solutions that "process frame data without prioritization and only based on storage time." Traditional solutions do not consider the impact of the correlation between frame data content on the continuity of the image, and key frames are easily discarded when the frame is full, resulting in image gaps. This application, however, uses a progressive processing of "frame type identification - similarity judgment - weight quantization - attribute binding" to give frame data a priority identifier based on content value. This provides a basis for "prioritizing the retention of high-value frames" when adjusting the buffer in the future, which meets the core requirement of the continuity of the image in Qt video playback scenarios and solves the problem of disordered frame data processing in traditional solutions.

[0020] Then, using the generated frame data with consistent weight labels as the processing object, the storage structure of the frame data is adapted based on the atomic operation mechanism—the storage format of the frame data is adjusted to adapt to the data structure of a circular queue (e.g., encapsulated in the field order of "frame identifier-weight label-pixel data"), to generate frame data with consistent weight labels that are adapted to the storage of the circular queue. Next, using the frame data with consistent weight labels that are adapted to the storage of the circular queue as the processing object, the read pointer and write pointer of the circular queue are synchronized using the atomic operation template class (std::atomic) provided by the C++ standard library—the write pointer is only updated when the child thread (frame input thread) writes frame data to the queue, and the read pointer is only updated when the main thread (rendering thread) reads frame data from the queue. The update avoids read / write pointer conflicts by leveraging the uninterruptible nature of atomic variables to generate frame data with coherent weight tags after pointer synchronization. Then, using this synchronized frame data with coherent weight tags as the processing object, the Qt framework's class (QAtomicInt) for performing integer atomic operations is used to perform real-time statistical processing on the number of frame data currently stored in the circular queue—incrementing the atomic integer value by 1 for each frame data written and decrementing it by 1 for each frame data read, to generate frame data with coherent weight tags and real-time count records. Subsequently, the data is processed using the synchronized frame data with coherent weight tags. The processing object is the frame data with sequential weight labels recorded in the time-to-time count. Capacity adjustment logic is set based on the frame data count information recorded by atomic integers: when the frame data count consistently exceeds the high-load threshold of the scene (e.g., the threshold can be set to 15 frames under normal load and 20 frames under high load), buffer capacity expansion is triggered (the expansion amount is determined based on the current frame data growth rate, e.g., if it increases by 3 frames every 50 milliseconds, then it expands by 5 frames); when the frame data count consistently falls below the low-load threshold of the scene (e.g., the threshold can be set to 5 frames under normal load), buffer capacity reduction is triggered (reduction). The storage capacity of the circular queue is dynamically adjusted using this logic (with the amplitude not less than the current frame data quantity) to generate frame data with coherent weight tags after capacity adaptation. Finally, the frame data with coherent weight tags after capacity adaptation is stored in a lock-free frame buffer built based on the circular queue, completing the lock-free buffer storage of the frame data, so as to generate frame data with coherent weight tags stored in the lock-free frame buffer. The frame data with coherent weight tags stored in the lock-free frame buffer will be used as the object of subsequent processing for subsequent buffer status monitoring.

[0021] Unlike traditional solutions that rely on a static buffer and lock mechanism for synchronization, this application achieves lock-free data interaction between the main thread and the child thread through a collaborative design of "atomic variable management of pointers - atomic integer counting of quantities - dynamic capacity adjustment," reducing the probability of main thread blocking. Furthermore, the fixed-capacity buffer cannot adapt to fluctuations in main thread load, leading to frequent frame drops when the frame is full.

[0022] Next, using the frame data with coherent weight tags stored in the unlocked frame buffer as the processing object, the current capacity of the unlocked frame buffer and the real-time frame data quantity are compared and monitored. This is done by reading the frame data quantity recorded in QAtomicInt and calculating the difference with the current dynamic capacity threshold of the buffer to generate a buffer status monitoring result (the buffer status monitoring result includes two types: "buffer full" and "buffer not full." When the frame data quantity is ≥ the current buffer capacity threshold, it is determined to be "buffer full," and when the frame data quantity is < the current buffer capacity threshold, it is determined to be "buffer not full"). Then, if the buffer status monitoring result is "buffer not full," the frame data with coherent weight tags stored in the unlocked frame buffer is directly used as the adjusted frame data with coherent weight tags stored in the unlocked frame buffer. This data will be used for subsequent Qt signal-slot adaptation processing. If the buffer status monitoring result is... If the buffer is full, the frame data with coherence weight tags stored in the unlocked frame buffer will be processed, and interval discarding will be performed: First, the coherence weight tag values ​​of all frame data will be sorted in descending order, and the frame data with the highest coherence weight tag values ​​will be retained (e.g., the first 60% of the frame data). Then, the remaining 40% of the frame data will be filtered at fixed intervals (e.g., every 3 frames as an interval unit) to select the frame data with the lowest coherence weight tag values ​​that are not intra-coded frames. The filtered frame data will be removed from the circular queue through an unlocked atomic exchange operation (e.g., using the std::atomic exchange method) to generate adjusted frame data with coherence weight tags stored in the unlocked frame buffer. This adjusted frame data with coherence weight tags stored in the unlocked frame buffer will be used for subsequent Qt signal and slot adaptation processing, so that the frame data entering the rendering stage can meet the capacity requirements and maintain the coherence of the picture to the greatest extent.

[0023] Unlike traditional solutions that simply discard the newest or oldest frame when the buffer is full (which doesn't consider frame data priority and can easily lead to dropped key frames and screen abruptness), this application uses a "state monitoring - situational adjustment - interval discarding of high-priority frames" process. When the buffer is full, it prioritizes retaining high-weight frames that have a significant impact on screen continuity and only discards low-weight, non-key frames. Furthermore, it avoids the discarding operation affecting the parallelism of read and write threads through lock-free atomic swaps. This solves the technical pain point of "disordered frame dropping disrupting screen continuity" in traditional solutions, making it particularly suitable for Qt applications with high requirements for screen continuity, such as real-time video surveillance and high-definition video on demand.

[0024] Optionally, the frame data with coherence weight labels is subjected to lock-free buffering storage processing, the number of frame data in the current buffer is recorded, and the buffer capacity is dynamically adjusted to generate frame data with coherence weight labels stored in the lock-free frame buffer, including: Based on the atomic operation mechanism, the frame data with coherent weight labels is subjected to lock-free storage structure adaptation processing to generate frame data with coherent weight labels that are adapted to circular queue storage. The frame data with coherent weight tags stored in an adapted circular queue is used as the processing object. The read and write pointers of the circular queue are synchronized through atomic variables to generate frame data with coherent weight tags after pointer synchronization. Based on atomic integers, the frame data with coherent weight labels after pointer synchronization is subjected to statistical processing to generate frame data with coherent weight labels and real-time quantity records. Based on the quantity information in the frame data with coherent weight labels that record real-time quantities, the storage capacity of the circular queue is dynamically adjusted to generate frame data with coherent weight labels that are adapted to the capacity. The capacity-adapted frame data with coherent weight labels is stored in the lock-free frame buffer to generate frame data with coherent weight labels stored in the lock-free frame buffer.

[0025] Specifically, taking the generated frame data with coherence weight labels as the processing object, the storage fields of this frame data are first structurally defined. Considering the rendering requirements of frame data in Qt client video playback scenarios, the frame data is decomposed into four core fields: "unique frame identifier (used to distinguish different frames), coherence weight labels (the priority information generated above), pixel data blocks (image information used for subsequent rendering), and frame timestamps (used to determine the frame playback order)," forming a standardized frame data structure. Then, based on the atomic operation mechanism... A mechanism (referring to operations that are not interrupted by the thread scheduling mechanism, ensuring the integrity of data interaction in a multi-threaded environment) is used to perform storage compatibility adaptation on the standardized frame data structure. This involves adjusting the memory alignment of the structure (e.g., optimizing field arrangement according to the default memory alignment rules of the Qt framework) to avoid data read anomalies caused by memory misalignment when storing in a circular queue. At the same time, atomic operation adaptation flags are added to the structure (used to identify data validity during subsequent read / write pointer synchronization). Finally, frame data with coherence weight tags adapted to circular queue storage is generated. This frame data with coherence weight tags adapted to circular queue storage will be used as the processing object for subsequent steps, for the synchronization management of subsequent circular queue read / write pointers.

[0026] Unlike traditional solutions that directly store raw frame data in a buffer (which fail to structurally adapt the frame data, leading to storage disorder in the circular queue due to inconsistent data formats and neglecting compatibility requirements for atomic operations, resulting in data truncation during subsequent multi-threaded read / write operations), this application employs a progressive processing approach of "structural field definition - memory alignment adaptation - atomic operation marking." This ensures that the frame data adapts to the circular queue storage structure and meets the synchronization requirements of subsequent atomic operations, laying the foundation for lock-free data interaction and resolving the technical problem of poor frame data storage adaptability in traditional solutions.

[0027] Then, using the frame data with sequential weight tags stored in the adapted circular queue as the processing object, the read and write pointers of the circular queue are first initialized—the initial value of the read pointer (used by the main thread to read frame data from the queue) is set to the queue start address, and the initial value of the write pointer (used by the child thread to write frame data to the queue) is also the queue start address. Both pointers are defined as the atomic operation template class (std::atomic, a template class that supports atomic operations and ensures that pointer update operations are not interrupted) provided by the C++ standard library. Next, the synchronization management logic of the read and write pointers is established: when the child thread writes frame data with sequential weight tags stored in the adapted circular queue to the circular queue, the atomic value of the current write pointer is first read through the load() method of std::atomic. After confirming that the queue position pointed to by the pointer is free, the frame data is written to that position, and then the atomic value of the write pointer is read through the load() method of std::atomic. The `tore()` method atomically updates the write pointer (making it point to the next free position in the queue). When the main thread reads frame data from the circular queue, it first reads the atomic value of the current read pointer using the `std::atomic`'s `load()` method. After confirming that there is valid frame data at the queue position pointed to by the pointer (determined by the atomic operation adaptation flag added above), it reads the frame data at that position and then atomically updates the read pointer using the `std::atomic`'s `store()` method (making it point to the next valid data position in the queue). At the same time, pointer boundary detection logic is set—when the read or write pointer reaches the end address of the queue, it automatically jumps to the beginning address of the queue, realizing the circular storage of the circular queue. Through the above synchronization management process, frame data with continuity weight tags after pointer synchronization is generated. This frame data with continuity weight tags after pointer synchronization will be used as the processing object for subsequent steps and for counting the number of subsequent frame data.

[0028] Unlike traditional solutions that rely on QMutex locks for synchronized read / write operations, where the main thread and child threads must compete for lock resources to operate on read / write pointers (e.g., when a child thread holds the lock, the main thread must wait for the lock to be released before reading data), this application directly manages read / write pointers using std::atomic atomic variables, making pointer update operations uninterruptible. Thread synchronization can be achieved without a lock mechanism, reducing the probability of the main thread being blocked while waiting for the lock. Simultaneously, pointer boundary detection ensures the circular queue is used cyclically, improving queue storage efficiency and enhancing the efficiency of multi-threaded frame data interaction in Qt scenarios.

[0029] Next, using the frame data with coherent weight tags generated above after pointer synchronization as the processing object, first initialize the variable of the class (QAtomicInt, an atomic integer class adapted to the Qt environment, supporting atomic increment, decrement, and read operations) provided by the Qt framework for performing atomic integer operations. Set the initial value of this variable to 0 to record the total number of frame data currently stored in the circular queue. Then, establish the linkage statistical logic between the number of frame data and read / write operations: when the child thread writes a frame data with coherent weight tags after pointer synchronization to the circular queue through the above write pointer synchronization operation, immediately call the fetchAndAddRelaxed(1) method of QAtomicInt to atomically increment the atomic integer value by 1, completing the update of the number after the frame data is written. When the main thread reads a frame of data with a coherent weight tag after pointer synchronization from the circular queue through the above-mentioned read pointer synchronization operation, it immediately calls the fetchAndSubRelaxed(1) method of QAtomicInt to atomically decrement the atomic integer value by 1, completing the quantity update after reading the frame data; at the same time, it sets the quantity reading interface - obtains the value of the current atomic integer in real time through the load() method of QAtomicInt to ensure that the accurate frame data quantity can be obtained when adjusting the capacity later; through the above quantity statistics processing, a frame of data with a coherent weight tag and real-time quantity record is generated. This frame of data with a coherent weight tag and real-time quantity record will be used as the subsequent processing object for the dynamic adjustment of the storage capacity of the circular queue.

[0030] Unlike traditional methods that count frames using ordinary integer variables, this application employs QAtomicInt for frame counting. Traditional methods lack atomicity in incrementing and decrementing ordinary integer variables, leading to counting discrepancies when multiple threads read and write simultaneously (e.g., simultaneous incrementing by a child thread and decrementing by the main thread might result in a count that is one less than expected). This application uses QAtomicInt to perform the count, ensuring atomic execution of both increment and decrement operations. This maximizes the accuracy of frame data counting and directly links the count to read and write operations, avoiding the delay caused by the traditional method of "reading and writing data first, then updating the count." This provides a reliable quantitative basis for subsequent dynamic adjustment of the buffer capacity and solves the problem of inaccurate frame data counting in traditional methods.

[0031] Next, using the generated frame data with real-time count records and coherence weight labels as the processing object, the real-time frame data count information recorded by QAtomicInt in the frame data is extracted first. Simultaneously, considering the load characteristics of the Qt client video playback scenario, basic parameters for capacity adjustment are set: including a basic capacity threshold (the minimum capacity to ensure normal playback in the scenario, such as 5 frames in a normal scenario), a high-load capacity threshold (the maximum capacity when the main thread is under high load, such as 20 frames), and a capacity adjustment step size (the number of frames to expand or shrink each time, such as 1-3 frames). Then, dynamic capacity adjustment logic is established: when the real-time frame data count is continuously (e.g., continuously for 50 milliseconds) higher than 80% of the high-load capacity threshold (e.g., continuously higher than 16 frames when the high-load threshold is 20 frames), the current buffer is determined to be in a high-load state, triggering expansion processing—increasing the storage capacity of the circular queue by the set step size until the capacity reaches the set threshold. When the high load capacity threshold is reached, or the number of real-time frame data drops below 80% of the high load capacity threshold; when the number of real-time frame data is continuously (e.g., continuously for 50 milliseconds) below 120% of the basic capacity threshold (e.g., continuously below 6 frames when the basic threshold is 5 frames), it is determined that the current buffer is in a low load state, triggering a shrinkage process—reducing the storage capacity of the circular queue by a set step size until the capacity drops to the basic capacity threshold, or the number of real-time frame data rises to more than 120% of the basic capacity threshold; at the same time, a capacity adjustment protection mechanism is set—expansion does not exceed the high load capacity threshold, and shrinkage does not fall below the basic capacity threshold, avoiding memory waste caused by excessive capacity or frame data overflow caused by insufficient capacity; through the above dynamic adjustment process, frame data with coherent weight tags after capacity adaptation is generated. This frame data with coherent weight tags after capacity adaptation will be used as the processing object for subsequent steps and used for storage of subsequent lock-free frame buffers.

[0032] Unlike traditional solutions that use a fixed buffer capacity, this application addresses the issue of buffer capacity not adapting to load fluctuations. Traditional solutions, where the buffer capacity is preset and cannot be adjusted, can lead to frame data loss due to insufficient capacity during high main thread loads and memory waste due to excessive capacity during low loads. Instead, this application employs a logic of "real-time quantity monitoring - load status determination - dynamic capacity adjustment - protection mechanism constraints" to flexibly adapt the buffer capacity to frame data loads. This avoids frame data overflow during high loads and reduces memory usage during low loads. Furthermore, by setting basic parameters based on Qt scenario characteristics, the capacity adjustment meets the actual needs of video playback, solving the problem of traditional solutions where buffer capacity cannot adapt to load fluctuations.

[0033] Finally, using the capacity-adapted frame data with coherent weight tags generated above as the processing object, the lock-free frame buffer is first initialized—based on the adjusted circular queue capacity, a corresponding memory space is allocated to the buffer (the memory space size is calculated as "single frame data volume × circular queue capacity"), and the buffer's status flag is set to "available" for subsequent buffer status monitoring to identify whether the buffer is functioning correctly. Next, the mapping storage logic between frame data and the buffer is established: according to the synchronized write pointer position, the capacity-adapted frame data with coherent weight tags is written one by one to the corresponding memory address of the lock-free frame buffer. During the writing process, the added unique frame identifier ensures the frame count. The data is stored without duplication, and the number of frame data in the buffer is updated in real time through the QAtomicInt variable mentioned above to ensure that the number record is consistent with the actual storage. After writing, the stored frame data is subjected to integrity verification processing - by comparing the field length of the frame data with the preset standardized structure length, it is determined whether the frame data is stored completely. If there is incomplete data, it is removed through the read pointer synchronization operation mentioned above to avoid data anomalies during subsequent rendering. Finally, frame data with coherent weight labels is generated and stored in the lock-free frame buffer. The frame data with coherent weight labels stored in the lock-free frame buffer will be used as the processing object for subsequent steps, and is used for subsequent buffer status monitoring and frame data adjustment.

[0034] Unlike traditional solutions that "directly store frame data without integrity verification," where frame data is not verified after being written to the buffer, leading to rendering errors due to storage anomalies and lacking deep integration with lock-free mechanisms, this application ensures the integrity and validity of frame data storage through a "buffer initialization-mapping storage-integrity verification" process. Furthermore, the entire storage operation is based on lock-free logic (atomic pointers, atomic integers) from the preceding steps, eliminating the need for locks and preventing thread blocking during storage. This allows for efficient operation of the lock-free frame buffer, resolving the issues of poor frame data integrity and reliance on locks in traditional solutions.

[0035] Optionally, atomic variables are atomic operation template classes provided by the C++ standard library, and atomic integers are classes provided by the Qt framework for performing atomic integer operations.

[0036] Optionally, the adjusted frame data with coherence weight labels is obtained based on the buffer monitoring results, including: If the buffer monitoring result indicates that the buffer is full, then the frame data with coherent weight labels stored in the unlocked frame buffer are dropped at intervals to generate adjusted frame data with coherent weight labels stored in the unlocked frame buffer. If the buffer monitoring result indicates that the buffer is not full, then the frame data with coherence weight labels stored in the unlocked frame buffer will be used as the adjusted frame data with coherence weight labels stored in the unlocked frame buffer.

[0037] Specifically, taking the buffer status monitoring result (determined as "buffer full") and the frame data with coherence weight tags stored in the unlocked frame buffer as the processing objects, the following steps are performed: First, the frame data with coherence weight tags stored in the unlocked frame buffer is sorted by weight—extracting the coherence weight tag value bound to each frame data and sorting all frame data in descending order of value to generate a frame data queue with weight sorting identifiers; Next, high-value frame filtering is performed based on the frame data queue with weight sorting identifiers—setting a retention ratio threshold (this threshold is determined based on the screen coherence requirements of the Qt client video playback scenario, such as 60% in a normal scenario), retaining the top 60% of high-weight frame data in the frame data queue with weight sorting identifiers, and filtering out the remaining 40% of low-weight frame data to generate a frame data set with high and low weight distinctions; Then, a second frame type filtering process is performed on the low-weight frame data in the frame data set with high and low weight distinctions—extracting the frame type identifier from the low-weight frame data (distinguishing Intra-coded frames). The system first extracts I-frames and predicted frames (P-frames), then removes intra-coded frames, retaining only low-weight predicted frames to generate a list of candidate frames to be discarded. Next, the high-weight frames in the weighted frame data set are merged and sorted with the candidate frame list—the playback order of the merged frames is restored according to their timestamps (used to determine the frame playback order), avoiding frame time order disorder caused by weight sorting, generating a queue of frames to be adjusted with timestamp sorting. Then, interval discarding is performed on this queue—a fixed interval unit is set (e.g., every 3 frames is an interval unit, determined based on the human eye's sensitivity to screen jumps), and one low-weight predicted frame is selected from the candidate frame list in each interval unit. A lock atomic swap operation (using the `exchange` method of the `std::atomic` template class provided by the C++ standard library) removes the frame from the lock-free frame buffer, ensuring that the discard operation does not affect the parallel access of the buffer by the main thread and child threads, generating a queue of frame data with discard marks. Finally, the queue of frame data with discard marks is processed to filter valid frames—all frame data with discard marks are removed, and the remaining high-weight frame data and the unselected low-weight frame data are retained, generating frame data with coherence weight tags that are adjusted and stored in the lock-free frame buffer. This adjusted frame data with coherence weight tags stored in the lock-free frame buffer will be used for subsequent Qt signal and slot adaptation processing, so that the frame data entering the rendering stage can maintain a high degree of image coherence.

[0038] Unlike traditional solutions that simply discard the newest or oldest frame when the buffer is full (traditional solutions do not consider the continuity weight of frame data and frame type when making discard decisions, easily leading to the discarding of high-value intra-coded frames or the continuous discarding of low-weight frames, causing visual abruptness), this application uses a progressive processing of "weight sorting - proportional filtering - type elimination - interval discarding." This prioritizes the retention of high-weight frames and intra-coded frames that have a greater impact on visual continuity, while avoiding visual breaks caused by continuous frame dropping through interval discarding. At the same time, lock-free atomic swaps ensure that the discarding operation does not block the thread, solving the problem of disordered frame dropping disrupting visual continuity in traditional solutions. This makes it particularly suitable for Qt applications with high requirements for visual continuity, such as real-time video surveillance and high-definition video on demand.

[0039] Then, using the buffer status monitoring result (determined as "buffer not full") and the frame data with coherent weight tags stored in the unlocked frame buffer as the processing objects, the frame data with coherent weight tags stored in the unlocked frame buffer is first subjected to integrity verification processing—extracting the frame unique identifier and field structure of each frame data, comparing the actual field length of the frame data with the preset standardized structure length (the "frame unique identifier-coherent weight tag-pixel data block-frame timestamp" structure defined above), determining whether the frame data is stored completely, and generating a frame data set with integrity verification results; next, invalid frame removal processing is performed on the frame data set with integrity verification results—removing frame data that fails integrity verification (such as frame data with missing fields or truncated data) to avoid screen anomalies during subsequent rendering, generating a frame data queue with valid frame identifiers; then, the frame data queue with valid frame identifiers is sorted by timestamp—by frame. The storage order of frame data is adjusted using timestamps to ensure that the frame data is arranged in the unlocked frame buffer in chronological order of playback time, avoiding rendering frame order errors caused by disordered storage order, and generating a valid frame data queue sorted by timestamps. Subsequently, the valid frame data queue sorted by timestamps undergoes buffer capacity adaptation confirmation processing—the storage capacity of the dynamically adjusted circular queue mentioned above is read, the number of frame data in the valid frame data queue sorted by timestamps is counted, and it is confirmed that this number is lower than the current buffer capacity threshold, ensuring that the buffer still has space to store subsequent frame data, and a frame data queue with a capacity adaptation confirmation flag is generated. Finally, the frame data queue with the capacity adaptation confirmation flag is directly used as the adjusted frame data with coherence weight tags stored in the unlocked frame buffer. This adjusted frame data with coherence weight tags stored in the unlocked frame buffer will be used for subsequent Qt signal slot adaptation processing, so that the frame data entering the rendering stage is both complete and meets the buffer capacity requirements.

[0040] Unlike traditional solutions that directly use the original frame data when the buffer is not full (which do not perform integrity checks and order adjustments on frame data when the buffer is not full, making them prone to rendering errors due to abnormal data storage of the original frame data (such as missing fields) or disordered order), this application uses a progressive process of "integrity check - invalid frame removal - timestamp sorting - capacity adaptation confirmation" to ensure that the frame data when the buffer is not full is both complete and conforms to the playback order. At the same time, it confirms the buffer capacity adaptation, reserving space for subsequent frame data writing, avoiding the problem of abnormal frame data affecting the rendering effect in traditional solutions, and further ensuring the smoothness of video playback on the Qt client.

[0041] Optionally, the frame data with coherence weight tags stored in the lock-free frame buffer is subjected to interval discarding to generate adjusted frame data with coherence weight tags stored in the lock-free frame buffer, including: The frame data with coherence weight tags stored in the lock-free frame buffer is subjected to interval discarding. The discarding operation is performed according to the rule of retaining the frame data with the highest coherence weight tag value and discarding the remaining frame data at fixed intervals the frames with the lowest coherence weight tag value that are not intra-coded frames. The discarding process is implemented through lock-free atomic swaps to generate adjusted frame data with coherence weight tags stored in the lock-free frame buffer.

[0042] Specifically, taking the frame data with coherent weight labels stored in the lock-free frame buffer in the scenario determined as "buffer full" as the processing object, the first step is to extract the weight information from this frame data—reading each frame data through the Qt Meta-Object system. The system binds the continuity weight label value and associates it with the unique frame identifier in the frame data (used to locate the storage location of the frame data in the buffer later), generating a frame data list with weight-identifier mapping; then, the frame data list with weight-identifier mapping is sorted by weight - the frame data is sorted in descending order of continuity weight label value, and the original storage location information of the frame data is retained during the sorting process to avoid the inability to locate the frame data in subsequent operations, generating a frame data queue with sorting priority; then, high-weight frame retention processing is performed based on the frame data queue with sorting priority - the retention ratio is set according to the requirements of the video playback scenario of the Qt client for the continuity of the picture (this ratio can be dynamically adjusted based on the scenario, such as 60% in real-time monitoring scenario to ensure the integrity of the picture, and 50% in on-demand scenario), the first N% (N is the set retention ratio) of the frame data in the frame data queue with sorting priority is selected as the core retained frames, and the remaining (100-N)% of the frame data is filtered as the frames to be filtered, generating a frame data set with retention-to-screening distinction. Subsequently, the frames to be screened in the frame data set with the distinction between retention and screening are processed for frame type identification—the frame type identifier of the frames to be screened is extracted (distinguishing between Intra-coded Frames, I-frames and Predicted Frames, P-frames), Intra-coded Frames in the frames to be screened are removed (because Intra-coded Frames contain complete image information, discarding them can easily cause image fragmentation), and only Predicted Frames in the frames to be screened are retained as candidate frames to be discarded, generating a list of frames to be discarded with frame type filtering; then, the core retained frames in the frame data set with the distinction between retention and screening and the list of frames to be discarded with frame type filtering are processed for timestamp integration—the original playback order of the two types of frame data is restored according to the frame timestamp (used to ensure the playback order of frame data), avoiding frame order disorder caused by weight sorting, and generating a mixed frame data queue with time order; Next, a fixed-interval partitioning process is performed on the time-series mixed frame data queue—a fixed interval unit is set (the number of interval units is determined based on the human eye's perception threshold for screen jumps, such as every 3 frames as an interval unit, to avoid visual breaks caused by continuous frame drops), the time-series mixed frame data queue is split into multiple sub-frame sequences according to the interval units, generating a set of sub-frame sequences with interval partitioning; then, a drop frame selection process is performed on each sub-frame sequence in the set of sub-frame sequences with interval partitioning—in each sub-frame sequence, one predicted frame is selected from the list of frames to be dropped with frame type filtering, and the predicted frame with the lowest coherence weight label value in the sub-frame sequence is selected as the target drop frame, generating a set of sub-frame sequences with target drop labels; Then, lock-free discarding processing is performed on the set of subframe sequences with target discarding marks. This involves using the `exchange` method of the atomic operation template class (`std::atomic`) provided by the C++ standard library to perform an atomic swap operation on the storage address of the target discarded frame in the lock-free frame buffer, setting the data at that address to an invalid flag (to avoid misreading by the main thread). This process does not block frame data writing by the child thread or frame data reading by the main thread, generating a set of frame data with discarding results. Finally, valid frame integration processing is performed on the set of frame data with discarding results. This involves removing all target discarded frames with invalid flags, retaining the core retained frames and unselected frames to be filtered, restoring the playback order of valid frames according to frame timestamps, and generating adjusted frame data with coherence weight labels stored in the lock-free frame buffer. This adjusted frame data with coherence weight labels stored in the lock-free frame buffer will serve as the input object for subsequent Qt signal-slot adaptation processing, ensuring high image coherence in the frame data entering the rendering stage.

[0043] Unlike traditional solutions that "discard the latest / oldest frame directly when the buffer is full, without distinguishing frame priority and type," traditional solutions do not consider the content correlation and type differences of frame data, and are prone to discarding key frames or continuously discarding low-weight frames, resulting in screen breaks or jumps. This application, through a progressive design of "weight sorting - proportional retention - type filtering - interval discarding - lock-free operation," prioritizes the retention of high-weight frames and intra-frame encoded frames that have a greater impact on screen continuity, avoids the visual problem of continuous frame loss by discarding at fixed intervals, and solves the problem of thread blocking during discarding operations under traditional locking mechanisms by using lock-free atomic swaps.

[0044] Optionally, the above processing method, after obtaining the adjusted frame data with coherence weight labels based on the buffer monitoring results, further includes: The frame data with coherent weight tags stored in the lock-free frame buffer after adjustment is subjected to Qt signal and slot adaptation processing. A custom frame ready signal is emitted for the adapted frame data. The custom frame ready signal is bound to the rendering slot function to generate frame data with coherent weight tags adapted to the Qt signal and slot mechanism and the corresponding signal and slot connection relationship. Based on the frame data with coherent weight labels adapted to the Qt signal-slot mechanism and the corresponding signal-slot connection relationship, the main thread rendering state is monitored and processed, and the playback mechanism is adjusted based on the main thread monitoring results.

[0045] Specifically, taking the frame data with coherent weight tags generated above and stored in the lock-free frame buffer as the processing object, the frame data is first subjected to signal adaptation preprocessing—extracting the coherent weight tag values ​​and frame unique identifiers from the frame data, and encapsulating them into a custom data structure conforming to the Qt signal transmission format (this structure contains a "frame unique identifier field, coherent weight tag field, and frame pixel data pointer field"), ensuring that the frame data can be completely transmitted through the Qt signal slot mechanism, generating frame data with signal adaptation structure; then, custom signal definition processing is performed on the frame data with signal adaptation structure—based on the Qt framework's signal declaration syntax, a custom frame ready signal is defined (the signal name can be set to frameReady, and the parameter is the above-mentioned custom data structure type), which is used to trigger the rendering slot function and generate frame data with custom signal association; Subsequently, weight threshold determination processing is performed on frame data with custom signal associations—a coherence weight threshold is set (this threshold is determined based on the rendering priority requirements of Qt client video playback scenarios; for example, it can be set to 8 in real-time monitoring scenarios to ensure fast rendering of key frames). The coherence weight label value of each frame data is compared with the set threshold, and the frame data with custom signal associations is divided into high-weight frame data (value ≥ threshold) and ordinary-weight frame data (value < threshold), generating a set of frame data with weight level divisions. Next, signal-slot connection type matching processing is performed on the set of frame data with weight level divisions—for high-weight frame data, the Qt framework's direct connection type (Qt::DirectConnection) is used to bind the custom frame ready signal and the rendering slot function, so that the rendering slot function is called in the sending thread immediately after the signal is emitted, reducing queue waiting time; for ordinary-weight frame data, the Qt framework's queue connection type (Qt::QueuedConnection) is used to bind the custom frame ready signal and the rendering slot function, and the call event is placed in the main thread event queue to ensure the safety of cross-thread data transmission as much as possible, generating signal-slot binding relationships with connection type tags; Next, the signal-slot binding relationships marked with connection types are associated and integrated with the frame data set divided by weight levels. Each connection type mark is mapped and stored with the corresponding frame data weight level to avoid mismatch between signal-slot connections and frame data, generating a signal-slot-frame data combination with associated mapping. Finally, the signal-slot-frame data combination with associated mapping is validated. The Qt Meta-ObjectSystem is used to check the legality of signal-slot connections (such as whether the slot function exists and whether the parameter types match), eliminating combinations that fail the validation and retaining valid combinations, generating frame data with coherent weight tags and corresponding signal-slot connections adapted to the Qt signal-slot mechanism. This frame data with coherent weight tags and corresponding signal-slot connections adapted to the Qt signal-slot mechanism will be used as the processing object for subsequent main thread rendering state monitoring and playback mechanism adjustment.

[0046] Unlike traditional solutions that use a single signal slot connection type to process all frame data, this application employs a "weight threshold division - connection type differentiation binding" design. This approach matches the optimal signal slot connection method to frame data of different priorities, improving the rendering response speed of high-priority frames while ensuring cross-thread safety for ordinary frames. Furthermore, by using a custom signal data structure and validity verification, it solves the problems of lost frame data attributes and connection failures in traditional solutions, thus meeting the dual requirements of "low latency for high-priority frames + secure transmission of ordinary frames" in Qt client video playback scenarios.

[0047] Next, using the frame data with coherent weight labels adapted to the Qt signal-slot mechanism and the corresponding signal-slot connection relationships generated above as the processing objects, the main thread rendering state monitoring parameters are first initialized—setting state monitoring indicators, including the main thread rendering interval (the time difference between the completion of rendering of two adjacent frames) and the rendering slot function response delay (the time difference between signal emission and slot function execution), and setting a judgment threshold for each indicator (e.g., the rendering interval threshold can be set to 20ms, determined based on the human eye's perception of screen stuttering threshold), generating a main thread state monitoring template with monitoring parameters; then, the main thread state monitoring template with monitoring parameters and the frame data with coherent weight labels adapted to the Qt signal-slot mechanism are correlated and monitored—the main thread rendering interval and slot function response delay data are periodically collected through the Qt framework's timer (QTimer), and the collected real-time data is compared with the threshold in the monitoring template to generate the main thread rendering state monitoring results (divided into smooth state: real-time data < threshold; blocked state: real-time data ≥ threshold); Subsequently, the main thread rendering state monitoring results are matched and analyzed with the frame data with coherent weight labels adapted to the Qt signal-slot mechanism. When the monitoring result is in a smooth state, the remaining quantity of frame data with coherent weight labels adapted to the Qt signal-slot mechanism in the lock-free frame buffer is counted. When the monitoring result is in a blocked state, the execution progress of the current rendering slot function is recorded to avoid interrupting the currently rendering frame data when adjusting the playback mechanism, and an analysis result with state-data matching information is generated. Then, the playback mechanism adjustment strategy selection process is performed on the analysis result with state-data matching information. If the playback is smooth and the number of frame data in the buffer is greater than the set number (e.g., 1 frame, to ensure the buffer is not empty), the accelerated playback mechanism is triggered—by adjusting the trigger interval of the Qt timer to shorten the frame rendering cycle, the rendering frame rate is temporarily increased to 1.2-1.5 times the base frame rate (not exceeding the maximum frame rate supported by the scene, such as 60fps), thus speeding up the consumption of frame data with coherent weight labels that are adapted to the Qt signal slot mechanism. If in a blocked state, maintain the normal playback mechanism—maintain the basic rendering frame rate and timer interval, respond to the custom frame ready signal sequentially according to the signal slot connection relationship through the rendering slot function, render the frame data with coherent weight tags adapted to the Qt signal slot mechanism, and avoid increasing the load on the main thread due to the increase in frame rate; generate playback control instructions with adjustment strategy flags. Next, instruction-data binding processing is performed on playback control instructions with adjustment strategy tags and frame data with coherence weight tags adapted to the Qt signal-slot mechanism. This process associates accelerated playback instructions with frame data to be consumed in the buffer and normal playback instructions with frame data in the current rendering progress, ensuring that instructions can accurately act on the target frame data and generating a playback control-frame data combination with binding relationships. Finally, execution effect feedback processing is performed on the playback control-frame data combination with binding relationships. This process collects changes in the number of frame data in the buffer after the execution of playback control instructions in real time (monitoring whether the number drops to the set value in smooth mode) and changes in the main thread rendering interval (monitoring whether it remains within the threshold in blocked mode). Based on the feedback results, instruction parameters are fine-tuned (e.g., if the frame rate is too high during accelerated playback, causing the interval to exceed the standard, it is appropriately reduced) to make the playback mechanism adjustment meet expectations, ultimately achieving the collaborative processing of frame data buffer and signal slot.

[0048] Unlike traditional solutions that rely on a fixed playback mechanism and don't dynamically adjust based on the main thread's state (which can lead to cache buildup when the main thread is running smoothly and exacerbate blockage by maintaining a high frame rate), this application uses a design of "multi-indicator state monitoring, dynamic strategy matching, and effect feedback fine-tuning" to enable the playback mechanism to flexibly adjust according to the main thread's state. It accelerates cache consumption to reduce latency when the main thread is running smoothly and maintains normal playback to avoid overload when the main thread is blocked. Furthermore, through instruction-data binding and progress recording, it solves the problems of disconnection between the adjustment mechanism and frame data, and rendering interruptions in traditional solutions. This aligns with the core requirement of "adaptive main thread load fluctuations" in Qt client video playback scenarios, effectively alleviating the dual problems of "cache buildup" and "exacerbated load" in traditional solutions.

[0049] Optionally, based on the main thread monitoring results, he adjusts the playback mechanism, including: If the main thread monitoring result is that the main thread is smooth, the accelerated playback mechanism is triggered to consume frame data with coherence weight tags that are adapted to the Qt signal slot mechanism until the number of frame data in the buffer drops to a set number. If the main thread monitoring result indicates that the main thread is blocked, the normal playback rhythm is maintained. The custom frame ready signal is responded to through the rendering slot function, and the frame data with coherence weight labels adapted to the Qt signal slot mechanism is rendered to achieve the coordinated processing of frame data buffering and signal slots.

[0050] Specifically, taking the frame data with coherent weight tags adapted to the Qt signal-slot mechanism and the corresponding signal-slot connection relationship, as well as the frame count statistics of the unlocked frame buffer (recorded via QAtomicInt) generated above as the processing objects, the first step is to perform threshold determination processing on the number of frame data in the unlocked frame buffer—setting a threshold for the number of buffered frames (this threshold is determined based on the low latency requirements of Qt client video playback scenarios, such as setting it to 1 frame in a normal scenario to ensure that there is no redundant backlog in the buffer), comparing the current number of frame data with the set threshold, if the current number > the threshold, then triggering the initialization of the accelerated playback mechanism and generating an accelerated playback trigger command; next, associating the accelerated playback trigger command with the frame data with coherent weight tags adapted to the Qt signal-slot mechanism—extracting the frame timestamp (used to calibrate the playback rhythm) from the frame data with coherent weight tags adapted to the Qt signal-slot mechanism, binding it with the accelerated playback trigger command, avoiding frame order disorder during acceleration, and generating an accelerated command-frame data combination with timestamp calibration; Subsequently, the frame rate adjustment parameter calculation process is performed on the acceleration instruction-frame data combination with timestamp calibration—setting the base rendering frame rate (e.g., 30fps in a normal scene), determining the frame rate increase based on the difference between the current number of frame data in the buffer and a threshold (the larger the difference, the higher the increase; for example, a difference of 5 frames increases the frame rate to 1.5 times the base frame rate, a difference of 2 frames increases it to 1.2 times, and the maximum does not exceed the maximum frame rate supported by the scene, such as 60fps), calculating the target accelerated frame rate, and generating acceleration control information with target frame rate parameters; then, the acceleration control information with target frame rate parameters is combined with the frame number with coherence weight labels adapted to the Qt signal-slot mechanism. The matching process is based on the signal-slot connection relationship. For high-weight frame data using the direct connection type (Qt::DirectConnection), the emission frequency of the custom frame ready signal is accelerated by shortening the trigger interval of the Qt timer (QTimer) (interval = 1000ms / target accelerated frame rate), thereby reducing the rendering wait time for high-weight frames. For ordinary-weight frame data using the queued connection type (Qt::QueuedConnection), the signal processing priority of this type of frame is increased in the main thread event queue to ensure that it can be scheduled first, generating an accelerated execution scheme with timer parameters and priority flags. Next, the effectiveness of the accelerated execution scheme with timer parameters and priority markers is verified—simulating the rendering load under the target accelerated frame rate, checking whether the main thread is still in a smooth state (rendering interval < set threshold, such as 20ms). If the verification passes, the scheme is executed; if the load exceeds the limit, the frame rate increase is appropriately reduced, and the verification is repeated until it passes, generating a verified accelerated execution scheme. Subsequently, based on the verified accelerated execution scheme, the frame data with coherence weight tags adapted to the Qt signal slot mechanism is accelerated and consumed—a custom frame ready signal is emitted according to the adjusted timer interval, and the rendering slot function is called through the corresponding signal slot connection relationship to continuously consume the frame data in the buffer. At the same time, the number of frame data recorded by QAtomicInt is read in real time to generate an accelerated consumption result with real-time quantity feedback. Finally, the acceleration consumption result with real-time quantity feedback is processed to determine the termination condition. When the number of real-time frame data drops to a set threshold (such as 1 frame), the accelerated playback mechanism is stopped, the basic rendering frame rate and timer interval are restored, and an accelerated playback termination instruction is generated. This accelerated playback termination instruction and the frame data with coherence weight labels adapted to the Qt signal slot mechanism after consumption are used as input for subsequent rendering processing, so that the buffer has no redundant backlog and can maintain a stable playback rhythm.

[0051] Then, taking the frame data with coherent weight labels adapted to the Qt signal and slot mechanism and the corresponding signal and slot connection relationship, as well as the main thread rendering status monitoring results (determined to be in a blocked state, rendering interval ≥ set threshold, such as 20ms) generated above as the processing objects, the first step is to perform preliminary determination of the cause of the main thread blocking state—collect the current task load information of the main thread (such as the number of other non-rendering tasks being executed, CPU utilization). If the load is high, it is determined that the blocking is caused by task overload, and the basic playback rhythm needs to be maintained to avoid the load from escalating, generating a blocking cause determination result; next, the blocking cause determination result is correlated with the frame data with coherent weight labels adapted to the Qt signal and slot mechanism—extract the coherent weight label values ​​in the frame data to ensure that high-weight frames are processed first during subsequent rendering, generating a blocking adaptation frame data set with weight priority; Subsequently, playback rhythm locking is performed on the weighted priority blocking adaptation frame data set—fixing the base rendering frame rate (e.g., 30fps) and the trigger interval of the Qt timer (QTimer) (e.g., 33ms), without adjusting the signal emission frequency to avoid further increasing the main thread load due to frequency increases, generating playback control parameters with fixed rhythm markers; next, the signal slot connection relationship between the playback control parameters with fixed rhythm markers and the frame data with coherent weight labels adapted to the Qt signal slot mechanism is matched—regardless of whether it is a high-weight frame or a normal-weight frame, a custom frame ready signal is emitted strictly according to the fixed timer interval, and the rendering slot function is called through the corresponding signal slot connection relationship (Qt::DirectConnection for high-weight frames and Qt::QueuedConnection for normal frames), ensuring that the signal emission matches the main thread processing capability, generating a rhythm-matched signal emission scheme; Next, the rendering order of the rhythm-matching signal emission scheme is optimized—the frame data with coherence weight labels adapted to the Qt signal slot mechanism is sorted from high to low according to the coherence weight label values. If the weights are the same, they are sorted by frame timestamp. This ensures that frame data that has a greater impact on the coherence of the picture is rendered first in the blocking state, reducing key frame latency and generating a rendering task list with an optimized order. Subsequently, stable rendering is performed based on the rendering task list with the optimized order—the rendering slot function is triggered by a custom frame ready signal according to the list order. After each frame is rendered, the main thread resources are briefly released through the Qt event loop mechanism (such as calling QCoreApplication::processEvents() to handle pending events) to avoid rendering tasks monopolizing the main thread and aggravating blocking, and a rendering result with resource release markers is generated. Simultaneously, the rendering results with resource release markers are monitored and processed in real time in conjunction with the main thread rendering status. After each frame is rendered, the main thread rendering interval is re-collected. If the interval is still greater than or equal to the set threshold, the normal playback rhythm is maintained. If the interval drops below the threshold (the main thread resumes smooth playback), a state switching instruction is triggered, and subsequent processing follows the aforementioned accelerated playback mechanism to generate a monitoring feedback result with state linkage. Finally, the monitoring feedback result with state linkage is processed to verify the rendering integrity. The rendered frame data is checked for screen breaks or pixel errors (e.g., comparing the pixel deviation of frame data before and after rendering). If any abnormalities are found, the frame data with continuity weight tags adapted to the Qt signal slot mechanism is re-called to re-render the slot function to ensure playback continuity and generate a complete rendering result. This complete rendering result realizes the collaborative processing of frame data buffering and signal slots, maintaining a better picture presentation effect even when the main thread is blocked.

[0052] Optionally, a custom frame ready signal can be bound to a rendering slot function to generate frame data with coherent weight labels and corresponding signal-slot connections adapted to the Qt signal-slot mechanism, including: When binding a custom frame ready signal to a rendering slot function, for frame data whose coherence weight label value reaches a set threshold, an additional direct connection type signal slot connection method is bound. For the remaining frame data, a queue connection type signal slot connection method is used to generate frame data with coherence weight labels and corresponding signal slot connection relationships that are adapted to the Qt signal slot mechanism.

[0053] Specifically, taking the weighted frame data set (including high-weight and ordinary-weight frames), the custom frame-ready signal, and the rendering slot function generated above as the processing objects, the first step is to perform coherence weight threshold calibration on the weighted frame data set. This is done by considering the priority requirements of Qt client video playback scenarios (e.g., real-time monitoring scenarios prioritize low latency for key frames, while high-definition on-demand scenarios balance smoothness and security). A coherence weight judgment threshold is set, ensuring that high-weight frames are those that significantly impact the continuity of the video (e.g., intra-coded frames, I-frames, and highly correlated predicted frames adjacent to I-frames). Frame (P-frame) is generated to form a weight judgment standard with scene adaptation threshold. Then, based on the weight judgment standard with scene adaptation threshold, a second weight verification process is performed on the frame data set with weight level division. The continuity weight label value of each frame data is extracted and compared with the set threshold to confirm the accuracy of the division between high weight frame data (value ≥ threshold) and ordinary weight frame data (value < threshold). Misjudged frame data is removed and reclassified to generate a weight level frame data set with verification mark. Subsequently, direct connection type binding is performed on the high-weight frame data in the set of weighted frame data with verification tags—the Qt framework's signal and slot connection function (QObject::connect) is called to bind the custom frameready signal (frameReady) to the rendering slot function using the direct connection type (Qt::DirectConnection). The core characteristic of this connection type is that "the rendering slot function is executed immediately in the sending thread (child thread) after the signal is emitted," without waiting for the main thread's event queue scheduling, thereby reducing the rendering wait time for high-weight frame data and generating a high-weight signal and slot binding relationship with the direct connection tag; at the same time, the high-weight frame data with verification tags is bound to the high-weight frame data. The ordinary weighted frame data in the set of frame data for verifying the weight level is bound using the queue connection type. This involves calling the QObject::connect function to bind the custom frameready signal (frameReady) to the rendering slot function using the queue connection type (Qt::QueuedConnection). This connection type encapsulates the rendering slot function call event into a Qt event object and places it in the main thread's event queue. The main thread's event loop schedules and executes the event objects sequentially, ensuring that no thread safety issues (such as data races or memory access exceptions) occur during cross-thread data transfer. This generates an ordinary weighted signal-slot binding relationship with the queue connection mark. Next, the binding relationships of high-weight signal slots with direct connection markers are associated and mapped with high-weight frame data. A one-to-one mapping table is established between "the unique frame identifier of high-weight frame data and the direct connection binding relationship" to avoid binding relationship confusion during subsequent signal transmission. This ensures that the custom frame ready signal triggered by high-weight frame data can accurately call the corresponding rendering slot function, generating a signal slot-frame data combination with high-weight mapping relationships. At the same time, the binding relationships of ordinary weight signal slots with queue connection markers are associated and mapped with ordinary weight frame data. A mapping table is established between "the unique frame identifier of ordinary weight frame data and the queue connection binding relationship" to generate a signal slot-frame data combination with ordinary weight mapping relationships. Next, connection validity verification is performed on the signal-slot-frame data combination with high-weight mapping and the signal-slot-frame data combination with ordinary weight mapping. This is done through the QMetaObject::checkConnectArgs function of the Qt Meta-ObjectSystem to check whether the parameter types of the signal and slot functions match (e.g., whether the parameters of the custom frame ready signal are consistent with the input parameters of the rendering slot function) and whether the slot function is callable (e.g., not destroyed, public permission). If the verification fails, the binding parameters are readjusted (e.g., the parameter types are corrected) and the verification is repeated until all combinations pass the verification, generating a set of signal-slot-frame data combinations with verification pass flags. Finally, the set of signal-slot-frame data combinations with verification passed is integrated—the mapping tables of high-weight and ordinary-weight data are merged, and the frame data with coherent weight labels adapted to the Qt signal-slot mechanism and the corresponding signal-slot connection relationships are stored uniformly. At the same time, a "connection type differentiation identifier" is added to this set to facilitate the rapid identification of the signal-slot connection methods corresponding to different frame data when monitoring the main thread status in subsequent steps. This generates the final frame data with coherent weight labels adapted to the Qt signal-slot mechanism and the corresponding signal-slot connection relationships. This final result will serve as the core input for subsequent steps, used for monitoring the main thread rendering status and adjusting the playback mechanism, ensuring that frame data of different priorities can achieve efficient and safe rendering processing through the adapted signal-slot connections.

[0054] Unlike traditional solutions that use a single signal-slot connection type to process all frame data, traditional solutions fail to adapt connection methods to the continuity and weight differences of frame data. If a uniform queue connection type (Qt::QueuedConnection) is used, high-weight frame data will experience rendering delays due to waiting for the main thread's event queue scheduling, affecting the continuity of the scene. If a uniform direct connection type (Qt::DirectConnection) is used, ordinary-weight frame data is prone to data contention when being passed across threads, leading to rendering anomalies. This application, through a closed-loop design of "scenario-based threshold division - differentiated connection binding - mapping verification integration," matches low-latency direct connections for high-weight frames and secure queue connections for ordinary-weight frames. At the same time, through Qt meta-object system verification and mapping table management, it solves the dual problems of "connection type and frame priority mismatch" and "incorrect signal-slot binding" in traditional solutions.

[0055] Optionally, the coherence weight labels calculated based on frame content relevance include: The coherence weight label of the intra-coded frame is set to the highest value. The coherence weight label of consecutive similar prediction frames decreases sequentially with the order of the prediction frames in the sequence. At the same time, a dependency chain weight is added to the prediction frames associated with the intra-coded frame. The value of the dependency chain weight decreases as the distance between the prediction frame and the intra-coded frame increases. The coherence weight label is bound to the frame data structure through the Qt meta-object system so that the coherence weight label attribute is not lost during the subsequent signal slot transmission process, so as to generate frame data with coherence weight label.

[0056] Taking the decoded frame data (including intra-coded frames and predicted frames) in a Qt client video playback scenario as the processing object, the first step is to perform accurate frame type identification processing on this frame data—by parsing the encoding header information of the frame data (such as the NAL unit type field in the H.264 / H.265 encoding format), the intra-coded frame (I-frame, containing complete image information and can be decoded independently) and the predicted frame (Predicted Frame) are distinguished. Frame (P-frame, depending on the decoding of preceding I-frame or P-frame), add a unique frame type identifier to each frame of data (e.g., "I-frame-Type01" "P-frame-Type02"), generating raw frame data with frame type identifiers; then, perform basic weight assignment processing on the I-frames in the raw frame data with frame type identifiers—combining the requirements of screen continuity in Qt video playback scenarios, set the continuity weight label value of the I-frame to the highest value adapted to the scene (this highest value must ensure that the I-frame is preferentially retained in subsequent buffer adjustments, such as 10 in general scenarios, which can be dynamically adjusted according to the scene), and add a "highest weight identifier" to this weight value to avoid it being mistakenly modified in subsequent processing, generating frame data with the highest weight of the I-frame. Subsequently, continuous similarity determination processing is performed on P-frames in the original frame data with frame type identifiers. This involves using an optimized pixel difference comparison algorithm (which adds local pixel block matching weights to the traditional inter-frame difference algorithm to reduce the interference of local image changes on similarity determination) to calculate the pixel difference between the current P-frame and the previous frame (I-frame or P-frame). If the difference is lower than a set similarity threshold (e.g., 30%, determined based on the human eye's perception threshold for image similarity), it is determined to be a continuously similar predicted frame, and a "continuous similarity identifier" is added to it. If the difference is higher than the threshold, it is determined to be non-continuously similar. Predict the frame, add a "non-contiguous similarity identifier" to generate P-frame data with similarity identifiers; then, perform basic weight reduction processing on the P-frame data with similarity identifiers—based on the highest weight value of the I-frame, set the weight reduction rule for consecutively similar predicted frames according to the order of their sequence (e.g., the weight of the first consecutively similar P-frame in the sequence is 80% of the highest value of the I-frame, the second is 50%, the third is 30%, and the reduction range can be adjusted according to the scenario), and assign basic weights to non-contiguously similar P-frame data at a fixed ratio (e.g., 60% of the highest value of the I-frame), generating frame data with P-frame basic weights. Next, dependency chain weight calculation is performed on the P-frames associated with the I-frame in the frame data with P-frame base weights. First, a dependency chain is established between the I-frame and subsequent associated P-frames (e.g., the first P-frame after the I-frame is the first level of the dependency chain, the second is the second level, and so on). Dependency chain weight addition rules are set: the closer the P-frame is to the I-frame, the higher the addition ratio (e.g., the first level P-frame adds 20%, the second level adds 15%, the third level adds 10%, and the addition ratio decreases as the distance increases). This addition ratio is multiplied by the base weight of the P-frame to obtain the dependency chain weight addition value, generating P-frame data with dependency chain addition value. Then, the final weight calculation is performed on the P-frame data with dependency chain addition value. The base weight of the P-frame is added to the dependency chain weight addition value to obtain the final value of the coherence weight label of the P-frame. At the same time, a "dependency chain addition identifier" is added to this value to clarify the weight composition, generating frame data with the final weight of the P-frame. Subsequently, the frame data with the highest weight of I-frames and the frame data with the final weight of P-frames are processed by weight integration. The coherent weight tag values ​​of the two types of frame data are uniformly encapsulated into a "weight information structure" containing three fields: "base weight value", "added weight value", and "weight type identifier". This avoids weight information confusion and generates frame data with weight structure. Next, the Qt Meta-Object System (a core feature provided by the Qt framework that supports dynamic property binding and signal-slot communication) is called to perform weight binding processing. Through the QObject::setProperty function, the "weight information structure" in the frame data with weight structure is bound to a custom property of the frame data (the property name is set to "coherenceWeightTag"). At the same time, the custom property is registered in the Qt Meta-Object System to ensure that the weight property can be completely transmitted with the frame data without loss when the frame data is passed across threads and signals and slots. This generates frame data with weight bound by the meta-object. Finally, the frame data with weights bound to meta-objects undergoes weight binding validity verification. This involves reading the "coherenceWeightTag" property of the frame data using the `QMetaObject::property` function, checking the completeness of the weight information structure (e.g., no missing fields, values ​​within a reasonable range), and simultaneously simulating the signal-slot transfer process (passing the frame data to the main thread via `Qt::QueuedConnection`). The verification checks if the weight attribute is still retained after the transfer. If the verification passes, the coherence weight tag binding is confirmed to be valid, and frame data with the coherence weight tag is generated. If the verification fails, the Qt meta-object binding process is re-executed until the verification passes. This frame data with the coherence weight tag will be used as the processing object for subsequent lock-free buffer storage processing, ensuring that the weight attribute remains valid and reflects the value of the frame data to the coherence of the frame during buffer adjustments and signal-slot transfers.

[0057] Optionally, priority marking processing is performed on the input frame data, and a coherence weight label calculated based on the relevance of frame content is bound to each frame data to generate frame data with coherence weight labels, including: The input frame data is subjected to frame content semantic parsing to extract the scene dynamic and static features contained in the frame data, so as to generate frame data with scene feature identifiers. Based on frame data with scene feature identifiers, the video scene type to which the frame data belongs is classified, and the scene type label corresponding to each frame data is determined to generate frame data with scene type labels. Using frame data with scene type labels as the processing object, and combining frame type and scene type labels, a coherence weight calculation model is constructed to differentiate and quantify the content correlation of frame data under different scene types, so as to generate frame data with differentiated weight values. Based on frame data with differentiated weight values, the differentiated weight values ​​are bound to the frame data structure through the Qt meta-object system, and a weight validity verification flag is added to generate frame data with continuous weight labels.

[0058] Specifically, taking the decoded raw frame data from a Qt client video playback scenario as the processing object, the raw frame data is first preprocessed at the pixel level—the pixel format of the frame data (such as YUV420, RGB888) is uniformly converted to the standard format supported by the Qt framework (such as QImage::Format_RGB888) to avoid feature extraction anomalies caused by format incompatibility, generating frame data with standardized format; then, scene dynamic feature extraction processing is performed on the standardized frame data—an optimized motion vector analysis algorithm (based on the traditional optical flow method, adding a fault-tolerant mechanism for matching adjacent frame pixel blocks to reduce the interference of image noise on motion judgment) is used to extract the motion trajectory of targets within the frame (such as the movement path of people, the direction of vehicle travel), motion speed (such as the offset of target position in adjacent frames), and scene switching indicators (such as the determination of scene switching when the pixel difference between frames exceeds a set ratio), generating frame data with dynamic feature parameters; Subsequently, static feature extraction processing is performed on the frame data with dynamic feature parameters. This involves using a gray-level co-occurrence matrix algorithm (optimized with added texture direction weights to make static features more closely match human perception of image details) to extract image textures (such as architectural textures and natural scene textures), color distributions (such as the statistical proportion of RGB three-channel pixel values), and key region contours (such as edge information of core targets within the frame), generating frame data with static feature parameters. Next, feature fusion processing is performed on the frame data with dynamic and static feature parameters. Dynamic feature parameters (motion trajectory, speed, scene transition markers) and static feature parameters (texture, color distribution, contours) are encapsulated into a unified feature structure, and a feature type identifier (such as "dynamic - motion trajectory" or "static - texture") is added to each feature parameter to avoid feature confusion, generating frame data with a feature structure. Finally, feature validity screening is performed on the frame data with feature structures—setting feature confidence thresholds (e.g., matching degree of motion trajectory in dynamic features ≥80%, completeness of texture extraction in static features ≥75%), eliminating invalid feature parameters with confidence below the threshold, retaining valid feature parameters, and generating frame data with scene feature labels; this frame data with scene feature labels will be used as the processing object in subsequent steps for subsequent video scene type classification processing.

[0059] Unlike traditional solutions that "extract only a single type of feature or use an unoptimized feature extraction algorithm," this application addresses the shortcomings of traditional methods. Traditional methods, if only dynamic features are extracted, are prone to classification errors in static scenes (such as fixed surveillance footage) due to insufficient features. Conversely, extracting only static features fails to distinguish between dynamic scenes (such as high-speed motion). Furthermore, traditional algorithms are susceptible to noise interference, resulting in low feature accuracy. This application employs a design that combines "unified format, collaborative extraction of dynamic and static features, optimized algorithms to reduce interference, and confidence level filtering." This comprehensive approach covers all scene features and improves feature quality through algorithmic optimization, solving the problems of "single features and low accuracy" in traditional solutions. It provides a reliable basis for subsequent scene classification, meeting the fundamental requirement of "accurate scene type identification" in Qt video playback scenarios.

[0060] Using the generated frame data with scene feature identifiers as the processing object, the first step is to standardize the feature dimensions of the scene feature structure in the frame data with scene feature identifiers—converting dynamic feature parameters (such as motion speed) and static feature parameters (such as color proportion) into a unified numerical range (such as normalized values ​​of 0-100) to avoid affecting the classification results due to differences in parameter magnitudes, thus generating frame data with standardized features. Next, a scene type classification rule base is constructed based on the frame data with standardized features—combining typical scenarios of Qt client video playback (such as real-time video monitoring, high-definition video on demand, and short video playback), feature thresholds are defined for each scenario: for example, for real-time monitoring scenarios, "dynamic features - motion speed ≤ medium, static features - key area outline integrity ≥ high"; for high-definition video on demand scenarios, "dynamic features - scene switching frequency ≤ low, static features - color distribution richness ≥ high"; and for short video playback scenarios, "dynamic features - scene switching frequency ≥ high, static features - texture complexity ≤ medium", thus generating a rule base with scene classification rules. Subsequently, feature-rule matching processing is performed on the frame data with standardized features and the rule base with scene classification rules. The standardized feature parameters of each frame data are compared with the feature thresholds of each scene in the rule base one by one, and the number of matching feature parameters is counted. If the proportion of matching parameters of a certain scene is greater than or equal to a set proportion (e.g., 60%), the frame data is determined to belong to that scene type, and frame data with preliminary scene type is generated. Then, scene type verification processing is performed on the frame data with preliminary scene type. The preliminary scene type of N consecutive frames (e.g., 5 frames, determined based on the minimum time interval of scene switching) is collected. If the proportion of the same scene type in the N consecutive frames is greater than or equal to 80%, the scene type is confirmed to be valid, avoiding classification errors caused by single frame feature anomalies, and frame data with verified scene type is generated. Finally, scene type label binding processing is performed on the frame data with verified scene types—a unique scene type label is assigned to each confirmed scene type (such as "monitoring scene-T01", "movie scene-T02", "short video scene-T03"). The scene type label is associated with the frame data through the structure field to ensure that the label is not lost in subsequent processing, generating frame data with scene type labels. This frame data with scene type labels will be used as the processing object for subsequent steps, for the construction of the subsequent coherent weight calculation model and differential quantization.

[0061] Unlike traditional solutions that classify scenarios based on a single feature or fixed rules (which fail to tailor classification rules to typical Qt video playback scenarios and lack verification of scenario consistency across consecutive frames, leading to classification fluctuations), this application employs a design that combines feature standardization, scenario-customized rule base, multi-parameter matching, and continuous frame verification. This approach ensures scenario classification aligns with real-world applications while avoiding single-frame errors, addressing the issues of poor classification specificity and low stability in traditional solutions. This provides a precise scenario-based basis for subsequent differentiated weight calculations.

[0062] Using the generated frame data with scene type labels as the processing object, the first step is to perform frame type identification processing on the frame data with scene type labels. This involves parsing the encoding identifier of the frame data (such as the frame type field in H.264 / H.265 encoding) to distinguish between intra-coded frames (I-frames) and predicted frames (P-frames), adding frame type identifiers (such as "I-frame-F01" and "P-frame-F02") to the frame data, and generating scene frame data with frame type identifiers. Next, the basic framework of the coherent weight calculation model is constructed based on the scene frame data with frame type identifiers. This model includes a three-layer structure: "scene weight coefficient layer", "frame type basic weight layer", and "relevance correction layer". Each layer passes parameters through a data interface to ensure the closed-loop logic of the model. Subsequently, parameter configuration processing is performed on the "Scene Weight Coefficient Layer"—based on the aforementioned scene type labels (such as surveillance scene, film and television scene, and short video scene), different scene weight coefficients are set: surveillance scene needs to prioritize ensuring the continuity of the image, so the dynamic feature weight coefficient is set to a high value (e.g., 0.6) and the static feature weight coefficient is set to a low value (e.g., 0.4); film and television scene needs to balance dynamic and static elements, so both weight coefficients are set to a medium value (e.g., 0.5); short video scene needs to highlight dynamic changes, so the dynamic feature weight coefficient is set to the highest value (e.g., 0.7) and the static feature weight coefficient is set to the lowest value (e.g., 0.3), generating a model layer with scene weight coefficients; Next, parameter configuration processing is performed on the "Frame Type Basic Weight Layer"—considering the impact of frame type on the continuity of the image: I-frames contain complete image information, and the basic weight is set to the highest (e.g., 10 points in a 10-point scale); P-frames depend on the preceding frame, and the basic weight decreases according to the distance from the I-frame (e.g., the first P-frame after the I-frame is set to 8 points, and the second is set to 6 points), generating a model layer with frame type basic weights; subsequently, the output parameters of the model layer with scene weight coefficients and the output parameters of the model layer with frame type basic weights are passed to the "Relevance Correction Layer" to perform correlation correction processing—calculating the correlation between the scene features of the frame data and the frame type (e.g., the correlation of static features of I-frames in a surveillance scene, and the correlation of dynamic features of P-frames in a film and television scene), and correcting the basic weights according to the correlation (e.g., if the correlation is ≥80%, the basic weight is increased by 10%, and if the correlation is <50%, the basic weight is decreased by 5%), generating frame data with corrected weights. Finally, weight normalization is performed on the frame data with corrected weights—the corrected weight values ​​are converted into a uniform range (such as integers from 1 to 10) to ensure that the weights of different scenes and frame types are comparable, generating frame data with differentiated weight values; this frame data with differentiated weight values ​​will be used as the processing object for subsequent steps, for binding the weights to the frame data structure.

[0063] Unlike traditional solutions that rely on "weight calculations without scene differentiation or model-based design"—which often employ fixed weights (e.g., high weight for I-frames and low weight for P-frames) without adjusting for different scene types (e.g., excessively reducing the weight of P-frames in surveillance scenes can lead to stuttering, while unbalanced dynamic and static characteristics in film and television scenes can cause weight biases—this new approach addresses the lack of a structured model and the resulting chaotic weight calculation logic. Instead, this application employs a "three-layer structured model - scene-customized coefficients - correlation correction - normalization" design, ensuring that weight calculations are both scene-appropriate and possess clear model logic. This solves the problems of "weights lacking scene adaptation and calculation lacking structure" in traditional solutions, enabling weights to accurately reflect the continuity value of frame data across different scenes.

[0064] Using the frame data with differentiated weight values ​​generated above as the processing object, the frame data structure parsing process is first performed on the frame data with differentiated weight values—extracting the original structure of the frame data (including fields such as pixel data, encoding information, and timestamps), determining the extended fields in the structure that can be used to bind weight values ​​(such as reserved custom parameter fields), avoiding the occupation of core fields that could cause frame data abnormalities, and generating frame data with structure parsing marks; then, the frame data with structure parsing marks and the frame data with differentiated weight values ​​are preprocessed for weight binding—converting the differentiated weight values ​​to a numeric type that adapts to the frame data structure (such as being consistent with the integer type of the custom fields in the structure), avoiding type incompatibility that could lead to binding failure, and generating frame data with adapted weight values. Subsequently, the Qt Meta-Object System is invoked to perform weight binding processing. The QMetaObject::invokeMethod function writes the weight values ​​from the frame data with adapted weight values ​​into an extended field of the frame data structure, and registers the weight values ​​as a dynamic attribute of the frame data (the attribute name is set to "coherenceWeight"). This ensures that the weight values ​​are retained as the frame data is passed through signal slots, generating frame data with weight binding attributes. Next, a weight validity check flag is added to the frame data with weight binding attributes. Based on the reasonable range of the weight values ​​(e.g., 1-10), a check rule is generated (e.g., if the weight value is within the range of 1-10, the check flag is set to "valid"; otherwise, it is set to "invalid"). The check rule and check flag are encapsulated into a check structure and bound to another extended field of the frame data structure, generating frame data with a weight check structure. Next, the frame data with the weighted verification structure undergoes binding integrity verification. This involves reading the "coherenceWeight" property of the frame data using the QObject::property function of the Qt meta-object system, checking if the weight value exists and is within a reasonable range, and verifying whether the verification flag matches the weight value (if the weight value is valid, the verification flag is "valid"). If the verification fails, the weight binding and verification flag addition process is repeated until the verification passes, generating frame data with a verification pass marker. Finally, the frame data with the verification pass marker undergoes final encapsulation processing—integrating the frame data structure (including pixel data, encoding information, and timestamp), the "coherenceWeight" weight attribute, and the weight verification structure to ensure complete association of all fields, generating frame data with coherent weight labels. This frame data with coherent weight labels will be used as the processing object for subsequent steps, for lock-free buffer storage processing.

[0065] Unlike traditional solutions that "do not bind weights through the Qt metaobject system or lack a verification mechanism," this application addresses the issue of weight loss during signal-slot transmission due to unregistered Qt object attributes. Traditional solutions, which directly write weights into frame data, are prone to weight loss because Qt object attributes are not registered. Without a verification mechanism, incorrect weights (such as those exceeding a reasonable range) can affect subsequent discard decisions and rendering priorities. This application employs a design of "structure parsing - type adaptation - Qt metaobject binding - verification flag - integrity verification," ensuring stable weight transmission with frame data while preventing interference from incorrect weights. This solves the problems of "weight loss and invalid weights" in traditional solutions, providing a reliable weight basis for subsequent lock-free buffering and priority scheduling, thus meeting the Qt environment's requirement for "safe cross-thread transmission of frame data attributes."

[0066] It should be understood that this application is not limited to the processes and structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.< / int> < / int>

Claims

1. A method for processing frame data, characterized in that, include: Priority marking is performed on the input frame data, and a coherence weight label calculated based on the correlation of frame content is bound to each frame data to generate frame data with coherence weight labels. The frame data with coherent weight labels is stored in a lock-free buffer. The number of frame data in the current buffer is recorded, and the buffer capacity is dynamically adjusted to generate frame data with coherent weight labels stored in the lock-free frame buffer. The frame data with coherence weight labels stored in the lock-free frame buffer is subjected to buffer state monitoring processing, and the adjusted frame data with coherence weight labels is obtained based on the buffer monitoring results.

2. The processing method according to claim 1, characterized in that, The process of performing lock-free buffering storage on frame data with coherence weight labels, recording the number of frame data in the current buffer, and dynamically adjusting the buffer capacity to generate frame data with coherence weight labels stored in the lock-free frame buffer includes: Based on the atomic operation mechanism, the frame data with coherent weight labels is subjected to lock-free storage structure adaptation processing to generate frame data with coherent weight labels that are adapted to circular queue storage. The frame data with coherent weight tags stored in an adapted circular queue is used as the processing object. The read and write pointers of the circular queue are synchronized through atomic variables to generate frame data with coherent weight tags after pointer synchronization. Based on atomic integers, the frame data with coherent weight labels after pointer synchronization is subjected to statistical processing to generate frame data with coherent weight labels and real-time quantity records. Based on the quantity information in the frame data with coherent weight labels that record real-time quantities, the storage capacity of the circular queue is dynamically adjusted to generate frame data with coherent weight labels that are adapted to the capacity. The capacity-adapted frame data with coherent weight labels is stored in the lock-free frame buffer to generate frame data with coherent weight labels stored in the lock-free frame buffer.

3. The processing method according to claim 2, characterized in that, The atomic variable is an atomic operation template class provided by the C++ standard library, and the atomic integer is a class provided by the Qt framework for performing atomic integer operations.

4. The processing method according to claim 1, characterized in that, The adjusted frame data with coherence weight labels obtained based on buffer monitoring results includes: If the buffer monitoring result indicates that the buffer is full, then the frame data with coherent weight labels stored in the unlocked frame buffer are dropped at intervals to generate adjusted frame data with coherent weight labels stored in the unlocked frame buffer. If the buffer monitoring result indicates that the buffer is not full, then the frame data with coherent weight labels stored in the unlocked frame buffer will be used as the adjusted frame data with coherent weight labels stored in the unlocked frame buffer.

5. The processing method according to claim 4, characterized in that, The step of periodically discarding frame data with coherence weight labels stored in the unlocked frame buffer to generate adjusted frame data with coherence weight labels stored in the unlocked frame buffer includes: The frame data with coherence weight tags stored in the lock-free frame buffer is subjected to interval discarding. The discarding operation is performed according to the rule of retaining the frame data with the highest coherence weight tag value and discarding the remaining frame data at fixed intervals the frames with the lowest coherence weight tag value that are not intra-coded frames. The discarding process is implemented through lock-free atomic swaps to generate adjusted frame data with coherence weight tags stored in the lock-free frame buffer.

6. The processing method according to claim 1, characterized in that, Also includes: The frame data with coherent weight tags stored in the lock-free frame buffer after adjustment is subjected to Qt signal and slot adaptation processing. A custom frame ready signal is emitted for the adapted frame data. The custom frame ready signal is bound to the rendering slot function to generate frame data with coherent weight tags adapted to the Qt signal and slot mechanism and the corresponding signal and slot connection relationship. Based on the frame data with coherent weight labels adapted to the Qt signal-slot mechanism and the corresponding signal-slot connection relationship, the main thread rendering state is monitored and processed, and the playback mechanism is adjusted based on the main thread monitoring results.

7. The processing method according to claim 6, characterized in that, The adjustment of the playback mechanism based on the main thread monitoring results includes: If the main thread monitoring result is that the main thread is smooth, then the accelerated playback mechanism is triggered to consume frame data with coherence weight tags that are adapted to the Qt signal slot mechanism until the number of frame data in the buffer drops to a set number. If the main thread monitoring result indicates that the main thread is blocked, the normal playback rhythm is maintained. The custom frame ready signal is responded to through the rendering slot function, and the frame data with coherence weight labels adapted to the Qt signal slot mechanism is rendered to achieve the collaborative processing of frame data buffering and signal slots.

8. The processing method according to claim 6, characterized in that, The step of binding a custom frame ready signal to a rendering slot function to generate frame data with coherent weight labels and corresponding signal slot connections adapted to the Qt signal slot mechanism includes: When binding a custom frame ready signal to a rendering slot function, for frame data whose coherence weight label value reaches a set threshold, an additional direct connection type signal slot connection method is bound. For the remaining frame data, a queue connection type signal slot connection method is used to generate frame data with coherence weight labels and corresponding signal slot connection relationships that are adapted to the Qt signal slot mechanism.

9. The processing method according to claim 1, characterized in that, The coherence weight label calculated based on frame content correlation includes: The coherence weight label value of the intra-coded frame is set to the highest value. The coherence weight label value of consecutive similar prediction frames decreases sequentially with the order of the prediction frame sequence. At the same time, a dependency chain weight is added to the prediction frames associated with the intra-coded frame. The value of the dependency chain weight decreases as the distance between the prediction frame and the intra-coded frame increases. The coherence weight label is bound to the frame data structure through the Qt meta-object system so that the coherence weight label attribute is not lost during the subsequent signal slot transmission process, so as to generate frame data with coherence weight label.

10. The processing method according to claim 1, characterized in that, The step of prioritizing the input frame data and binding each frame data with a coherence weight label calculated based on the frame content correlation to generate frame data with coherence weight labels includes: The input frame data is subjected to frame content semantic parsing to extract the scene dynamic and static features contained in the frame data, so as to generate frame data with scene feature identifiers. Based on frame data with scene feature identifiers, the video scene type to which the frame data belongs is classified, and the scene type label corresponding to each frame data is determined to generate frame data with scene type labels. Using frame data with scene type labels as the processing object, and combining frame type and scene type labels, a coherence weight calculation model is constructed to differentiate and quantify the content correlation of frame data under different scene types, so as to generate frame data with differentiated weight values. Based on frame data with differentiated weight values, the differentiated weight values ​​are bound to the frame data structure through the Qt meta-object system, and a weight validity verification flag is added to generate frame data with continuous weight labels.

Images