A playback frame rate adjustment method, device and computer readable storage medium

Abstract

The application discloses a playback frame rate adjusting method, device and computer readable storage medium, and the method comprises the following steps: detecting the importance of data code streams in each code stream channel; dividing the code stream channel into multiple gears based on the importance of the data code streams; and sequentially performing frame dropping on the data code streams of the code stream channel of each gear from low to high according to the importance. Through the above method, the decoding capability can be more effectively utilized, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a playback frame rate adjustment method, device, and computer-readable storage medium. Background Technology

[0002] In recent years, with the rapid development of computer, network, image processing, and transmission technologies, video surveillance technology has also made significant progress. However, the decoding capability of each backend storage hard disk recorder is fixed, and the number of channels it supports for playback is also fixed. Due to the limited decoding capability, full-frame playback cannot be guaranteed. Existing playback frame dropping strategies uniformly drop frames from all channels, which may lead to a waste of decoding capability and the low recording frame rate issue that users are concerned about, resulting in a poor user experience. Summary of the Invention

[0003] The main technical problem solved by this invention is to provide a playback frame rate adjustment method, device and computer-readable storage medium that can more effectively utilize decoding capabilities and improve user experience.

[0004] To solve the above-mentioned technical problems, the present invention adopts a technical solution as follows: a playback frame rate adjustment method is provided, which includes: detecting and acquiring the importance of data streams in each bitstream channel; dividing the bitstream channel into multiple levels based on the importance of the data streams; and dropping frames in the data streams of each level of bitstream channel in order of increasing importance.

[0005] The process of detecting and acquiring the importance of data streams in each stream channel includes: detecting and acquiring the proportion of dynamic images in the data stream; and acquiring the importance of the data stream based on the proportion of dynamic images, with a higher proportion of dynamic images indicating a higher importance of the data stream.

[0006] The method of dividing the bitstream channel into multiple levels based on the importance of the data bitstream includes: sorting the bitstream channels by the proportion of dynamic images in the data bitstream of the bitstream channel and dividing the bitstream channel into N levels, wherein the proportion of dynamic images in the data bitstream of the bitstream channel in level N is less than the proportion of dynamic images in the data bitstream of the bitstream channel in level N-1.

[0007] The process of dropping frames in the data streams of each bitstream channel in order of importance from low to high includes: performing gradient frame dropping on the data streams of each bitstream channel until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream. Gradient frame dropping refers to dropping frames at different proportions on the data streams of each bitstream channel simultaneously according to predetermined rules.

[0008] The gradient frame dropping for the data stream of each bitstream channel includes: starting from the Nth bitstream, alternatingly dropping frames once or multiple times for the data stream of each bitstream channel; and in each switching cycle, dropping frames once or multiple times for the data stream of the current bitstream channel according to a predetermined ratio.

[0009] Starting from the Nth level, the data stream of each level's bitstream channel is alternately dropped one or more times, including: dropping 1 / 4 frame of the data stream of the Nth level's bitstream channel, and determining whether the playback device's decoding capability is greater than the decoding capability required to decode the data stream; if the playback device's decoding capability is less than the decoding capability required to decode the data stream, then dropping 1 / 2 frame of the data stream of the Nth level's bitstream channel, and determining whether the playback device's decoding capability is greater than the decoding capability required to decode the data stream; if the playback device's decoding capability is less than the decoding capability required to decode the data stream, then dropping 1 / 4 frame of the data stream of the (N-1)th level's bitstream channel, and determining whether the playback device's decoding capability is greater than the decoding capability required to decode the data stream; if the playback device's decoding capability is greater than the decoding capability required to decode the data stream, then dropping 1 / 4 frame of the data stream of the (N-1)th level's bitstream channel, and determining whether the playback device's decoding capability is greater than the decoding capability required to decode the data stream; if the playback device's decoding capability is greater than the decoding capability required to decode the data stream, then dropping 1 / 2 .... If the decoding capability of the playback device is less than the decoding capability required for the data stream, then drop 1 / 2 frame of the data stream in the (N-1)th bitstream channel, and determine if the decoding capability of the playback device is greater than the decoding capability required for the data stream. If the decoding capability of the playback device is less than the decoding capability required for the data stream, then drop 1 / 4 frame of the data stream in the N-2th bitstream channel, and determine if the decoding capability of the playback device is greater than the decoding capability required for the data stream. Continue dropping frames in this manner until the decoding capability of the playback device is greater than or equal to the decoding capability required for the data stream.

[0010] The gradient frame dropping for the data stream of each bitstream channel includes: sequentially dropping frames in one or more rounds for the data stream of each bitstream channel in each bitstream channel. In each round of frame dropping, in the order from bitstream N to bitstream 1, a predetermined proportion of frames are dropped for the data stream of each bitstream channel in each bitstream channel. In the same round of frame dropping, the frame dropping proportion is the same for each bitstream channel.

[0011] The process of sequentially dropping frames in one or more rounds for each bitstream channel includes: dropping 1 / 4 frame from the bitstream channel of the Nth bitstream, and determining whether the decoding capability of the playback device is greater than the decoding capability required to decode the data bitstream; if the decoding capability of the playback device is less than the decoding capability required to decode the data bitstream, dropping 1 / 4 frame from the bitstream channel of the (N-1)th bitstream, and determining whether the decoding capability of the playback device is greater than the decoding capability required to decode the data bitstream; if the decoding capability of the playback device is less than the decoding capability required to decode the data bitstream, dropping 1 / 4 frame from the bitstream channel of the (N-2)th bitstream, and determining whether the decoding capability of the playback device is greater than the decoding capability required to decode the data bitstream. If the decoding capability of the playback device is greater than the decoding capability required to decode the data stream, then, starting from the Nth level, drop frames sequentially until 1 / 4 frames are dropped from the data stream channel of the first level. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then, starting from the Nth level, drop 1 / 2 frames from the data stream channel of each level sequentially until 1 / 2 frames are dropped from the data stream channel of the first level. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then, starting from the Nth level, drop 1 frames from the data stream channel of each level sequentially until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

[0012] The process of dropping frames from the data stream channels of each tier in order of importance from low to high includes: dropping frames from the data stream channels of each tier in reverse order of tier until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream. The process of dropping frames from the current tier means dropping frames from the data stream channels of the current tier until all data stream channels of the current tier have been dropped into I-frames. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then the next tier of frame dropping is performed.

[0013] The centralized frame dropping of data streams in each bitstream channel of each bitstream level includes: dropping frames in one or more rounds according to a predetermined ratio for each bitstream channel of the current bitstream level until the data streams of each bitstream channel of the current bitstream level are dropped as I-frames or the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

[0014] The process of dropping frames in one or more rounds according to a predetermined ratio for each bitstream channel in the current tier includes: dropping 1 / 4 frame of the data stream in the Nth tier bitstream channel, and determining whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream; if the decoding capability of the playback device is less than the decoding capability required to decode the data stream, dropping 1 / 2 frame of the data stream in the Nth tier bitstream channel, and determining whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream; if the decoding capability of the playback device is less than the decoding capability required to decode the data stream, dropping the data stream in the Nth tier bitstream channel into an I-frame, and determining whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream; if the decoding capability of the playback device is less than the decoding capability required to decode the data stream, starting from dropping 1 / 4 frame of the data stream in the (N-1)th tier bitstream channel, dropping frames sequentially until the data stream in the (N-1)th tier bitstream channel is dropped into an I-frame or the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

[0015] Before performing gradient frame dropping or concentrated frame dropping on the data streams of each bitstream channel, the process includes: detecting whether the dynamic frame ratio of the data stream in the Nth bitstream channel is zero; if the dynamic frame ratio of the data stream in the Nth bitstream channel is zero, then the data stream in the Nth bitstream channel is dropped as an I-frame, and gradient frame dropping or concentrated frame dropping is performed on the data streams of each bitstream channel starting from the N-1th bitstream channel; if the dynamic frame ratio of the data stream in the Nth bitstream channel is not zero, then gradient frame dropping or concentrated frame dropping is performed on the data streams of each bitstream channel starting from the Nth bitstream channel.

[0016] The process of determining whether the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream after a frame is dropped includes: detecting whether the remaining decoding capability of the playback device is greater than the decoding capability required for 1 / 4 frame of the data stream of the most important decoding channel in the dropped frames; if the remaining decoding capability of the playback device is greater than the decoding capability required for 1 / 4 frame of the data stream of the most important decoding channel in the dropped frames, then 1 / 4 frame is added to the data stream of the most important decoding channel.

[0017] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a playback frame rate adjustment device, the data device including a processor, the processor being used to execute the above-mentioned playback frame rate adjustment method.

[0018] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a computer-readable storage medium for storing instruction / program data, which can be executed to implement the above-mentioned playback frame rate adjustment method.

[0019] The beneficial effects of this invention are as follows: Unlike the prior art, this invention obtains decoded data channels with attention gradients by dividing them into batches according to their importance. Under the condition that the decoding capability of the playback device is certain, the bitstream channels of each level are dropped in batches according to their importance from low to high. Priority is given to ensuring that the bitstream channels with high importance are decoded and displayed at high frame rates, while the bitstream channels with low importance are decoded and displayed at low frame rates. This can make more effective use of decoding capabilities and improve the user experience. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating one implementation method of the playback frame rate adjustment method of this application;

[0021] Figure 2 This is a flowchart illustrating another implementation of the playback frame rate adjustment method of this application;

[0022] Figure 3 This is a flowchart illustrating one implementation of the frame dropping strategy of this application;

[0023] Figure 4 This is a schematic diagram of the playback frame rate adjustment device in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the playback frame rate adjustment device in the embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and effects of the present invention clearer and more explicit, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0027] When a video recorder captures image data, it encodes and stores the data from each stream channel on a backend hard drive. When playback is required, the data stream from each channel is decoded. However, each playback device has a fixed decoding capability. When the decoding capability of a playback device is less than the required decoding capability for the data stream, frame dropping is necessary. Specifically, the decoding capability required for full-frame playback of the current stream is calculated and compared with the total decoding capability supported by the playback device. If the required decoding capability is less than or equal to the total decoding capability of the playback device, playback is performed directly at the current frame rate. If the required decoding capability exceeds the total decoding capability of the playback device, further analysis is required. In one specific implementation, the required decoding capability of the data stream is 8*1080p@30fps (i.e., 240 1080p frames), and the total decoding capability of the playback device is 2*1080p@30fps (i.e., 60 1080p frames). This indicates that the required decoding capability of the data stream is greater than the total decoding capability of the playback device, and frame dropping processing is required for the data stream.

[0028] This application provides a playback frame rate adjustment method by dropping frames to varying degrees on the data streams of different bitstream channels. For details, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating one embodiment of the playback frame rate adjustment method of this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes:

[0029] S110: Detect the importance of the data stream in each bitstream channel.

[0030] Once the data stream is stored, the number of stream channels is fixed. The importance of each channel is determined based on the user's actual needs. For example, when performing dynamic detection on the data streams of each channel, dynamic streams are considered more important than static streams. Similarly, when performing face detection on the data streams of each channel, streams containing faces are considered more important than those without faces.

[0031] S130: Divide the bitstream channel into multiple levels based on the importance of the data bitstream.

[0032] Generally, there are many bitstream channels. After sorting the importance of the data bitstreams, the bitstream channels are divided into multiple levels, with each level containing at least one bitstream channel.

[0033] S150: Drop frames from the data stream of each bitstream channel in order of importance from low to high.

[0034] The frame dropping method for the current bitstream channel is determined based on the importance of the data bitstream. In this implementation, different frame dropping methods are used for bitstream channels of different importance. One level is used as a frame dropping unit, and frame dropping is prioritized for the data bitstream channels in the lower importance level.

[0035] In this implementation, by dividing the data into batches according to importance, decoded data channels with attention gradients are obtained. Given a certain decoding capability of the playback device, the bitstream channels of each level are dropped in batches according to their importance from low to high. Priority is given to ensuring that the bitstream channels with high importance are decoded and displayed at a high frame rate, while the bitstream channels with low importance are decoded and displayed at a low frame rate. This can make more effective use of decoding capabilities and improve the user experience.

[0036] In one embodiment, the importance of each data stream channel is distinguished based on the dynamic detection of the data stream. The higher the proportion of dynamic images, the higher the importance of the data stream. The proportion of dynamic images can be divided to obtain a tiered importance classification of the data streams for each channel. In one embodiment, a dynamic proportion threshold can be used to divide the importance of the data stream into multiple levels. For example, data streams with a dynamic proportion less than a first threshold are considered low in importance; data streams with a dynamic proportion greater than the first threshold but less than or equal to a second threshold are considered medium in importance; and data streams with a dynamic proportion greater than the second threshold are considered high in importance. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the playback frame rate adjustment method of this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 2 The illustrated process sequence is limited. For example... Figure 2 As shown, this embodiment includes:

[0037] S210: Detect the proportion of dynamic images in the acquired data stream.

[0038] After the video recorder captures image data, it performs dynamic detection on the data stream before encoding and storage to obtain the percentage of dynamic images in the entire captured image. This percentage is then encoded and stored in the stream header information of that stream channel. In one specific implementation, there are eight stream channels, Chn1-Chn8, with the following dynamic image percentages: Chn1: 0%; Chn2: 35%; Chn3: 45%; Chn4: 13%; Chn5: 64%; Chn6: 0%; Chn7: 0%; Chn8: 85%. Chn1, Chn6, and Chn8 are all static images, so their dynamic image percentage is 0%.

[0039] S230: Sort the bitstream channels by the proportion of dynamic images in the data bitstream in the bitstream channel, and divide the bitstream channels into N levels.

[0040] After sorting multiple bitstream channels, they are divided into N tiers according to a certain ratio. Each tier contains at least one bitstream channel, meaning the number of tiers is less than or equal to the number of bitstream channels. The proportion of dynamic images in the data stream of the bitstream channels in tier N is less than the proportion of dynamic images in the data stream of the bitstream channels in tier N-1. In one specific implementation, the proportion of dynamic images in the 8 bitstream channels is sorted and divided into 4 tiers. Tier 1 is assigned to channels with a dynamic image proportion greater than 60%, including Chn8 and Chn5; Tier 2 is assigned to channels with a dynamic image proportion greater than 30% and less than or equal to 60%, including Chn3 and Chn2; Tier 3 is assigned to channels with a dynamic image proportion greater than zero and less than or equal to 30%, including Chn4; and Tier 4 is assigned to channels that are completely still, i.e., have a dynamic image proportion of zero, including Chn1, Chn6, and Chn7.

[0041] S250: Detect whether the proportion of dynamic images in the data stream of the N-level bitstream channel is zero; if the proportion of dynamic images in the data stream of the N-level bitstream channel is zero, then drop the data stream of the N-level bitstream channel as an I-frame.

[0042] In the last N-level, frames are extracted from the data stream starting from the last level. When the proportion of dynamic images in the data stream of the N-level bitstream channel is zero, it means that the images in the bitstream channel of that level are all static images. In this case, the data stream of all bitstream channels in the N-level is directly dropped as I-frames.

[0043] S270: If the proportion of dynamic images in the data stream of the N-level bitstream channel is not zero, then the data streams of each bitstream channel are dropped in order of importance from low to high.

[0044] This application can perform gradient frame dropping on the data stream of each bitstream channel until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream. Gradient frame dropping refers to simultaneously dropping frames at different proportions on the data stream of each bitstream channel according to predetermined rules. There are various predetermined rules, and this application will specifically describe them using the following two implementation methods.

[0045] In the first embodiment, starting from the Nth level, frames are dropped one or more times alternately in the data stream channels of each level. In each switching cycle, frames are dropped one or more times in the data stream channels of the current level according to a predetermined ratio. However, in this embodiment, the principle of dropping frames in the data stream channels of each level in order of increasing importance is still followed. For example, when the data stream in the current level's data stream channel is dropped as an I-frame, all data streams in the data stream channels with lower importance than the current level have already been dropped as I-frames.

[0046] For detailed implementation methods, please refer to Figure 3 , Figure 3 This is a flowchart illustrating one implementation of the frame dropping strategy of this application. In this flowchart, N is the last frame, N-1 is the frame preceding the last frame, M is the current frame, and M-1 is the frame preceding the current frame. This implementation performs frame dropping detection starting from N. When the data stream in N is detected to be static, all frames in N are dropped as I-frames, and frame dropping begins from N-1, meaning N-1 is the current frame (M). Conversely, when the data stream in N is detected to be non-static, frame dropping begins from N, meaning N is the current frame (M). Before and after each frame loss, it is determined whether the decoding capability of the playback device is less than the decoding capability required for the data stream. 1 / 4 frame is dropped, then 1 / 2 frame is dropped, sequentially for the M-level bitstream channel. When M-level is N-level, the same frame dropping is performed on M-1 level (i.e., N-1 level). When M-level is not N-level, 1 / 4 frame is dropped, then 1 / 2 frame is dropped, sequentially for the M-level bitstream channel. For the M+1 level bitstream channel, I frames are dropped. This process continues until the decoding capability of the playback device is greater than or equal to the decoding capability required for the data stream.

[0047] The following explanation uses N-level as the current M-level as an example. If the playback device's decoding capability is less than the required decoding capability for the data stream, 1 / 4 frame is dropped from the N-level stream channel. The playback device's decoding capability is then checked to see if it exceeds the required decoding capability. If the playback device's decoding capability is less than the required decoding capability, 1 / 2 frame is dropped from the N-level stream channel. The playback device's decoding capability is then checked again. If the playback device's decoding capability is less than the required decoding capability, 1 / 4 frame is dropped from the (N-1)-level stream channel. The playback device's decoding capability is then checked again. If the playback device's decoding capability is less than the required decoding capability, 1 / 2 frame is dropped from the (N-1)-level stream channel. The playback device's decoding capability is then checked again. If the required decoding capability of the bitstream is determined, then half a frame is dropped from the data stream of the (N-1)th bitstream channel. The playback device's decoding capability is then checked to see if it exceeds the required decoding capability. If the playback device's decoding capability is less than the required decoding capability, then an I-frame is dropped from the Nth bitstream channel. The playback device's decoding capability is then checked again to see if it exceeds the required decoding capability. If the playback device's decoding capability is still less than the required decoding capability, then a quarter frame is dropped from the data stream of the (N-2)th bitstream channel. The playback device's decoding capability is then checked to see if it exceeds the required decoding capability. This process of dropping frames continues until the playback device's decoding capability is greater than or equal to the required decoding capability.

[0048] In the second specific implementation, one or more rounds of frame dropping are performed sequentially on the data streams of each bitrate channel. In each round of frame dropping, a predetermined proportion of frames are dropped sequentially on the data streams of each bitrate channel, from bitrate N to bitrate 1. The frame dropping proportion is the same for each bitrate in the same round. In this implementation, the principle of dropping frames on the data streams of each bitrate channel in ascending order of importance is still followed, but the frame dropping proportion is the same each time.

[0049] In a specific implementation, frame drop detection is performed starting from level N. When the data stream at level N is detected to be static, all frames at level N are dropped as I-frames, and frame dropping begins from level N-1, i.e., level N-1 becomes the current level M. Conversely, when the data stream at level N is detected to be non-static, frame dropping begins from level N, i.e., level N becomes the current level M. Before and after each frame drop, it is determined whether the decoding capability of the playback device is less than the decoding capability required for the data stream. Quarter frames are dropped sequentially for levels M, M-1, M-2, etc. After one round of frame dropping, another half frame is dropped, resulting in I-frames.

[0050] The following explanation uses N-level as the current M-level as an example. If the playback device's decoding capability is less than the required decoding capability for the data stream, 1 / 4 frame is dropped from the data stream channel in the N-level. The playback device's decoding capability is then checked to see if it exceeds the required decoding capability. If the playback device's decoding capability is less than the required decoding capability, 1 / 4 frame is dropped from the data stream channel in the (N-1)-level. The playback device's decoding capability is then checked again. If the playback device's decoding capability is less than the required decoding capability, 1 / 4 frame is dropped from the data stream channel in the (N-2)-level. If the playback device's decoding capability is greater than the decoding capability required to decode the data stream, then, starting from the Nth level, the playback device will drop 1 / 2 frames of the data stream in each level sequentially until 1 / 2 frames of the data stream in the first level are dropped. If the playback device's decoding capability is still less than the decoding capability required to decode the data stream, then, starting from the Nth level, the playback device will drop 1 frames of the data stream in each level sequentially until the playback device's decoding capability is greater than or equal to the decoding capability required to decode the data stream.

[0051] This application can also perform concentrated frame dropping on the data streams of each bitstream channel in reverse order of the bitstream level, until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream. Concentrated frame dropping means that the data streams of the current bitstream channel are dropped in a concentrated manner until all the data streams of the current bitstream channel are dropped into I-frames. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then the next bitstream level is dropped.

[0052] In one embodiment, the data stream of each bitstream channel in the current gear is dropped in one or more rounds according to a predetermined ratio until the data stream of each bitstream channel in the current gear is dropped as an I-frame or the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

[0053] In a specific implementation, frame drop detection is performed starting from the N-level. When the N-level data stream is detected to be static, all N-level frames are dropped as I-frames, and frame dropping begins from the N-1-level, which is the current M-level. Conversely, when the N-level data stream is detected to be non-static, frame dropping begins from the N-level, which is the current M-level. Before and after each frame drop, it is determined whether the decoding capability of the playback device is less than the decoding capability required to decode the data stream. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, 1 / 4 frame, 1 / 2 frame, and then I-frame are dropped sequentially from the M-level, and frame dropping continues from the M-1-level.

[0054] The following explanation uses N-level as the current level and M-level as an example. If the playback device's decoding capability is less than the required decoding capability for the data stream, 1 / 4 frame is dropped from the data stream channel in level N. The playback device's decoding capability is then checked to see if it exceeds the required decoding capability. If so, 1 / 2 frame is dropped from the data stream channel in level N. The playback device's decoding capability is then checked again. If still less than the required decoding capability, the data stream channel in level N is dropped to an I-frame. The playback device's decoding capability is then checked again. If still less than the required decoding capability, the data stream channel in level N-1 is dropped starting with 1 / 4 frame. This process continues until the data stream channel in level N-1 is dropped to an I-frame or the playback device's decoding capability is greater than or equal to the required decoding capability.

[0055] In the aforementioned frame dropping method, optionally, if the last frame dropping level includes multiple bitstream channels, and it is not necessary to uniformly drop frames on all bitstream channels to satisfy that the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data bitstream, then according to the importance of the data bitstream in the bitstream channels, starting from the bitstream channel with the lowest importance, frames are dropped sequentially for each bitstream channel in the current level. After frame dropping is completed, there may be a situation where the decoding capability of the playback device remains. In order to preserve the data bitstream to the greatest extent possible with the support of the playback device, the remaining decoding capability of the playback device is detected and analyzed. The highest importance bitstream channel in the highest level of frame dropping is detected, and it is checked whether the remaining decoding capability of the playback device is greater than the decoding capability required to decode 1 / 4 frame of the data bitstream of the highest importance bitstream channel in the dropped frames. If the remaining decoding capability of the playback device is greater than the decoding capability required to decode 1 / 4 frame of the data bitstream of the highest importance bitstream channel in the dropped frames, then 1 / 4 frame is added to the data bitstream of the bitstream channel.

[0056] S290: Reacquire the dynamic frame ratio of the data stream and adjust the frame dropping method.

[0057] After displaying for a period of time using the current decoding and frame dropping method, the dynamic frame ratio of the data stream is re-detected. If the dynamic frame ratio changes, the bitstream channel settings are readjusted, and the frame dropping method is adjusted accordingly to ensure real-time updates.

[0058] This implementation proposes multiple frame dropping methods, dropping frames in batches according to their importance from low to high. Priority is given to ensuring that high-importance bitstream channels are decoded and displayed at high frame rates, while low-importance bitstream channels are decoded and displayed at low frame rates. This can more effectively utilize decoding capabilities and improve user experience. At the same time, after the frame dropping ends, the remaining decoding capabilities can be re-analyzed and used to supplement high-priority bitstream channels, maximizing the utilization of decoding capabilities even when the device's decoding capabilities are limited.

[0059] In one specific implementation, the data streams of the eight bitstream channels require a decoding capability of 1080p@30fps, and the total decoding capability of the playback device is 2*1080p@30fps. Level 1 includes Chn8 and Chn5; Level 2 includes Chn3 and Chn2; Level 3 includes Chn4; and Level 4 includes Chn1, Chn6, and Chn7. Since the dynamic proportion of Level 4 is zero, Level 4 is directly dropped as an I-frame, and frame dropping continues for Level 3. Please refer to Table 1, which is a calculation table for a specific frame dropping method.

[0060] Table 1 - Calculation Table for Specific Frame Dropping Methods

[0061] 1 4 levels: Chn6, Chn7, Chn1 Dropped as an I-frame 5*1080P@30 + 3*1080P@1 = 153 1080P images 2 Tier 3: Chn4 Drop 1 / 4 frame 153 1080P images - 1 * 1080P @ 7 = 146 1080P images 2 Tier 3: Chn4 Drop 1 / 2 frame 153 1080P images - 1 * 1080P @ 15 = 138 1080P images 3 Tier 2: Chn3 > Chn2 Drop 1 / 4 frame 138 1080P images - 2 * 1080P images @ 7 = 124 1080P images 5 Tier 2: Chn3 > Chn2 Drop 1 / 2 frame 138 1080P images - 2 * 1080P images @ 15 = 108 1080P images 6 Tier 3: Chn4 Dropped as an I-frame 108 1080P images - 1 * 1080P @ 14 = 94 1080P images 7 Tier 1: Chn8 > Chn5 Drop 1 / 4 frame 94 1080P images - 2 * 1080P images @ 7 = 80 1080P images 8 Tier 1: Chn8 > Chn5 Drop 1 / 2 frame 94 1080P images - 2 * 1080P images @ 15 = 64 1080P images 9 Tier 2: Chn2 Dropped as an I-frame 64 1080P images - 1 * 1080P @ 14 = 50 1080P images 10 Tier 1: Chn8 Add 1 / 4 frame 50 1080P images + 1 * 1080P @ 8 = 58 1080P images

[0062] First, drop frames from the three bitstream channels of the 4-level bitstream, all dropping them as I-frames. The decoding capability required for the 5 full-frame bitstream channels and the 3 I-frame bitstream channels is 5*1080P@30+3*1080P@1=153 1080P. After dropping 1 / 4 of a frame from the 3-level bitstream channel, the decoding capability required is 153 1080P-1*1080P@7=146 1080P. After dropping another 1 / 4 frame, i.e., dropping 1 / 2 of a frame, the decoding capability required is 153 1080P-1*1080P@15=138 1080P. Following this logic, after dropping 1 / 2 frames for level 1, level 2 needs to be dropped as I-frames. Level 2 has two bitstream channels, with Chn2 being less important than Chn3. Dropping frames from only one bitstream channel is sufficient to make the playback device's decoding capability greater than or equal to the decoding capability required for the data stream. Therefore, Chn2 is prioritized for dropping as I-frames. After this, the required decoding capability is 64 1080P - 1 * 1080P@14 = 50 1080P. The playback device's total decoding capability is 2 * 1080p@30fps = 60 1080P. At this point, the playback device has 10 1080P remaining. Therefore, 1 / 4 frame is added to Chn8 of the highest level (level 1). After this, the required decoding capability is 50 1080P + 1 * 1080P@8 = 58 1080P. The display is then decoded according to the current frame rate. After displaying for a period of time using the current decoding and frame dropping method, the dynamic picture ratio of the data stream is re-detected, or the importance ranking is re-acquired when the focus changes. If there are changes, the bitstream channel settings are readjusted, and the aforementioned frame dropping method is adjusted accordingly to ensure real-time updates.

[0063] Please see Figure 4 , Figure 4 This is a schematic diagram of the playback frame rate adjustment device according to an embodiment of this application. In this embodiment, the playback frame rate adjustment device includes a detection module 41, a segmentation module 42, and a frame dropping module 43.

[0064] The device comprises three modules: a detection module 41 for detecting the importance of data streams in each stream channel; a grading module 42 for dividing the stream channels into multiple gradations based on their importance; and a frame dropping module 43 for dropping frames from each gradation of the stream channels in ascending order of importance. This playback frame rate adjustment device obtains decoded data channels with a gradient of importance by batch-dividing them according to their importance. Given a fixed decoding capability of the playback device, it drops frames from each gradation of the stream channels in batches according to their importance, prioritizing high-importance stream channels for high frame rate decoding and display, while low-importance stream channels are decoded and displayed at a low frame rate. This more effectively utilizes decoding capabilities and improves the user experience.

[0065] Please see Figure 5 , Figure 5 This is a schematic diagram of the playback frame rate adjustment device according to an embodiment of this application. In this embodiment, the playback frame rate adjustment device 51 includes a processor 52.

[0066] Processor 52 can also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor, or processor 52 can be any conventional processor.

[0067] The playback frame rate adjustment device 51 may further include a memory (not shown) for storing instructions and data required for the processor 52 to run.

[0068] The processor 52 is used to execute instructions to implement the method provided by any embodiment and any non-conflicting combination of the playback frame rate adjustment method of this application.

[0069] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. The computer-readable storage medium 61 in this embodiment stores instruction / program data 62. When executed, this instruction / program data 62 implements the method provided by any embodiment of the playback frame rate adjustment method and any non-conflicting combination thereof. The instruction / program data 62 can be formed into a program file and stored in the storage medium 61 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium 61 includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0071] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0072] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for adjusting playback frame rate, characterized in that, The method includes: Detect the importance of the data streams in each stream channel; The data stream channel is divided into multiple levels based on the importance of the data stream; Frames are dropped from the data stream of each bitstream channel in order of importance from low to high. The step of dropping frames from the data stream channels of each tier according to their importance from low to high includes: performing gradient frame dropping on the data stream channels of each tier until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream. Gradient frame dropping refers to simultaneously dropping frames at different proportions on the data stream channels of each tier according to a predetermined rule; or The step of dropping frames from the data stream of each channel in order of importance from low to high includes: dropping frames from the data stream of each channel in each channel in reverse order of importance until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream. The dropped frames mean that the data stream of the current channel is dropped first until all data streams of the current channel are dropped into I-frames. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then the next channel is dropped. Before performing gradient frame dropping or concentrated frame dropping on the data stream of each bitstream channel in each level, the process includes: detecting whether the dynamic frame ratio of the data stream of the bitstream channel in the Nth level is zero; if the dynamic frame ratio of the data stream of the bitstream channel in the Nth level is zero, then dropping the data stream of the bitstream channel in the Nth level into I frames, and performing gradient frame dropping or concentrated frame dropping on the data stream of each bitstream channel starting from the (N-1)th level; if the dynamic frame ratio of the data stream of the bitstream channel in the Nth level is not zero, then performing gradient frame dropping or concentrated frame dropping on the data stream of each bitstream channel starting from the Nth level.

2. The playback frame rate adjustment method according to claim 1, characterized in that, The importance of detecting and acquiring the data stream in each stream channel includes: Detect the dynamic frame ratio of the acquired data stream; The importance of the data stream is determined based on the proportion of the dynamic image; the higher the proportion of the dynamic image, the higher the importance of the data stream.

3. The playback frame rate adjustment method according to claim 2, characterized in that, The method of dividing the bitstream channel into multiple levels based on the importance of the data bitstream includes: The bitstream channels are sorted by the proportion of dynamic images in the data bitstream of the bitstream channel, and the bitstream channels are divided into N levels, wherein the proportion of dynamic images in the data bitstream of the bitstream channel in level N is less than the proportion of dynamic images in the data bitstream of the bitstream channel in level N-1.

4. The playback frame rate adjustment method according to claim 1, characterized in that, The gradient frame dropping process for the data streams of each bitstream channel includes: Starting from the Nth level, the data stream of the bitstream channel in each level is dropped one or more times alternately. In each switching cycle, the data stream of the bitstream channel in the current level is dropped one or more times according to a predetermined ratio.

5. The playback frame rate adjustment method according to claim 4, characterized in that, Starting from the Nth level, the alternating dropping of frames in the data stream channels of each level, one or more times, includes: Drop 1 / 4 frame of the data stream in the Nth bitstream channel, and determine whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then 1 / 2 frame of the data stream in the Nth bitstream channel is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then 1 / 4 frame of the data stream in the (N-1)th bit channel is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then 1 / 2 frame of the data stream in the (N-1)th bit channel is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then the data stream of the N-level stream channel is dropped as an I-frame, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then 1 / 4 frame of the data stream in the stream channel of the N-2th level is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. Frames are dropped sequentially until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

6. The playback frame rate adjustment method according to claim 1, characterized in that, The gradient frame dropping process for the data streams of each bitstream channel includes: The data stream of each bitstream channel in each bitstream position is dropped in one or more rounds in sequence. In each round of dropping frames, the data stream of each bitstream channel in each bitstream position is dropped in sequence according to the order from bitstream position N to bitstream position 1. In the same round of dropping frames, the dropping frame ratio of each bitstream position is the same.

7. The playback frame rate adjustment method according to claim 6, characterized in that, The sequential frame dropping process for the data streams of each bitstream channel in each bitstream position includes: Drop 1 / 4 frame of the data stream in the Nth bitstream channel, and determine whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then 1 / 4 frame of the data stream in the (N-1)th bit channel is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then 1 / 4 frame of the data stream in the N-2th bitstream channel is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream after dropping frames sequentially until 1 / 4 frames of the data stream channel in the first level are dropped, then starting from the Nth level, 1 / 2 frames of the data stream channel in each level are dropped sequentially until 1 / 2 frames of the data stream channel in the first level are dropped. If the decoding capability of the playback device is still less than the decoding capability required to decode the data stream, then starting from the Nth level, the data stream of the stream channel in each level will be dropped as an I-frame until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

8. The playback frame rate adjustment method according to claim 1, characterized in that, The centralized frame dropping of the data stream in each of the bitstream channels in each gear includes: According to a predetermined ratio, the data stream of each bitstream channel in the current gear is dropped in one or more rounds until the data stream of each bitstream channel in the current gear is dropped as an I-frame or the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

9. The playback frame rate adjustment method according to claim 8, characterized in that, The step of dropping frames in one or more rounds according to a predetermined ratio for the data stream of each bitstream channel in the current gear includes: Drop 1 / 4 frame of the data stream in the Nth bitstream channel, and determine whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then 1 / 2 frame of the data stream in the Nth bitstream channel is dropped, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then the data stream of the stream channel in the Nth bit is dropped as an I frame, and it is determined whether the decoding capability of the playback device is greater than the decoding capability required to decode the data stream. If the decoding capability of the playback device is less than the decoding capability required to decode the data stream, then the data stream of the stream channel in the (N-1)th level will start dropping 1 / 4 frames and drop frames sequentially until the data stream of the stream channel in the (N-1)th level is dropped as an I-frame or the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream.

10. The playback frame rate adjustment method according to any one of claims 5-8, characterized in that, The frame dropping until the decoding capability of the playback device is greater than or equal to the decoding capability required to decode the data stream includes: Detect whether the remaining decoding capability of the playback device is greater than the decoding capability required for 1 / 4 of the data stream of the data stream channel with the highest decoding importance among the lost frames; If the remaining decoding capacity of the playback device is greater than the decoding capacity required for 1 / 4 frame of the data stream of the highest decoding importance channel among the lost frames, then 1 / 4 frame is added to the data stream of the highest decoding importance channel.

11. A playback frame rate adjustment device, characterized in that, Includes a processor, the processor being configured to execute instructions to implement the playback frame rate adjustment method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instruction / program data that can be executed to implement the playback frame rate adjustment method as described in any one of claims 1-10.

Images