Method, apparatus, electronic device, and storage medium for obtaining decoding rendering configuration

Abstract

Provided are a method, an apparatus, an electronic device, and a storage medium for obtaining a decoding rendering configuration, which relate to the field of decoding rendering in image processing technology, and include: obtaining a default configuration; under this default configuration, obtaining a video bitstream and performing decoding rendering on the video bitstream to obtain an output frame rate and a single-frame delay; if the difference between the input frame rate under this default configuration and the output frame rate under this default configuration is greater than or equal to a first threshold, and / or, the single-frame delay under this default configuration is greater than or equal to a second threshold, adjusting this default configuration to obtain an optimal configuration; wherein, the difference between the input frame rate under this optimal configuration and the output frame rate under this optimal configuration is less than this first threshold, and the single-frame delay under this optimal configuration is less than this second threshold; the method provided by this application can determine an optimal configuration that takes into account both the output frame rate and the delay requirements for some business scenarios that require an extreme experience, so as to exert the limit ability of the current device.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of decoding and rendering in image processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for obtaining decoding and rendering configurations. Background Art

[0002] Currently, video applications are generally divided into three categories: video call applications, video player applications, and screen sharing applications. Video call applications are oriented to low-latency scenarios, have no special requirements for high resolution and high frame rate, and are more concerned about the single-frame decoding latency. Generally, in this scenario, the resolution of the video is between 480p and 720p, and the frame rate of the video is 30fps. For the single-frame decoding latency, generally, the single-frame decoding latencies of different decoding chips (H264 / H265 / ...) are compared to find the configuration with the minimum latency; of course, some decoding chips may have the situation of decoding and storing frames, and this factor will also be considered when selecting the optimal configuration. Video player applications are oriented to high-resolution scenarios and have no strict requirements for the frame rate and single-frame decoding latency of the video. Generally, the frame rate of the video in this scenario only needs to reach 30fps, and they don't care about the decoding and storing frames of the chip, as long as the decoded output frame rate is stable; compared with the frame rate and single-frame decoding latency of the video, video player applications are more concerned about the highest resolution supported by the decoding chip (H264 / H265 / ...). Screen sharing-related applications are oriented to high-resolution and low-latency scenarios, and the decoding frame rate is generally required to be between 15fps and 30fps. In addition, screen sharing-related applications will consider not only the highest resolution supported by the decoding chip (H264 / H265 / ...), but also the frame storage situation of the decoding chip.

[0003] However, for some business scenarios that require an extreme experience, such as business scenarios that have requirements for both output frame rate and single-frame decoding latency, the solutions for the above three scenarios cannot meet the experience requirements. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, electronic device, and storage medium for obtaining decoding and rendering configurations, which can determine the optimal configuration that takes into account both the output frame rate and latency requirements to exert the limit capabilities of the current device.

[0005] On the one hand, a method for obtaining a decoding and rendering configuration is provided, including:

[0006] Obtain the default configuration;

[0007] Under this default configuration, obtain the video bitstream and perform decoding and rendering on the video bitstream to obtain the output frame rate and single-frame latency;

[0008] If the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, and / or the single-frame delay under the default configuration is greater than or equal to the second threshold, adjust the default configuration to obtain an optimal configuration; wherein, the difference between the input frame rate under the optimal configuration and the output frame rate under the optimal configuration is less than the first threshold, and the single-frame delay under the optimal configuration is less than the second threshold.

[0009] On the other hand, a device for obtaining a decoding and rendering configuration is provided, including:

[0010] An obtaining unit, configured to obtain a default configuration;

[0011] A decoding and rendering unit, configured to obtain a video bitstream and perform decoding and rendering on the video bitstream under the default configuration to obtain an output frame rate and a single-frame delay;

[0012] An adjustment unit, configured to adjust the default configuration to obtain an optimal configuration if the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, and / or the single-frame delay under the default configuration is greater than or equal to the second threshold; wherein, the difference between the input frame rate under the optimal configuration and the output frame rate under the optimal configuration is less than the first threshold, and the single-frame delay under the optimal configuration is less than the second threshold.

[0013] On the other hand, an electronic device is provided, including:

[0014] A processor and a memory, the memory is configured to store a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the above method for obtaining a decoding and rendering configuration.

[0015] On the other hand, a computer-readable storage medium is provided, the computer-readable storage medium stores computer instructions, and when the computer instructions are read and executed by a processor of a computer device, the computer device is caused to execute the above method for obtaining a decoding and rendering configuration.

[0016] Based on the above technical solutions, the terminal device obtains the default configuration and decodes and renders the acquired video stream under the default configuration to obtain the output frame rate under the default configuration and the single-frame delay under the default configuration. If the difference between the input frame rate and the output frame rate under the default configuration is greater than or equal to the first threshold, and / or the single-frame delay under the default configuration is greater than or equal to the second threshold, the terminal device adjusts the default configuration to obtain the optimal configuration. Equivalently, the terminal device first probes the obtained default configuration to obtain the output frame rate and the single-frame delay under the default configuration, and then determines whether the output frame rate and the single-frame delay obtained under the default configuration meet the compliance conditions of the optimal configuration. If both the output frame rate and the single-frame delay obtained under the default configuration meet the standards, the default configuration is determined as the optimal configuration. If any one or both of the output frame rate and the single-frame delay obtained under the default configuration do not meet the standards, the default configuration is adjusted to obtain the optimal configuration that meets both the output frame rate and the single-frame delay. On the one hand, since the difference between the input frame rate and the output frame rate under the optimal configuration is less than the first threshold, equivalently, considering various decoding service scenarios that need to meet the frame rate requirements, it can ensure that the output frame rate obtained under the optimal configuration meets the requirements of the preset output frame rate, which has a certain universality. On the other hand, since the single-frame delay under the optimal configuration is less than the second threshold, equivalently, considering various decoding service scenarios that need to meet the delay requirements, it can ensure that the single-frame delay obtained under the optimal configuration meets the requirements of the preset delay, which also has a certain universality.

[0017] That is, the method provided in this application checks whether the output frame rate and the single-frame delay obtained under the default configuration meet the standards. If any one or both of the output frame rate and the single-frame delay obtained under the default configuration do not meet the standards, the default configuration is adjusted until the optimal configuration that meets both the output frame rate and the single-frame delay is obtained. Equivalently, in the case where any one or both of the output frame rate and the single-frame delay under the default configuration do not meet the standards, a limited number of adjustments are made on the basis of the default configuration to obtain the optimal configuration that meets the requirements of the output frame rate and the single-frame delay. This not only meets the needs of some extreme services for balancing the output frame rate and the single-frame delay, but also exerts the limit ability of the current device and improves the utilization rate of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1It is a schematic diagram for detecting the static capabilities of hardware decoding and rendering provided by an embodiment of the present application.

[0020] Figure 2 It is a schematic diagram for detecting the dynamic capabilities of hardware decoding and rendering provided by an embodiment of the present application.

[0021] Figure 3 It is a schematic flowchart of a method for obtaining a decoding and rendering configuration provided by an embodiment of the present application.

[0022] Figure 4 It is another schematic flowchart of a method for obtaining a decoding and rendering configuration provided by an embodiment of the present application.

[0023] Figure 5 It is a schematic block diagram of a device for obtaining a decoding and rendering configuration provided by an embodiment of the present application.

[0024] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0026] The present application can be applied to the fields of image decoding, video decoding, hardware video decoding, dedicated circuit video decoding, and real-time video decoding. For example, the solution of the present application can be combined with the Audio Video coding Standard (AVS). For example, the H.264 / Audio Video coding (AVC) standard, the H.265 / High Efficiency Video Coding (HEVC) standard, and the H.266 / Versatile Video Coding (VVC) standard.

[0027] More specifically, the present application relates to a method for obtaining a decoding and rendering configuration, which can be implemented by a device for obtaining a decoding and rendering configuration. It should be noted that the device for obtaining a decoding and rendering configuration provided in the embodiments of the present application can be integrated in a terminal device. The terminal device includes any device that integrates the device for obtaining a decoding and rendering configuration, has the ability to access the Internet, usually runs various operating systems, and has strong video processing capabilities. The terminal device includes but is not limited to smart mobile phones, tablet computers, and other small personal portable devices, such as personal digital assistants (PDAs), e-books, etc. The present application does not make specific limitations in this regard. Of course, the device provided in the embodiments of the present application can also be integrated in a server. The server can include an independent running server or a distributed server, and can also include a server cluster or a distributed system composed of multiple servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The servers can be directly or indirectly connected through wired or wireless communication methods. The present application does not make limitations in this regard.

[0028] For the convenience of understanding the solution of the present application, the following explains the related terms involved in the present application.

[0029] Decoding frame buffering: refers to the phenomenon that the hardware decoder starts to output decoded images only after a certain number of video frames are input.

[0030] Single-frame delay: refers to the time difference between the input of a video frame by the hardware decoder and the output of the video image of this frame by the hardware decoder; for a decoding chip without frame buffering, the single-frame decoding delay is the actual decoding delay; for a decoding chip with frame buffering, the single-frame decoding delay will include the frame buffering time of the previous few frames.

[0031] Input frame rate: refers to the frequency at which the video bitstream is sent to the decoder.

[0032] Output frame rate: refers to the frequency at which the decoder outputs video image frames.

[0033] Single-frame reference: means that when the video frame is encoded, it only refers to the image content of the current frame and the previous frame.

[0034] Multi-frame reference: means that when the video frame is encoded, it refers to the image content of the current frame and the previous few frames; for some models with frame buffering, multi-frame reference will cause an increase in the number of frames buffered by the chip.

[0035] To meet the requirements of high-resolution, high-frame-rate, and low-latency video applications, it is necessary to try different decoding and rendering configurations to explore the maximum capabilities of the device and obtain the optimal configuration. To facilitate the understanding of the decoding and rendering configuration in the solution of this application, the hardware decoding and rendering configuration will be described below.

[0036] The decoding and rendering configuration may include: chip type, resolution of the video stream, input frame rate, type of rendering control, frame dropping strategy, and reference method for encoding parameters.

[0037] Among them, the chip type may include: H264, H265, VP9, AVS, etc.

[0038] The resolution of the video stream may include: 720p, 1080p, 2K, 4K resolutions.

[0039] The input frame rate may include: 50fps, 60fps, etc.

[0040] The type of rendering control may vary according to different systems. Taking the Android platform as an example, the type of rendering control may include two types: Surface View and Texture View.

[0041] The frame dropping strategy may include: frame dropping and no frame dropping. For example, the display refreshes the screen according to the vertical synchronization signal. For a 60Hz display device, the vertical synchronization signal occurs every 16ms. Among them, in the case of frame dropping, that is, when multiple video images are continuously sent within two vertical synchronization signals, only the last video image sent is displayed; the opening of frame dropping will reduce the smoothness of the picture; it should be noted that the rendering frequency of most current displays is 60Hz. In the case of no frame dropping, the decoding frame rate can only support up to 60fps at most. However, if there is decoding jitter or network jitter, due to the accumulation of latency, the decoding and rendering latency will gradually increase. At this time, frame dropping also needs to be enabled to reduce the overall low latency. For a decoding frame rate of 50fps, since it does not reach the upper limit frequency of the device's rendering, even if there is decoding jitter or network jitter, frame dropping does not need to be enabled.

[0042] The reference method for encoding parameters may include: multi-frame reference and single-frame reference. Since specific encoding semantics will affect the single-frame decoding latency, for some chips, the number of encoding reference frames will affect the number of frames stored in the chip during decoding, thus affecting the decoding latency. Therefore, for such chips, the single-frame reference method needs to be used.

[0043] Optionally, the decoding and rendering configuration is a decoding and rendering configuration for hardware decoding. Hardware decoding is different from software decoding. Software decoding is equivalent to a decompression process during video playback, which can be handled by the central processing unit (CPU). However, as the resolution increases, an ordinary CPU is difficult to support the data decompression work, and a graphics processing unit (GPU) etc. are needed to handle such repetitive decoding work of large amounts of data.

[0044] To facilitate the understanding of the decoding and rendering configuration in the solution of this application, the detection of the decoding and rendering configuration will be described in detail below in combination with Figures 1 to 2 the detection of the decoding and rendering configuration.

[0045] It should be noted that the detection of the decoding and rendering configuration may include the detection of the static capabilities of the device and the detection of the dynamic capabilities of the device.

[0046] Figure 1 is a schematic block diagram of the static capabilities provided by an embodiment of this application.

[0047] As Figure 1 shown, the static capabilities may include hardware decoding capabilities and rendering capabilities. Further, the hardware decoding capabilities may include chip type and chip capabilities. Optionally, the chip capabilities may be the resolutions supported by the chip. The rendering capabilities may include rendering control type and rendering control capabilities. Optionally, the rendering control capabilities may include frame dropping strategies.

[0048] It should be noted that static capabilities detection refers to directly obtaining detailed hardware parameter information from the hardware interface of the device, that is, the device can obtain the static capabilities of the device according to the method of static capabilities detection.

[0049] For example, the chip type may include H264 / H265 / VP9 / AVS, etc., and the chip capabilities may be parameters such as the resolutions supported by each chip. For example, the rendering control type may include Surface View, Texture View, etc. on the Android platform, and the rendering control capabilities may include the frame dropping strategies supported by the control, where the frame dropping strategies may include frame dropping or no frame dropping.

[0050] Figure 2 is a schematic flowchart of the detection method 200 for dynamic capabilities provided by an embodiment of this application.

[0051] As Figure 2 shown, the method 200 includes:

[0052] S210, set the input frame rate.

[0053] S220, set the reference method for encoding parameters.

[0054] S230, detect the decoding and rendering capabilities in real time.

[0055] Specifically, based on the static capabilities already obtained by the device, the method 200 can detect or obtain dynamic capabilities in real time based on the set input frame rate and the reference method of the set encoding parameters. That is, the video bitstream is input in real time at the set input frame rate, and the encoding parameters are determined in the set reference method, and the video bitstream is decoded and rendered to obtain data such as the output frame rate and single-frame latency of the device in real time, so as to detect the dynamic capabilities of the device. Since an increase in the input frame rate will cause the device operating frequency to increase, resulting in faster decoding, generally, the single-frame latency of most devices will change with the input frame rate. The higher the input frame rate and the faster the decoding speed, the lower the single-frame latency. Therefore, different input frame rates will have different single-frame latencies. That is, the input frame rate needs to be detected as a configuration during the device dynamic capability detection stage.

[0056] To sum up, combined with Figures 1 to 2 , first, the static capabilities of the device can be obtained; second, the dynamic capabilities of the device can be detected to obtain the current configuration of the device.

[0057] The detection of the static capabilities and dynamic capabilities of the device will be illustrated by examples below.

[0058] For example, on a TV device of a certain Android platform, it is necessary to find the optimal configuration that meets high resolution, high frame rate, and low latency.

[0059] The detection of the static capabilities of this device is as follows:

[0060] The chip type supports H264 and H265, the decoding resolution supports 720p and 1080p, the rendering control types include Surface View and Texture View, and the frame dropping strategy supports frame dropping and no frame dropping. Therefore, there are a total of 16 combinations.

[0061] During the device dynamic capability detection stage, it is necessary to detect the performance of the device at two decoding input frame rates (50fps and 60fps), and at the same time, it is also necessary to detect the influence of the reference method of the encoding parameters (single-frame reference method and multi-frame reference method). Therefore, there are a total of 64 combinations in the overall configuration of the static capability detection of the device's hardware decoding and rendering. In order to explore the ultimate decoding and rendering capabilities of the chip, it is necessary to try all the above decoding and rendering configurations. Since 64 times of integrity detection are required to obtain the optimal configuration of this device, if the detection time for each configuration is 10 seconds, the entire detection process will take 640 seconds, and the efficiency is relatively low.

[0062] Based on this, the embodiments of the present application provide a method, an apparatus, an electronic device, and a storage medium for obtaining a decoding and rendering configuration. In the method provided by the embodiments of the present application, complete decoding and rendering configuration tests are performed on multiple devices in advance, and a default configuration is obtained after the tests are completed. When a new device is first started, it will be adjusted based on the default configuration to obtain an optimal configuration, and the optimal configuration that suits the device will be stored for subsequent direct use of the locally stored optimal configuration during formal startup. That is, the method provided by the present application selects the configuration with the highest frequency of occurrence and the best decoding and rendering effect among multiple devices as the default configuration. When both the output frame rate and the single-frame delay under the default configuration do not meet the standards, the default configuration is adjusted, which can reduce the number of detections for obtaining the optimal configuration, save detection time, and improve detection efficiency.

[0063] Figure 3 It is a schematic flowchart of the method 300 for obtaining a decoding and rendering configuration provided by the embodiments of the present application.

[0064] It should be understood that the method for obtaining the decoding and rendering configuration can be executed by a terminal device integrating the apparatus for obtaining the decoding and rendering configuration, or can be executed by a server integrating the apparatus for obtaining the decoding and rendering configuration. The present application does not make specific limitations on this. Hereinafter, taking the execution by a terminal device integrating the apparatus for obtaining the decoding and rendering configuration as an example, the method for obtaining the decoding and rendering configuration provided by the present application will be described in detail.

[0065] S301, obtain the default configuration;

[0066] S302, under the default configuration, obtain a video bitstream and perform decoding and rendering on the video bitstream to obtain an output frame rate and a single-frame delay;

[0067] S303, if the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to a first threshold, and / or the single-frame delay under the default configuration is greater than or equal to a second threshold, adjust the default configuration to obtain an optimal configuration; wherein, the difference between the input frame rate and the output frame rate under the optimal configuration is less than the first threshold, and the single-frame delay under the optimal configuration is less than the second threshold.

[0068] In other words, the terminal device obtains the default configuration and performs decoding and rendering on the obtained video bitstream under the default configuration to obtain the output frame rate and the single-frame delay under the default configuration. If the difference between the input frame rate and the output frame rate under the default configuration is greater than or equal to the first threshold, and / or the single-frame delay under the default configuration is greater than or equal to the second threshold, the terminal device adjusts the default configuration to obtain the optimal configuration.

[0069] For example, the first threshold value can be within 10% of the input frame rate under the default configuration. If the input frame rate is 60fps, the first threshold value can be 5fps or 3fps, which means that the input frame rate and output frame rate under the optimal configuration are very close, and the output of the optimal configuration is a high frame rate video image; of course, the first threshold value can also be set according to the needs of the business scenario, and it does not necessarily have to achieve a high frame rate. As long as it meets the needs of the actual business scenario, this application does not impose specific restrictions on this. For example, the second threshold value can be the reciprocal of the input frame rate of the video stream; for another example, the second threshold value can be the product of the reciprocal of the input frame rate of the video stream and the number of frames accumulated by the chip when decoding and rendering the video stream; in addition, the default configuration can be any combination of configurations in the device, or it can be the configuration with the best decoding and rendering effect detected by other devices. For example, the default configuration can be a combination of Figure 1 and Figure 2 The best configuration for decoding and rendering of other devices is obtained; of course, the default configuration can also limit only some parameters of the default configuration, and the remaining parameters are determined by dynamic detection. For example, there are some models where H265 decoding capability is stronger than H264, but the decoding output frame rate and single-frame decoding delay of H264 / H265 can meet the standards under the default configuration. In this case, the chip type of the default configuration can be set to an unknown type. In the dynamic detection stage, the optimal decoding chip type is first detected on the device to determine the optimal default configuration, and then the subsequent detection logic is performed.

[0070] It should be noted that the default configuration is also adjusted based on other methods to obtain the optimal configuration; for example, if the resolution under the default configuration is greater than the third threshold, the default configuration is adjusted to obtain the optimal configuration; this is equivalent to taking into account the business requirements for resolution in the business scenario. Based on this, the first threshold, the second threshold, and the third threshold can be used to determine the configuration with the best decoding and rendering effect that meets the output frame rate, single-frame delay, and resolution requirements. Of course, the first threshold, the second threshold, and the third threshold can be set to detect the optimal configuration that takes into account high frame rate, high resolution, and low delay to meet business needs of high output frame rate, low single-frame delay, and high resolution.

[0071] Based on the above technical solutions, the terminal device first probes the obtained default configuration to obtain the output frame rate and single-frame delay under this default configuration, and then determines whether the output frame rate and single-frame delay obtained under this default configuration meet the compliance conditions of the optimal configuration. If both the output frame rate and single-frame delay obtained under the default configuration meet the standards, the default configuration is determined as the optimal configuration. If either one or both of the output frame rate and single-frame delay obtained under this default configuration do not meet the standards, the default configuration is adjusted to obtain an optimal configuration that meets both the output frame rate and single-frame delay; on the one hand, since the difference between the input frame rate under this optimal configuration and the output frame rate under this optimal configuration is less than the first threshold, equivalently, considering various decoding service scenarios that need to meet the frame rate requirements, it can ensure that the output frame rate obtained under the optimal configuration meets the requirements of the preset output frame rate, which has a certain universality; on the other hand, since the single-frame delay under this optimal configuration is less than the second threshold, equivalently, considering various decoding service scenarios that need to meet the delay requirements, it can ensure that the single-frame delay obtained under the optimal configuration meets the requirements of the preset delay, which also has a certain universality.

[0072] In addition, if the default configuration is the configuration that appears most frequently among multiple devices and has the best decoding and rendering effect, then compared with trying all decoding and rendering configurations to determine the optimal configuration, performing a limited number of probes based on the default configuration to determine the optimal configuration can improve the detection efficiency.

[0073] That is, the method provided in this application checks whether the output frame rate and single-frame delay obtained under the default configuration meet the standards. If either one or both of the output frame rate and single-frame delay obtained under the default configuration do not meet the standards, the default configuration is adjusted until an optimal configuration that meets both the output frame rate and single-frame delay is obtained. Equivalently, when either one or both of the output frame rate and single-frame delay under the default configuration do not meet the standards, a limited number of adjustments are made based on the default configuration to obtain an optimal configuration that meets the requirements of the output frame rate and single-frame delay, which not only meets the needs of some extreme services for balancing the output frame rate and single-frame delay, but also exerts the limit ability of the current device and improves the utilization rate of the device.

[0074] In some embodiments of this application, S303 may include:

[0075] If the difference between the input frame rate under this default configuration and the output frame rate under this default configuration is greater than or equal to the first threshold, then the output frame rate under this default configuration is adjusted by at least one of adjusting the chip type, reducing the input frame rate, and reducing the resolution to obtain the optimal configuration;

[0076] If the default single-frame delay is greater than or equal to the second threshold, the single-frame delay in the default configuration is adjusted by adjusting at least one of the rendering control type and the frame dropping strategy to obtain the optimal configuration.

[0077] In other words, if the difference between the input frame rate in the default configuration and the output frame rate in the default configuration is greater than or equal to the first threshold, the terminal device adjusts the output frame rate in the default configuration by adjusting at least one of the chip type, reducing the input frame rate, and reducing the resolution, until the difference between the input frame rate in the adjusted configuration and the output frame rate in the adjusted configuration is less than the first threshold, that is, the adjusted configuration is the optimal configuration; similarly, if the default single-frame delay is greater than or equal to the second threshold, the terminal device adjusts the single-frame delay in the default configuration by adjusting at least one of the rendering control type and the frame dropping strategy, until the single-frame delay in the adjusted configuration is less than the second threshold, that is, the adjusted configuration is the optimal configuration.

[0078] For example, the chip type can be H264 / H265, etc.; for another example, the rendering control types of devices on the Android platform may include Surface View or Texture View; the frame dropping strategy may include frame dropping or no frame dropping.

[0079] In some embodiments of the present application, S303 may include:

[0080] If the difference between the input frame rate in the default configuration and the output frame rate in the default configuration is greater than or equal to the first threshold, the chip type in the default configuration is adjusted to obtain a first configuration;

[0081] If the difference between the input frame rate in the first configuration and the output frame rate in the first configuration is less than the first threshold, the first configuration is determined as the optimal configuration;

[0082] If the difference between the input frame rate in the first configuration and the output frame rate in the first configuration is greater than or equal to the first threshold, the output frame rate in the first configuration is adjusted to obtain the optimal configuration.

[0083] In other words, if the difference between the input frame rate in the default configuration and the output frame rate in the default configuration is greater than or equal to the first threshold, the terminal device adjusts the chip type in the default configuration to obtain a first configuration; if the difference between the input frame rate in the first configuration and the output frame rate in the first configuration is less than the first threshold, the terminal device determines the first configuration as the optimal configuration; if the difference between the input frame rate in the first configuration and the output frame rate in the first configuration is greater than or equal to the first threshold, the terminal device adjusts the output frame rate in the first configuration to obtain the optimal configuration.

[0084] In some embodiments of the present application, if the difference between the input frame rate and the output frame rate under the first configuration is greater than or equal to the first threshold, the chip type under the first configuration is adjusted to the chip type with the highest output frame rate to obtain a second configuration; the output frame rate under the second configuration is adjusted by reducing at least one of the input frame rate and the resolution to obtain the optimal configuration.

[0085] In other words, if the difference between the input frame rate and the output frame rate under the first configuration is greater than or equal to the first threshold, the terminal device adjusts the chip type under the first configuration to the chip type with the highest output frame rate to obtain a second configuration; the terminal device then adjusts the output frame rate under the second configuration by reducing at least one of the input frame rate and the resolution to obtain the optimal configuration.

[0086] If adjusting the chip type can no longer meet the compliance conditions for the output frame rate and single-frame delay under the optimal configuration, the chip type is adjusted to the chip type with the highest output frame rate to obtain a second configuration, and on this basis of the second configuration, at least one of the input frame rate and the resolution is reduced to adjust the output frame rate to obtain the optimal configuration; on the one hand, compared with the existing solutions that more consider the influence of the chip type on the decoding configuration, the solution provided by the present application takes into account the influence of the input frame rate and the resolution on the decoding configuration. On the other hand, if the difference between the input frame rate and the output frame rate still cannot meet the requirement that the difference is less than the first threshold after attempting to adjust the chip type, the chip type is adjusted to the chip type with the highest output frame rate, and on this basis, at least one of the input frame rate and the resolution is further reduced. Compared with directly reducing the input frame rate or the resolution, it can relatively ensure that the output frame rate is a high frame rate or relatively ensure that the resolution of the video stream is a high resolution.

[0087] In some embodiments of the present application, the output frame rate under the second configuration is adjusted by reducing it at least once to obtain a third configuration; if the difference between the input frame rate and the output frame rate under the third configuration is less than the first threshold, the third configuration is determined as the optimal configuration; if the difference between the input frame rate and the output frame rate under the third configuration is greater than or equal to the first threshold, the output frame rate under the third configuration is adjusted to obtain the optimal configuration.

[0088] In other words, the terminal device reduces the input frame rate under the second configuration at least once to obtain a third configuration; if the difference between the input frame rate and the output frame rate under the third configuration is less than the first threshold, the terminal device determines the third configuration as the optimal configuration; if the difference between the input frame rate and the output frame rate under the third configuration is greater than or equal to the first threshold, the terminal device adjusts the output frame rate under the third configuration to obtain the optimal configuration.

[0089] In some embodiments of the present application, the resolution in the second configuration is reduced at least once to obtain a fourth configuration; if the difference between the input frame rate and the output frame rate in the fourth configuration is less than the first threshold, the fourth configuration is determined as the optimal configuration; if the difference between the input frame rate and the output frame rate in the fourth configuration is greater than or equal to the first threshold, the output frame rate in the fourth configuration is adjusted to obtain the optimal configuration.

[0090] In other words, the terminal device reduces the resolution in the second configuration at least once to obtain a fourth configuration; if the difference between the input frame rate and the output frame rate in the fourth configuration is less than the first threshold, the terminal device determines the fourth configuration as the optimal configuration; if the difference between the input frame rate and the output frame rate in the fourth configuration is greater than or equal to the first threshold, the terminal device adjusts the output frame rate in the fourth configuration to obtain the optimal configuration.

[0091] In some embodiments of the present application, the input frame rate and the resolution in the second configuration are sequentially reduced at least once to obtain the optimal configuration; or the resolution and the input frame rate in the second configuration are sequentially reduced at least once to obtain the optimal configuration.

[0092] In other words, the terminal device adjusts the output frame rate in the second configuration by sequentially reducing the input frame rate in the second configuration at least once and then reducing the resolution in the second configuration to obtain the optimal configuration; or the terminal device adjusts the output frame rate in the second configuration by sequentially reducing the resolution in the second configuration at least once and then reducing the input frame rate in the second configuration to obtain the optimal configuration.

[0093] In some embodiments of the present application, S303 may include:

[0094] If the single-frame delay in the default configuration is greater than or equal to the second threshold, the rendering control type in the default configuration is adjusted to obtain a fifth configuration;

[0095] If the single-frame delay in the fifth configuration is less than the second threshold, the fifth configuration is determined as the optimal configuration;

[0096] If the single-frame delay in the fifth configuration is greater than or equal to the second threshold, the fifth configuration is adjusted to obtain the optimal configuration.

[0097] In other words, if the single-frame delay in the default configuration is greater than or equal to the second threshold, the terminal device adjusts the type of rendering control in the default configuration to obtain a fifth configuration; if the single-frame delay in the fifth configuration is less than the second threshold, the terminal device determines the fifth configuration as the optimal configuration; if the single-frame delay in the fifth configuration is greater than or equal to the second threshold, the terminal device adjusts the fifth configuration to obtain the optimal configuration.

[0098] For example, for devices on the Android platform, the type of rendering control may include Surface View and Texture View, and there are differences in rendering performance under different rendering control configurations.

[0099] It should be noted that the terminal device can pre-configure the type of rendering control into the decoding and rendering device. When the type of rendering control needs to be adjusted, the terminal device can directly adjust the type of rendering control in the default configuration.

[0100] When adjusting the single-frame delay in the default configuration, the method provided in this application takes into account that the rendering control of the device may be insufficient in performance at the current rendering frame rate and needs to switch the type of rendering control. That is, the solution provided in this application can determine the optimal configuration that meets the requirements of the delay service scenario from the perspective of modifying the type of rendering control.

[0101] In some embodiments of this application, S303 may include:

[0102] If the single-frame delay in the default configuration is greater than or equal to the second threshold, adjust the frame dropping strategy in the default configuration to obtain a sixth configuration;

[0103] If the single-frame delay in the sixth configuration is less than the second threshold, determine the sixth configuration as the optimal configuration;

[0104] If the single-frame delay in the sixth configuration is greater than or equal to the second threshold, adjust the sixth configuration to obtain the optimal configuration.

[0105] In other words, if the single-frame delay in the default configuration is greater than or equal to the second threshold, the terminal device adjusts the frame dropping strategy in the default configuration to obtain a sixth configuration; if the single-frame delay in the sixth configuration is less than the second threshold, the terminal device determines the sixth configuration as the optimal configuration; if the single-frame delay in the sixth configuration is greater than or equal to the second threshold, the terminal device adjusts the sixth configuration to obtain the optimal configuration.

[0106] It should be noted that the frame loss strategy may include frame loss and no frame loss. In addition, since most models have an upper limit of 60Hz display frequency, the decoding frame rate is often required to be 60fps in high frame rate scenarios, and the display frequency upper limit of many devices is also 60fps. In the case of jitter in decoding and network, it will cause the decoding and rendering frequency to exceed 60fps in a short period of time. Therefore, the method provided in this application takes into account the use of appropriate rendering frame loss strategy, which can effectively reduce latency and reduce the cumulative effect of latency caused by network jitter. It should be noted that the terminal device can configure the frame loss strategy to the decoding and rendering device in advance. When the frame loss strategy needs to be adjusted, the terminal device can directly adjust the frame loss strategy under the default configuration.

[0107] When adjusting the single-frame delay under the default configuration, the method provided in the present application takes into account the possible rendering bottleneck of the device and requires adjustment of the frame loss strategy. That is, the solution provided in the present application can determine the optimal configuration that meets the needs of the delayed business scenario from the perspective of adjusting the frame loss strategy, and by adjusting the frame loss strategy, the cumulative effect of delay caused by network jitter can also be reduced.

[0108] In some embodiments of the present application, method 300 may further include:

[0109] If the chip under the optimal configuration is a multi-frame chip, the encoding parameters are determined based on a multi-frame reference method;

[0110] If the chip under the optimal configuration is a non-frame chip, the encoding parameters are determined based on a single frame reference.

[0111] For example, for the Android platform, for most frame-stacking chips, specific encoding semantics (for example, the number of reference frames) will affect the number of frame-stacking chips for decoding. For some frame-stacking chips, using a single-frame reference method will result in fewer frames being stored than using multiple-frame references, thereby reducing the single-frame decoding delay. For chips that do not store frames, in order to achieve better video quality, a multiple-frame reference method can be used to determine the encoding parameters.

[0112] It should be noted that there is generally no frame hoarding phenomenon on Windows and Mac, but there is still a phenomenon that different encoding parameters affect the decoding and rendering performance. At this time, it is still necessary to detect the optimal encoding parameters in this link.

[0113] By considering the impact of encoding parameters on the decoding and rendering capabilities of the device under the optimal configuration, the optimal encoding configuration is determined, and finally the optimal configuration that takes into account encoding, decoding, and rendering is achieved.

[0114] In some embodiments of the present application, before S301, the method may further include:

[0115] Determine the configuration with the highest occurrence frequency among multiple configurations as the default configuration; the multiple configurations include the configurations with the best decoding and rendering effects when performing decoding and rendering tests for each of multiple devices.

[0116] Exemplarily, to select the default configuration of a TV device on the Android platform, it is necessary to perform integrity tests on a certain range of TV devices on the Android platform in advance, and select the configuration with the highest occurrence frequency and the best decoding and rendering effect as the default configuration of the TV device on the Android platform; for example, the default configuration can be H264 + 1080p + 60fps + SurfaceView + no frame loss + multi-frame reference for video encoding; where H264 is the chip type under the default configuration, 1080p is the resolution of the video bitstream under the default configuration, 60fps is the input frame rate under the default configuration, Surface View is the control type under the default configuration, no frame loss is the frame loss strategy under the default configuration, and multi-frame reference for video encoding is the reference method of the encoding parameters under the default configuration.

[0117] Since the configuration with the highest occurrence frequency among the multiple configurations with the best decoding and rendering effects corresponding to multiple devices is determined as the default configuration, that is, the default configuration is summarized based on a certain number of devices and has a certain universality. For most devices, the default configuration is the optimal configuration; for the remaining devices, only partial configuration adjustments need to be made on the default configuration to find the optimal configuration. Therefore, by determining the configuration with the highest occurrence frequency among the multiple configurations with the best decoding and rendering effects corresponding to multiple devices as the default configuration, the number of detections for finding the optimal configuration can be reduced, and the detection efficiency can be improved.

[0118] In some embodiments of the present application, the second threshold is the reciprocal of the input frame rate of the video bitstream; or the second threshold is the product of the reciprocal of the input frame rate of the video bitstream and the number of frames stored by the chip when decoding and rendering the video bitstream.

[0119] In other words, for a chip that does not store frames during decoding, the second threshold can be 1 / input frame rate; for a chip that stores frames during decoding, the single-frame delay needs to consider the number of frames stored by the chip. For example, for a chip that stores 3 frames during decoding, only when the video bitstream of the (n + 4)-th frame is input, the video image of the n-th frame will be output from the decoder. Because at this time, the single-frame delay will additionally calculate the frame storage time of the previous 3 frames, that is, 1 / input frame rate * 3, so the second threshold can be 1 / (decoding input frame rate) * (1 + number of frames stored).

[0120] By determining the second threshold as the reciprocal of the input frame rate of the video stream; or by determining the second threshold as the product of the reciprocal of the input frame rate of the video stream and the number of frames buffered by the chip when decoding and rendering the video stream. That is, the single-frame delay in the optimal configuration is less than the theoretical value of the time difference between the hardware decoder receiving a frame of video and the hardware decoder outputting the frame of video. That is, low delay is achieved in the optimal configuration, which can meet the low-delay service requirements in extreme service scenarios.

[0121] Figure 4 It is another schematic flowchart of the method 400 for obtaining the decoding and rendering configuration provided by the embodiments of the present application. It should be understood that the method for obtaining the decoding and rendering configuration can be executed by a terminal device integrating the device for obtaining the decoding and rendering configuration, or can be executed by a server integrating the device for obtaining the decoding and rendering configuration. The present application does not make specific limitations on this. Hereinafter, taking the execution by a terminal device integrating the device for obtaining the decoding and rendering configuration as an example, another schematic process of the method for obtaining the decoding and rendering configuration provided by the present application will be described in detail.

[0122] S401, the terminal device obtains the default configuration.

[0123] S402, the terminal device obtains the output frame rate under the default configuration and the single-frame delay under the default configuration.

[0124] S403, does the terminal device determine whether the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is less than the first threshold? When the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, execute S404; when the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is less than the first threshold, execute S411.

[0125] S404, the terminal device adjusts the chip type under the default configuration to obtain the first configuration.

[0126] S405, does the terminal device determine whether the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is less than the first threshold? When the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is greater than or equal to the first threshold, execute S406; when the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is less than the first threshold, execute S411.

[0127] S406, the terminal device adjusts the chip type under the first configuration to the chip type with the highest output frame rate to obtain the second configuration.

[0128] S407, the terminal device reduces the input frame rate under the current configuration.

[0129] S408. Does the terminal device determine whether the difference between the input frame rate under the configuration after reducing the input frame rate and the output frame rate under the configuration after reducing the input frame rate is less than the first threshold? When the difference between the input frame rate under the configuration after reducing the input frame rate and the output frame rate under the configuration after reducing the input frame rate is greater than or equal to the first threshold, execute S409; when the difference between the input frame rate under the configuration after reducing the input frame rate and the output frame rate under the configuration after reducing the input frame rate is less than the first threshold, execute S411.

[0130] S409. Reduce the resolution under the configuration after reducing the input frame rate to obtain a new current configuration.

[0131] S410. Does the terminal device determine whether the difference between the input frame rate under the new current configuration and the output frame rate under the new current configuration is less than the first threshold? When the difference between the input frame rate under the new current configuration and the output frame rate under the new current configuration is greater than or equal to the first threshold, determine the new current configuration as the current configuration and execute S407; when the difference between the input frame rate under the new current configuration and the output frame rate under the new current configuration is less than the first threshold, execute S411.

[0132] S411. Does the terminal device determine whether the single-frame delay under the default configuration is less than the second threshold? When the single-frame delay under the default configuration is greater than or equal to the second threshold, execute S412; when the single-frame delay under the default configuration is less than the second threshold, then the default configuration is the optimal configuration and execute S415.

[0133] S412. The terminal device adjusts the rendering control type and frame-drop strategy to obtain a sixth configuration.

[0134] S413. Does the terminal device determine whether the single-frame delay under the sixth configuration is less than the second threshold? If the single-frame delay under the sixth configuration is less than the second threshold, then the sixth configuration is the optimal configuration and execute S415; if the single-frame delay under the sixth configuration is greater than or equal to the second threshold, then execute S414.

[0135] S414. The terminal device adjusts the sixth configuration to obtain the optimal configuration.

[0136] S415. Does the terminal device determine whether the reference method of the encoding parameters under the optimal configuration needs to be modified? If the reference method of the encoding parameters under the optimal configuration needs to be modified, then execute S416; if the reference method of the encoding parameters under the optimal configuration does not need to be modified, then execute S417.

[0137] S416. The terminal device modifies the reference method of the encoding parameters.

[0138] S417. The terminal device obtains the optimal configuration that takes into account the best of encoding, decoding, and rendering.

[0139] In other words, the terminal device first detects the default configuration and obtains the output frame rate and single-frame delay under the default configuration. If both the output frame rate and the single-frame delay obtained under the default configuration meet the standards, the default configuration is the optimal configuration, and the optimal encoding parameters are detected based on the optimal configuration. If the output frame rate under the default configuration meets the standard and the single-frame decoding delay under the default configuration does not meet the standard, the configuration parameters supporting the rendering ability need to be modified. If the decoded output frame rate under the default configuration does not meet the standard, first modify the decoding chip type under the default configuration, then reduce the decoding input frame rate, and then reduce the decoding resolution until the decoded output frame rate that meets the conditions of the optimal configuration is detected.

[0140] It should be noted that the default configuration can be the configuration with the highest frequency of occurrence among the configurations with the best decoding and rendering effects corresponding to multiple devices. For example, on a TV device of a certain Android platform, the configurations to be detected are as follows: the chip types include H264 and H265, the decoding resolutions include 1080p and 720p, the input frame rates include 60fps and 50fps, the rendering control types include Surface View and Texture View, the rendering frame-drop strategies include frame dropping and no frame dropping, and the reference methods for encoding parameters include single-frame reference method and multi-frame reference method. There are a total of 64 combinations, and a complete detection requires 64 times. However, under this default configuration, which can be the configuration with the highest frequency of occurrence among the configurations with the best decoding and rendering effects corresponding to multiple devices, after the first default configuration detection is completed, the results of most devices are that the output frame rate meets the standard, the single-frame delay meets the standard, and there is no frame accumulation phenomenon during decoding. At this time, the default configuration is the optimal configuration, that is, only one detection is required at this time. Based on the optimal configuration, for the frame-accumulating chip, an additional detection of the encoding parameter reference method is required, and a total of two detections are required. In the case where the output frame rate meets the standard and the single-frame delay does not meet the standard (that is, the decoding ability is sufficient and the rendering performance is insufficient), only 3 additional detections are required (one detection for adjusting the rendering control type, one detection for adjusting the frame-drop strategy, and one detection for adding the encoding parameter reference method for the frame-accumulating chip) to select the optimal rendering configuration. Even in the worst case (both the output frame rate and the single-frame delay do not meet the standard), it is far less than the 64 times of complete detection.

[0141] Based on the above technical solution, the terminal device first probes the obtained default configuration to obtain the output frame rate and single-frame delay under this default configuration, and then determines whether the output frame rate and single-frame delay obtained under this default configuration meet the standards. If both meet the standards, the default configuration is determined as the optimal configuration. If any one or both of the output frame rate and single-frame delay obtained under this default configuration do not meet the standards, the default configuration is adjusted to obtain an optimal configuration that meets both the output frame rate and single-frame delay; on the one hand, since the difference between the input frame rate under this optimal configuration and the output frame rate under this optimal configuration is less than the first threshold, equivalently, considering various decoding service scenarios that need to meet the frame rate requirements, it can ensure that the output frame rate obtained under the optimal configuration meets the requirements of the preset output frame rate, having a certain universality; on the other hand, since the single-frame delay under this optimal configuration is less than the second threshold, equivalently, considering various decoding service scenarios that need to meet the delay requirements, it can ensure that the single-frame delay obtained under the optimal configuration meets the requirements of the preset delay, also having a certain universality.

[0142] In addition, if the default configuration is the configuration that appears most frequently among multiple devices and has the best decoding and rendering effect, then compared with trying all decoding and rendering configurations to determine the optimal configuration, performing a limited number of probes based on the default configuration to determine the optimal configuration can improve the detection efficiency.

[0143] That is, the method provided in this application checks whether the output frame rate and single-frame delay obtained under the default configuration meet the standards. If any one or both of the output frame rate and single-frame delay obtained under the default configuration do not meet the standards, the default configuration is adjusted until an optimal configuration that meets both the output frame rate and single-frame delay is obtained. Equivalently, in the case where any one or both of the output frame rate and single-frame delay under the default configuration do not meet the standards, a limited number of adjustments are made based on the default configuration to obtain an optimal configuration that meets the requirements of the output frame rate and single-frame delay, which not only meets the needs of some extreme services for balancing the output frame rate and single-frame delay, but also exerts the limit ability of the current device and improves the utilization rate of the device.

[0144] It should be noted that in the above description, the "meeting the standards" involved includes "the output frame rate meets the standards" and "the single-frame delay meets the standards"; "the output frame rate meets the standards" means that when the difference between the input frame rate under a certain configuration and the output frame rate under this configuration is less than the first threshold, the output frame rate under this configuration meets the standards; similarly, "the single-frame delay meets the standards" means that the single-frame delay under a certain configuration is less than the second threshold, then the single-frame delay under this configuration meets the standards.

[0145] It should be noted that in the above description, the terms "first / second / third / fourth / fifth / sixth" only distinguish similar objects and do not represent a specific order for the objects. It should be understood that "first / second / third / fourth / fifth / sixth" can be interchanged in a specific order or sequence when permitted, and should not be a limitation to this application.

[0146] It should be noted that in the embodiments of the present invention, the term "and / or" only describes the association relationship of associated objects and indicates that there can be three relationships. Specifically, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this text generally represents an "or" relationship between the associated objects before and after.

[0147] The preferred embodiments of this application have been described in detail above in conjunction with the accompanying drawings. However, this application is not limited to the specific details in the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, in the various specific technical features described in the above specific embodiments, they can be combined in any appropriate manner without contradiction. To avoid unnecessary repetition, this application will not explain various possible combination methods separately. Another example is that any combination can be made between various different embodiments of this application, as long as it does not violate the idea of this application, it should also be regarded as the content disclosed in this application.

[0148] It should also be understood that in the various method embodiments of this application, the magnitudes of the serial numbers of the above processes do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0149] The method provided by the embodiments of this application has been described above, and the device provided by the embodiments of this application will be described below.

[0150] Figure 5 It is a schematic block diagram of an acquisition device 500 for decoding and rendering configuration provided by the embodiments of this application.

[0151] As Figure 5 shown, the acquisition device 500 for decoding and rendering configuration may include but is not limited to:

[0152] An acquisition unit 510, configured to acquire a default configuration;

[0153] A decoding and rendering unit 520, configured to acquire a video bitstream and perform decoding and rendering on the video bitstream under the default configuration to obtain an output frame rate and a single-frame delay;

[0154] An adjustment unit 530, configured to adjust the default configuration to obtain an optimal configuration if the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to a first threshold, and / or the single-frame delay under the default configuration is greater than or equal to a second threshold; wherein, the difference between the input frame rate under the optimal configuration and the output frame rate under the optimal configuration is less than the first threshold, and the single-frame delay under the optimal configuration is less than the second threshold.

[0155] In some embodiments of the present application, the adjustment unit 530 can be used to:

[0156] If the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, adjust the output frame rate under the default configuration by at least one of adjusting the chip type, reducing the input frame rate, and reducing the resolution to obtain the optimal configuration;

[0157] If the default single-frame delay is greater than or equal to the second threshold, adjust the single-frame delay under the default configuration by at least one of adjusting the rendering control type and the frame dropping strategy to obtain the optimal configuration.

[0158] In some embodiments of the present application, the adjustment unit 530 can also be used to:

[0159] If the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, adjust the chip type under the default configuration to obtain a first configuration;

[0160] If the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is less than the first threshold, determine the first configuration as the optimal configuration;

[0161] If the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is greater than or equal to the first threshold, adjust the output frame rate under the first configuration to obtain the optimal configuration.

[0162] In some embodiments of the present application, the adjustment unit 530 can also be used to:

[0163] If the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is greater than or equal to the first threshold, adjust the chip type under the first configuration to the chip type with the highest output frame rate to obtain a second configuration;

[0164] Adjust the output frame rate under the second configuration by at least one of reducing the input frame rate and reducing the resolution to obtain the optimal configuration.

[0165] In some embodiments of the present application, the adjustment unit 530 can also be used to:

[0166] By reducing the input frame rate in the second configuration at least once to obtain a third configuration;

[0167] If the difference between the input frame rate in the third configuration and the output frame rate in the third configuration is less than the first threshold, then determine the third configuration as the optimal configuration;

[0168] If the difference between the input frame rate in the third configuration and the output frame rate in the third configuration is greater than or equal to the first threshold, then adjust the output frame rate in the third configuration to obtain the optimal configuration.

[0169] In some embodiments of the present application, the adjustment unit 530 may further be configured to:

[0170] By reducing the resolution in the second configuration at least once to obtain a fourth configuration;

[0171] If the difference between the input frame rate in the fourth configuration and the output frame rate in the fourth configuration is less than the first threshold, then determine the fourth configuration as the optimal configuration;

[0172] If the difference between the input frame rate in the fourth configuration and the output frame rate in the fourth configuration is greater than or equal to the first threshold, then adjust the output frame rate in the fourth configuration to obtain the optimal configuration.

[0173] In some embodiments of the present application, the adjustment unit 530 may further be configured to:

[0174] Successively reduce the input frame rate and the resolution in the second configuration at least once to obtain the optimal configuration; or

[0175] Successively reduce the resolution and the input frame rate in the second configuration at least once to obtain the optimal configuration.

[0176] In some embodiments of the present application, the adjustment unit 530 may further be configured to:

[0177] If the single-frame delay in the default configuration is greater than or equal to the second threshold, then adjust the rendering control type in the default configuration to obtain a fifth configuration;

[0178] If the single-frame delay in the fifth configuration is less than the second threshold, then determine the fifth configuration as the optimal configuration;

[0179] If the single-frame delay in the fifth configuration is greater than or equal to the second threshold, then adjust the fifth configuration to obtain the optimal configuration.

[0180] In some embodiments of the present application, the adjustment unit 530 may further be configured to:

[0181] If the single-frame delay under the default configuration is greater than or equal to the second threshold, adjust the frame dropping policy under the default configuration to obtain a sixth configuration;

[0182] If the single-frame delay under the sixth configuration is less than the second threshold, determine the sixth configuration as the optimal configuration;

[0183] If the single-frame delay under the sixth configuration is greater than or equal to the second threshold, adjust the sixth configuration to obtain the optimal configuration.

[0184] In some embodiments of the present application, the apparatus 500 may further include:

[0185] A determination unit, specifically configured to, if the chip under the optimal configuration is a frame-storing chip, determine encoding parameters based on a multi-frame reference method; if the chip under the optimal configuration is a non-frame-storing chip, determine the encoding parameters based on a single-frame reference method.

[0186] In some embodiments of the present application, the determination unit may further be configured to:

[0187] Determine the configuration with the highest occurrence frequency among multiple configurations as the default configuration; the multiple configurations include the configurations with the best decoding and rendering effects when performing decoding and rendering tests for each of multiple devices.

[0188] In some embodiments of the present application, the determination unit may further be configured to: the second threshold is the reciprocal of the input frame rate of the video stream; or the second threshold is the product of the reciprocal of the input frame rate of the video stream and the number of frame storages of the chip when decoding and rendering the video stream.

[0189] It should be understood that the apparatus embodiments and the method embodiments can correspond to each other, and similar descriptions can refer to the method embodiments. To avoid repetition, it will not be elaborated here. For example, the apparatus 500 may correspond to the corresponding main body in the methods 300 to 400 of the embodiments of the present application, and each unit in the apparatus 500 respectively implements the corresponding processes in the methods 300 to 400. For the sake of brevity, it will not be elaborated here.

[0190] It should also be understood that each unit in the device 500 involved in the embodiments of the present application can be separately or wholly combined into one or several other units to form, or some of the units can be further split into multiple smaller units with functional division to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the device 500 may also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units. According to another embodiment of the present application, the device 500 involved in the embodiments of the present application can be constructed by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method on a general computing device of a general computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), and the method for obtaining the decoding and rendering configuration of the embodiments of the present application can be realized. Among them, the computer program can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device through the computer-readable storage medium, and run therein to realize the corresponding method of the embodiments of the present application.

[0191] In other words, the units mentioned above can be implemented in the form of hardware, can also be implemented by instructions in software form, or can be implemented in a combination of software and hardware. Specifically, the respective steps of the method embodiments in the embodiments of the present application can be completed by the integrated logic circuit in the hardware of the processor and / or instructions in software form. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software in the decoding processor. Optionally, the software can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, and a register. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps in the above method embodiments.

[0192] Figure 6 It is a schematic structural diagram of the electronic device 600 provided by the embodiments of the present application.

[0193] Such as Figure 6As shown, the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. Among them, the processor 610 and the computer-readable storage medium 620 can be connected through a bus or other means. The computer-readable storage medium 620 is used to store a computer program 621, and the computer program 621 includes computer instructions. The processor 610 is used to execute the computer instructions stored in the computer-readable storage medium 620. The processor 610 is the computing core and control core of the electronic device 600, and it is suitable for implementing one or more computer instructions. Specifically, it is suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0194] As an example, the processor 610 can also be referred to as a Central Processing Unit (CPU). The processor 610 may include, but is not limited to: a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on.

[0195] By way of example, the computer-readable storage medium 620 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one computer-readable storage medium located far from the aforementioned processor 610. Specifically, the computer-readable storage medium 620 includes but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synch link dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0196] In one implementation, the electronic device 600 can be Figure 5 the acquisition device 500 of the decoding and rendering configuration shown; computer instructions are stored in the computer-readable storage medium 620; the computer instructions stored in the computer-readable storage medium 620 are loaded and executed by the processor 610 to implement Figures 3 to 4 the corresponding steps in the method embodiments shown; in a specific implementation, the computer instructions in the computer-readable storage medium 620 are loaded and executed by the processor 610 to perform the corresponding steps. To avoid repetition, they are not described here again.

[0197] According to another aspect of the present application, embodiments of the present application further provide a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in the electronic device 600 and is used to store programs and data. For example, the computer-readable storage medium 620. It can be understood that the computer-readable storage medium 620 here can include both the built-in storage medium in the electronic device 600 and, of course, the extended storage medium supported by the electronic device 600. The computer-readable storage medium provides a storage space, and this storage space stores the operating system of the electronic device 600. And, in this storage space, one or more computer instructions suitable for being loaded and executed by the processor 610 are also stored. These computer instructions can be one or more computer programs 621 (including program codes).

[0198] The electronic device 600 may further include: a transceiver 630, and the transceiver 630 may be connected to the processor 610 or the computer-readable storage medium 620.

[0199] Among them, the computer-readable storage medium 620 can control the transceiver 630 to communicate with other devices. Specifically, it can send information or data to other devices, or receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include an antenna, and the number of antennas may be one or more.

[0200] According to another aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and these computer instructions are stored in a computer-readable storage medium. For example, the computer program 621. At this time, the electronic device 600 may be a computer. The processor 610 reads the computer instructions from the computer-readable storage medium 620, and the processor 610 executes the computer instructions, so that the computer executes the method for obtaining the decoding and rendering configuration provided in the above various optional manners.

[0201] In other words, when implemented using software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes of the embodiments of the present application are run in whole or in part, or the functions of the embodiments of the present application are implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.).

[0202] Those of ordinary skill in the art can realize that the units and process steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0203] Finally, it should be noted that the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for obtaining a decoding and rendering configuration, characterized in that, it includes: Obtaining a default configuration; Under the default configuration, obtaining a video bitstream and performing decoding and rendering on the video bitstream to obtain the output frame rate under the default configuration and the single-frame delay under the default configuration; If the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to a first threshold, and / or, the single-frame delay under the default configuration is greater than or equal to a second threshold, adjusting the default configuration to obtain an optimal configuration; wherein, the difference between the input frame rate under the optimal configuration and the output frame rate under the optimal configuration is less than the first threshold, and the single-frame delay under the optimal configuration is less than the second threshold; Adjusting the default configuration to the optimal configuration.

2. The method according to claim 1, characterized in that, the step of if the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to a first threshold, and / or, if the single-frame delay under the default configuration is greater than or equal to a second threshold, then adjusting the default configuration to obtain an optimal configuration includes: If the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, then adjusting the output frame rate under the default configuration by adjusting at least one of the chip type, reducing the input frame rate, and reducing the resolution to obtain the optimal configuration; If the single-frame delay under the default configuration is greater than or equal to the second threshold, then adjusting the single-frame delay under the default configuration by adjusting at least one of the rendering control type and the frame dropping strategy to obtain the optimal configuration.

3. The method according to claim 1, characterized in that, the step of if the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to a first threshold, and / or, if the single-frame delay under the default configuration is greater than or equal to a second threshold, then adjusting the default configuration to obtain an optimal configuration includes: If the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, then adjusting the chip type under the default configuration to obtain a first configuration; If the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is less than the first threshold, then determining the first configuration as the optimal configuration; If the difference between the input frame rate under the first configuration and the output frame rate under the first configuration is greater than or equal to the first threshold, then adjusting the output frame rate under the first configuration to obtain the optimal configuration.

4. The method according to claim 3, characterized in that, the step of adjusting the output frame rate under the first configuration to obtain the optimal configuration includes: Adjusting the chip type under the first configuration to the chip type with the highest output frame rate to obtain a second configuration; Adjusting the output frame rate under the second configuration by reducing at least one of the input frame rate and the resolution to obtain the optimal configuration.

5. The method according to claim 4, It is characterized in that adjusting the output frame rate in the second configuration by at least one of reducing the input frame rate and reducing the resolution to obtain the optimal configuration, including: obtaining a third configuration by reducing the input frame rate in the second configuration at least once; if the difference between the input frame rate in the third configuration and the output frame rate in the third configuration is less than the first threshold, determining the third configuration as the optimal configuration; if the difference between the input frame rate in the third configuration and the output frame rate in the third configuration is greater than or equal to the first threshold, adjusting the output frame rate in the third configuration to obtain the optimal configuration.

6. The method according to claim 4, It is characterized in that adjusting the output frame rate in the second configuration by at least one of reducing the input frame rate and reducing the resolution to obtain the optimal configuration, including: obtaining a fourth configuration by reducing the resolution in the second configuration at least once; if the difference between the input frame rate in the fourth configuration and the output frame rate in the fourth configuration is less than the first threshold, determining the fourth configuration as the optimal configuration; if the difference between the input frame rate in the fourth configuration and the output frame rate in the fourth configuration is greater than or equal to the first threshold, adjusting the output frame rate in the fourth configuration to obtain the optimal configuration.

7. The method according to claim 4, It is characterized in that adjusting the output frame rate in the second configuration by at least one of reducing the input frame rate and reducing the resolution to obtain the optimal configuration, including: successively obtaining the optimal configuration by reducing the input frame rate in the second configuration and the resolution in the second configuration at least once; or successively obtaining the optimal configuration by reducing the resolution in the second configuration and the input frame rate in the second configuration at least once.

8. The method according to any one of claims 1 to 7, It is characterized in that if the difference between the input frame rate in the default configuration and the output frame rate in the default configuration is greater than or equal to the first threshold, and / or if the single-frame delay in the default configuration is greater than or equal to the second threshold, adjusting the default configuration to obtain the optimal configuration, including: if the single-frame delay in the default configuration is greater than or equal to the second threshold, adjusting the rendering control type in the default configuration to obtain a fifth configuration; if the single-frame delay in the fifth configuration is less than the second threshold, determining the fifth configuration as the optimal configuration; if the single-frame delay in the fifth configuration is greater than or equal to the second threshold, adjusting the fifth configuration to obtain the optimal configuration.

9. The method according to any one of claims 1 to 7, It is characterized in that if the difference between the input frame rate in the default configuration and the output frame rate in the default configuration is greater than or equal to the first threshold, and / or if the single-frame delay in the default configuration is greater than or equal to the second threshold, adjusting the default configuration to obtain the optimal configuration, including: If the single-frame delay under the default configuration is greater than or equal to the second threshold, adjust the frame-drop strategy under the default configuration to obtain a sixth configuration; If the single-frame delay under the sixth configuration is less than the second threshold, determine the sixth configuration as the optimal configuration; If the single-frame delay under the sixth configuration is greater than or equal to the second threshold, adjust the sixth configuration to obtain the optimal configuration.

10. The method according to any one of claims 1 to 7, wherein, the method further includes: If the chip under the optimal configuration is a frame-storing chip, determine the encoding parameters based on the multi-frame reference method; If the chip under the optimal configuration is a non-frame-storing chip, determine the encoding parameters based on the single-frame reference method.

11. The method according to any one of claims 1 to 7, wherein, before obtaining the default configuration, the method further includes: Determine the configuration with the highest occurrence frequency among multiple configurations as the default configuration; the multiple configurations include the configurations with the best decoding and rendering effects when performing decoding and rendering tests for each of multiple devices.

12. The method according to any one of claims 1 to 7, wherein, the second threshold is the reciprocal of the input frame rate of the video bitstream; or the second threshold is the product of the reciprocal of the input frame rate of the video bitstream and the number of frame storages of the chip when decoding and rendering the video bitstream.

13. An apparatus for obtaining a decoding and rendering configuration, wherein, comprises: An obtaining unit, configured to obtain a default configuration; A decoding and rendering unit, configured to obtain a video bitstream and perform decoding and rendering on the video bitstream under the default configuration to obtain the output frame rate under the default configuration and the single-frame delay under the default configuration; An adjustment unit, configured to, if the difference between the input frame rate under the default configuration and the output frame rate under the default configuration is greater than or equal to the first threshold, and / or, the single-frame delay under the default configuration is greater than or equal to the second threshold, adjust the default configuration to obtain an optimal configuration; wherein, the difference between the input frame rate under the optimal configuration and the output frame rate under the optimal configuration is less than the first threshold, and the single-frame delay under the optimal configuration is less than the second threshold; The adjustment unit is further configured to adjust the default configuration to the optimal configuration.

14. An electronic device, wherein, comprises: A processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, wherein, used to store a computer program, and the computer program enables a computer to execute the method according to any one of claims 1 to 12.

Images