Method and device for optimizing cloud application screen freeze

By obtaining the parameters related to the decoding capability of the terminal device, calculating the new resolution, frame rate and code rate, and adjusting the encoder's encoding code rate, the problem of screen stuttering on terminal devices with weak performance is solved, and a smooth cloud application experience is achieved.

CN119276848BActive Publication Date: 2025-03-04HAIMA CLOUD TIANJIN INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411794224.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-09
Publication Date
2025-03-04
Estimated Expiration
2044-12-09

AI Technical Summary

Technical Problem

Cloud applications frequently appear on terminal devices with weak performance, such as increasing decoding delay and frame dropping, resulting in stuttering in the cloud application screen seen by users.

Method used

By obtaining the decoding capability related parameters of the terminal device, the new resolution, frame rate and code rate are calculated, and the encoder's encoding code rate and maximum encoding code rate are adjusted according to these parameters to match the decoding capability of the terminal device.

Benefits of technology

It realizes dynamic adjustment of encoding parameters according to the decoding capabilities of the terminal device, thereby reducing screen lag and ensuring that users have a clear and smooth cloud application experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119276848B_ABST
    Figure CN119276848B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for optimizing the freezing of cloud application screens, the method comprising: obtaining relevant parameters reflecting the decoding capability statistically obtained by a terminal device, and when the relevant parameters reflecting the decoding capability indicate that the decoding capability of the terminal device is insufficient, calculating a new resolution or a new frame rate, and a new bit rate; judging whether the new bit rate is greater than or equal to the bandwidth estimated by a bandwidth estimation BWE module, if the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, setting the maximum encoding bit rate of the encoder to the new bit rate, or if the new bit rate is less than the bandwidth estimated by the BWE module, adjusting the encoding bit rate of the encoder to the new bit rate, and setting the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate, and being able to adaptively adjust the encoding parameters according to the decoding capability of the terminal device, so that users can obtain a clear and smooth cloud application experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cloud applications, and in particular to a method and device for optimizing screen freezes in cloud applications. Background Art

[0002] With the rapid development of cloud computing and network technology, cloud applications (such as cloud games) have become one of the important trends in the application industry as a new application usage method. Cloud applications migrate all the complex computing and rendering tasks handled by user terminals in traditional applications to cloud servers, and transmit the computing results to user terminals in the form of video streams through efficient network transmission. This architecture significantly reduces the hardware requirements for user devices, allowing users to easily enjoy high-quality application experience through a variety of devices such as mobile phones, tablets, and smart TVs. For users, cloud applications eliminate the performance limitations of hardware devices. As long as network conditions permit, they can easily experience large-scale applications with high complexity and high performance requirements.

[0003] However, the user experience of cloud applications is highly dependent on network bandwidth, latency, and the decoding capabilities of terminal devices. WebRTC (Web Real-Time Communication) technology can already handle network-related issues such as bandwidth and latency very well, and has a certain degree of adaptability to the encoding capabilities of server devices, but there is no corresponding feedback and adjustment for the decoding capabilities of terminal devices, which will cause increased decoding delays and frame drops on terminal devices with weaker performance, causing the cloud application screen seen by users to freeze. Therefore, how to dynamically adjust the encoding parameters according to the decoding capabilities of the terminal device to ensure that each user has a smooth and high-quality application experience on their terminal device has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the embodiments of the present application provide a method and device for optimizing the freeze of cloud application screens.

[0005] In a first aspect, an embodiment of the present application provides a method for optimizing the freeze of a cloud application screen, comprising:

[0006] Obtain relevant parameters reflecting decoding capability counted by the terminal device, and when the relevant parameters reflecting decoding capability indicate that the decoding capability of the terminal device is insufficient, calculate a new resolution or a new frame rate, and a new bit rate, wherein the new resolution is smaller than the old resolution, the new frame rate is smaller than the old frame rate, and the new bit rate is calculated according to the new resolution or the new frame rate;

[0007] Determine whether the new bit rate is greater than or equal to the bandwidth estimated by the bandwidth estimation BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, set the maximum encoding bit rate of the encoder to the new bit rate; or if the new bit rate is less than the bandwidth estimated by the BWE module, adjust the encoding bit rate of the encoder to the new bit rate, and set the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate.

[0008] In a second aspect, the embodiment of the present application further provides a cloud application screen freeze optimization device, comprising:

[0009] a calculation unit, configured to obtain relevant parameters reflecting decoding capability counted by the terminal device, and when the relevant parameters reflecting decoding capability indicate that the decoding capability of the terminal device is insufficient, calculate a new resolution or a new frame rate, and a new bit rate, wherein the new resolution is smaller than the old resolution, the new frame rate is smaller than the old frame rate, and the new bit rate is calculated according to the new resolution or the new frame rate;

[0010] The processing unit is used to determine whether the new bit rate is greater than or equal to the bandwidth estimated by the bandwidth estimation BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, the maximum encoding bit rate of the encoder is set to the new bit rate, or if the new bit rate is less than the bandwidth estimated by the BWE module, the encoding bit rate of the encoder is adjusted to the new bit rate, and the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module are set to the new bit rate.

[0011] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cloud application screen freeze optimization method as described in the first aspect are executed.

[0012] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the cloud application screen freeze optimization method as described in the first aspect.

[0013] In summary, the cloud application screen freeze optimization method and device, electronic device and storage medium provided in the embodiments of the present application calculate a new resolution or a new frame rate when the relevant parameters reflecting the decoding capability statistically represented by the terminal device indicate that the decoding capability of the terminal device is insufficient, and calculate a new bit rate based on the new resolution or the new frame rate, and compare the new bit rate with the bandwidth estimated by the BWE module. When the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, the maximum encoding bit rate of the encoder is set to the new bit rate; and when the new bit rate is less than the bandwidth estimated by the BWE module, the encoding bit rate of the encoder is adjusted to the new bit rate, and the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module are set to the new bit rate. In this way, the encoding parameters can be adaptively adjusted according to the decoding capability of the terminal device, so that users can obtain a clear and smooth cloud application experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of an embodiment of a method for optimizing a cloud application screen freeze provided in an embodiment of the present application;

[0015] Figure 2 A structural diagram of an embodiment of a cloud application screen freeze optimization device provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0018] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0019] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0020] Reference Figure 1 As shown, an embodiment of the present application provides a flow chart of a method for optimizing a cloud application screen freeze, the method comprising:

[0021] S10, obtaining relevant parameters reflecting decoding capability counted by the terminal device, and when the relevant parameters reflecting decoding capability indicate that the decoding capability of the terminal device is insufficient, calculating a new resolution or a new frame rate, and a new bit rate, wherein the new resolution is smaller than the old resolution, the new frame rate is smaller than the old frame rate, and the new bit rate is calculated according to the new resolution or the new frame rate;

[0022] In this embodiment, it should be noted that the executor of steps S10 and S11 is the cloud server, and the terminal can report relevant parameters reflecting the decoding capability to the cloud server according to the set period. The relevant parameters may include at least one of the receiving frame rate, decoding frame rate, decoding delay and rendering frame rate. The receiving frame rate, decoding frame rate, decoding delay and rendering frame rate are terms in the web real-time communication WebRTC technology, and their meanings are not repeated here. When the relevant parameters reflecting the decoding capability indicate that the decoding capability of the terminal device is insufficient, a new resolution or a new frame rate is calculated, and a new bit rate is calculated based on the new resolution or the new frame rate. After calculating the new resolution or the new frame rate, it is necessary to adjust the encoding resolution of the encoder to the new resolution or adjust the encoding frame rate of the encoder to the new frame rate.

[0023] S11. Determine whether the new bit rate is greater than or equal to the bandwidth estimated by the bandwidth estimation BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, set the maximum encoding bit rate of the encoder to the new bit rate. Or if the new bit rate is less than the bandwidth estimated by the BWE module, adjust the encoding bit rate of the encoder to the new bit rate, and set the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate.

[0024] In this embodiment, it should be noted that the BWE module is a module in the WebRTC technology for estimating bandwidth, and the encoding bit rate of the encoder is the smaller value between the new bit rate and the bandwidth estimated by the BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, the current encoding bit rate of the encoder cannot be adjusted, and only the maximum encoding bit rate of the encoder needs to be set to the new bit rate; if the new bit rate is less than the bandwidth estimated by the BWE module, it is necessary to adjust the current encoding bit rate of the encoder to the new bit rate, and set the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate.

[0025] The cloud application screen freeze optimization method provided in the embodiment of the present application calculates a new resolution or a new frame rate when the relevant parameters reflecting the decoding capability statistically represented by the terminal device indicate that the decoding capability of the terminal device is insufficient, and calculates a new bit rate based on the new resolution or the new frame rate, and by comparing the new bit rate with the bandwidth estimated by the BWE module, when the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, sets the maximum encoding bit rate of the encoder to the new bit rate; and when the new bit rate is less than the bandwidth estimated by the BWE module, adjusts the encoding bit rate of the encoder to the new bit rate, and sets the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate. In this way, the encoding parameters can be adaptively adjusted according to the decoding capability of the terminal device, so that users can obtain a clear and smooth cloud application experience.

[0026] Based on the foregoing method embodiment, when the relevant parameters reflecting the decoding capability meet the preset conditions, it is determined that the decoding capability of the terminal device is insufficient;

[0027] Among them, the preset conditions may include at least one of the first condition, the second condition, the third condition, the fourth condition and the fifth condition. The first condition is that the ratio of the decoding frame rate to the receiving frame rate is less than the first threshold, the second condition is that the ratio of the rendering frame rate to the receiving frame rate is less than the second threshold, the third condition is that the decoding frame rate variance is greater than the third threshold, the fourth condition is that the rendering frame rate variance is greater than the fourth threshold, and the fifth condition is that the decoding delay variance is greater than the fifth threshold.

[0028] In this embodiment, it should be noted that when judging whether the relevant parameters reflecting the decoding capability meet the preset conditions, the judgment is based on the data in a moving time window of a fixed length (such as 5 seconds). For example, assuming that the preset conditions include the first condition and the third condition, there are 5 groups of data in the current time window, namely (receiving frame rate 1, decoding frame rate 1), (receiving frame rate 2, decoding frame rate 2), (receiving frame rate 3, decoding frame rate 3), (receiving frame rate 4, decoding frame rate 4) and (receiving frame rate 5, decoding frame rate 5), then when the ratio of decoding frame rate 1 to receiving frame rate 1 is less than the first threshold, when the ratio of decoding frame rate 2 to receiving frame rate 2 is less than the first threshold, when the ratio of decoding frame rate 3 to receiving frame rate 3 is less than the first threshold, when the ratio of decoding frame rate 4 to receiving frame rate 4 is less than the first threshold, when the ratio of decoding frame rate 5 to receiving frame rate 5 is less than the first threshold, and the variances corresponding to decoding frame rate 1, decoding frame rate 2, decoding frame rate 3, decoding frame rate 4 and decoding frame rate 5 are all greater than the third threshold, it is determined that the relevant parameters reflecting the decoding capability meet the preset conditions, wherein the variance corresponding to the decoding frame rate can be obtained by inputting the decoding frame rate into the update function in the WebRTC technology. When the preset conditions are other conditions, the method for judging whether the relevant parameters reflecting the decoding capability meet the preset conditions is consistent with the judgment method in this example, and will not be repeated here.

[0029] Based on the aforementioned method embodiment, the calculation formula of the width new_width of the new resolution may be new_width=old_width×α, and the calculation formula of the height new_height of the new resolution may be new_height=old_height×β, wherein old_width and old_height are respectively the width and the height of the old resolution, α and β are adjustment parameters, α∈(0,1), β∈(0,1);

[0030] The calculation formula of the new bit rate new_bitrate can be new_bitrate=new_width×new_height×old_bitrate / (old_width×old_height)×M, where old_bitrate is the old bit rate, M is the adjustment parameter, M∈(1,+∞).

[0031] In this embodiment, it should be noted that α and β can be the same or different, and the values ​​can be set as needed, for example, α and β are both set to 2 / 3. The value of M can be set as needed. In a preferred embodiment, M=1.2×m, m=log2(old_width×old_height / (new_width×new_height)).

[0032] Based on the aforementioned method embodiment, the calculation formula of the new frame rate new_fps may be new_fps=old_fps×γ, where old_fps is the old frame rate, γ is the adjustment parameter, γ∈(0,1);

[0033] The calculation formula of the new bit rate new_bitrate can be new_bitrate=new_fps×old_bitrate / old_fps×N, where old_bitrate is the old bit rate, N is the adjustment parameter, N∈(1,+∞).

[0034] In this embodiment, it should be noted that the value of γ can be set as needed, for example, it can be set to 1 / 2. The value of N can be set as needed. In a preferred embodiment, N=1.2×n, n=log2(old_fps / new_fps).

[0035] On the basis of the foregoing method embodiment, setting the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to a new bit rate may include:

[0036] Set the maximum bandwidth limit estimated by the BWE module to the new bitrate.

[0037] In this embodiment, it should be noted that the maximum limited bandwidth estimated by the BWE module is set to the new bit rate, so as to ensure that the encoding bit rate of the encoder and the sending bit rate of the sending module will not exceed the new bit rate.

[0038] Reference Figure 2 , which is a schematic diagram of the structure of a cloud application screen freeze optimization device provided by an embodiment of the present application, and the device includes:

[0039] The calculation unit 20 is used to obtain relevant parameters reflecting the decoding capability of the terminal device, and when the relevant parameters reflecting the decoding capability indicate that the decoding capability of the terminal device is insufficient, calculate a new resolution or a new frame rate, and a new bit rate, wherein the new resolution is smaller than the old resolution, the new frame rate is smaller than the old frame rate, and the new bit rate is calculated according to the new resolution or the new frame rate;

[0040] The processing unit 21 is used to determine whether the new bit rate is greater than or equal to the bandwidth estimated by the bandwidth estimation BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, the maximum encoding bit rate of the encoder is set to the new bit rate; if the new bit rate is less than the bandwidth estimated by the BWE module, the encoding bit rate of the encoder is adjusted to the new bit rate, and the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module are set to the new bit rate.

[0041] The cloud application screen freeze optimization device provided in the embodiment of the present application calculates a new resolution or a new frame rate when the relevant parameters reflecting the decoding capability statistically represented by the terminal device indicate that the decoding capability of the terminal device is insufficient, and calculates a new bit rate based on the new resolution or the new frame rate, and by comparing the new bit rate with the bandwidth estimated by the BWE module, when the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, sets the maximum encoding bit rate of the encoder to the new bit rate; and when the new bit rate is less than the bandwidth estimated by the BWE module, adjusts the encoding bit rate of the encoder to the new bit rate, and sets the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate. In this way, the encoding parameters can be adaptively adjusted according to the decoding capability of the terminal device, so that users can obtain a clear and smooth cloud application experience.

[0042] Based on the above-mentioned device embodiment, when the relevant parameters reflecting the decoding capability meet the preset conditions, it is determined that the decoding capability of the terminal device is insufficient;

[0043] Among them, the preset conditions may include at least one of the first condition, the second condition, the third condition, the fourth condition and the fifth condition. The first condition is that the ratio of the decoding frame rate to the receiving frame rate is less than the first threshold, the second condition is that the ratio of the rendering frame rate to the receiving frame rate is less than the second threshold, the third condition is that the decoding frame rate variance is greater than the third threshold, the fourth condition is that the rendering frame rate variance is greater than the fourth threshold, and the fifth condition is that the decoding delay variance is greater than the fifth threshold.

[0044] Based on the aforementioned device embodiment, the calculation formula of the width new_width of the new resolution may be new_width=old_width×α, and the calculation formula of the height new_height of the new resolution may be new_height=old_height×β, wherein old_width and old_height are the width and height of the old resolution respectively, α and β are adjustment parameters, α∈(0,1), β∈(0,1);

[0045] The calculation formula of the new bit rate new_bitrate can be new_bitrate=new_width×new_height×old_bitrate / (old_width×old_height)×M, where old_bitrate is the old bit rate, M is the adjustment parameter, M∈(1,+∞).

[0046] Based on the aforementioned device embodiment, the calculation formula of the new frame rate new_fps may be new_fps=old_fps×γ, where old_fps is the old frame rate, γ is the adjustment parameter, γ∈(0,1);

[0047] The calculation formula of the new bit rate new_bitrate can be new_bitrate=new_fps×old_bitrate / old_fps×N, where old_bitrate is the old bit rate, N is the adjustment parameter, N∈(1,+∞).

[0048] Based on the above device embodiment, the processing unit may be used for:

[0049] Set the maximum bandwidth limit estimated by the BWE module to the new bitrate.

[0050] The cloud application screen freeze optimization device provided in the embodiment of the present application has an implementation process that is consistent with the cloud application screen freeze optimization method provided in the embodiment of the present application, and the effect that can be achieved is also the same as the cloud application screen freeze optimization method provided in the embodiment of the present application, and will not be repeated here.

[0051] like Figure 3 As shown, an electronic device provided by an embodiment of the present application includes: a processor 30, a memory 31 and a bus 32, wherein the memory 31 stores machine-readable instructions executable by the processor 30, and when the electronic device is running, the processor 30 communicates with the memory 31 through the bus 32, and the processor 30 executes the machine-readable instructions to perform the steps of the above-mentioned cloud application screen freeze optimization method.

[0052] Specifically, the above-mentioned memory 31 and processor 30 can be general-purpose memory and processor, which are not specifically limited here. When the processor 30 runs the computer program stored in the memory 31, the above-mentioned cloud application screen freeze optimization method can be executed.

[0053] Corresponding to the above-mentioned cloud application screen freeze optimization method, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned cloud application screen freeze optimization method are executed.

[0054] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for optimizing the freezing of cloud application screens, characterized in that: include: Obtain relevant parameters reflecting decoding capability counted by the terminal device, and when the relevant parameters reflecting decoding capability indicate that the decoding capability of the terminal device is insufficient, calculate a new resolution or a new frame rate, and a new bit rate, wherein the new resolution is smaller than the old resolution, the new frame rate is smaller than the old frame rate, and the new bit rate is calculated according to the new resolution or the new frame rate; Determine whether the new bit rate is greater than or equal to the bandwidth estimated by the bandwidth estimation BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, set the maximum encoding bit rate of the encoder to the new bit rate; or if the new bit rate is less than the bandwidth estimated by the BWE module, adjust the encoding bit rate of the encoder to the new bit rate, and set the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module to the new bit rate.

2. The method according to claim 1, characterized in that When the relevant parameters reflecting the decoding capability meet the preset conditions, determining that the decoding capability of the terminal device is insufficient; Among them, the preset conditions include at least one of the first condition, the second condition, the third condition, the fourth condition and the fifth condition. The first condition is that the ratio of the decoding frame rate to the receiving frame rate is less than the first threshold, the second condition is that the ratio of the rendering frame rate to the receiving frame rate is less than the second threshold, the third condition is that the decoding frame rate variance is greater than the third threshold, the fourth condition is that the rendering frame rate variance is greater than the fourth threshold, and the fifth condition is that the decoding delay variance is greater than the fifth threshold.

3. The method according to claim 1 or 2, characterized in that The calculation formula of the width new_width of the new resolution is new_width=old_width×α, and the calculation formula of the height new_height of the new resolution is new_height=old_height×β, where old_width and old_height are the width and height of the old resolution respectively, α and β are adjustment parameters, α∈(0,1), β∈(0,1); The calculation formula of the new bit rate new_bitrate is new_bitrate=new_width×new_height×old_bitrate / (old_width×old_height)×M, where old_bitrate is the old bit rate and M is the adjustment parameter, M∈(1,+∞).

4. The method according to claim 1 or 2, characterized in that: The calculation formula of the new frame rate new_fps is new_fps=old_fps×γ, where old_fps is the old frame rate, γ is the adjustment parameter, γ∈(0,1); The calculation formula of the new bit rate new_bitrate is new_bitrate=new_fps×old_bitrate / old_fps×N, where old_bitrate is the old bit rate, N is the adjustment parameter, N∈(1,+∞).

5. The method according to claim 1, characterized in that The setting of the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module as new bit rates includes: Set the maximum bandwidth limit estimated by the BWE module to the new bitrate.

6. A cloud application screen freeze optimization device, characterized in that: include: a calculation unit, configured to obtain relevant parameters reflecting decoding capability counted by the terminal device, and when the relevant parameters reflecting decoding capability indicate that the decoding capability of the terminal device is insufficient, calculate a new resolution or a new frame rate, and a new bit rate, wherein the new resolution is smaller than the old resolution, the new frame rate is smaller than the old frame rate, and the new bit rate is calculated according to the new resolution or the new frame rate; The processing unit is used to determine whether the new bit rate is greater than or equal to the bandwidth estimated by the bandwidth estimation BWE module. If the new bit rate is greater than or equal to the bandwidth estimated by the BWE module, the maximum encoding bit rate of the encoder is set to the new bit rate, or if the new bit rate is less than the bandwidth estimated by the BWE module, the encoding bit rate of the encoder is adjusted to the new bit rate, and the maximum encoding bit rate of the encoder and the maximum sending bit rate of the sending module are set to the new bit rate.

7. The device according to claim 6, characterized in that When the relevant parameters reflecting the decoding capability meet the preset conditions, determining that the decoding capability of the terminal device is insufficient; Among them, the preset conditions include at least one of the first condition, the second condition, the third condition, the fourth condition and the fifth condition. The first condition is that the ratio of the decoding frame rate to the receiving frame rate is less than the first threshold, the second condition is that the ratio of the rendering frame rate to the receiving frame rate is less than the second threshold, the third condition is that the decoding frame rate variance is greater than the third threshold, the fourth condition is that the rendering frame rate variance is greater than the fourth threshold, and the fifth condition is that the decoding delay variance is greater than the fifth threshold.

8. The device according to claim 6 or 7, characterized in that The calculation formula of the width new_width of the new resolution is new_width=old_width×α, and the calculation formula of the height new_height of the new resolution is new_height=old_height×β, where old_width and old_height are the width and height of the old resolution respectively, α and β are adjustment parameters, α∈(0,1), β∈(0,1); The calculation formula of the new bit rate new_bitrate is new_bitrate=new_width×new_height×old_bitrate / (old_width×old_height)×M, where old_bitrate is the old bit rate and M is the adjustment parameter, M∈(1,+∞).

9. The device according to claim 6 or 7, characterized in that The calculation formula of the new frame rate new_fps is new_fps=old_fps×γ, where old_fps is the old frame rate, γ is the adjustment parameter, γ∈(0,1); The calculation formula of the new bit rate new_bitrate is new_bitrate=new_fps×old_bitrate / old_fps×N, where old_bitrate is the old bit rate, N is the adjustment parameter, N∈(1,+∞).

10. The device according to claim 6, characterized in that The processing unit is used for: Set the maximum bandwidth limit estimated by the BWE module to the new bitrate.

Citation Information

Patent Citations

  • Video bit rate transmission control device and control method

    CN107623851A

  • Remote video transmission method and transmission device, storage medium and electronic equipment

    CN113747194A