Cloud application picture lag optimization method and device
By dynamically adjusting the resolution or code rate of the encoder on the cloud server, the frame rate of the real -time coding resources of the cloud application video is solved, and the problem of the screen stuck caused by the insufficient encoding resources, and the stable picture quality and frame rate are achieved.
Patent Information
- Application Number
- CN202411886533.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-01-21
- Estimated Expiration
- 2044-12-20
AI Technical Summary
The real-time video encoding resources of cloud applications on cloud servers are insufficient, resulting in a decrease in frame rate or unstable, which in turn causes the problem of terminal playback screen stuttering.
By obtaining the current encoding resource usage or code rate of the encoder, the corresponding relationship of the preset encoding attribute and the VMAF value is used to dynamically adjust the resolution or code rate of the encoder to match the target coding resource usage or code rate to ensure the ensuring The picture quality is stable and reduced.
While ensuring clear image quality, by dynamically adjusting encoding attributes, the frame rate can be maintained or stabilized, and lags in terminal playback of cloud application images can be reduced.
Smart Images

Figure CN119342043A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cloud applications, and in particular to a method and device for optimizing screen freezes in cloud applications. Background Art
[0002] Cloud services based on video generation and processing technologies have flourished in recent years and are widely used in cloud applications, live streaming, short videos, and long videos. These services require real-time video processing on cloud servers. Only by generating video streams that meet the end-user playback requirements in real time can the user experience be guaranteed. However, due to limitations in the cloud server's encoder performance and / or internal factors, insufficient real-time encoding resources can lead to frame rate drops or instabilities, and / or encoding output bitrates exceeding the set bitrate, resulting in interruptions such as lag when playing back cloud applications on the end-user.
[0003] In view of this, how to dynamically adjust the encoding properties based on the performance of the cloud server to reduce the lag of terminal playback of cloud application images has become a technical problem that needs to be solved urgently. Summary of the Invention
[0004] In response to the technical problems existing in the prior art, the embodiments of the present application provide a method and device for optimizing cloud application screen freezes.
[0005] In a first aspect, an embodiment of the present application provides a method for optimizing screen freezes in cloud applications, which is applied to a cloud server and includes: Obtaining a current encoding resource usage of the encoder, and when the current encoding resource usage is greater than a first value, determining a target encoding resource usage from a first correspondence between a preset first encoding attribute, an encoding resource usage, and a VMAF value, determining at least one piece of data in the first correspondence whose encoding resource usage is less than or equal to a second value, and if the number of the at least one piece of data is greater than 1, determining at least one piece of data having a maximum corresponding VMAF value among the at least one piece of data, determining target data from the at least one piece of data having the maximum corresponding VMAF value, determining a target resolution based on a resolution of the target data, and adjusting the resolution of the encoder to the target resolution, wherein the first correspondence includes multiple pieces of data, each piece of the multiple pieces of data includes a set of first encoding attributes, an encoding resource usage, and a VMAF value, the first encoding attribute includes resolution, the second value is determined based on the current encoding resource usage and the target encoding resource usage, and the target data is one piece of data among the at least one piece of data having the maximum corresponding VMAF value; and / or Obtaining a current bit rate of the encoder, and when the current bit rate is greater than a third value, determining a target bit rate from a second correspondence between a preset second coding attribute and a VMAF value, determining at least one data item in the second correspondence whose bit rate is less than or equal to a fourth value, and if the number of the at least one data item is greater than 1, determining at least one data item with a maximum corresponding VMAF value among the at least one data item, determining target data from the at least one data item with the maximum corresponding VMAF value, determining a target resolution based on a resolution of the target data, and adjusting the resolution of the encoder to the target resolution, wherein the second correspondence includes multiple data items, each of the multiple data items includes a set of second coding attributes and a VMAF value, the second coding attributes include resolution and bit rate, the fourth value is determined based on the current bit rate and the target bit rate, and the target data item is one of the at least one data item with the maximum corresponding VMAF value.
[0006] In a second aspect, an embodiment of the present application further provides a cloud application screen freeze optimization device, which is applied to a cloud server and includes: a first adjustment unit, configured to obtain a current encoding resource usage of the encoder, and when the current encoding resource usage is greater than a first value, determine a target encoding resource usage from a first correspondence between a preset first encoding attribute, an encoding resource usage, and a VMAF value, determine at least one piece of data in the first correspondence whose encoding resource usage is less than or equal to a second value, and if the number of the at least one piece of data is greater than 1, determine at least one piece of data having a maximum corresponding VMAF value among the at least one piece of data, determine target data from the at least one piece of data having a maximum corresponding VMAF value, determine a target resolution based on a resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the first correspondence includes a plurality of pieces of data, each piece of the plurality of pieces of data includes a set of first encoding attributes, an encoding resource usage, and a VMAF value, the first encoding attribute includes resolution, the second value is determined based on the current encoding resource usage and the target encoding resource usage, and the target data is one piece of data among the at least one piece of data having a maximum corresponding VMAF value; and / or The second adjustment unit is configured to obtain a current bit rate of the encoder, determine a target bit rate from a second correspondence between a preset second coding attribute and a VMAF value when the current bit rate is greater than a third value, determine at least one data item in the second correspondence whose bit rate is less than or equal to a fourth value, determine, if the number of the at least one data item is greater than 1, determine at least one data item with a maximum corresponding VMAF value among the at least one data item, determine target data from the at least one data item with the maximum corresponding VMAF value, determine a target resolution based on a resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the second correspondence includes multiple data items, each of the multiple data items includes a set of second coding attributes and a VMAF value, the second coding attributes include resolution and bit rate, the fourth value is determined based on the current bit rate and the target bit rate, and the target data item is one of the at least one data item with the maximum corresponding VMAF value.
[0007] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cloud application screen freeze optimization method as described in the first aspect are executed.
[0008] In a fourth aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the cloud application screen freeze optimization method as described in the first aspect.
[0009] In summary, the cloud application screen freeze optimization method and device, electronic device and storage medium provided by the embodiments of the present application, when the current encoding resource usage of the encoder is greater than the first value, the target encoding resource usage is determined from the first correspondence relationship between the preset first encoding attribute, the encoding resource usage and the VMAF value, and at least one data in the first correspondence relationship whose encoding resource usage is less than or equal to the second value is determined based on the target encoding resource usage, and the target data is determined from the at least one data, and the resolution of the encoder is adjusted according to the target data. When the real-time encoding resources are insufficient, the frame rate can be maintained / stabilized while ensuring clear image quality; and / or when the current bit rate of the encoder is greater than the third value, the target bit rate is determined from the second correspondence relationship between the preset second encoding attribute and the VMAF value, and at least one data in the second correspondence relationship whose bit rate is less than or equal to the fourth value is determined based on the target bit rate, and the target data is determined from the at least one data, and the resolution of the encoder is adjusted according to the target data. When the encoding output bit rate of the encoder exceeds the set bit rate, the encoding output bit rate can be reduced while ensuring clear image quality. That is, through the above scheme, the freeze of the cloud application screen played by the terminal can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 A flowchart of an embodiment of a method for optimizing screen freezes in cloud applications provided by an embodiment of the present application; Figure 2 A schematic diagram of the structure of an embodiment of a cloud application screen freeze optimization device provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0011] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.
[0012] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.
[0013] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0014] Reference Figure 1 As shown, an embodiment of the present application provides a flow chart of a method for optimizing a cloud application screen freeze, which is applied to a cloud server and includes: S10. Obtain a current encoding resource usage of the encoder. When the current encoding resource usage is greater than a first value, determine a target encoding resource usage from a first correspondence between a preset first encoding attribute, encoding resource usage, and VMAF value, determine at least one piece of data in the first correspondence whose encoding resource usage is less than or equal to a second value, and if the number of the at least one piece of data is greater than 1, determine at least one piece of data having a maximum corresponding VMAF value among the at least one piece of data, determine target data from the at least one piece of data having a maximum corresponding VMAF value, determine a target resolution based on the resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the first correspondence includes multiple pieces of data, each piece of the multiple pieces of data includes a set of first encoding attributes, a encoding resource usage, and a VMAF value, the first encoding attribute includes resolution, the second value is determined based on the current encoding resource usage and the target encoding resource usage, and the target data is one piece of data among the at least one piece of data having a maximum corresponding VMAF value; and / or S11. Obtain a current bit rate of the encoder. When the current bit rate is greater than a third value, determine a target bit rate from a second correspondence between a preset second coding attribute and a VMAF value. Determine at least one data item in the second correspondence whose bit rate is less than or equal to a fourth value. If the number of the at least one data item is greater than 1, determine at least one data item with a maximum corresponding VMAF value among the at least one data item. Determine target data from the at least one data item with the maximum corresponding VMAF value. Determine a target resolution based on a resolution of the target data. Adjust the resolution of the encoder to the target resolution. The second correspondence includes multiple data items. Each of the multiple data items includes a set of second coding attributes and a VMAF value. The second coding attributes include resolution and bit rate. The fourth value is determined based on the current bit rate and the target bit rate. The target data item is one of the at least one data item with the maximum corresponding VMAF value.
[0015] In this embodiment, it should be noted that before executing step S10 , the first corresponding relationship needs to be determined; before executing step S11 , the second corresponding relationship needs to be determined. When determining the first and second correspondences, multiple resolutions (the multiple resolutions are between the lowest and highest resolutions, or between the resolutions consisting of the short side of the lowest and highest resolutions and the resolutions consisting of the long side of the lowest and highest resolutions) can be generated based on the lowest and highest resolutions of the cloud application online service. Multiple bitrates (the multiple bitrates are between the lowest and highest bitrates) can be generated based on the lowest and highest bitrates of the cloud application online service. Multiple encoding types (for example, including H264, H265, and VP8), multiple resolutions, and multiple bitrates can be combined to obtain multiple sets of encoding properties. The multiple sets of encoding properties are used to configure the encoder, obtain the encoding resource usage during encoding, and obtain the video output by the encoder. The VMAF value of the video can then be calculated. This results in multiple sets of data (each set of data includes an encoding type, a resolution, a bitrate, an encoding resource usage, and a VMAF value). The first and second correspondences can be determined based on the multiple sets of data, for example, using the multiple sets of data as the first and second correspondences. The first value is a preset threshold for encoding resource usage; the third value is a preset threshold for bitrate. The encoding resource usage may be a usage rate of an encoding unit of a GPU.
[0016] In addition, if the target data includes other coding attributes in addition to resolution, the value of the coding attribute corresponding to the encoder can be adjusted according to the value of the other coding attributes contained in the target data. For example, if the target data also includes bit rate, the bit rate of the encoder can be adjusted according to the bit rate contained in the target data. Specifically, the bit rate of the encoder can be adjusted to the bit rate contained in the target data.
[0017] The cloud application screen freeze optimization method provided in an embodiment of the present application, when the current encoding resource usage of the encoder is greater than a first value, determines a target encoding resource usage from a first correspondence between a preset first encoding attribute, encoding resource usage, and VMAF value, determines at least one data in the first correspondence whose encoding resource usage is less than or equal to a second value based on the target encoding resource usage, determines target data from the at least one data, adjusts the encoder resolution based on the target data, and can maintain / stabilize the frame rate while ensuring clear image quality when real-time encoding resources are insufficient; and / or when the current bit rate of the encoder is greater than a third value, determines a target bit rate from a second correspondence between a preset second encoding attribute and VMAF value, determines at least one data in the second correspondence whose bit rate is less than or equal to a fourth value based on the target bit rate, determines target data from the at least one data, adjusts the encoder resolution based on the target data, and can reduce the encoding output bit rate while ensuring clear image quality when the encoding output bit rate of the encoder exceeds the set bit rate. That is, through the above solution, the freeze of the cloud application screen played by the terminal can be reduced.
[0018] Based on the aforementioned method embodiment, determining the target coding resource usage from the first correspondence between the preset first coding attribute, the coding resource usage, and the VMAF value may include: Determining a comparison order of target data types in the first corresponding relationship, wherein the target data types include a resolution width and a resolution height; According to the comparison order of the target data types, traverse each target data type in the comparison order, and filter the data whose value under the target data type in the first corresponding relationship is the same as the value under the target data type currently under the encoder or the absolute value of the difference is the smallest, to obtain filtered data, wherein the data source of the latter traversal and filtering is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type in the at least two data is smaller than the value under the target data type currently under the encoder; First intermediate data is determined according to the finally obtained screening data, and the coding resource usage in the first intermediate data is determined as the target coding resource usage.
[0019] In this embodiment, a specific example is used to illustrate the process of determining the target coding resource usage. It is assumed that the target data type includes the coding type, the width of the resolution, the height of the resolution, and the bit rate, and the comparison order of the target data type is coding type, the width of the resolution, the height of the resolution, and the bit rate from front to back. First, the data containing the same encoding type as the current encoding type of the encoder can be filtered out from the first corresponding relationship to obtain the filtered data of the first traversal; then the data containing the smallest absolute value of the difference between the width of the resolution and the width of the current resolution of the encoder can be filtered out from the filtered data of the first traversal to obtain the filtered data of the second traversal. It should be noted that if the data containing the smallest absolute value of the difference between the width of the resolution and the width of the current resolution of the encoder in the filtered data of the first traversal includes at least two data, then the filtered data of the second traversal is the data containing the width of the resolution smaller than the width of the current resolution of the encoder in the at least two data; then the data containing the smallest absolute value of the difference between the height of the resolution and the height of the current resolution of the encoder can be filtered out from the filtered data of the second traversal to obtain the filtered data of the third traversal. If the data containing the smallest absolute value of the difference between the height of the resolution and the height of the current resolution of the encoder in the filtered data of the second traversal includes at least two data, then the filtered data of the third traversal is the data containing the height of the resolution smaller than the height of the current resolution of the encoder in the at least two data. Finally, the data with the smallest absolute value of the difference between the code rate and the current code rate of the encoder can be filtered out from the filtered data of the third traversal to obtain the first intermediate data. If the data with the smallest absolute value of the difference between the code rate and the current code rate of the encoder contained in the filtered data of the third traversal includes one piece of data, then the data is the first intermediate data, or if the data with the smallest absolute value of the difference between the code rate and the current code rate of the encoder contained in the filtered data of the third traversal includes at least two pieces of data, then the first intermediate data is the data with a code rate less than the current code rate of the encoder contained in the at least two pieces of data. In addition, it should be noted that in the aforementioned traversal process, if a unique piece of data is determined when traversing to a certain target data type, then the data is the first intermediate data. After the first intermediate data is determined, the coding resource usage in the first intermediate data is the target coding resource usage.
[0020] Based on the aforementioned method embodiment, the second value=target coding resource usage-(current coding resource usage-first value).
[0021] Based on the aforementioned method embodiment, determining the target bitrate from the second correspondence between the preset second encoding attribute and the VMAF value may include: Determining a comparison order of target data types in the second corresponding relationship, wherein the target data types include resolution width, resolution height, and bit rate; According to the comparison order of the target data types, traverse each target data type in the comparison order, and filter the data whose value under the target data type in the first corresponding relationship is the same as the value under the target data type currently under the encoder or the absolute value of the difference is the smallest, to obtain filtered data, wherein the data source of the latter traversal and filtering is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type in the at least two data is smaller than the value under the target data type currently under the encoder; Second intermediate data is determined based on the finally obtained filtered data, and the bit rate in the second intermediate data is determined as the target bit rate.
[0022] The process of determining the second intermediate data in this embodiment is consistent with the process of determining the first intermediate data in the previous embodiment, and will not be repeated here.
[0023] Based on the foregoing method embodiment, the fourth value=target bit rate-(current bit rate-third value).
[0024] Based on the above method embodiment, the calculation formula of the width W2 of the target resolution can be: W2 = (W1×H1×W / H) 1 / 2 The calculation formula for the target resolution height H2 can be: H2= (W1×H1×H / W) 1 / 2 , where W1 and H1 are the width and height of the target data resolution, and W and H are the width and height of the encoder's current resolution.
[0025] Reference Figure 2 FIG. 1 is a schematic diagram of a cloud application screen freeze optimization device provided by an embodiment of the present application. The device is applied to a cloud server and includes: A first adjustment unit 20 is configured to obtain a current encoding resource usage of the encoder, and when the current encoding resource usage is greater than a first value, determine a target encoding resource usage from a first correspondence relationship among a preset first encoding attribute, encoding resource usage, and VMAF value, determine at least one piece of data in the first correspondence relationship whose encoding resource usage is less than or equal to a second value, and if the number of the at least one piece of data is greater than 1, determine at least one piece of data with a maximum corresponding VMAF value among the at least one piece of data, determine target data from the at least one piece of data with the maximum corresponding VMAF value, determine a target resolution based on the resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the first correspondence relationship includes multiple pieces of data, each piece of the multiple pieces of data includes a set of first encoding attributes, a encoding resource usage, and a VMAF value, the first encoding attribute includes resolution, the second value is determined based on the current encoding resource usage and the target encoding resource usage, and the target data is one piece of data among the at least one piece of data with the maximum corresponding VMAF value; and / or The second adjustment unit 21 is configured to obtain a current bit rate of the encoder, and when the current bit rate is greater than a third value, determine a target bit rate from a second correspondence between a preset second coding attribute and a VMAF value, determine at least one data item in the second correspondence whose bit rate is less than or equal to a fourth value, and if the number of the at least one data item is greater than one, determine at least one data item with a maximum corresponding VMAF value among the at least one data item, determine target data among the at least one data item with the maximum corresponding VMAF value, determine a target resolution based on a resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the second correspondence includes multiple data items, each of the multiple data items includes a set of second coding attributes and a VMAF value, the second coding attributes include resolution and bit rate, the fourth value is determined based on the current bit rate and the target bit rate, and the target data item is one of the at least one data item with the maximum corresponding VMAF value.
[0026] The cloud application screen freeze optimization device provided in an embodiment of the present application determines a target encoding resource usage from a first correspondence between a preset first encoding attribute, encoding resource usage, and VMAF value when the current encoding resource usage of the encoder is greater than a first value, determines at least one data in the first correspondence whose encoding resource usage is less than or equal to a second value based on the target encoding resource usage, determines target data from the at least one data, adjusts the encoder resolution based on the target data, and can maintain / stabilize the frame rate while ensuring clear image quality when real-time encoding resources are insufficient; and / or when the current bit rate of the encoder is greater than a third value, determines a target bit rate from a second correspondence between a preset second encoding attribute and VMAF value, determines at least one data in the second correspondence whose bit rate is less than or equal to a fourth value based on the target bit rate, determines target data from the at least one data, adjusts the encoder resolution based on the target data, and can reduce the encoding output bit rate while ensuring clear image quality when the encoding output bit rate of the encoder exceeds the set bit rate. In other words, through the above solution, freezes when the terminal plays the cloud application screen can be reduced.
[0027] Based on the aforementioned device embodiment, the first adjustment unit may be used to: Determining a comparison order of target data types in the first corresponding relationship, wherein the target data types include a resolution width and a resolution height; According to the comparison order of the target data types, traverse each target data type in the comparison order, and filter the data whose value under the target data type in the first corresponding relationship is the same as the value under the target data type currently under the encoder or the absolute value of the difference is the smallest, to obtain filtered data, wherein the data source of the latter traversal and filtering is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type in the at least two data is smaller than the value under the target data type currently under the encoder; First intermediate data is determined according to the finally obtained screening data, and the coding resource usage in the first intermediate data is determined as the target coding resource usage.
[0028] Based on the aforementioned device embodiment, the second adjustment unit may be used to: Determining a comparison order of target data types in the second corresponding relationship, wherein the target data types include resolution width, resolution height, and bit rate; According to the comparison order of the target data types, traverse each target data type in the comparison order, and filter the data whose value under the target data type in the first corresponding relationship is the same as the value under the target data type currently under the encoder or the absolute value of the difference is the smallest, to obtain filtered data, wherein the data source of the latter traversal and filtering is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type in the at least two data is smaller than the value under the target data type currently under the encoder; Second intermediate data is determined based on the finally obtained filtered data, and the bit rate in the second intermediate data is determined as the target bit rate.
[0029] Based on the above device embodiment, the calculation formula of the width W2 of the target resolution can be: W2 = (W1×H1×W / H) 1 / 2 The calculation formula for the target resolution height H2 can be: H2= (W1×H1×H / W) 1 / 2 , where W1 and H1 are the width and height of the target data resolution, and W and H are the width and height of the encoder's current resolution.
[0030] The cloud application screen jamming optimization device provided in the embodiment of the present application has the same implementation process as the cloud application screen jamming optimization method provided in the embodiment of the present application, and the effect that can be achieved is also the same as the cloud application screen jamming optimization method provided in the embodiment of the present application, which will not be repeated here.
[0031] like Figure 3 As shown, an electronic device provided by an embodiment of the present application includes: a processor 30, a memory 31 and a bus 32, wherein the memory 31 stores machine-readable instructions executable by the processor 30. When the electronic device is running, the processor 30 communicates with the memory 31 through the bus 32, and the processor 30 executes the machine-readable instructions to perform the steps of the above-mentioned cloud application screen freeze optimization method.
[0032] Specifically, the above-mentioned memory 31 and processor 30 can be general-purpose memories and processors, which are not specifically limited here. When the processor 30 runs the computer program stored in the memory 31, it can execute the above-mentioned cloud application screen freeze optimization method.
[0033] Corresponding to the above-mentioned cloud application screen jamming optimization method, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the above-mentioned cloud application screen jamming optimization method are executed.
[0034] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.
Claims
1. A method for optimizing the freezing of cloud application screens, characterized in that: Applied to cloud servers, including: Obtaining a current encoding resource usage of the encoder, and when the current encoding resource usage is greater than a first value, determining a target encoding resource usage from a first correspondence relationship among a preset first encoding attribute, an encoding resource usage, and a VMAF value, determining at least one piece of data in the first correspondence relationship whose encoding resource usage is less than or equal to a second value, and if the number of the at least one piece of data is greater than 1, determining at least one piece of data with a maximum corresponding VMAF value among the at least one piece of data, determining target data from the at least one piece of data with a maximum corresponding VMAF value, determining a target resolution according to a resolution of the target data, and adjusting the resolution of the encoder to the target resolution, wherein the first correspondence relationship includes a plurality of pieces of data, each of the plurality of pieces of data includes a set of first encoding attributes, an encoding resource usage, and a VMAF value, the first encoding attribute includes a resolution, the second value is determined according to the current encoding resource usage and the target encoding resource usage, and the target data is one piece of data among the at least one piece of data with a maximum corresponding VMAF value; and / or The method comprises: obtaining a current bit rate of the encoder, and when the current bit rate is greater than a third value, determining a target bit rate from a second correspondence relationship between a preset second encoding attribute and a VMAF value, determining at least one data in the second correspondence relationship whose bit rate is less than or equal to a fourth value, and if the number of the at least one data is greater than 1, determining at least one data with a maximum corresponding VMAF value among the at least one data, determining target data from the at least one data with a maximum corresponding VMAF value, determining a target resolution according to a resolution of the target data, and adjusting the resolution of the encoder to the target resolution, wherein the second correspondence relationship includes multiple data, each of the multiple data includes a set of second encoding attributes and a VMAF value, the second encoding attribute includes a resolution and a bit rate, the fourth value is determined according to the current bit rate and the target bit rate, and the target data is one data among the at least one data with a maximum corresponding VMAF value.
2. The method according to claim 1, characterized in that The determining the target coding resource usage from a first correspondence between a preset first coding attribute, a coding resource usage, and a VMAF value includes: Determining a comparison order of target data types in the first corresponding relationship, wherein the target data types include a resolution width and a resolution height; According to the comparison order of the target data types, traverse each bit of the target data type in the comparison order, filter the data whose value under the target data type of the bit in the first corresponding relationship is the same as the value under the target data type of the encoder at present or the data whose absolute value of the difference is the smallest, to obtain the filtered data, wherein the data source filtered in the latter traversal is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type of the bit in the at least two data is smaller than the value under the target data type of the encoder at present; The first intermediate data is determined according to the finally obtained screening data, and the coding resource usage in the first intermediate data is determined as the target coding resource usage.
3. The method according to claim 1 or 2, characterized in that The second value = target coding resource usage - (current coding resource usage - first value).
4. The method according to claim 1, characterized in that The determining the target bit rate from a second correspondence between a preset second encoding attribute and a VMAF value includes: Determining a comparison order of target data types in the second corresponding relationship, wherein the target data types include resolution width, resolution height, and bit rate; According to the comparison order of the target data types, traverse each bit of the target data type in the comparison order, filter the data whose value under the target data type of the bit in the first corresponding relationship is the same as the value under the target data type of the encoder at present or the data whose absolute value of the difference is the smallest, to obtain the filtered data, wherein the data source filtered in the latter traversal is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type of the bit in the at least two data is smaller than the value under the target data type of the encoder at present; The second intermediate data is determined according to the finally obtained screening data, and the bit rate in the second intermediate data is determined as the target bit rate.
5. The method according to claim 1 or 4, characterized in that The fourth value = target bit rate - (current bit rate - third value).
6. The method according to claim 1, characterized in that The calculation formula for the width W2 of the target resolution is: W2 = (W1×H1×W / H) 1 / 2 The calculation formula for the target resolution height H2 is: H2 = (W1×H1×H / W) 1 / 2 , where W1 and H1 are the width and height of the target data resolution, and W and H are the width and height of the encoder's current resolution.
7. A cloud application screen freeze optimization device, characterized in that: Applied to cloud servers, including: a first adjustment unit, configured to obtain a current encoding resource usage of the encoder, and when the current encoding resource usage is greater than a first value, determine a target encoding resource usage from a first correspondence relationship among a preset first encoding attribute, an encoding resource usage, and a VMAF value, determine at least one piece of data in the first correspondence relationship whose encoding resource usage is less than or equal to a second value, and if the number of the at least one piece of data is greater than 1, determine at least one piece of data with a maximum corresponding VMAF value among the at least one piece of data, determine target data from the at least one piece of data with a maximum corresponding VMAF value, determine a target resolution according to a resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the first correspondence relationship includes a plurality of pieces of data, each of the plurality of pieces of data includes a set of first encoding attributes, an encoding resource usage, and a VMAF value, the first encoding attribute includes a resolution, the second value is determined according to the current encoding resource usage and the target encoding resource usage, and the target data is one piece of data among the at least one piece of data with a maximum corresponding VMAF value; and / or The second adjustment unit is used to obtain the current bit rate of the encoder, and when the current bit rate is greater than the third value, determine the target bit rate from the second correspondence between the preset second encoding attribute and the VMAF value, determine at least one data in the second correspondence whose bit rate is less than or equal to the fourth value, if the number of the at least one data is greater than 1, determine at least one data with the largest corresponding VMAF value among the at least one data, determine the target data from the at least one data with the largest corresponding VMAF value, determine the target resolution according to the resolution of the target data, and adjust the resolution of the encoder to the target resolution, wherein the second correspondence includes multiple data, each of the multiple data includes a set of second encoding attributes and a VMAF value, the second encoding attribute includes resolution and bit rate, the fourth value is determined according to the current bit rate and the target bit rate, and the target data is one data among the at least one data with the largest corresponding VMAF value.
8. The device according to claim 7, characterized in that The first adjustment unit is used to: Determining a comparison order of target data types in the first corresponding relationship, wherein the target data types include a resolution width and a resolution height; According to the comparison order of the target data types, traverse each bit of the target data type in the comparison order, filter the data whose value under the target data type of the bit in the first corresponding relationship is the same as the value under the target data type of the encoder at present or the data whose absolute value of the difference is the smallest, to obtain the filtered data, wherein the data source filtered in the latter traversal is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type of the bit in the at least two data is smaller than the value under the target data type of the encoder at present; The first intermediate data is determined according to the finally obtained screening data, and the coding resource usage in the first intermediate data is determined as the target coding resource usage.
9. The device according to claim 7, characterized in that The second adjustment unit is used for: Determining a comparison order of target data types in the second corresponding relationship, wherein the target data types include resolution width, resolution height, and bit rate; According to the comparison order of the target data types, traverse each bit of the target data type in the comparison order, filter the data whose value under the target data type of the bit in the first corresponding relationship is the same as the value under the target data type of the encoder at present or the data whose absolute value of the difference is the smallest, to obtain the filtered data, wherein the data source filtered in the latter traversal is the filtered data obtained in the previous traversal, and if the data with the smallest absolute value of the difference includes at least two data, then the corresponding filtered data is the data whose value under the target data type of the bit in the at least two data is smaller than the value under the target data type of the encoder at present; The second intermediate data is determined according to the finally obtained screening data, and the bit rate in the second intermediate data is determined as the target bit rate.
10. The device according to claim 7, characterized in that The calculation formula for the width W2 of the target resolution is: W2 = (W1×H1×W / H) 1 / 2 The calculation formula for the target resolution height H2 is: H2 = (W1×H1×H / W) 1 / 2 , where W1 and H1 are the width and height of the target data resolution, and W and H are the width and height of the encoder's current resolution.
Citation Information
Patent Citations
VMAF image quality evaluation method, terminal, host, system and storage medium
CN114866840A
Method and device for optimizing video communication image quality, electronic equipment and medium
CN117528090A
Dynamic bandwidth resource management method and device
CN118449863A
Multimedia data coding method, video data coding method and respective devices
CN119110073A
Methods and devices for producing a bit rate ladder for video streaming
WO2024038019A1