Method, system and computer program product for streaming

By obtaining the receiver's resource information in streaming and combining super-resolution reconstruction technology, the transmission quality of the stream is dynamically adjusted, which solves the problem of insufficient resource utilization in adaptive adjustment of transmission quality, and improves user experience and resource utilization.

CN116017004BActive Publication Date: 2024-11-12EMC IP HLDG CO LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111228331.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-21
Publication Date
2024-11-12
Estimated Expiration
2041-10-21

AI Technical Summary

Technical Problem

In streaming, the adaptive adjustment of transmission quality mainly depends on network resources, resulting in a decline in the user experience of the receiver when the network conditions are poor, and the existing technology cannot effectively utilize the receiver's resources for compensation, resulting in poor resource utilization full.

Method used

During the transmission process of the stream, the resource information of the receiver that can be used to compensate for the degradation of the transmission quality, and determine the target transmission quality of the stream based on the receiver and network resources. Combined with super-resolution reconstruction technology, the transmission quality of the stream is dynamically adjusted to optimize the transmission quality of the stream by combining with the super-resolution reconstruction technology. Resource utilization and user experience.

Benefits of technology

By dynamically adjusting resource utilization at the receiver, improving the output quality of streams, reducing network interruptions, optimizing resource utilization and user experience, and reducing operating costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116017004B_ABST
    Figure CN116017004B_ABST
Patent Text Reader

Abstract

Embodiments of the present disclosure relate to a method, system, and computer program product for streaming transmission. The method includes: during the transmission of a stream, obtaining information indicating resources of a receiver of the stream that can be used to compensate for the degradation of the transmission quality of the stream; and determining at least a target transmission quality of the stream based at least on the resources of the receiver and network resources that can be used to transmit the stream. The solution provides a more flexible adaptive trade-off mechanism for streaming transmission, further optimizing the utilization of various resources and user experience in streaming transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to network transmission technology, and more particularly, to methods, systems, and computer program products for streaming transmission. Background Art

[0002] In streaming, it is usually necessary to adjust the quality of the stream being transmitted to dynamically adapt to network conditions. Traditional methods of adjusting and adapting to network conditions mainly automatically change the transmission quality of the stream (for example, the transmission resolution of a dynamic video stream) according to the availability of network resources (such as bandwidth). According to this method, when the network conditions are poor, the quality of the stream received by the receiver will be reduced, which, for example, will reduce the user experience of using the stream at the receiver. Therefore, an optimization method is needed to better balance the utilization of various resources and the user experience in streaming. Summary of the invention

[0003] The embodiments of the present disclosure provide a solution for streaming transmission.

[0004] In a first aspect of the present disclosure, a method for streaming transmission is provided, the method comprising: during transmission of the stream, obtaining information indicating resources of a receiver of the stream that can be used to compensate for degradation in transmission quality of the stream; and determining at least a target transmission quality of the stream based at least on resources of the receiver and network resources that can be used to transmit the stream.

[0005] In a second aspect of the present disclosure, an electronic device is provided, comprising a processor and a memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, causes the device to perform actions, the actions comprising: during transmission of a stream, obtaining information indicating resources of a receiver of the stream that can be used to compensate for degradation in transmission quality of the stream; and determining at least a target transmission quality of the stream based at least on resources of the receiver and network resources that can be used to transmit the stream.

[0006] In a third aspect of the present disclosure, there is provided a computer program product tangibly stored on a computer readable medium and comprising machine executable instructions which, when executed, cause a machine to perform the method according to the first aspect of the present disclosure.

[0007] Please note that the present summary is provided to introduce a selection of concepts in a simplified form, which concepts will be further described in the following detailed description. The present summary is not intended to identify key features or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0009] Figure 1 A schematic diagram illustrating an example environment in which some embodiments of the present disclosure may be implemented;

[0010] Figure 2 A flowchart of a method for streaming according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram showing a portion of an example output interface at a receiving party when a flow of a method according to an embodiment of the present disclosure is output on the example output interface;

[0012] Figure 4 A schematic diagram showing options for transmission quality and output quality of a stream determined according to an embodiment of the present disclosure; and

[0013] Figure 5 A schematic block diagram of a device that can be used to implement an embodiment of the present disclosure is shown.

[0014] Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] The term "streaming" as used in this article refers to the continuous, real-time transmission of time-based media data (such as sound, image or animation, etc.) from a sender (such as an audio and video server) to a receiver (such as a user's computer). In streaming, the receiver does not have to wait until the entire data file is downloaded. Instead, it only needs to transmit for a few seconds or a dozen seconds. Once the downloaded data reaches a certain amount, it can be output (for example, the received video stream is played on the display). Streaming plays an important role in network transmission. In 2020, 82% of the world's bandwidth was used for video streaming, and this proportion is increasing with the adoption of new video applications such as remote AR / VR and remote gaming.

[0017] The term "including" and its variations used in this document are open-ended, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment". The relevant definitions of other terms will be given in the following description.

[0018] In recent years, due to the rise of streaming-based application platforms (such as video streaming services, remote games, teleconferencing, etc.) that require high network resources, network pressure has continued to increase, and some streaming platforms have to limit the quality of the streams allowed to be transmitted and output. On the other hand, the network bandwidth usage pattern has a strong seasonal effect. For example, short video services have a strong daily seasonality, and their network utilization fluctuates between saturation and underutilization at different times of the day. Therefore, an adaptive trade-off mechanism is needed during streaming to adapt to the dynamic network environment.

[0019] Conventional streaming makes an adaptive trade-off between transmission quality and network resources to ensure the continuity of streaming media. For example, in a video streaming platform, different transmission qualities (e.g., video resolution) can be automatically switched to adapt to network conditions (e.g., bandwidth conditions). When bandwidth is limited, the sender will transmit a compressed, lower-quality video stream to the receiver (e.g., the user's terminal device) to save bandwidth, thereby ensuring that the user has a continuous experience in streaming media. Since the compression technology applied (such as H.264) is a pre-applied static compression (i.e., the compression occurs before and cannot be changed without recalculation), the computing requirements cannot be adjusted at runtime. Therefore, when the transmission quality of the stream decreases, the quality of the stream that can be output at the receiver also decreases.

[0020] Techniques such as super-resolution can be used to compensate for the quality degradation of the stream. "Super-resolution" refers to the use of technical means such as models, algorithms and processing flows to construct higher-resolution data from given lower-resolution data. The term "super-resolution reconstruction" refers to the use of technical means to restore compressed data (such as media data such as video, images and audio) to a higher resolution (such as original resolution). Combining super-resolution reconstruction with conventional video compression for streaming, and compensating the quality of the stream at the receiver for the compressed stream that is transmitted at a quality lower than the expected output quality, can improve the output quality of the content of the stream while reducing bandwidth usage, but at the cost of performing additional calculations.

[0021] Among them, although some calculations including training of the model for reconstruction can be performed offline on the cloud or in a data center, various resources at the receiver such as the edge or user end are still required to perform reconstruction on the received stream to compensate for the quality degradation of the stream. As an example, a deep neural network (DNN) model for reconstructing a video can be trained in the cloud. When an end user or cache server requests a video, the sender transmits a lower quality stream of the video and a corresponding DNN model, and the DNN model is used to reconstruct the video. This inevitably introduces additional resource consumption at the receiver, and methods such as DNN-based reconstruction that can provide good output quality but are computationally intensive may have high resource requirements. For example, intensive calculations in DNNs require specialized accelerators to match computing requirements, and accelerators are very expensive compared to other hardware components. This makes it sometimes very expensive or difficult to support use cases with real-time or even super real-time requirements (such as cloud games). Therefore, the use of receiver resources in streaming is not unlimited.

[0022] To at least partially solve the above problems and other potential problems, an embodiment of the present disclosure proposes a solution for streaming. In the adaptive trade-off mechanism of streaming, the method takes into account the resources of the receiver in addition to the available network resources. During the transmission of the stream, the solution at least determines the quality of the stream to be transmitted based on the resources of the receiver that can be used to compensate for the degradation of the transmission quality of the stream and the network resources that can be used to transmit the stream. To some extent, the solution can be regarded as a trade-off calculation between the saving of the receiver's resources and the network resources, which adaptively adjusts the transmission quality of the stream according to the current utilization and available amount of these two resources, and optionally adjusts the quality of the content of the stream output at the receiver.

[0023] The solution according to the embodiment of the present disclosure is a supplement to the traditional adaptive trade-off based on transmission quality adjustment. The solution combines the technology of compensating for the quality of the stream and introduces the resources of the receiver of the stream as a new dimension of flexibility, so that the operator of the streaming platform can provide a series of more flexible options for streaming video according to more different conditions. The solution can be used in combination with other network optimization technologies such as content distribution networks (CDNs) that help reduce seasonal effects by reducing repeated transmissions to further optimize the utilization of various resources and user experience in streaming.

[0024] The following will describe the embodiments of the present disclosure in detail with reference to the accompanying drawings. Through this detailed description, the advantages of the solutions of the present disclosure will become more obvious. Figure 1 A schematic diagram of an example environment 100 is shown in which various embodiments of the present disclosure can be implemented.

[0025] like Figure 1As shown, the example environment 100 includes a sender 110 of a stream 130 and a receiver 120 of the stream 130. Depending on the deployment scenario, for example, the sender 110 can be a server of a streaming platform, and the receiver 120 can be a terminal device capable of playing streaming media, an edge device that provides a network entry point for the terminal device, or a cache server that is closer to the user than the sender 110 in the network. The sender 110 and the receiver 120 can also be any other fixed or mobile device capable of sending and receiving streams. In addition to sending and receiving the stream 130, the sender 110 and the receiver 120 can also transmit other messages and data to each other.

[0026] A third party 140 that communicates with the sender 110 and the receiver 120 may also optionally exist in the example environment 100. The third party 140 may be any suitable device that can communicate with the sender 110 and the receiver 120. For example, the third party 140 may be some control component or resource allocation component of a streaming cloud platform.

[0027] Based on, for example, a request from the receiver 120, the sender 110 may transmit the stream 130 to the receiver 120. In some cases, due to dynamic changes in network conditions that may be used to transmit the stream 130, the transmission quality of the stream 130 may increase or decrease during the transmission process. In some cases, the receiver 120 may use its available computing and data resources to compensate for the received stream 130 (e.g., super-resolution reconstruction of the stream 130 of the video content) to meet the quality requirements of the content of the stream 130 being output at the receiver.

[0028] It should be understood that the architecture and functions in the example environment 100 are described only for exemplary purposes, and do not imply any limitation on the scope of the present disclosure. The sender 110, the receiver 120, or the third party 140 may be a single device, a distributed device, or a device in any other suitable form. Other devices, systems, or components not shown may also exist in the example environment 100. The embodiments of the present disclosure may also be applied to other environments with different structures and / or functions.

[0029] Figure 2 2 is a flow chart showing an example method 200 for streaming according to some embodiments of the present disclosure. The example method 200 may be performed by, for example, Figure 1 The method 200 may be performed by any of the sender 110, the receiver 120, or the third party 140 shown. It should be understood that the method 200 may also include additional actions not shown, and the scope of the present disclosure is not limited in this respect. Figure 1 The method 200 will be described in detail.

[0030] At box 210, during the transmission of stream 130, information indicating the resources of the receiver 120 of stream 130 that can be used to compensate for the degradation of the transmission quality of stream 130 is obtained. In some embodiments, the information may include, for example, various indicators related to the computing power of the receiver. For example, the information may include a quantified value of the computing power of the receiver and its utilization rate. For example, the information may also include the proportion of the total computing power at the receiver that can be allocated to compensate for the quality of stream 130. In some embodiments of using a model to compensate for stream 130 (e.g., optionally using a model such as a DNN to reconstruct stream 130 of video content), the information may also include relevant parameters about the performance of the model used, etc. Any other applicable information may also be obtained.

[0031] At block 220, a target transmission quality may be determined based at least on the resources of the recipient 120 and the network resources available for transmitting the stream 130, which target transmission quality should be used when transmitting the stream 130 to the recipient 120. For example, based on the network bandwidth being greater than the original uncompressed quality of the content of the stream 130 at the sender and no resources available at the recipient 120 for compensation, the target transmission quality of the stream 130 may be determined to be its original quality.

[0032] In some embodiments, the transmitted stream 130 should be output at the receiver 120 at a specified output quality. The output quality can be specified by the user. In these embodiments, the method 200 may also include obtaining the specified output quality of the stream 130, and determining the target transmission quality of the stream 130 based on the output quality, the resources of the receiver 120, and the network resources available for transmitting the stream 130. The output quality of the stream 130 can be specified by the user in various ways. For example, the output quality of the stream 130 can be specified by the user by selecting an output quality option (such as selecting a specific playback resolution on a video playback application interface) via an interface of an output device at the receiver 120. For example, the output quality of the stream 130 can also be specified by the user by agreeing to a certain agreement (e.g., a specific user service level agreement).

[0033] In some embodiments, when the transmission quality of the stream 130 supported by the network resources is less than the specified output quality, the target transmission quality of the stream 130 can be determined based on the transmission quality of the stream 130 that can be compensated by the resources of the receiver 120. Taking a video stream that can be reconstructed using super-resolution as an example, when the resources of the receiver 120 are insufficient to compensate the stream 130 of the video content from the resolution that can be transmitted by the network bandwidth to the resolution specified for playback by the user in real time or near real time, the resolution of the stream 130 that can be supported by the current network bandwidth at most can be used as the transmission resolution of the stream 130. In this case, the content of the stream 130 played at the receiver 120 may be paused to wait for the receiver 120 to reconstruct the transmitted stream 130 to have the specified resolution.

[0034] In some embodiments, the target transmission quality of stream 130 may be determined in a manner that complies with the following constraints: the resources of the receiver 120 required to compensate the stream 130 with the target transmission quality to have a specified output quality do not exceed the resources available at the receiver 120; and the network resources used to transmit the stream 130 with the target transmission quality do not exceed the available network resources. In some embodiments, the target transmission quality of stream 130 may be determined while complying with the aforementioned constraints using a modeling method such as linear programming. As an example, the following transmission quality may be found as the target transmission quality of stream 130 while complying with the aforementioned constraints: transmitting stream 130 with this transmission quality minimizes the operating cost for the operator of the transmission platform to transmit and compensate the transmitted stream 130 to have a specified resolution. The scope of the present disclosure is not limited to the specific model or instance thereof used to determine the target transmission quality in the above manner.

[0035] The above-mentioned embodiments of the present disclosure allow a trade-off between using more receiver resources to save more network resources and using more network resources to save receiver resources without adjusting the output quality at the receiver, thereby providing multiple options to meet the same stream output quality requirements to reduce interruptions in stream output and optimize resource utilization while meeting the requirements.

[0036] As a demonstration, Figure 3 FIG. 1 is a schematic diagram of an embodiment of the present disclosure, which shows a portion 300 of an example output interface when a stream is output on an example output interface at a receiving end when a user specifies the output quality of the stream. For example, the example output interface may be a playback interface of a stream-based video player. For demonstration purposes, the following will be discussed in the context of video stream playback. Figure 3 , with resolution as the quality measure of the stream and bandwidth as the measure of network resources. Here the receiver can be Figure 1 The receiver 120 in the embodiment of the present invention and the video stream may be stream 130. Figure 1Section 300 will now be described in detail.

[0037] Portion 300 includes a progress bar 310 when the video content of stream 130 is played on an example output interface at receiver 120. For illustrative purposes, segments 310-1, 310-2, and 310-3 of progress bar 310 are shown. At the intersection of the time periods corresponding to the segments, the network conditions and the resource conditions of receiver 120 have changed. It should be understood that during the period when the content of stream 130 is played, the network conditions and the resource conditions of receiver 120 may change more or less times, and in different ways.

[0038] During the time period corresponding to segment 310-1, the network connection is good, the stream 130 is transmitted to the receiver 120 at the playback resolution specified by the user, and the receiver 120 plays the received stream 130 in the playback interface having part 300 without quality compensation (e.g., super-resolution reconstruction). At the beginning of the time period corresponding to segment 320-1, the bandwidth is reduced due to network congestion. Using the method disclosed in the present invention, the stream 130 is transmitted at another resolution lower than the playback resolution specified by the user during the time period, and the receiver plays the received stream 130 at the specified playback resolution after quality compensation. At the beginning of the time period corresponding to segment 320-3, the utilization rate of the resources (e.g., the computing resources of the edge server at the receiver) that can be used to compensate for the quality of the stream 130 at the receiver reaches saturation, and the network connection becomes good again. Therefore, during this time period, the stream 130 is transmitted again at the specified playback resolution and is played directly. During the whole process, the transmission resolution of stream 130 and the resources used to compensate the receiver of stream 130 are dynamically adjusted, so that the content of stream 130 played to the user maintains the user-specified resolution and avoids freezes, thereby improving the user experience.

[0039] Continue to refer Figure 2 In other embodiments of the present disclosure, there may be no output quality of the stream 130 at the receiver 120 specified, for example, by a user. For example, a user at the receiver of the video stream may select an automatic mode resolution to allow the resolution at which the video stream is played to be dynamically adjusted during playback. In this case, the example method 200 may also include determining a target transmission quality of the stream 130 and an output quality of the output stream 130 at the receiver 120 based on resources of the receiver 120 and network resources available for transmitting the stream 130.

[0040] In some embodiments, the target transmission quality of the stream 130 and the output quality of the stream 130 output at the receiver 120 may be determined based on a comparison of the amount of network resources and the amount of resources of the receiver 120 with corresponding thresholds. For example, based on the network resources required to transmit the stream 130 at a certain quality falling within a certain threshold range, the quality may be used as the target transmission quality of the stream 130. For example, based on the resources of the receiver 120 required to compensate the stream 130 transmitted at the determined target transmission quality for a certain output quality falling within a certain threshold range, the output quality may be used as the output quality of the stream 130 at the receiver 120.

[0041] In some embodiments, the target transmission quality and the output quality of the output stream 130 at the receiver 120 may be determined in a manner that complies with the following constraints: the resources of the receiver 120 required to compensate the stream 130 of the target transmission quality to have the output quality do not exceed the resources available at the receiver 120; and the network resources used to transmit the stream 130 at the target transmission quality do not exceed the available network resources. In some embodiments, a modeling method such as linear programming may be used to determine the target transmission quality of the stream 130 while complying with the aforementioned constraints. As an example, the following transmission quality and output quality may be found as the target transmission quality of the stream 130 and the output quality of the output stream 130 at the receiver 120 respectively while complying with the aforementioned constraints: transmitting the stream 130 at the transmission quality and outputting the stream 130 at the receiver 120 at the output quality will minimize the operating costs of the operator of the streaming platform for transmitting the stream 130 and compensating for the transmitted stream 130. The scope of the present disclosure is not limited to the specific model or instance thereof used to determine the target transmission quality of the stream 130 in the above manner.

[0042] Together with the aforementioned embodiments and other embodiments of the present disclosure, embodiments of dynamically determining the output quality of a stream according to the method of the present disclosure can provide a more flexible streaming infrastructure platform framework. This is beneficial to both end users and service platforms. Users can obtain higher quality stream outputs, or even "super quality" stream outputs (for example, video streams with a resolution greater than the transmission resolution supported by the network bandwidth) with fewer network interruptions. The platform can balance between computing and network bandwidth based on the usage status of the transmission pipeline, and adaptively adjust the mode of providing streaming services through more flexible additional options to achieve better overall resource utilization and lower operating costs.

[0043] As a demonstration, Figure 4 A sequential schematic diagram 400 is shown, which shows examples of streaming options that a streaming platform can provide in a mode of dynamically adjusting output quality according to an embodiment of the present disclosure in a coordinate system 401. The streaming platform can use these options to transmit Figure 1the stream 130 therein and output it at the receiving end. The following describes the diagram 400 with reference to Figure 1 for illustration.

[0044] As Figure 4 shown, the horizontal axis of the coordinate system 401 indicates the computing resources at the receiving end that need to be consumed by the usage option, and the vertical axis of the coordinate system 401 indicates the network resources that need to be consumed by the usage option. The positions of the options 410-450 in the coordinate system 401 are ordinal, that is, the position indicates the relative magnitude between the options, rather than an exact value.

[0045] When using the option 410, the platform transmits the stream 130 at a5 and outputs the stream at the receiving end 120 at a5. When using the option 420, the platform transmits the stream 130 at a1 and outputs the stream at the receiving end 120 at a5. When using the option 430, the platform transmits the stream 130 at a2 and outputs the stream at the receiving end 120 at a6. When using the option 440, the platform transmits the stream 130 at a3 and outputs the stream at the receiving end 120 at a7. When using the option 450, the platform transmits the stream 130 at a4 and outputs the stream at the receiving end 120 at a8. Among them, a1 < a2 < a3 < a4 < a5 < a6 < a7 < a8. It can be seen from the diagram 400 that these options have different resource requirements. Using the solution provided by the present disclosure, it is possible to determine which option should be used to provide the stream 130 under real-time resource conditions based on the network resources and the resources of the stream receiver and using the method described in combination with the embodiments described above. For example, methods such as planning according to thresholds or under constraints as described in the embodiments above can be used to determine which option to use. It should be understood that Figure 4 the options in are only examples, and more options, fewer options, and different options can also be provided for streaming.

[0046] Referring again to Figure 2 . As described above, the example method 200 can be executed by the sender 110 or the receiver 120. When the example method 200 is executed by the sender 110, the sender 110 can also use the determined target transmission quality of the stream 130 to transmit the stream 130 to the receiver 120.

[0047] When the exemplary method 200 is performed by the receiver 120, the receiver 120 may also send the determined target transmission quality of the stream 130 to the sender of the stream 130, and may receive the stream 130 having the target transmission quality. In some embodiments, when the target transmission quality of the stream 130 received by the receiver 120 is less than the specified output quality of the stream 130, the receiver 120 uses its resources to compensate the stream 130 to have the specified output quality. In other embodiments, when the target transmission quality of the stream 130 received by the receiver 120 is less than the output quality of the output stream 130 of the receiver 120 determined based on the method of the present disclosure, the receiver 120 uses its resources to compensate the stream 130 to have the determined output quality.

[0048] Figure 5 Schematic block diagram of an example device 500 that can be used to implement embodiments of the present disclosure is shown. Figure 1 The receiver 120 shown may be implemented by the device 500. Figure 5 As shown, the device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 or computer program instructions loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0049] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0050] The various processes and processing described above, such as methods 200 and / or 400, may be performed by the processing unit 501. For example, in some embodiments, the methods 200 and / or 400 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the CPU 501, one or more actions of the methods 200 and / or 400 described above may be performed.

[0051] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0052] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination of the above. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0053] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0054] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0055] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0056] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0057] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0058] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0059] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for streaming transmission, comprising: During the transmission of the stream, obtaining information indicating computing resources of a receiver of the stream that can be used to compensate for the degradation of the transmission quality of the stream; as well as determining at least a target transmission quality for the stream based at least on the computing resources of the recipient and network resources available for transmitting the stream; wherein the information indicating computing resources of the receiver of the stream that can be used to compensate for the transmission quality degradation of the stream comprises at least information characterizing one or more available portions of computing capabilities of the receiver of the stream; and Wherein at least determining the target transmission quality of the stream includes selecting between at least the following items: (i) a first transmission configuration, wherein the first transmission configuration utilizes a first level of network resources to transmit the stream to the receiver and utilizes a first level of computing power of the receiver to compensate for quality degradation of the transmitted stream, and (ii) a second transmission configuration, wherein the second transmission configuration utilizes a second level of network resources to transmit the stream to the receiver and utilizes a second level of computing power of the receiver to compensate for quality degradation of the transmitted stream, the second level of network resources being different from the first level of network resources, and the second level of computing power being different from the first level of computing power.

2. The method according to claim 1, wherein at least determining the target transmission quality comprises: Obtaining an output quality of the stream specified by a user; as well as The target transmission quality is determined based on the computing resources of the receiver, the network resources available for transmitting the stream, and the output quality specified by the user.

3. The method according to claim 2, wherein at least determining the target transmission quality comprises: When the transmission quality of the stream supported by the network resources is less than the output quality specified by the user, the target transmission quality is determined based on the transmission quality of the stream that can be compensated by the computing resources of the receiver.

4. The method according to claim 3, wherein at least determining the target transmission quality comprises: The target transmission quality is determined in the following manner: An amount of computing resources of the recipient required to compensate the target transmission quality of the stream to the output quality specified by the user does not exceed the computing resources available at the recipient; as well as An amount of network resources used to transmit the stream at the target transmission quality does not exceed the network resources that are available.

5. The method according to any one of claims 2 to 4, wherein the method is performed by the receiver of the stream, and the method further comprises: Sending the determined target transmission quality of the stream to a sender of the stream; receiving the stream having the target transmission quality; as well as When the target transmission quality of the received stream is less than the output quality specified by the user, the stream is compensated to have the specified output quality by using the computing resources of the receiver.

6. The method of claim 1, wherein at least determining the target transmission quality comprises: A target transmission quality of the stream and an output quality of the stream output at the receiver are determined based on the computing resources of the receiver and network resources available for transmitting the stream.

7. The method of claim 6, wherein determining the target transmission quality and the output quality comprises: The target transmission quality and the output quality are determined based on a comparison of the amount of the network resources and the amount of the computing resources of the receiver with corresponding thresholds.

8. The method of claim 6, wherein determining the target transmission quality and the output quality comprises: The target transmission quality and the output quality of the stream output at the receiving end are determined in the following manner: An amount of computing resources of the recipient required to compensate the target transmission quality of the stream to the output quality does not exceed the computing resources available at the recipient; as well as An amount of network resources used to transmit the stream at the target transmission quality does not exceed the network resources that are available.

9. The method according to any one of claims 6 to 8, wherein the method is performed by the receiver of the stream, and the method further comprises: Sending the determined target transmission quality of the stream to a sender of the stream; receiving the stream having the target transmission quality; as well as When the target transmission quality of the received stream is less than the output quality of the stream, the stream is compensated to have the output quality by using the computing resources of the receiver.

10. The method according to claim 1, wherein the method is performed by a sender of the stream, and the method further comprises: The stream is transmitted using the determined target transmission quality for the stream.

11. An electronic device, comprising: processor; as well as a memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, causing the device to perform actions, the actions comprising: During the transmission of the stream, obtaining information indicating computing resources of a receiver of the stream that can be used to compensate for the degradation of the transmission quality of the stream; and determining at least a target transmission quality for the stream based at least on the computing resources of the recipient and network resources available for transmitting the stream; wherein the information indicating computing resources of the receiver of the stream that can be used to compensate for the transmission quality degradation of the stream comprises at least information characterizing one or more available portions of computing capabilities of the receiver of the stream; and Wherein at least determining the target transmission quality of the stream includes selecting between at least the following items: (i) a first transmission configuration, wherein the first transmission configuration utilizes a first level of network resources to transmit the stream to the receiver and utilizes a first level of computing power of the receiver to compensate for quality degradation of the transmitted stream, and (ii) a second transmission configuration, wherein the second transmission configuration utilizes a second level of network resources to transmit the stream to the receiver and utilizes a second level of computing power of the receiver to compensate for quality degradation of the transmitted stream, the second level of network resources being different from the first level of network resources, and the second level of computing power being different from the first level of computing power.

12. The apparatus of claim 11, wherein at least determining the target transmission quality comprises: Obtaining an output quality of the stream specified by a user; as well as The target transmission quality is determined based on the computing resources of the receiver, the network resources available for transmitting the stream, and the output quality specified by the user.

13. The apparatus of claim 12, wherein at least determining the target transmission quality comprises: When the transmission quality of the stream supported by the network resources is less than the output quality specified by the user, the target transmission quality is determined based on the transmission quality of the stream that can be compensated by the computing resources of the receiver.

14. The apparatus of claim 13, wherein at least determining the target transmission quality comprises: The target transmission quality is determined in the following manner: An amount of computing resources of the recipient required to compensate the target transmission quality of the stream to the output quality specified by the user does not exceed the computing resources available at the recipient; as well as An amount of network resources used to transmit the stream at the target transmission quality does not exceed the network resources that are available.

15. The apparatus according to any one of claims 12 to 14, wherein the apparatus is the receiver of the stream, and the actions further comprise: Sending the determined target transmission quality of the stream to a sender of the stream; receiving the stream having the target transmission quality; as well as When the target transmission quality of the received stream is less than the output quality specified by the user, the stream is compensated to have the specified output quality by using the computing resources of the receiver.

16. The apparatus of claim 11, wherein at least determining the target transmission quality comprises: A target transmission quality of the stream and an output quality of the stream output at the receiver are determined based on the computing resources of the receiver and network resources available for transmitting the stream.

17. The apparatus of claim 16, wherein determining the target transmission quality and the output quality comprises: The target transmission quality and the output quality are determined based on a comparison of the amount of the network resources and the amount of the computing resources of the receiver with corresponding thresholds.

18. The apparatus of claim 16, wherein determining the target transmission quality and the output quality comprises: The target transmission quality and the output quality of the stream output at the receiving end are determined in the following manner: An amount of computing resources of the recipient required to compensate the target transmission quality of the stream to the output quality does not exceed the computing resources available at the recipient; as well as Transmitting the stream at the target transmission quality uses no more network resources than are available.

19. The device according to any one of claims 16 to 18, wherein the device is the receiver of the stream, and the actions further comprise: Sending the determined target transmission quality of the stream to a sender of the stream; receiving the stream having the target transmission quality; as well as When the target transmission quality of the received stream is less than the output quality of the stream, the stream is compensated to have the output quality by using the computing resources of the receiver.

20. The device of claim 11, wherein the device is a sender of the stream, and the actions further comprise: The stream is transmitted using the determined target transmission quality for the stream.

21. A computer program product tangibly stored on a computer readable medium and comprising machine executable instructions which, when executed, cause a machine to perform the method according to any one of claims 1 to 10.