Reduction of Bandwidth Consumption by Adversarial Generative Network

Abstract

The computer-implemented method includes determining a desired resolution quality of the media file, determining available resources of a computing device, selecting one of a plurality of generative adversarial network (GAN) plug-ins based on the determined available resources of the computing device, and estimating a second resolution quality of the media file corresponding to the selected GAN plug-in, the second resolution quality being lower than the desired resolution quality. The method further includes communicating with a server to receive the media file at the estimated second resolution quality, converting the received media file from the second resolution quality to the desired resolution quality by using the selected GAN plug-in, and outputting the converted media file at the desired resolution quality.

Description

Background Art

[0001] Often, a user uses a browser or other application running on the user's device to stream media content such as video. In some cases, the streamed media content may pause or be delayed due to insufficient bandwidth to handle the file size of the streamed media content. For example, a user may desire to view a streamed video in high definition (HD) quality, which has a larger file size than standard definition (SD) quality. Depending on the available bandwidth, problems such as buffering delays can occur in HD quality video, resulting in a decrease in user satisfaction when consuming the streamed video.

Summary of the Invention

[0002] Aspects of the present disclosure may include a computer-implemented method, a computer program product, and a system. An example of the method includes determining a desired resolution quality of a media file, determining available resources of a computing device, selecting one of a plurality of adversarial generative network (GAN) plugins based on the determined available resources of the computing device, and estimating a second resolution quality of the media file corresponding to the selected GAN plugin. The second resolution quality is lower than the desired resolution quality. The method further includes communicating with a server to receive the media file at the estimated second resolution quality, using the selected GAN plugin to convert the received media file from the second resolution quality to the desired resolution quality, and outputting the converted media file at the desired resolution quality.

[0003] The drawings illustrate exemplary embodiments only and should not be construed as limiting the scope. With this understanding, the exemplary embodiments will be described more specifically and in detail with reference to the accompanying drawings.

Brief Description of the Drawings

[0004]

Figure 1

Figure 2

Figure 3

Modes for Carrying Out the Invention

[0005] By general convention, the various features described are not drawn to scale and are drawn to emphasize specific features relevant to the exemplary embodiments.

[0006] In the following detailed description, reference is made to the accompanying drawings which form a part hereof, and in which are shown by way of illustration specific exemplary embodiments. However, it should be understood that other embodiments may be utilized and that logical, mechanical, and electrical changes may be made. Further, the methods presented in the drawings and the specification should not be construed as limiting the order in which the individual steps are performed. Accordingly, the following detailed description should not be construed in a limiting sense.

[0007] As used herein, "a number of" when used with respect to an item means one or more items. For example, "a number of different types of networks" is one or more different types of networks.

[0008] Furthermore, the phrases "at least one", "one or more", and "or... or both (and / or)" are open-ended expressions that are conjunctive and disjunctive in operation. For example, the expressions "at least one of A, B, and C", "at least one of A, B, or C", "one or more of A, B, and C", "one or more of A, B, or C", and "A, B, or C or a combination thereof" each mean only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. That is, "at least one of...", "one or more of...", and "or... or both (and / or)" mean that any combination of items and any number of items from the list may be used, but not all items in the list are required. The items may be specific objects, things, or categories. Further, the amount or number of each item in the combination of listed items need not be the same. For example, in some illustrative examples, "at least one of A, B, and C" may be, for example, but not limited to, 2 of item A, 1 of item B, and 10 of item C, or 0 of item A, 4 of item B, and 7 of item C, or other suitable combinations.

[0009] Furthermore, the term "a" or "an" entity refers to one or more of that entity. Thus, the terms "a" (or "an"), "one or more", and "at least one" may be used synonymously herein. It should also be noted that the terms "comprise", "include", and "have" may be used synonymously.

[0010] Furthermore, as used herein, the term "automated" and variations thereof refer to any process or operation that is performed without significant human input when the process or operation is executed. However, even if significant or insignificant human input is used in the execution of a process or operation, the process or operation may be automated if the input is received prior to the execution of the process or operation. Human input is considered significant if it affects the manner in which the process or operation is executed. Human input that merely consents to the execution of the process or operation is not considered "significant".

[0011] As described above, due to network bandwidth limitations, problems such as latency buffering can occur when streaming media content requires more bandwidth than is available for a desired media quality (e.g., HD, full HD, quad HD, ultra HD, etc.) to play that desired media quality. Embodiments described herein address this problem through the use of a selectable adversarial generative network (GAN) plugin, reducing the amount of network bandwidth consumed by the streamed media content while providing the media content at the media quality desired by the user.

[0012] FIG. 1 is a block diagram of an exemplary embodiment of a system 100. The system 100 includes a client device 102 communicatively coupled to a server 110 via a network 108. The network 108 can be implemented using any number of any suitable physical communication topologies or logical communication topologies or both. The network 108 may include one or more private computing networks or public computing networks. For example, the network 108 may include a private network. Alternatively or additionally, the network 108 may include a public network such as the Internet. Thus, the network 108 may form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet or a combination thereof. The network 108 can include one or more servers, networks, or databases and can transfer data between the server 110 and the client device 102 using one or more communication protocols.

[0013] Further, although the network 108 is shown as a single entity in FIG. 1, in other examples, it may include multiple networks such as a public network or a private network or a combination thereof. The communication network 108 can include various types of physical communication channels or “links”. The links can be wired, wireless, optical, or any other suitable media or a combination thereof. Additionally, the communication network 108 can include various network hardware and network software for performing routing, switching, and other functions such as routers, switches, base stations, bridges, or any other devices that may be useful for facilitating communication of data.

[0014] Server 110 stores media file 112, and media file 112 can be provided to client device 102 via network 108, such as in response to a request from client device 102 for media file 112. For simplicity of explanation, FIG. 1 depicts a single client device 102, a single server 110, and a single media file 112, but it should be understood that in other embodiments, two or more client devices 102, two or more servers 110, or two or more media files 112, or combinations thereof, can be used. For example, in some embodiments, copies of the same media file 112 can be stored across multiple servers 110. Further, in some embodiments, a single media file 112 can be distributed across multiple servers 110. Additionally, multiple different media files can be stored on the same server 110, across multiple servers 110, or both. In addition, in some embodiments, multiple client devices 102 can request the same media file 112 or different media files 112 from the same or different servers 110.

[0015] The client device 102 includes a plurality of GAN plugins 104 and a resource monitor 106. A GAN is a type of neural network that can use unsupervised machine learning to convert low-quality images / videos into high-quality images / videos. Specifically, the GAN architecture includes a generator model and a discriminator model. The generator model is used to generate new realizable examples of the sample domain, for example, new pixels of an image. The discriminator model is used to determine which pixels are actual or original pixels and which pixels are fake or generated pixels. The two models can be trained together with the aim of reaching a level where the discriminator model cannot distinguish between the generated pixels and the actual or original pixels. However, a given GAN model does not necessarily need to reach a level where the discriminator cannot distinguish between the generated pixels and the original pixels in all cases.

[0016] The generator model and the discriminator model can be implemented using a convolutional neural network (CNN). A convolutional neural network (CNN) is a class of deep feedforward artificial neural networks that has been successfully applied to the analysis of visual images. A convolutional neural network is composed of an artificial structure or a structure like neurons with learnable weights and biases. Each neuron receives several inputs and performs a dot product. A convolutional neural network architecture typically comprises a stack of layers that operate to receive an input (e.g., a single vector) and transform that input through a series of hidden layers. Each hidden layer is composed of a set of neurons, each neuron having learnable weights and biases, each neuron being capable of being fully connected to all neurons in the previous layer, and the neurons within a single layer being able to function independently without shared connections. The last layer is a fully connected output layer, and in a classification setting, the output layer represents class scores, which can be any real-valued number or a real-valued target (e.g., in regression). However, it should be understood that the embodiments described herein are not limited to the use of a CNN for implementing a GAN.

[0017] The accuracy of the GAN network depends on the GAN architecture, such as the network size or the number of layers of the GAN being used. A larger GAN requires more processing resources of the client devices 102 than a smaller GAN in terms of network size. The plurality of GAN plugins 104 includes GANs of various sizes. As a result, each of the GAN plugins 104 has different requirements for processing resources such as memory and CPU. Further, since each GAN plugin 104 has a different network size, each GAN plugin 104 can convert inputs of different sizes. For example, in some embodiments, the largest GAN plugin with the highest resource requirements can convert a 144-pixel (144p) video into a 1080p video, while the smallest GAN plugin with the lowest resource requirements can only convert a 720p video into a 1080p video. Therefore, the size or resolution quality of the required input is inversely proportional to the size of the GAN plugin network (i.e., the smaller the GAN plugin, the larger the number of pixels or video quality required for conversion to the desired resolution).

[0018] In some embodiments, there are five GAN plugins for which the client device 102 selects a GAN plugin. In some such embodiments, the first and smallest GAN plugin is configured to convert a 720p video to 1080p. The second or next GAN plugin is configured to convert a 480p video to 1080p. The third GAN plugin is configured to convert a 360p video to 1080p. The fourth GAN plugin is configured to convert a 244p video to 1080p. The fifth or largest GAN plugin is configured to convert a 144p video to 1080p. It should be understood that the video quality / size used above is provided for illustrative purposes only, and other input video sizes and output video sizes can be used in other embodiments. Further, it should be understood that in other embodiments, various sizes of GAN plugins more than five or less than five can be used.

[0019] The client device 102 further includes a resource monitor 106. The resource monitor 106 is configured to monitor the processing capacity of the client device 102. That is, the resource monitor 106 tracks the processes and applications being executed, as well as the amount of resources being utilized (e.g., CPU, memory, bus bandwidth, etc.). The resource monitor 106 can determine the amount of resources available for use by one of the GAN plugins 104 based on the amount of resources being used and the total resource capacity of the client device 102. Accordingly, the client device 102 compares the amount of available resources with the respective amounts of resources required by each of the GAN plugins 104 to select one of the GAN plugins 104 for processing the media content received from the server 110.

[0020] After selecting one of the GAN plugins 104, the client device 102 estimates the minimum video resolution required for the selected GAN plugin 104 to convert the video to the desired resolution quality. As described above, each GAN plugin 104 has a minimum video resolution quality required as an input for converting to the desired output resolution quality. Accordingly, the client device 102 can estimate the minimum input resolution quality required based on the selected GAN plugin 104 and the desired output quality. The client device 102 communicates with the server 110 to instruct the server 110 to transmit the video file at the minimum input resolution determined by the client device 102. The video file is received at the requested minimum input resolution, and the selected GAN plugin 104 processes the received video file and converts it to the desired resolution quality for playback / viewing by the user.

[0021] It should be understood that the conversion using the selected GAN plugin 104 can be performed in real time while the video is being received. For example, in some embodiments, while a video file is being streamed from server 110, the user is consuming or viewing the video file. However, in some other embodiments, the user can completely download or receive the video file from server 110 before viewing the video file. For example, the user may specify the time when they want to view the video file. In any situation, in the embodiments described herein, since the video file can be transmitted at a resolution quality lower than the output resolution quality at which the user views the video file, the amount of bandwidth required to transmit the video file from server 110 to client device 102 is reduced. Therefore, since the file size becomes smaller as the resolution quality decreases, the amount of bandwidth used to transmit the video file is reduced.

[0022] Furthermore, in some embodiments, the available resources of the client device 102 are not the same or static throughout the process of receiving the video file from the server 110. For example, the client device 102 may start using more resources for other applications or background processes compared to when the media file 112 was first requested from the server 110, thereby reducing the amount of resources available to the GAN plugin 104. Alternatively, the client device 102 may start using fewer resources for other processes or applications compared to when the media file 112 was first requested from the server 110, thereby increasing the amount of resources available to the GAN plugin 104. Thus, in some embodiments, the resource monitor 106 updates the amount of resources available for use by the GAN plugin 104 when the amount of resources utilized by other processes / applications changes. The client device 102 can then select a different GAN plugin 104 based on the changed available resources and communicate with the server 110 to change the corresponding resolution of the media file 112 being sent to the client device 102.

[0023] Furthermore, in some embodiments, the resource monitor 106 maintains a history of resource usage data, which can include various factors such as, but not limited to, which processes / applications are being run, the average time or the longest / shortest time or both that each process / application has been run, the average, maximum, or minimum amount of each processing resource utilized by each process / application, or combinations thereof. In this way, the client device 102 can predict the amount of resources available over a given period. For example, in some implementations, the client device 102 does not recognize how large a media file 112 is, or how long a video is. However, the client device 102 can use the historical data to predict the resources available over a given period such as, but not limited to, a period of 45 seconds. For example, the period can be measured in seconds, minutes, etc.

[0024] Therefore, the total time for receiving the media file 112 can be divided into a plurality of consecutive periods. The periods for dividing the total time can be of the same length or of different sizes. For each period, the client device 102 predicts the available resources during that period, selects a GAN plugin based on the predicted available resources, and communicates with the server 110 to adjust the resolution quality of the media file 112 for that period so that the corresponding resolution quality of the media file 112 is transmitted during each period. In this way, during the process of receiving the media file 112, the client device 102 can adjust the input resolution quality of the received media file 112 and, while adjusting the GAN plugin 104 used to adapt to changes in available resources, provide a relatively consistent output resolution quality to the user. For example, a background process continuously collects information about resource usage over time. If the available resources increase or are predicted to increase after the first period, the client device 102 can upgrade to a larger GAN plugin 104 for the next consecutive period. Similarly, if the available resources decrease or are predicted to decrease after the first period, the client device 102 can downgrade to a smaller GAN plugin 104 for the next consecutive period.

[0025] In some embodiments, each GAN plugin 104 can be pre-trained and loaded onto the client device 102 after training. Additionally, the GAN plugin 104 can be implemented as a plugin for an Internet browser application or as a plugin for another application or software program that can stream media file 112 from server 110 and / or receive it in some other way. Further, it should be understood that in FIG. 1, the GAN plugin 104 and the resource monitor 106 are depicted as separate entities for simplicity of explanation. However, the functions described above with respect to the GAN plugin 104, the client device 102, and the resource monitor 106 can be performed by a single entity, divided among multiple entities, or both, in ways other than those described in the example of FIG. 1. For example, in some embodiments, the function of monitoring and predicting available resources can be performed by a part of a selected GAN plugin 104 that runs during a given period.

[0026] FIG. 2 is a block diagram of one embodiment of a computing device 200 configured to implement the functionality of a client device 102 that includes a GAN plugin 104 and a resource monitor 106. The components of the computing device 200 shown in FIG. 2 include one or more processors 202, a memory 204, a storage interface 216, an input / output (“I / O”) device interface 212, and a network interface 218, all of which are communicatively coupled, either directly or indirectly, for component-to-component communication via a memory bus 206, an I / O bus 208, a bus interface unit (“IF”) 209, and an I / O bus interface unit 210.

[0027] In the embodiment shown in FIG. 2, computing device 200 includes one or more general-purpose programmable central processing units (CPUs) 202A and 202B, which are collectively referred to herein as processor 202. In some embodiments, computing device 200 includes multiple processors. However, in other embodiments, computing device 200 is a single CPU system. Each processor 202 executes instructions stored in memory 204. Also, although embodiments are described with respect to central processing unit chips, it should be understood that the embodiments described herein are also applicable to computer systems that utilize a digital signal processor (DSP) or a graphics processing unit (GPU) chip or both in addition to or instead of a CPU chip. Thus, references herein to a processor or processing unit may refer to a CPU chip, a GPU chip, or a DSP or a combination thereof.

[0028] In some embodiments, memory 204 includes a random access semiconductor memory, a storage device, or a storage medium (either volatile or non-volatile) for storing or encoding data and programs. For example, memory 204 stores instructions 211, resource data 213, and a plurality of GAN plugins 215. Instructions 211, when executed by a processor such as processor 202, cause the processor 202 to perform the functions and calculations described herein with respect to monitoring, collecting resource data 213, predicting resource usage, selecting one of the GAN plugins 215, and executing the selected GAN plugin 215 to transform the video quality of the received input media file.

[0029] In some embodiments, memory 204 represents the entire virtual memory of computing device 200 and may also include the virtual memory of other computer devices coupled to computing device 200 via a network. In some embodiments, memory 204 is a single monolithic entity, while in other embodiments, memory 204 includes a hierarchy of caches and other memory devices. For example, memory 204 can exist at multiple levels of cache, and these caches can be further divided by function used by the processor such that one cache holds instructions and another cache holds non-instruction data. Memory 204 may be further distributed and associated with different processing units or sets of processing units, as is known, for example, in various so-called non-uniform memory access (NUMA) computer architectures. Thus, in the example shown in FIG. 2 for illustrative purposes, instructions 240, resource data 213, and GAN plugin 215 are stored in the same memory 204, but it should be understood that in other embodiments they may be implemented differently. For example, instructions 240, resource data 213, or GAN plugin 215 or combinations thereof may be distributed across multiple physical media.

[0030] In the embodiment shown in FIG. 2, the computing device 200 includes a processor 202, a memory 204, a display system 224, and a bus interface unit 209 for handling communication between the display system 224 and the I / O bus interface unit 210. The I / O bus interface unit 210 is coupled to an I / O bus 208 for transferring data between various I / O units. Specifically, the I / O bus interface unit 210 can communicate with a plurality of I / O interface units 212, 216, and 218, also known as I / O processors (IOPs) or I / O adapters (IOAs), via the I / O bus 208. The display system 224 includes a display controller, a display memory, or both. The display controller can provide video, still images, audio, or a combination thereof to the display device 226. The display memory may be a dedicated memory for buffering video data. The display system 224 is coupled to the display device 226. In some embodiments, the display device 226 also includes one or more speakers for rendering audio. Alternatively, one or more speakers for rendering audio may be coupled to an I / O interface unit. In alternative embodiments, one or more functions provided by the display system 224 are implemented on an integrated circuit that also includes the processor 202. Additionally, in some embodiments, one or more of the functions provided by the bus interface unit 209 are implemented on an integrated circuit that also includes the processor 202.

[0031] The I / O interface unit supports communication with various storage and I / O devices. For example, the I / O device interface unit 212 supports connection of one or more user I / O devices 220, and the one or more user I / O devices 220 may include a user output device and a user input device (such as a keyboard, mouse, keypad, touchpad, trackball, button, light pen, or other pointing device). The user can operate the user input device using the user interface to provide input data and commands to the user I / O device 220. Further, the user can receive output data via the user output device. For example, the user interface may be presented via the user I / O device 220, such as being displayed on a display device or played via a speaker.

[0032] The storage interface 216 supports connection of one or more storage devices 228, such as flash memory. The contents of the memory 204 or any portion thereof may be stored in the storage device 228 and retrieved from the storage device 228 as needed. The network interface 218 provides one or more communication paths from the computing device 200 to other digital and computer devices. For example, in some embodiments, the computing device 200 can communicate with a server, such as server 110, via the network interface 218 to request and receive media files. Further, the computing device 200 can communicate with the server via the network interface 218 to indicate the desired resolution quality of the media file.

[0033] The computing device 200 shown in FIG. 2 illustrates a particular bus structure that provides direct communication paths between a processor 202, a memory 204, a bus interface unit 209, a display system 224, and an I / O bus interface unit 210. However, in alternative embodiments, the computing device 200 may include different buses or communication paths, which may be arranged in any of a variety of forms, such as point-to-point links in a hierarchical configuration, a star configuration, or a web configuration, multiple hierarchical buses, parallel paths and redundant paths, or any other suitable type of configuration. Further, although the I / O bus interface unit 210 and the I / O bus 208 are shown as a single individual unit, in other embodiments, the computing device 200 may include multiple I / O bus interface units 210 or multiple I / O buses 208 or both. Multiple I / O interface units are shown that separate the I / O bus 208 from various communication paths extending to various I / O devices, but in other embodiments, some or all of the I / O devices are directly connected to one or more system I / O buses.

[0034] As described above, in some embodiments, one or more of the components and data shown in FIG. 2 include instructions or statements that are executed on the processor 202 or interpreted by instructions or statements executed on the processor 202 to perform the functions described herein. In other embodiments, one or more of the components shown in FIG. 2 are implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, or other physical hardware devices or combinations thereof, instead of or in addition to a processor-based system. Additionally, in other embodiments, it is possible to omit some of the components shown in FIG. 2, include other components, or both.

[0035] FIG. 3 is a flowchart illustrating an exemplary method 300 for reducing bandwidth consumption via a Generative Adversarial Network (GAN). The method 300 may be implemented by a computing device such as the computing device 200 described above. It should be understood that the order of actions in the exemplary method 300 is provided for illustrative purposes and that in other embodiments the method may be executed in a different order. Similarly, it should be understood that in other embodiments some actions may be omitted or additional actions may be included.

[0036] At 302, a plurality of GAN plugins are created and trained. As described herein, each of the plurality of GAN plugins can utilize a different network size and can have different requirements for the minimum input resolution to achieve a selected output resolution. Further, each GAN plugin can have requirements for processing resources such as memory, CPU, etc., that are different from other GAN plugins. In some embodiments, five GAN plugins are created, each having a different network size. Each GAN plugin can be pre-trained before being loaded onto the computing device. Further, in some embodiments, the GAN plugin can be an Internet browser plugin. In other embodiments, the GAN plugin can be a stand-alone application, or a plugin incorporated into another program capable of obtaining media files from a server, or both.

[0037] In 304, the desired resolution quality of a media file (e.g., an image or video file) is determined. For example, a user can select a media file to obtain from a server. Such a server can include, but is not limited to, a video sharing website, a social media platform, a File Transfer Protocol (FTP) server, or other remote locations accessible via a wide area network such as a local area network or the Internet. In some embodiments, in response to detecting the user's selection, the user can be prompted to enter an input to a question for selecting the desired resolution quality of the media file. In other embodiments, the user can empirically enter the desired default resolution quality of the media file. In still other embodiments, the system can learn from the user's responses and automatically determine, based on the historical data, which resolution quality should be provided to the user. The resolution quality can be indicated using conventional notations such as 1080p, 720p, etc.

[0038] At 306, the available resources of the user's computing device are determined. As described above, the available resources can include, but are not limited to, the CPU, memory, bus bandwidth, etc. The available resources used herein refer to the resources that are not utilized or reserved by another process or application and are thus available for use by the GAN plugin to process and convert the received media file. In some embodiments, the currently available resources are determined. In other embodiments, in addition to the currently available resources, future available resources are predicted as described above. For example, as described above, the resource usage on the user's computing device can be monitored over a long period of time to accumulate a record of historical data. In some embodiments, future available resources can be determined based on the historical data. Additionally, in some embodiments, predicting future available resources includes predicting the future available resources in a first period, and before the completion of the first period, the second future available resources in a second period are predicted. For example, as described above, the file size of the media file is unknown, and thus the time for downloading or receiving the media file is divided into shorter periods, such as 45 seconds. The future available resources in the first 45 seconds can be predicted, then the future available resources in the next 45 seconds can be predicted, and so on until the media file is fully received.

[0039] At 308, based on the determined available resources of the computing device, one of a plurality of adversarial generation network (GAN) plugins is selected. As described above, each GAN plugin can have a different size. Thus, each GAN plugin can have different processing requirements or resource requirements based on the size of the GAN. In some embodiments, the same GAN plugin is used to process the entire media file. However, in other embodiments, multiple GAN plugins can be used. For example, in embodiments where the time for downloading or receiving a media file is divided into successive periods that are smaller, the computing device can select a GAN plugin for each successive period based on the currently available resources or the predicted available resources or both in each period. For example, in the above example regarding the first period and the second period, based on the predicted first future available resources, a first GAN plugin for use during the first period can be selected, and based on the predicted second future available resources, a second GAN plugin for use during the second period can be selected. If the predicted first future available resources and the predicted second future available resources are similar or the same, the same GAN plugin can be selected (i.e., the first GAN plugin and the second GAN plugin are the same). However, if the predicted first future available resources and the predicted second future available resources are different, different GAN plugins can be selected (i.e., the first GAN plugin is different from the second GAN plugin).

[0040] At 310, based on the selected GAN plugin, a second resolution quality of the media file is estimated. As described above, each GAN plugin has a minimum resolution quality required to convert the media file to a desired resolution quality. For example, the minimum input resolution quality of a small GAN plugin needs to be higher than that of a large GAN plugin. As described above, the second resolution quality is lower than the desired resolution quality so that the bandwidth usage for receiving the media file is reduced.

[0041] At 312, the computing device communicates with the server to receive the media file at the estimated second resolution quality. For example, the computing device can contact the server and request that the media file be sent at the second resolution quality. Further, as described above, in some embodiments, different GAN plugins having different sizes can be used at different times when receiving the media file. When a new GAN plugin is selected, the computing device can communicate with the server to request the media file at a new resolution quality (e.g., a third resolution quality) corresponding to the new GAN plugin. In this way, when the available resources of the computing device increase, a larger GAN plugin can be selected and the media file can be received at a resolution quality lower than the desired resolution quality or the second resolution quality. In contrast, when the available resources of the computing device decrease, a smaller GAN plugin that requires relatively fewer resources than a large GAN plugin can be selected. Thus, the third resolution quality is higher than the second resolution quality but may still be lower than the desired resolution quality, resulting in a further reduction in bandwidth usage compared to receiving the media file at the desired resolution quality.

[0042] Furthermore, in some embodiments, it should be understood that a media file can be requested at a resolution quality that is not the minimum resolution quality required by the selected GAN plugin. For example, if the minimum resolution quality of the selected GAN plugin is 240p, in some embodiments, the computing device can estimate a second resolution quality that is higher than the minimum 240p but lower than the desired resolution quality, such as 360p or 480p.

[0043] At 314, the received media file is converted from the second resolution quality to the desired resolution quality using the selected GAN plugin. As described above, in embodiments where multiple different sized GAN plugins are continuously selected based on changes in available resources, each portion of the received media file is converted using the corresponding GAN plugin. The media file is converted to the desired resolution quality and output at the desired resolution quality at 316. For example, the converted media file can be displayed on a display screen of the computing device. Furthermore, in some embodiments, it should be understood that the actions performed at 306 - 314 can be performed in real - time substantially simultaneously with the media file being output at 316. That is, while a portion of the media file is being output / displayed, the computing device monitors the availability of resources, detects changes in the availability of resources, and adjusts the selected GAN as needed while subsequent other portions are received and converted. In other embodiments, the user can specify a scheduled time to view the media file or download the media file without playing the media file simultaneously. In such embodiments, the conversion can be performed before the media file is output.

[0044] Accordingly, the embodiments described herein enable reduction of the bandwidth usage amount when acquiring a media file. Since problems such as buffering delay are reduced by this reduction of the bandwidth usage amount, the user satisfaction when viewing the media file can be improved. Further, the reduction of the bandwidth usage amount can bring other advantages in embodiments where the bandwidth usage amount is metered. However, even though the bandwidth is reduced and the resolution quality of the acquired media file is degraded, the output resolution quality of the media file still remains at the resolution quality desired by the user.

[0045] The present invention may be a system, a method, or a computer program product or a combination thereof at any possible technical detail integration level. The computer program product may include a computer-readable storage medium (or multiple computer-readable storage media) having computer-readable program instructions for causing a processor to implement aspects of the present invention.

[0046] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction-executing device. The computer-readable storage medium can be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, but is not limited thereto. A non-exhaustive list of more specific examples of computer-readable storage media includes the following, namely, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards, or mechanically encoded devices such as raised structures within grooves in which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed to be a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through an electrical wire.

[0047] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices or to an external computer or an external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include a copper transmission cable, an optical transmission fiber, a wireless transmission, a router, a firewall, a switch, a gateway computer, or an edge server, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0048] Computer-readable program instructions for carrying out the operations of the present invention may be any combination of source code or object code written in one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or any combination of one or more programming languages such as object-oriented programming languages like Smalltalk(R), C++, and procedural programming languages like the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer as a stand-alone software package, partly on the user's computer, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to carry out aspects of the present invention, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0049] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0050] Instructions executed via the processor of a computer or other programmable data processing apparatus may provide means for creating a machine to perform the functions / acts specified in one or more blocks of a flowchart or block diagram or both, such that the processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus creates a machine. These computer-readable program instructions may also be stored in a computer-readable storage medium that includes a manufactured article that includes instructions for implementing the aspects of the functions / acts specified in one or more blocks of a flowchart or block diagram or both, and can be instructed to function in a particular manner with respect to a computer, programmable data processing apparatus, or other device or combination thereof.

[0051] The computer-readable program instructions may also be loaded onto a computer, other programmable apparatus, or other device to create a computer-implemented process such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / acts specified in one or more blocks of a flowchart or block diagram or both, causing a series of operational steps to be executed on the computer, other programmable apparatus, or other device.

[0052] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may be performed in an order different than that noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently depending on the functionality involved, or the blocks may sometimes be executed in the reverse order. It should also be noted that each block of the block diagrams or flowcharts, or both, and combinations of blocks in the block diagrams or flowcharts, or both, can be implemented by a dedicated hardware-based system that performs the specified function or acts, or by a combination of dedicated hardware and computer instructions.

[0053] Although specific embodiments have been illustrated and described herein, it will be understood by those skilled in the art that any configuration determined to achieve the same purpose may be used instead of the specific embodiments shown. Accordingly, it is clearly intended that the present invention be limited only by the claims and their equivalents.

Claims

A method for information processing by a computer, comprising: Determining a desired resolution quality of a media file; Determining available resources of a computing device; Based on the determined available resources of the computing device, selecting one of a plurality of adversarial generation network (GAN) plugins; Estimating a second resolution quality of the media file corresponding to the selected GAN plugin, wherein the second resolution quality is lower than the desired resolution quality; Communicating with a server to receive the media file at the estimated second resolution quality; Using the selected GAN plugin to convert the received media file from the second resolution quality to the desired resolution quality; Outputting the converted media file at the desired resolution quality A method comprising: Claim 2 The method according to claim 1, wherein the second resolution quality is a minimum resolution quality associated with the selected GAN plugin. Claim 3 The method according to claim 1 or 2, wherein the plurality of GAN plugins includes five GAN plugins, and each of the plurality of GAN plugins has a different size from other GAN plugins within the plurality of GAN plugins. Claim 4 The method according to any one of claims 1 to 3, wherein the plurality of GAN plugins are Internet browser plugins that are pre-trained before being loaded onto the computing device. Claim 5 Determining available resources of a computing device includes predicting future available resources of the computing device based on historical data, Selecting one of the plurality of GAN plugins includes selecting one of the plurality of GAN plugins based on the predicted future available resources, The method according to any one of claims 1 to 4. Claim 6 Predicting future available resources of the computing device includes Predicting first future available resources of the computing device in a first period; Predicting, before the completion of the first period, second future available resources of the computing device during a second period including selecting one of the plurality of GAN plugins, selecting a first GAN plugin to be used during the first period based on the predicted first future available resources, selecting a second GAN plugin to be used during the second period based on the predicted second future available resources The method according to claim 5, comprising:

7. The method according to claim 6, wherein the first GAN plugin is different from the second GAN plugin.

8. Monitoring available resources of the computing device while receiving the media file from the server to detect changes in the available resources of the computing device; In response to detecting the change in the available resources of the computing device, selecting a second GAN plugin from the plurality of GAN plugins based on the detected change in the available resources of the computing device; estimating a third resolution quality of the media file corresponding to the selected second GAN plugin, the third resolution quality being lower than the desired resolution quality and different from the second resolution quality; communicating with the server to receive the media file at the estimated third resolution quality; converting the received media file from the third resolution quality to the desired resolution quality by using the selected second GAN plugin The method according to any one of claims 1 to 7, further comprising:

9. A computing device, comprising: an interface communicatively coupled to a server; a processor communicatively coupled to the interface, the processor being configured to: determine a desired resolution quality of a media file; determine available resources of the computing device; select one of a plurality of generative adversarial network (GAN) plugins based on the determined available resources of the computing device estimating a second resolution quality of the media file corresponding to the selected GAN plugin, wherein the second resolution quality is lower than the desired resolution quality; communicating with the server via the interface to request the media file at the estimated second resolution quality; using the selected GAN plugin to convert the media file received from the server at the second resolution quality to the desired resolution quality; outputting the converted media file at the desired resolution quality A computing device configured to perform the above. **Claim 10**: A computer-executable computer program for causing a computer to execute the method according to any one of Claims 1 to 8. **Claim 11**: A computer-readable recording medium storing the computer program according to Claim 10.

Images