Apparatus, method, and storage medium for providing video service

Abstract

This device of a network node may comprise: a memory for storing instructions; and at least one processor. The instructions, when executed individually or collectively by the at least one processor, may instruct the device to: acquire compressed data including feature points of a first frame for a video service between a first terminal and a second terminal; identify a neural network for performing decompression of the compressed data in the network node or a load related to the second terminal; according to the load exceeding a reference load, transmit the compressed data to the second terminal in order to enable the second terminal to perform decompression of the compressed data; according to the load less than the reference load, generate a second frame for the video service by performing decompression of the compressed data on the basis of the neural network; and transmit the second frame to the second terminal.

Description

Device, method, and storage medium for providing video service

[0001] The descriptions below relate to devices, methods, and storage media for providing video services.

[0002] An electronic device may provide a video service. For example, a frame for the video service may include an image. For example, the frame for the video service may be provided to an external electronic device that provides the video service. For example, the frame may be provided as compressed data from the electronic device to the external electronic device.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0004] According to embodiments of the present disclosure, a device of a network node may include a memory that stores instructions. The device may include at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the device to obtain compressed data including feature points of a first frame for a video service between a first terminal and a second terminal. The instructions, when individually or collectively executed by the at least one processor, may cause the device to identify a load associated with a neural network for performing decompression of the compressed data within the network node. The instructions, when individually or collectively executed by the at least one processor, may cause the device to transmit the compressed data to the second terminal to enable the second terminal to perform decompression of the compressed data in response to the load exceeding a reference load. The above instructions, when individually or collectively executed by the at least one processor, may cause the device to generate a second frame for the video service by decompressing the compressed data based on the neural network according to the load being less than the reference load, and to transmit the second frame to the second terminal.

[0005] According to embodiments of the present disclosure, a first terminal may include at least one transceiver. The first terminal may include a memory storing instructions and including one or more storage media. The first terminal may include at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to identify a value indicative of a quality of a channel between the first terminal and a network node associated with a video service between the first terminal and a second terminal. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to transmit, to the network node, first compressed data of a frame generated by performing compression on the frame for the video service according to the value exceeding a reference value. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to generate second compressed data including feature points by performing compression on the frame based on a neural network within the first terminal according to the value being less than the reference value, and to transmit the second compressed data to the network node.

[0006] Figure 1 illustrates an example of a wireless communication system.

[0007] Figure 2 illustrates an example of a functional configuration of an electronic device.

[0008] Figure 3a illustrates an example of a network for providing video services between a first terminal and a second terminal.

[0009] Figure 3b shows an example of a neural network for decompression.

[0010] FIG. 4 illustrates an example of a signal flow for a method of compressing a frame for a video service between a first terminal and a second terminal and decompressing the compressed data.

[0011] Figure 5a illustrates an example of an operational flow for a method of performing decompression on compressed data.

[0012] Figure 5b illustrates an example of how a second terminal performs decompression on compressed data.

[0013] Figure 5c illustrates an example of how a network node performs decompression on compressed data.

[0014] Figure 6a illustrates an example of an operational flow for how a first terminal performs compression on a frame.

[0015] Figure 6b shows examples of frames for a video service.

[0016] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.

[0017] The various embodiments of the present disclosure described below illustrate a hardware-based approach as an example. However, since the various embodiments of the present disclosure include techniques utilizing both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0018] In the following description, terms referring to signals (e.g., packet, message, signal, information, signaling), terms referring to resources (e.g., section, symbol, slot, subframe, radio frame, subcarrier, RE (resource element), RB (resource block), BWP (bandwidth part), band, spectrum), terms for operational states (e.g., step, operation, procedure), terms referring to data (e.g., packet, message, user stream, information, bit, symbol, codeword), terms referring to channels, terms referring to network entities (distributed unit (DU), radio unit (RU), central unit (CU), control plane (CU-CP), user plane (CU-UP), open radio access network (O-RAN) DU (O-DU), O-RAN RU (O-RU), Terms such as O-CU (O-RAN CU), O-CU-UP (O-RAN CU-CP), O-CU-CP (O-RAN CU-CP)), referring to components of the device, are examples for convenience of explanation. Therefore, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used. In addition, terms such as '... part', '... device', '... object', '... body', etc. used below may mean at least one shape structure or a unit that processes a function.

[0019] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled, but this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." A condition described as "more than" may be replaced with "more than," a condition described as "less than" may be replaced with "less than," and a condition described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of the elements from A (including A) to B (including B). hereinafter, "C" and / or "D" mean at least one of "C" or "D," that is, including {"C", "D", "C" and "D"}. hereinafter, the meaning of "about E" may be replaced with a value within a margin of error of ±5% or ±10% based on E.

[0020] Although this disclosure describes embodiments using terminology used in certain communication standards (e.g., 3rd Generation Partnership Project (3GPP)), this is merely an example for illustrative purposes. Embodiments of this disclosure can also be applied to other communication and broadcasting systems.

[0021] Figure 1 illustrates an example of a wireless communication system.

[0022] Referring to FIG. 1, FIG. 1 illustrates a base station (110) and a terminal (120) as some of the nodes utilizing a wireless channel in a wireless communication system. Although FIG. 1 illustrates only one base station, the wireless communication system may further include other base stations identical or similar to the base station (110).

[0023] The base station (110) is a network infrastructure that provides wireless access to the terminal (120). The base station (110) has coverage defined based on the distance at which a signal can be transmitted. In addition to the base station, the base station (110) may be referred to as an 'access point (AP)', 'eNodeB (eNB)', '5th generation node', 'next generation nodeB (gNB)', 'wireless point', 'transmission / reception point (TRP)', or other terms having equivalent technical meanings.

[0024] The terminal (120) is a device used by a user and communicates with the base station (110) via a wireless channel. The link from the base station (110) to the terminal (120) is referred to as a downlink (DL), and the link from the terminal (120) to the base station (110) is referred to as an uplink (UL). In addition, although not shown in FIG. 1, the terminal (120) and another terminal may communicate with each other via a wireless channel. In this case, the link between the terminal (120) and another terminal (device-to-device link, D2D) is referred to as a sidelink, and the sidelink may be used interchangeably with the PC5 interface. In some other embodiments, the terminal (120) may be operated without the involvement of a user. In one embodiment, the terminal (120) is a device that performs machine type communication (MTC) and may not be carried by the user. Additionally, according to one embodiment, the terminal (120) may be an NB (narrowband)-IoT (internet of things) device.

[0025] The terminal (120) may be referred to as a terminal, or other terms such as 'user equipment (UE),' 'customer premises equipment (CPE),' 'mobile station,' 'subscriber station,' 'remote terminal,' 'wireless terminal,' 'electronic device,' or 'user device,' or other terms having equivalent technical meanings.

[0026] The base station (110) and the terminal (120) can perform beamforming. The base station (110) and the terminal (120) can transmit and receive wireless signals in a relatively low frequency band (e.g., FR 1 (frequency range 1) of NR). In addition, the base station (110) and the terminal (120) can transmit and receive wireless signals in a relatively high frequency band (e.g., FR 2 (or, FR 2-1, FR 2-2, FR 2-3), FR 3 of NR), millimeter wave (mmWave) band (e.g., 28 GHz, 30 GHz, 38 GHz, 60 GHz)). To improve channel gain, the base station (110) and the terminal (120) can perform beamforming. Here, the beamforming can include transmission beamforming and reception beamforming. The base station (110) and the terminal (120) can impart directionality to the transmitted or received signal. To this end, the base station (110) and the terminal (120) can select serving beams through a beam search or beam management procedure. After the serving beams are selected, subsequent communication can be performed through resources that have a QCL relationship with the resource that transmitted the serving beams.

[0027] If large-scale characteristics of a channel carrying a symbol on a first antenna port can be inferred from a channel carrying a symbol on a second antenna port, the first antenna port and the second antenna port can be evaluated to have a QCL relationship. For example, the large-scale characteristics may include at least one of delay spread, Doppler spread, Doppler shift, average gain, average delay, and a spatial receiver parameter.

[0028] Although both the base station (110) and the terminal (120) are described as performing beamforming in FIG. 1, the embodiments of the present disclosure are not necessarily limited thereto. In some embodiments, the terminal may or may not perform beamforming. Furthermore, the base station may or may not perform beamforming. That is, either only one of the base station and the terminal may perform beamforming, or neither the base station nor the terminal may perform beamforming.

[0029] In the present disclosure, a beam refers to a spatial flow of a signal in a wireless channel, and is formed by one or more antennas (or antenna elements), and this forming process may be referred to as beamforming. Beamforming may include at least one of analog beamforming and digital beamforming (e.g., precoding). Reference signals transmitted based on beamforming may include, for example, a demodulation-reference signal (DM-RS), a channel state information-reference signal (CSI-RS), a synchronization signal / physical broadcast channel (SS / PBCH), and a sounding reference signal (SRS). In addition, as a configuration for each reference signal, an IE such as a CSI-RS resource or an SRS-resource may be used, and this configuration may include information associated with the beam. Information associated with a beam may mean whether the configuration (e.g., a CSI-RS resource) uses the same spatial domain filter as another configuration (e.g., another CSI-RS resource within the same CSI-RS resource set) or a different spatial domain filter, or whether it is quasi-co-located (QCL) with a reference signal, and if so, what type it is (e.g., QCL type A, B, C, D).

[0030] In the past, in communication systems with relatively large cell radius of base stations, each base station was installed to include the functions of a digital processing unit (or distributed unit (DU)) and a radio frequency (RF) processing unit (or radio unit (RU)). However, as higher frequency bands are used in 4G (4th generation) and / or subsequent communication systems (e.g., 5G) and the cell coverage of base stations decreases, the number of base stations to cover a specific area has increased. The burden of installation costs on operators for installing base stations has also increased. In order to minimize the installation cost of base stations, a structure has been proposed in which the DU and RU of a base station are separated, one or more RUs are connected to one DU via a wired network, and one or more RUs are geographically distributed to cover a specific area.

[0031] For example, network entities according to a distributed deployment may include a digital unit (DU) and a radio unit (RU) (or a massive multiple input multiple output (MMU) unit). For example, the network entities may be connected via a fronthaul. Unlike the backhaul between a base station and a core network, the fronthaul refers to entities (e.g., DU, RU) between a wireless LAN and a base station. Although a fronthaul structure between a DU and a single RU is exemplified, this is merely for convenience of description and the present disclosure is not limited thereto. In other words, embodiments of the present disclosure can also be applied to a fronthaul structure between a single DU and a plurality of RUs. For example, embodiments of the present disclosure can be applied to a fronthaul structure between a single DU and two RUs. Furthermore, embodiments of the present disclosure can also be applied to a fronthaul structure between a single DU and three RUs.

[0032] For example, the base station (110) may include a DU and an RU. The fronthaul between the DU and the RU may be operated via an Fx interface. For the operation of the fronthaul, an interface such as an enhanced common public radio interface (eCPRI) or radio over ethernet (ROE) may be used. Depending on the implementation example, the DU may be referred to as a baseband unit (BBU), a digital BBU, a baseband digital unit, a digital processing unit, a digital processing circuit, a baseband processing circuit, a baseband processing unit, and / or equivalent technical terms in addition to a DU (digital unit). Depending on the implementation example, the RU may be referred to as a remote unit, a radio demote head (RRH), a radio processing circuit, a radio processing unit, an antenna integrated radio, an air radio device, an air scale communication device, a radio device, a radio communication device, and / or equivalent technical terms in addition to the RU (radio unit). In addition, depending on the implementation example, although a network entity connected to a DU in the present disclosure is described as an RU, it is to be understood that a massive multiple input multiple output (MMU) unit may be connected to and used with the DU instead of the RU.

[0033] As communications technology advances, mobile data traffic increases, significantly increasing the bandwidth requirements for the fronthaul between the digital unit and the radio unit. In deployments such as C-RAN (centralized / cloud radio access network), the DU performs functions for the packet data convergence protocol (PDCP), radio link control (RLC), media access control (MAC), and physical layer (PHY), while the RU can be implemented to perform additional functions for the PHY layer in addition to its radio frequency (RF) functions.

[0034] A DU may be responsible for upper layer functions of a wireless network. For example, a DU may perform functions of a MAC layer and a part of a PHY layer. Here, a part of the PHY layer refers to functions performed at a higher level among the functions of the PHY layer, and may include, for example, channel encoding (or channel decoding), scrambling (or descrambling), modulation (or demodulation), and layer mapping (or layer demapping). In one embodiment, if a DU complies with the O-RAN standard, it may be referred to as an O-DU (O-RAN DU). If necessary, a DU may be represented by being replaced with a first network entity for a base station (e.g., gNB) in embodiments of the present disclosure.

[0035] An RU may be responsible for lower layer functions of a wireless network. For example, an RU may perform a part of a PHY layer, an RF function. Here, a part of the PHY layer refers to functions of the PHY layer that are performed at a relatively lower level than a DU, and may include, for example, iFFT transform (or FFT transform), CP insertion (CP removal), and digital beamforming. An RU may be referred to as an 'access unit (AU)', an 'access point (AP)', a 'transmission / reception point (TRP)', a 'remote radio head (RRH)', a 'radio unit (RU)', or other terms having equivalent technical meanings. In one embodiment, when an RU complies with the O-RAN standard, it may be referred to as an O-RU (O-RAN RU). An RU may be represented by being replaced with a second network entity for a base station (e.g., gNB) in embodiments of the present disclosure, as needed.

[0036] In the above example, the base station (110) is described as including a DU and an RU, but the embodiments of the present disclosure are not limited thereto. The base station (110) according to the embodiments may be implemented in a distributed deployment according to a centralized unit (CU) configured to perform functions of upper layers of an access network (e.g., packet data convergence protocol (PDCP), radio resource control (RRC)) and a distributed unit (DU) configured to perform functions of lower layers. For example, the digital unit (DU) may be implemented by separating it into a centralized unit (CU) and a distributed unit (DU). Between a core (e.g., 5G core (5GC) or next generation core (NGC)) network and a radio network (RAN), the base station (110) may be implemented in a structure in which a centralized unit (CU), a distributed unit (DU), and a radio unit (RU) are arranged in that order. The interface between the CU (centralized unit) and the DU (distributed unit) can be referred to as the F1 interface.

[0037] A centralized unit (CU) may be connected to one or more distributed units (DUs) and may be responsible for functions at a higher layer than the distributed units (DUs). For example, the CU may be responsible for functions at the radio resource control (RRC) and packet data convergence protocol (PDCP) layers, while the DU and RU may be responsible for functions at lower layers. The DU may perform some functions (high PHY) of the radio link control (RLC), media access control (MAC), and physical (PHY) layers, while the RU may be responsible for the remaining functions (low PHY) of the PHY layer. In addition, for example, a digital unit (DU) may be included in a distributed unit (DU) depending on the implementation of a distributed deployment of the base station. Hereinafter, unless otherwise defined, the operations of DU (digital unit) and RU are described, but various embodiments of the present disclosure can be applied to both a base station arrangement including a CU and an arrangement in which a DU is directly connected to a core network (i.e., a base station in which the CU and DU are integrated into a single entity (e.g., an NG-RAN node)).

[0038] Figure 2 illustrates an example of a functional configuration of an electronic device.

[0039] The configuration of the electronic device (200) illustrated in FIG. 2 may be understood as a configuration of a base station (110), a terminal (120), a CU, a DU, or a network node (e.g., the first network node (330) or the second network node (340) of FIG. 3A). Terms such as "...unit" and "...unit" used hereinafter mean a unit that processes at least one function or operation, and this may be implemented by hardware, software, or a combination of hardware and software.

[0040] Referring to FIG. 2, the electronic device (200) may include a transceiver (210), a memory (220), and a processor (230). However, the present disclosure is not limited thereto. For example, the electronic device (200) may not include at least some of the components illustrated in FIG. 2, or may include further components not illustrated in FIG. 2. For example, the electronic device (200) may not include the transceiver (210).

[0041] The transceiver (210) can perform functions for transmitting and receiving signals in a wired communication environment. The transceiver (210) can include a wired interface for controlling direct connections between devices via a transmission medium (e.g., copper wire, optical fiber). For example, the transceiver (210) can transmit electrical signals to other devices via copper wire, or perform conversion between electrical signals and optical signals.

[0042] The transceiver (210) may perform functions for transmitting and receiving signals in a wireless communication environment. For example, the transceiver (210) may perform a conversion function between baseband signals and bit streams according to the physical layer specifications of the system. For example, when transmitting data, the transceiver (210) encodes and modulates the transmitted bit stream to generate complex-valued symbols. Furthermore, when receiving data, the transceiver (210) demodulates and decodes the baseband signal to restore the received bit stream. Furthermore, the transceiver (210) may include multiple transmission and reception paths.

[0043] The transceiver (210) transmits and receives signals as described above. Accordingly, all or part of the transceiver (210) may be referred to as a "communication unit," a "transmitter," a "receiver," or a "transmitter-receiver unit." Furthermore, in the following description, transmission and reception performed via a wireless channel are used to mean that the transceiver (210) performs the processing described above.

[0044] Although not illustrated in FIG. 2, the transceiver (210) may further include a backhaul transceiver for connection to the core network or other base stations. The backhaul transceiver provides an interface for communicating with other nodes within the network. That is, the backhaul transceiver converts a bit stream transmitted from the base station to other nodes, such as other access nodes, other base stations, upper nodes, the core network, etc., into a physical signal, and converts a physical signal received from other nodes into a bit stream.

[0045] The memory (220) stores data such as basic programs, application programs, and setting information for the operation of the electronic device (200). The memory (220) may be referred to as a storage unit. The memory (220) may be composed of volatile memory, nonvolatile memory, or a combination of volatile memory and nonvolatile memory. In addition, the memory (220) provides stored data upon request from the processor (230).

[0046] For example, the processor (230) may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits including at least one processor, one or more of which may be configured to individually and / or collectively perform the various functions described below in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms encompass, for example, and without limitation, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions, and also situations where one processor may perform all of the recited functions. Additionally, the at least one processor may include a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0047] The processor (230) controls the overall operations of the electronic device (200). The processor (230) may be referred to as a control unit. For example, the processor (230) transmits and receives signals via the transceiver (210) (or via the backhaul communication unit). In addition, the processor (230) records and reads data from the memory (220). In addition, the processor (230) may perform functions of a protocol stack required by a communication standard. Although only the processor (230) is illustrated in FIG. 2, the electronic device (200) may include two or more processors according to other implementation examples.

[0048] For example, the terminal (120) can provide a video service to the user. As a non-limiting example, the video service may include a video call service or a video streaming service. The following description will be based on the video service, which is a video call service, but the present disclosure is not limited thereto.

[0049] For example, for the aforementioned video service, a relatively high bandwidth may be required for relatively high data transmission. As user demand for the aforementioned video service increases, a large amount of network resources may be required to provide relatively high-quality video calls.

[0050] Within the framework for providing the above video service, frames for the video service may be compressed. When compression of the frames based on a compression algorithm is used, reducing the size of the compressed data of each of the frames may be limited. In other words, reducing the size of the compressed data below a certain size may be difficult. For example, the compression algorithm may include a discrete cosine transform, motion compensation, or entropy coding. Alternatively, for example, the compression algorithm may include H.264 (or AVC (advanced video coding)), H.265 (or HEVC (high efficiency video coding)), VP9, ​​etc. As described above, since the video service has high data requirements, a communication connection according to a communication technique of 4G or higher is required. In this case, if the size of the compressed data can be further reduced, the video service can be provided even through a communication connection according to a communication technique that provides a lower bandwidth (e.g., 2G).

[0051] To further reduce the size of the compressed data, compression based on a neural network may be utilized. For example, the neural network may be referred to as an artificial intelligence (AI) model, engine, or module. For example, compression based on the neural network may be referred to as neural compression. While neural compression can provide a high level of compression, it may be difficult to apply to the network (or mobile network) between the terminal (120) and the base station (110). For example, although the neural network is utilized to compress and decompress frames for the video service, the roles of nodes within the network (e.g., the terminal (120), the base station (110), or nodes of the core network) may be unclear. When the terminal (120) performs frame compression based on the neural network, high computational resources of the terminal (120) are required. When the base station (110) performs compression on a frame based on the neural network, the load of the base station (110) may not be taken into consideration. In addition, the compression on the frame based on the neural network may not utilize the location information (e.g., global positioning system (GPS) coordinates) of the terminal (120). The compression on the frame based on the neural network is performed using feature points of a specific object (e.g., a face) within the frame, but may not consider a background image other than the specific object. In other words, the compression on the frame based on the neural network is focused on reducing the size of the compressed data, and a method for more efficient compression and reducing the load on nodes within the network needs to be considered.

[0052] Currently, the base station (110) can provide the compressed data provided from the terminal (120) to other terminals. As technology advances, the base station (110) implemented through a virtual RAN (vRAN) or cloud RAN (cRAN) can actively participate in the operation of the framework for the video service. The base station (110) implemented as a vRAN or cRAN can provide the high-quality video service within a service area that provides connectivity or low bandwidth by utilizing high computational performance.

[0053] Hereinafter, the present disclosure proposes a framework for the video service using a neural network. For example, the present disclosure may be applied to a network including a first terminal (e.g., terminal 120 of FIG. 1) transmitting a frame for the video service, a second terminal (e.g., terminal 120 of FIG. 1) receiving the frame for the video service, and at least one base station (e.g., base station 110 of FIG. 1) between the first terminal and the second terminal. For example, the first terminal may be referred to as a transmitting terminal, Tx UE, or sender. For example, the second terminal may be referred to as a receiving terminal, Rx UE, or receiver. The present disclosure may reduce the size of compressed data of a frame transmitted from the first terminal. For example, by compressing the frame based on the neural network, compressed data including feature points representing specific objects (e.g., a face or a background image) may be generated. For example, the size of the compressed data can be reduced by transmitting compressed data containing the feature points instead of the compressed data of the frame itself. The compressed data can then be reconstructed into a decompressed frame based on another neural network.

[0054] In addition, the present disclosure can perform neural network-based compression and neural network-based decompression using the load of the base station, battery information of the terminal, or channel information of the channel between the base station and the terminal. For example, the base station can perform the neural network-based decompression instead if the second terminal is not in a state where it can perform the neural network-based decompression. For example, the state may be related to resources related to the decompression of the second terminal or battery information of the second terminal. In addition, the present disclosure can utilize location information for the compression and decompression. For example, the location information may include location information of the first terminal that performed the compression. For example, when the decompression is performed, a background image mapped (or stored) to the location information of the first terminal can be synthesized for the decompressed frame. In other words, the decompressed frame is decompressed from compressed data including feature points of a specific object (e.g., a face) of the frame, and a background image can be synthesized based on the location information for a portion other than the specific object (e.g., a background).

[0055] As described above, the present disclosure can provide the video service even in unstable network environments or using communication techniques with low bandwidth. Furthermore, the present disclosure can increase the usage time of a terminal by reducing battery usage. Furthermore, the present disclosure can reduce the resource usage of a base station (or network) by reducing the load on the base station. Accordingly, the present disclosure can provide an improved user experience.

[0056] Specific details regarding a network for providing the video service according to the present disclosure are exemplified and described below with reference to FIG. 3a.

[0057] Figure 3a illustrates an example of a network for providing video services between a first terminal and a second terminal.

[0058] FIG. 3A illustrates a network (300) for providing a video service (305) between a first terminal (310) and a second terminal (320). For example, the network (300) may include a first terminal (310), a second terminal (320), a first network node (330), a second network node (340), and a core network (350). However, the present disclosure is not limited thereto. For example, the network (300) may further include other network nodes, or may not include some network nodes.

[0059] For example, the first terminal (310) may be an example of the terminal (120) of FIG. 1. For example, the second terminal (320) may be an example of the terminal (120) of FIG. 1. The first terminal (310) or the second terminal (320) may include an IoT (internet of things) device. For example, the first network node (330) may be an example of the base station (110) (or CU, DU) of FIG. 1. For example, the second network node (340) may be an example of the base station (110) (or CU, DU) of FIG. 1. For example, the core network (350) may perform routing and data forwarding between the first network node (330) and the second network node (340).

[0060] For example, a connection for a video service (305) may be established between a first terminal (310) and a second terminal (320). Establishing the connection for the video service (305) between the first terminal (310) and the second terminal (320) may include establishing a connection between the first terminal (310), the first network node (330), the second network node (340), and the second terminal (320).

[0061] For example, the first terminal (310) can generate a frame for a video service (305). For example, the first terminal (310) can perform compression on the generated frame. For example, the first terminal (310) can perform compression on the frame based on a neural network (315). Accordingly, the first terminal (310) can generate compressed data including feature points of the frame. For example, the neural network (315) included in the first terminal (310) can be referred to as a compression engine, a compression module, a compression AI model, a convolution neural network (CNN), a compression network, or a first neural network.

[0062] Alternatively, for example, the first terminal (310) may perform compression on the frame using a compression algorithm without using the neural network (315). For example, the first terminal (310) may generate compressed data for the frame using the compression algorithm. The compressed data generated using the compression algorithm may not include feature points and may have a compressed size from the frame. For example, the compressed data generated using the compression algorithm may have a compression format according to the compression algorithm. For example, the compression algorithm may include a discrete cosine transform, motion compensation, or entropy coding.

[0063] For example, the first terminal (310) may transmit the compressed data, which is generated based on the neural network (315) or using the compression algorithm, to the first network node (330). For example, the first network node (330) may transmit the received compressed data to the second network node (340) via the core network (350). In FIG. 3A, the first network node (330) is illustrated as transmitting the compressed data to the second network node (340) via the core network (350), but the present disclosure is not limited thereto. For example, the network (300) of FIG. 3A may include one network node (e.g., the first network node (330) or the second network node (340)), and both the first terminal (310) and the second terminal (320) may be connected to one network node. That is, when a first terminal (310) and a second terminal (320) are located within a cell provided by the one network node, the one network node may transmit the compressed data received from the first terminal (310) to the second terminal (320), or the one network node may transmit the compressed data received from the first terminal (310) to the core network (350) and then transmit the compressed data received again from the core network (350) to the second terminal (320).

[0064] For example, the second network node (340) may obtain (or receive) the compressed data of the frame for the video service (305) transmitted from the first terminal (310). For example, the second network node (340) may decompress the compressed data based on a neural network (345) for decompression of the compressed data. For example, the second network node (340) may generate a reconstructed frame from the compressed data by decompression of the compressed data based on the neural network (345). For example, the neural network (345) may be referred to as a decompression engine, a decompression module, a decompression AI model, a generative adversarial network (GAN), a decompression network, or a second neural network. The second network node (340) may transmit the reconstructed frame to the second terminal (320). For example, the second terminal (320) may provide a video service (305) using the received reconstructed frame. For example, providing the video service (305) may include displaying the reconstructed frame on the display of the second terminal (320).

[0065] In the above example, the second network node (340) is described as performing the decompression based on the neural network (345), but the present disclosure is not limited thereto. For example, the second network node (340) may transmit the received compressed data to the second terminal (320). For example, the second terminal (320) may perform decompression on the compressed data based on the neural network (325). For example, the second terminal (320) may generate a reconstructed frame from the compressed data by performing decompression on the compressed data based on the neural network (325). For example, the neural network (325) may be referred to as a decompression engine, a decompression module, a decompression AI model, a generative adversarial network (GAN), a decompression network, or a second neural network. The second terminal (320) may provide a video service (305) using the reconstructed frame. For example, providing a video service (305) may include displaying the reconstructed frame on a display of a second terminal (320).

[0066] In FIG. 3A, an example is illustrated in which compression is performed on the frame at the first terminal (310), and decompression is performed on the compressed data of the frame at the second network node (340) or the second terminal (320), but the present disclosure is not limited thereto. For example, compression may be performed on the frame at the second terminal (320), and decompression may be performed on the compressed data of the frame at the first network node (330) or the first terminal (310). Accordingly, the first terminal (310) may further include a neural network for decompression (e.g., a neural network (325)), and the first network node (330) may further include a neural network for decompression (e.g., a neural network (345)). In addition, the second terminal (320) may further include a neural network for compression (e.g., a neural network (315)). In the following, for convenience of explanation, it is assumed that the first terminal (310) performs compression on a frame, and the second network node (340) or the second terminal (320) decompresses the compressed data of the frame.

[0067] For example, a neural network for compression (e.g., neural network (315)) may include multiple components (or stages). For example, a neural network for compression may use a video sequence as input. For example, the video sequence may be data that requires compression. For example, the video sequence may include frames. The frames within the video sequence may be composed of images in chronological order. For example, the frames may be preprocessed, for example, before being input to the neural network for compression. For example, the preprocessing may include color space conversion, resizing, normalization, and other operations to prepare the frames for compression. For example, a neural network for compression may include an encoder, a latent space, and a decoder. For example, the encoder may be used to compress the frames. For example, the encoder may be composed of convolutional layers, recurrent layers, or a combination thereof. For example, the recurrent layers may include long short-term memory (LSTM) or gated recurrent units (GRU). The encoder may be trained to extract spatial and temporal features from the frame. For example, the latent space may represent a low-dimensional representation in which the most important information of the compressed frame is preserved. For example, the decoder may be used to reconstruct the frame from the compressed frame. For example, the decoder may be configured to mirror the architecture of the encoder or to recover compressed data from the frame.For example, a neural network for compression may include a bitrate control module that dynamically adjusts the compression level based on the content complexity of the frame and the available bandwidth. The bitrate control module may determine the amount of bits to be allocated to encode each frame or video sequence. For example, a neural network for compression may perform rate-distortion optimization. A module within the neural network for compression for rate-distortion optimization may determine how to allocate bits to different parts of the video to minimize quality loss within the bitrate. For example, a neural network for compression may perform further processing on the compressed data after performing neural network-based compression. For example, the further processing may include an entropy coding technique such as arithmetic coding or Huffman coding. For example, within the further processing, the neural network for compression may generate a compressed bitstream suitable for storage or transmission. For example, a neural network for compression may include a system for decompression. For example, the system may perform bitstream parsing. For example, the system may parse a compressed bitstream to extract compressed data. For example, the system may perform entropy decoding. For example, the system may reconstruct a frame by passing the compressed data to the decoder of the neural network. For example, the system may utilize location information (e.g., geolocation and GPS coordinates) associated with the frame to render the background of the frame. For example, the system may perform postprocessing to improve the visual quality of the reconstructed frame.For example, a video sequence output from the above system of a neural network for compression may be formed similarly to the original input video. For example, the neural network for compression may optionally include a quality assessment module for assessing the visual quality of the decompressed frames. For example, the neural network for compression may convert the compressed frames (or video sequences) into a commonly used video codec (e.g., H.264, H.265) for playback on various devices to ensure compatibility with existing video standards and devices.

[0068] Figure 3b shows an example of a neural network for decompression.

[0069] Referring to FIG. 3B, a neural network for decompression (e.g., neural network (325) or neural network (345)) may include a decompression module (360). For example, the decompression module may include a generator (G) (361) and a discriminator (D) (362). Training of the decompression module (360) may be referred to as example (365).

[0070] Referring to example (365), the generator (G) (361) of the decompression module (360) can use feature points (or vectors) as input. The generator (G) (361) can be trained to output decompressed frames using the feature points. For example, the discriminator (D) (362) of the decompression module (360) can be trained to evaluate the performance of the generator (G) (361). More specifically, the neural network for decompression (e.g., the neural network (325) or the neural network (345)) can be trained according to the following operations. For example, samples for training the neural network for decompression can be collected from a data set and used to train the discriminator (D) (362) of the neural network for decompression. In one example, a noise vector of a specific length (e.g., 100) can be generated using a Gaussian noise distribution. For example, a generator (G) (361) of a neural network for decompression can generate a fake image using the generated noise vector. The fake image is generated using a condition vector, and the condition vector can be formed of feature points (or pivot points). For example, the feature points generated (or acquired) according to the compression of a frame can be used by the generator (G) (361) to generate a fake image representing the frame. For example, a discriminator (D) (362) can use the feature points to discriminate an image generated by the generator (G) (361). For example, the discriminator (D) (362) can classify an image as a fake image generated by the generator or a real image acquired from a dataset. For example, for the above classification, a loss value (J) can be used. For example, the loss value (J) can be calculated by the least square error.For example, the loss value (J) may be back-propagated to update the weights of the discriminator (D) (362). For example, m noise vectors having different specific lengths (e.g., 10) may be generated using a Gaussian noise distribution. For example, the generator (G) (361) may generate images using the m noise vectors. For example, the accuracy of the generated images may be calculated using the least square error. For example, based on the accuracy, the weights (or parameter weights) of the generator (G) (361) may be updated. Specific details related to the learning of the decompression module (360) for decompression may be exemplified in the table below.

[0071]

[0072] The details of learning a neural network for decompression shown in the above table are merely exemplary, and the learning method of a neural network for decompression of the present disclosure is not limited to the above examples.

[0073] For example, the learned decompression module (360) can use compressed data including feature points as input (370). For example, the learned generator (G) (361) of the decompression module (360) can perform decompression on the input (370). For example, the generator (G) (361) can generate an output (380) which is a frame by performing decompression on the input (370). For example, the frame generated as the output (380) can include a decompressed frame for the video service.

[0074] In the network (300) of FIG. 3a, specific details on how the first terminal (310) performs compression on a frame for a video service, and how the second terminal (320) or the second network node (340) performs decompression on the compressed data may be referred to in FIGS. 4 to 6b below.

[0075] FIG. 4 illustrates an example of a signal flow for a method of compressing a frame for a video service between a first terminal and a second terminal and decompressing the compressed data.

[0076] FIG. 4 illustrates an example of a signal flow for a method in which a first terminal (310) determines a compression method for a frame for a video service and a second network node (340) determines a decompression method for compressed data. The first terminal (310) of FIG. 4 may be an example of the first terminal (310) of FIG. 3A . The second terminal (320) of FIG. 4 may be an example of the second terminal (320) of FIG. 3A . The first network node (330) of FIG. 4 may be an example of the first network node (330) of FIG. 3A . The second network node (340) of FIG. 4 may be an example of the second network node (340) of FIG. 3A .

[0077] Although FIG. 4 illustrates a first network node (330) transmitting the compressed data to a second network node (340) via a core network, the present disclosure is not limited thereto. For example, the example signal flow of FIG. 4 can be substantially equally applied to a case where a single network node (e.g., the first network node (330) or the second network node (340)) is included, and both the first terminal (310) and the second terminal (320) are connected to the single network node.

[0078] Referring to FIG. 4, in operation (400), a connection for a video service (e.g., video service (305) of FIG. 3A) may be established. For example, a connection for the video service may be established between a first terminal (310) and a second terminal (320). For example, the connection for the video service between the first terminal (310) and the second terminal (320) may include establishing a connection between the first terminal (310), a first network node (330), a second network node (340), and the second terminal (320).

[0079] In operation (405), the first network node (330) may transmit first information related to compression to the first terminal (310). For example, the first information related to compression may include information related to compression of frames for the video service.

[0080] For example, the first information may include information about the load of the first network node (330) (or the second network node (340)). For example, the load may include at least one of resources of at least one processor used to perform decompression based on a neural network (e.g., the second network node (345) of FIG. 3A) of the first network node (330) (or the second network node (340)), resources of a memory used to perform decompression based on a neural network (e.g., the second network node (345) of FIG. 3A) of the first network node (330) (or the second network node (340)), or resources of a physical channel allocated for the video service within the neural network (e.g., the second network node (345) of FIG. 3A) of the first network node (330) (or the second network node (340)).

[0081] For example, the first information may include a value indicating the quality of a channel between the first network node (330) and the first terminal (310). For example, the first information may include a value indicating the quality of the channel, which may include channel state information. For example, the channel state information may include a channel quality indicator (CQI), a precoding matrix indicator (PMI), a rank indicator (RI), a signal to interference plus noise ratio (SINR), a reference signal received power (RSRP), or a received signal strength indicator (RSSI). However, the present disclosure is not limited thereto. For example, the value indicating the quality of the channel may include a result of channel estimation estimated based on a reference signal (e.g., a sounding reference signal (SRS), a demodulation reference signal (DMRS)).

[0082] For example, the first information may indicate the number of feature points included in the compressed data to be generated as a result of compression of the frame when compression is performed on the frame based on the neural network (315) of the first terminal (310). For example, the number of feature points may be referenced as a compression ratio or a compression ratio value. For example, the number of feature points may be determined (or adjusted) based on at least one of information about the load of the first information, the value indicating the quality of the channel of the first information, or the remaining battery amount of the first terminal (310). For example, as the number of feature points increases, the size of the compressed data may increase, and the quality of the video service may increase.

[0083] In the example of FIG. 4, the first information related to compression is illustrated as being provided from the first network node (330) to the first terminal (310), but the present disclosure is not limited thereto. For example, the first terminal (310) may perform operation (410) without receiving the first information from the first network node (330). In other words, operation (405) may be omitted.

[0084] In operation (410), the first terminal (310) may determine a compression method for the frame. For example, the first terminal (310) may determine a compression method for the frame for the video service. For example, the compression method may include compression based on the neural network (315) of the first terminal (310) or compression using a compression algorithm.

[0085] For example, the first terminal (310) can determine whether a criterion for compression on a frame is satisfied. For example, if the criterion is satisfied, the first terminal (310) can perform compression on the frame for the video service based on the neural network (315). Alternatively, if the criterion is not satisfied, the first terminal (310) can perform compression on the frame for the video service using the compression algorithm. For example, performing compression using the compression algorithm may indicate not performing compression based on the neural network (315). Specific details related to the criterion are described below in FIG. 6A.

[0086] In operation (415), the first terminal (310) may perform compression on the frame. For example, the first terminal (310) may perform compression on the frame for the video service using the compression method determined in operation (410). For example, the first terminal (310) may generate compressed data by performing compression on the frame based on the neural network (315). Alternatively, for example, the first terminal (310) may generate compressed data by performing compression on the frame using the compression algorithm.

[0087] For example, the compressed data according to the compression performed based on the neural network (315) may include feature points of the frame. For example, the feature points may include feature points for representing one or more objects of the frame. In one example, the one or more objects may include a designated object. For example, the designated object may include an object corresponding to a region of interest (ROI) related to the video service. As a non-limiting example, in a video call service, the designated object may include a user's face. For example, the one or more objects may include a background portion (or a background image) other than the designated object. In one example, the first terminal (310) may generate the compressed data including feature points of the designated object or the compressed data including feature points of the frame (or the one or more objects of the frame) based on the neural network (315) depending on whether the frame includes the designated object. Specific details related to this are described in Fig. 6a below.

[0088] Alternatively, for example, the first terminal (310) may generate compressed data including feature points of the background image of the frame when the frame is a reference frame, or may generate compressed data including feature points of the specified object of the frame when the frame is not a reference frame. For example, the reference frame may be referred to as an I frame. For example, a frame that is not a reference frame may be referred to as a B frame or a P frame. For example, a frame that is not a reference frame may be dependent on the reference frame. Specific details related thereto are described below in FIG. 6B.

[0089] For example, the compressed data resulting from compression performed based on the compression algorithm may include data having a compressed size from the frame. For example, the compressed data resulting from compression performed based on the compression algorithm may include feature points.

[0090] In operation (420), the first terminal (310) may transmit compressed data to the first network node (330). For example, the first network node (330) may receive the compressed data from the first terminal (310) and transmit it to the second network node (340) via the core network.

[0091] In operation (425), the second network node (340) may receive second information related to decompression from the second terminal (320). For example, the second information related to decompression may include information related to decompression of the compressed data of the frame for the video service.

[0092] For example, the second information may include battery information of the second terminal (320). For example, the battery information may include the remaining battery capacity of the second terminal (320). For example, the remaining battery capacity may be used to determine whether to perform the decompression based on the neural network (325) of the second terminal (320).

[0093] For example, the second information may include a value indicating the quality of a channel between the second network node (340) and the second terminal (320). For example, the second information may include channel state information, which may include, for example, CQI, PMI, RI, SINR, RSRP, or RSSI. However, the present disclosure is not limited thereto. For example, the value indicating the quality of the channel may include a result of channel estimation estimated based on a reference signal (e.g., SRS, DMRS).

[0094] In the example of FIG. 4, the second information related to decompression is illustrated as being provided from the second terminal (320) to the second network node (340), but the present disclosure is not limited thereto. For example, the second network node (340) may perform operation (430) without receiving the second information from the second terminal (320). In other words, operation (425) may be omitted. Even if the second information is not received, the second network node (340) may be aware that the second terminal (320) is capable of performing decompression based on the neural network (325) of the second terminal (320).

[0095] In operation (430), the second network node (340) may determine a decompression method of compressed data. For example, the second network node (340) may determine a decompression method of the compressed data of the frame transmitted from the first terminal (310). For example, the decompression method may include decompression based on a neural network (345) of the second network node (340) or decompression based on a neural network (325) of the second terminal (320).

[0096] For example, the second network node (340) can determine whether the criteria for performing decompression on compressed data are satisfied. For example, if the criteria are satisfied, the second network node (340) can perform decompression on the compressed data based on the neural network (345). Specific details related to this are exemplified in operations (460) of FIG. 4. Alternatively, if the criteria are not satisfied, the second network node (340) can authorize the second terminal (320) to perform decompression on the compressed data. For example, the second network node (340) can transmit the compressed data to the second terminal (320) without performing decompression on the compressed data. Specific details related to this are exemplified in operations (440) of FIG. 4. Specific details related to the criteria for performing decompression are described below in FIG. 5A.

[0097] Operations (440) may be performed when the criteria for performing decompression are not satisfied. In operation (445), the second network node (340) may transmit the compressed data of the frame to the second terminal (320) upon determining that the criteria for performing the decompression are not satisfied. For example, the second network node (340) may enable the second terminal (320) to perform the decompression on the compressed data. In operation (450), the second terminal (320) may perform the decompression on the compressed data based on the neural network (325). By performing the decompression on the compressed data based on the neural network (325), the second terminal (320) may reconstruct the frame for the video service. For specific details related thereto, reference may be made to FIG. 5B below.

[0098] Operations (460) may be performed when a criterion for performing decompression is satisfied. In operation (465), the second network node (340) may perform decompression on the compressed data based on the neural network (345) upon determining that the criterion for performing decompression is satisfied. For example, the second network node (340) may reconstruct a frame for the video service by performing decompression on the compressed data based on the neural network (345). In operation (470), the second network node (340) may transmit the frame to the second terminal (320). For example, the second network node (340) may transmit the reconstructed frame to the second terminal (320). For specific details related thereto, reference may be made to FIG. 5C below.

[0099] In operation (475), the second terminal (320) may provide a video service. For example, the second terminal (320) may provide the video service using the reconstructed frame generated according to operation (450) or the reconstructed frame received according to operation (470). For example, the second terminal (320) may display a screen (or image) according to the reconstructed frame on the display of the second terminal (320). For example, the screen according to the reconstructed frame may be included in the UI (user interface) of a specific software application. For example, the specific software application may include an application for the video service. For example, the specific software application may include a video call application.

[0100] In the above example, compressed data for a frame for a video service is transmitted from a first terminal (310) to a second network node (340), and an example is illustrated in which the second network node (340) or the second terminal (320) decompresses the compressed data to generate a reconstructed frame, but the present disclosure is not limited thereto. For example, the first terminal (310) may transmit location information of the first terminal (310) together with the compressed data. For example, the second network node (340) or the second terminal (320) may identify a background image using the received location information of the first terminal (310). For example, the background image may be mapped or stored with respect to the location information of the first terminal (310) for the video service. For example, the second network node (340) or the second terminal (320) may synthesize the background image into the reconstructed frame. For example, a frame in which the background image and the reconstructed frame are synthesized may include an image in which the specified object and the background image are synthesized from feature points of the specified object.

[0101] Figure 5a illustrates an example of an operational flow for a method of performing decompression on compressed data.

[0102] At least a portion of the method of FIG. 5A may be performed by a network node of FIG. 3A (e.g., the second network node (340) of FIG. 3A). For example, at least a portion of the method may be controlled by a processor of the network node (e.g., the processor (230) of FIG. 2). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0103] Referring to operation (500), the network node may obtain compressed data including feature points of the first frame. For example, the network node may receive the compressed data including feature points of the first frame for a video service from another network node (e.g., the first network node (330)) through a core network (e.g., the core network (350) of FIG. 3A), or from the first terminal (310) that transmitted the first frame. For example, the compressed data may include data compressed from the first frame based on a compression algorithm or a neural network (315) of the first terminal (310).

[0104] In operation (505), the network node may determine whether a criterion for performing decompression is satisfied. For example, the network node may determine whether the criterion for performing decompression on the compressed data of the first frame is satisfied. For example, the criterion for performing decompression may include that a load associated with a neural network of the network node (e.g., the neural network (345) of FIG. 3A) is less than (or less than) a reference load. For example, the criterion for performing decompression may include that a remaining battery amount of a second terminal (320) connected to the network node is less than (or less than) a reference battery amount. For example, the criterion for performing decompression may include that a value representing the quality of a channel between the network node and the second terminal (320) is less than (or less than) a reference value.

[0105] In the above example, the load, the value, and the remaining battery amount related to the criterion are exemplified, but the present disclosure is not limited thereto. For example, terminal parameters and network parameters related to the criterion may be considered. For example, the terminal parameters may include battery information of the terminal. For example, the network parameters may include the load of the network node, the resource availability of the network node, or the connectivity (or channel status) between the network node and the terminal.

[0106] For example, the network node may determine whether the load associated with the neural network (e.g., the neural network (345) of FIG. 3A) is less than (or below) the reference load. For example, the load may include at least one of resources of at least one processor of the network node used to perform the decompression based on the neural network, resources of a memory used to perform the decompression based on the neural network, or resources of a physical channel allocated for the video service. For example, the network node may determine that the reference is satisfied based on the load being less than (or below) the reference load. Alternatively, the network node may determine that the reference is not satisfied based on the load exceeding the reference load.

[0107] For example, the network node may receive (or acquire) battery information from a second terminal (320) connected to the network node. For example, the battery information may include the remaining battery amount of the second terminal (320). For example, the network node may determine whether the remaining battery amount is less than (or less than) the reference battery amount. For example, the network node may determine that the criterion is satisfied based on the remaining battery amount being less than (or less than) the reference battery amount. For example, the network node may determine that the criterion is not satisfied based on the remaining battery amount exceeding the reference battery amount.

[0108] For example, the network node may obtain channel information of a channel between a second terminal (320) and the network node. For example, the network node may identify the channel information or receive a report including the channel information from the second terminal (320). For example, the network node may identify the value indicating the quality of the channel based on the channel information. For example, the network node may determine that the criterion is satisfied based on the value being less than (or less than) the criterion value. For example, the network node may determine that the criterion is not satisfied based on the value exceeding the criterion value.

[0109] In the above examples, the network node is described as determining whether the criterion is satisfied for each of the load, the remaining battery amount, and the value, but the present disclosure is not limited thereto. For example, the network node may also determine whether the criterion is satisfied for a combination of at least one of the load, the remaining battery amount, or the value.

[0110] As a non-limiting example, the network node may determine that the criterion is satisfied based on the load being less than (or below) the reference load or the remaining battery amount being less than (or below) the reference battery amount. For example, the network node may determine that the criterion is not satisfied based on the load exceeding the reference load and the remaining battery amount exceeding the reference battery amount.

[0111] Additionally, as a non-limiting example, the network node may determine that the criterion is satisfied based on the value being less than (or equal to) the reference value, the load being less than (or equal to) the reference load, or the remaining battery amount being less than (or equal to) the reference battery amount. For example, the network node may determine that the criterion is not satisfied based on the value exceeding the reference value, the load exceeding the reference load, and the remaining battery amount exceeding the reference battery amount.

[0112] Additionally, as a non-limiting example, the network node may determine that the criterion is satisfied based on the value being less than (or equal to) the reference value, or the remaining battery amount being less than (or equal to) the reference battery amount. For example, the network node may determine that the criterion is not satisfied based on the value exceeding the reference value and the remaining battery amount exceeding the reference battery amount.

[0113] In operation (505), if the network node determines that the criterion is satisfied, the network node may perform operation (515). For example, satisfying the criterion may indicate that the network node can decompress the compressed data. Conversely, in operation (505), if the network node determines that the criterion is not satisfied, the network node may perform operation (510). For example, not satisfying the criterion may indicate that the network node cannot decompress the compressed data.

[0114] In operation (510), the network node may transmit compressed data. For example, if the criterion is not satisfied, the network node may transmit the compressed data to the second terminal (320). For example, if the criterion is not satisfied, the network node may transmit the compressed data to the second terminal (320) to authorize the second terminal (320) to decompress the compressed data.

[0115] In operation (515), the network node may perform decompression based on the neural network. For example, if the criterion is satisfied, the network node may perform decompression on the compressed data based on the neural network. For example, the network node may generate a second frame by performing decompression on the compressed data. For example, the second frame may represent a frame reconstructed based on the neural network from the compressed data of the first frame.

[0116] In operation (520), the network node may transmit the second frame. For example, the network node may transmit the second frame, which is a reconstructed frame, to the second terminal (320).

[0117] Although not illustrated in FIG. 5A, the network node may obtain location information of the first terminal (310) together with the compressed data of the first frame. For example, the location information of the first terminal (310) may be used to generate a background image of the first frame. For example, the network node may identify a mapped background image (or a stored background image) with respect to the location information of the first terminal (310). For example, the network node may synthesize the identified background image with respect to the second frame. For example, the network node may transmit the synthesized second frame to the second terminal (320). In the example, the first terminal (310) may transmit the first frame and the location information, rather than the reference frame, to the network node. For example, upon receiving a command from the network node to transmit location information of the first terminal (310), the first terminal (310) may transmit the location information of the first terminal (310) together with the compressed data for the first frame. For example, the network node may transmit the command to the first terminal (310) when the available resources of the network node are insufficient. For example, the compressed data of the first frame, which is not the reference frame, may be compressed data including feature points of designated objects excluding a background image. In this case, the second frame may represent the designated objects. For example, the network node may generate the synthesized second frame including the background image and the designated object. Thereafter, the network node may transmit the synthesized second frame to the second terminal (320).

[0118] In one example, when the network node determines that the remaining battery amount of the second terminal (320) is less than or equal to a reference battery amount, the network node may generate the second frame based on a neural network and transmit the second frame to the second terminal (320). Thereafter, when the remaining battery amount of the second terminal (320) exceeds the reference battery amount, the second terminal (320) may request that the decompression performed by the network node be stopped. Thereafter, the second terminal (320) may receive the compressed data from the network node according to operation (510) and perform the decompression based on the neural network (325).

[0119] Figure 5b illustrates an example of how a second terminal performs decompression on compressed data.

[0120] FIG. 5b illustrates an example (530) of a method for decompressing compressed data of a first frame (531) for a video service between a first terminal (310) and a second terminal (320) at a second terminal (320). In the example (530) of FIG. 5b, for convenience of explanation, a case is illustrated where both the first terminal (310) and the second terminal (320) are connected to a second network node (340), but the present disclosure is not limited thereto.

[0121] Referring to example (530), the first terminal (310) can generate a first frame (531) and use the generated first frame (531) as an input to the neural network (315). For example, the first terminal (310) can generate compressed data (533) by performing compression on the first frame (531) based on the neural network (315). For example, the compressed data (533) can be transmitted to the second network node (340). Although not shown in example (530) of FIG. 5b, location information of the first terminal (310) can also be transmitted to the second network node (340) together with the compressed data (533).

[0122] For example, the second network node (340) may transmit (or forward) compressed data (533) to the second terminal (320). For example, the second network node (340) may transmit the compressed data (533) to the second terminal (320) upon determining that the criteria for decompression are not satisfied.

[0123] For example, the second terminal (320) can receive compressed data (541). For example, the compressed data (541) can be substantially the same as (or correspond to) the compressed data (533). In other words, although the example (530) of FIG. 5B illustrates the compressed data (541) and the compressed data (533) as different, the compressed data (541) and the compressed data (533) can be substantially the same. However, the present disclosure is not limited thereto. For example, the compressed data (541) can also include the compressed data (533). For example, the second terminal (320) can use the received compressed data (541) as an input to the neural network (325). For example, the second terminal (320) can generate the second frame (543) by decompressing the compressed data (541) based on the neural network (325). Although not shown in FIG. 5b, the second terminal (320) can provide the video service based on the second frame (543).

[0124] For example, the second terminal (320) can identify a background image to be synthesized for the second frame (543) using the location information of the first terminal (310). For example, the second terminal (320) can identify the background image mapped (or stored) for the location information of the first terminal (310). For example, the second terminal (320) can synthesize the background image for the second frame (543). At this time, the first frame (531) is a frame other than the reference frame, and the compressed data (533) of the first frame (531) can include feature points of a designated object. Accordingly, the second frame (543) generated by the decompression of the compressed data (541) (or the compressed data (533)) can represent the designated object. The second terminal (320) can provide the video service by using the second frame (543) representing the specified object and the frame in which the background image is synthesized (or combined).

[0125] Although not illustrated in FIG. 5b, the second network node (340) (and / or the first network node (330)) may change the compression and decompression algorithm based on the parameters of the network. For example, the second network node (340) (and / or the first network node (330)) may change the number of feature points included in the compressed data to be generated when performing the compression based on the neural network (315) of the first terminal (310) based on the load. For example, the second network node (340) (and / or the first network node (330)) may perform tuning of the neural network (345) of the second network node (340) or provide information for tuning the neural network (345) of the second terminal (320) based on the load.

[0126] Figure 5c illustrates an example of how a network node performs decompression on compressed data.

[0127] FIG. 5c illustrates an example (550) of a method for decompressing compressed data of a first frame (551) for a video service between a first terminal (310) and a second terminal (320) at a second network node (340). In the example (550) of FIG. 5c, for convenience of explanation, a case is illustrated where both the first terminal (310) and the second terminal (320) are connected to the second network node (340), but the present disclosure is not limited thereto.

[0128] Referring to example (550), the first terminal (310) can generate a first frame (551) and use the generated first frame (551) as an input to the neural network (315). For example, the first terminal (310) can generate compressed data (553) by performing compression on the first frame (551) based on the neural network (315). For example, the compressed data (553) can be transmitted to the second network node (340). Although not shown in example (550) of FIG. 5c, location information of the first terminal (310) can also be transmitted to the second network node (340) together with the compressed data (553).

[0129] For example, the second network node (340) may receive compressed data (561). For example, the compressed data (561) may be substantially the same as (or correspond to) the compressed data (553). In other words, although the example (550) of FIG. 5c illustrates the compressed data (561) and the compressed data (553) as being different, the compressed data (561) and the compressed data (553) may be substantially the same. However, the present disclosure is not limited thereto. For example, the compressed data (561) may also include the compressed data (553).

[0130] For example, the second network node (340) may perform decompression on the compressed data (553) based on the neural network (345) upon determining that the above criteria for decompression are satisfied. For example, the second network node (340) may use the received compressed data (561) as input to the neural network (345). For example, the second network node (340) may generate the second frame (563) by performing decompression on the compressed data (561) based on the neural network (345). For example, the second network node (340) may transmit the second frame (563) to the second terminal (320). Although not illustrated in FIG. 5C, the second terminal (320) may provide the video service based on the second frame (563).

[0131] For example, the second network node (340) can identify a background image to be synthesized for the second frame (563) using the location information of the first terminal (310). For example, the second network node (340) can identify the background image mapped (or stored) for the location information of the first terminal (310). For example, the second network node (340) can synthesize the background image for the second frame (563). At this time, the first frame (551) is a frame other than the reference frame, and the compressed data (553) of the first frame (551) can include feature points of a specified object. Accordingly, the second frame (563) generated by the decompression of the compressed data (561) (or the compressed data (553)) can represent the specified object. The second network node (340) can transmit a second frame (563) representing the specified object and a frame in which the background image is synthesized (or combined) to the second terminal (320).

[0132] Figure 6a illustrates an example of an operational flow for how a first terminal performs compression on a frame.

[0133] At least some of the above methods of FIG. 6A may be performed by the first terminal (310) of FIG. 3A. For example, at least some of the above methods may be controlled by a processor of the first terminal (310) (e.g., processor (230) of FIG. 2). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0134] In operation (600), the first terminal (310) may generate a frame for a video service. For example, the first terminal (310) may generate the frame for the video service between the first terminal (310) and the second terminal (320).

[0135] In operation (605), the first terminal (310) may determine whether a criterion for compression for the frame is satisfied. For example, the criterion for compression for the frame may be used to determine whether to perform compression for the frame based on a neural network (315). For example, the criterion for compression for the frame may include that a load related to the video service of a network node (e.g., the first network node (310) and / or the second network node (340) of FIG. 3A) is less than (or below) a reference load. For example, the criterion for compression for the frame may include that the remaining battery amount of the first terminal (310) is less than (or below) a reference battery amount. For example, the criteria for compression for the frame may include that a value representing the quality of a channel between a network node (e.g., the first network node (310) and / or the second network node (340) of FIG. 3A) and the first terminal (310) is equal to or less than a criteria value.

[0136] In the above example, the load, the value, and the remaining battery amount related to the criterion are exemplified, but the present disclosure is not limited thereto. For example, terminal parameters and network parameters related to the criterion may be considered. For example, the terminal parameters may include battery information of the terminal. For example, the network parameters may include the load of the network node, the resource availability of the network node, or the connectivity (or channel status) between the network node and the terminal.

[0137] For example, the first terminal (310) can obtain channel information of a channel between the first terminal (310) and the network node. For example, the first terminal (310) can identify the channel information or receive information including the channel information from the network node. For example, the first terminal (310) can identify the value indicating the quality of the channel based on the channel information. For example, the first terminal (310) can determine that the criterion is satisfied based on the value being less than (or less than) the criterion value. For example, the first terminal (310) can determine that the criterion is not satisfied based on the value exceeding the criterion value.

[0138] For example, the first terminal (310) can identify battery information of the first terminal (310). For example, the battery information can include the remaining battery amount of the first terminal (310). For example, the first terminal (310) can determine whether the remaining battery amount is less than (or less than) the reference battery amount. For example, the first terminal (310) can determine that the criterion is satisfied based on the remaining battery amount being less than (or less than) the reference battery amount. For example, the first terminal (310) can determine that the criterion is not satisfied based on the remaining battery amount exceeding the reference battery amount.

[0139] For example, the first terminal (310) may determine whether the load related to the video service of the network node is less than (or below) the reference load. For example, the load may include at least one of the resources of at least one processor of the network node used to perform neural network-based decompression of the network node, the resources of a memory used to perform neural network-based decompression of the network node, or the resources of a physical channel allocated for the video service. For example, the first terminal (310) may determine that the reference is satisfied based on the load being less than (or below) the reference load. Alternatively, the first terminal (310) may determine that the reference is not satisfied based on the load exceeding the reference load.

[0140] In the above examples, the first terminal (310) is described as determining whether the criterion is satisfied for each of the load, the remaining battery amount, and the value, but the present disclosure is not limited thereto. For example, the network node may also determine whether the criterion is satisfied for a combination of at least one of the load, the remaining battery amount, or the value.

[0141] As a non-limiting example, the first terminal (310) may determine that the criterion is satisfied based on the value being less than (or equal to) the reference value or the remaining battery amount being less than (or equal to) the reference battery amount. For example, the first terminal (310) may determine that the criterion is not satisfied based on the value exceeding the reference value and the remaining battery amount exceeding the reference battery amount.

[0142] Additionally, as a non-limiting example, the first terminal (310) may determine that the criterion is satisfied based on the load being less than (or less than) the reference load, the value being less than (or less than) the reference value, or the remaining battery amount being less than (or less than) the reference battery amount. For example, the first terminal (310) may determine that the criterion is not satisfied based on the load exceeding the reference load, the value exceeding the reference value, and the remaining battery amount exceeding the reference battery amount.

[0143] Additionally, as a non-limiting example, the first terminal (310) may determine that the criterion is satisfied based on the load being less than (or below) the reference load, or the remaining battery amount being less than (or below) the reference battery amount. For example, the network node may determine that the criterion is not satisfied based on the load exceeding the reference load and the remaining battery amount exceeding the reference battery amount.

[0144] In operation (605), the first terminal (310) may perform operation (620) if it determines that the above criterion is satisfied. For example, satisfying the criterion may include performing compression on the frame based on the neural network (315) of the first terminal (310). Alternatively, in operation (605), the first terminal (310) may perform operation (610) if it determines that the above criterion is not satisfied. For example, not performing compression on the frame based on the neural network (315) of the first terminal (310) may include performing compression on the frame using a compression algorithm.

[0145] In operation (610), the first terminal (310) may perform compression on the frame using the compression algorithm. For example, the compression algorithm may include discrete cosine transform, motion compensation, or entropy coding. Or, for example, the compression algorithm may include H.264 (or AVC (advanced video coding)), H.265 (or HEVC (high efficiency video coding)), VP9, ​​etc. For example, the first terminal (310) may generate first compressed data by performing compression on the frame using the compression algorithm.

[0146] In operation (615), the first terminal (310) may transmit the first compressed data. For example, the first terminal (310) may transmit the first compressed data to the network node. For example, the first compressed data may be transmitted to the second terminal (320) via the network node, or a frame decompressed from the first compressed data may be provided to the second terminal (320).

[0147] In operation (620), the first terminal (310) may determine whether the frame includes a designated object. For example, if the compression criteria for the frame are satisfied, the first terminal (310) may determine whether the frame includes the designated object. For example, the designated object may include a user's face. However, the present disclosure is not limited thereto. For example, the designated object may include an object determined (or selected) in relation to the video service.

[0148] In operation (620), the first terminal (310) may perform operation (625) if the frame includes the specified object. Alternatively, in operation (620), the first terminal (310) may perform operation (630) if the frame does not include the specified object.

[0149] In the above example, the operation of the first terminal (310) is described depending on whether the frame includes the designated object, but the present disclosure is not limited thereto. For example, the first terminal (310) can determine whether the frame is a reference frame. For example, if the frame is the reference frame, the first terminal (310) can perform operation (630). Alternatively, if the frame is not the reference frame, the first terminal (310) can perform operation (625). For specific details related thereto, reference may be made to FIG. 6B below.

[0150] In operation (625), the first terminal (310) may perform compression on the specified object based on the neural network (315). For example, the first terminal (310) may generate second compressed data including feature points of the specified object by performing compression on the specified object based on the neural network (315).

[0151] In operation (630), the first terminal (310) may perform compression on the frame based on the neural network (315). For example, the first terminal (310) may perform compression on the frame based on the neural network (315), thereby generating second compressed data including feature points of the frame. For example, the feature points of the frame may include feature points of objects (e.g., background) of the frame.

[0152] The second compressed data generated in operation (625) and the second compressed data generated in operation (630) may have a smaller size than the first compressed data generated in operation (615). The quality of a frame decompressed from the second compressed data generated in operation (625) and the second compressed data generated in operation (630) may be lower than the quality of a frame decompressed from the first compressed data generated in operation (615).

[0153] In operation (635), the first terminal (310) may transmit the second compressed data. For example, the second compressed data generated in operation (625) or the second compressed data generated in operation (630) may be transmitted to the second terminal (320) via the network node, or a frame decompressed from the second compressed data may be provided to the second terminal (320).

[0154] Although not illustrated in FIG. 6A, the first terminal (310) can adjust the number of feature points of the second compressed data. For example, the first terminal (310) can determine the number of feature points of the compressed data generated by performing compression based on the neural network (315) using at least one of the value that is less than or equal to the reference value, the remaining battery amount that is less than or equal to the reference battery amount, or the load that is less than or equal to the reference load. In one example, when the load is a first load that is less than the reference load, the first terminal (310) can generate compressed data including a first number (e.g., 24) of feature points based on the neural network (315). In contrast, the first terminal (310) may generate compressed data including a second number (e.g., 48) of feature points exceeding the first number based on the neural network (315) when the load is a second load that is less than the first load. Alternatively, in one example, the first terminal (310) may generate compressed data including the second number (e.g., 48) of feature points based on the neural network (315) when the remaining battery amount is a first battery amount that is less than the reference battery amount. In contrast, the first terminal (310) may generate compressed data including the first number (e.g., 24) of feature points that are less than the second number based on the neural network (315) when the remaining battery amount is a second battery amount that is less than the first battery amount. Alternatively, in one example, the first terminal (310) may generate compressed data including the second number (e.g., 48) of feature points based on the neural network (315) when the value is a first value that is less than the reference value.Alternatively, the first terminal (310) may generate compressed data including the first number (e.g., 24) of feature points that are less than the second number based on the neural network (315) when the value is a second value that is less than the first value. In the above example, an example is described in which the first terminal (310) adjusts the number of feature points based on each of the value, the load, and the remaining battery amount, but the present disclosure is not limited thereto. For example, the first terminal (310) may also adjust the number of feature points based on at least one of the value, the load, or the remaining battery amount.

[0155] Alternatively, in one example, upon receiving a signal for adjusting the number of feature points from a network node (e.g., the first network node (310) or the second network node (340)) connected to the first terminal (310), the first terminal (310) may adjust the number of feature points. For example, if the network node (e.g., the first network node (310) or the second network node (340)) connected to the first terminal (310) is servicing a large number of terminals and thus cannot provide a high-quality video service, the signal instructing the first terminal (310) to reduce the number of feature points may be transmitted.

[0156] In addition, although FIG. 6A illustrates that the first terminal (310) determines whether the frame satisfies the criteria for compression, the present disclosure is not limited thereto. For example, the first terminal (310) may receive information about the criteria determined by the network node and perform an operation based on whether the criteria indicated by the information are satisfied. For example, the network node may determine whether the criteria are satisfied and transmit the information indicating that the criteria are satisfied or the information indicating that the criteria are not satisfied to the first terminal (310). Accordingly, the first terminal (310) may perform operation (610) or operation (620).

[0157] Figure 6b shows examples of frames for a video service.

[0158] FIG. 6B illustrates an example (650) of frames (660, 670, 680) for the video service between a first terminal (310) and a second terminal (320). The example (650) of FIG. 6B is merely an example for convenience of explanation, and the present disclosure is not limited thereto. For example, the number or period (or interval) of the reference frames (660, 680) may be changed. For example, the number of one or more objects included in each of the frames (660, 670) may be changed. In addition, in FIG. 6B, for convenience of explanation, the first terminal (310) is illustrated as being directly connected to the second network node (340), but the present disclosure is not limited thereto. For example, the first terminal (310) may also be connected to the second network node (340) via the first network node (340).

[0159] In one example, the number of reference frames (660, 680) may be changed based on a value indicating the remaining battery amount of the first terminal (310), the load of the network node, or the quality of the channel between the first terminal (310) and the network node. For example, if the remaining battery amount is high, the load is low, or the quality is good, the number of reference frames (660, 680) may increase.

[0160] Referring to example (650), the first terminal (310) can generate frames (660, 670, 680) for the video service. If the video service is a video call service, each of the frames (660, 670, 680) can include an image representing a screen for the video call. The frames (660, 670, 680) can include reference frames (660, 680). For example, compressed data for the reference frame (660) can include feature points of a specified object (661) and feature points for a background image (663).

[0161] For example, among the frames (660, 670, 680), a frame (670) (or a general frame) other than the reference frames (660, 680) may include a designated object (671). For example, the compressed data for the frame (670) may include the feature points of the designated object (671) among the feature points of the designated object (671) and the feature points of the background image (663). In other words, the compressed data for the frame (670) may not include the background image (663).

[0162] For example, the first terminal (310) may generate second compressed data from the reference frame (660) based on the neural network (315). For example, the first terminal (310) may generate second compressed data from the frame (670) following the reference frame (660) based on the neural network (315). For example, the second compressed data generated from the frame (670) may include feature points for a designated object (671). For example, each of the feature points for the designated object (671) of the second compressed data generated from the frame (670) may include a vector value indicating a change from each of the other feature points for the designated object (661) of the reference frame (660). For example, the vector value indicating the change may indicate a direction indicating a changed position of the other feature points corresponding to each of the feature points, and a changed degree.

[0163] In one example, the first terminal (310) may transmit the location information of the first terminal (310) together with the second compressed data for the frame (670) to the second network node (340). For example, the location information of the first terminal (310) may be used to identify the background image (663). For example, in relation to the video service, the location information of the first terminal (310) may be mapped to the background image (663). Accordingly, when the second network node (340) (or the second terminal (320)) decompresses the second compressed data generated from the frame (670), the second network node (340) may use the location information of the first terminal (310) to identify the background image (663) and perform synthesis on the decompressed frame. For example, upon receiving a command from a second network node (340) to transmit location information of the first terminal (310), the first terminal (310) may transmit the location information of the first terminal (310) together with the second compressed data for the frame (670). For example, the second network node (340) may transmit the command to the first terminal (310) when the available resources of the second network node (340) are insufficient.

[0164] The electronic device, method, and storage medium according to the present disclosure can reduce battery consumption of a terminal (e.g., a first terminal (310) or a second terminal (320)) while providing a video service having a quality level or higher. For example, the electronic device, method, and storage medium according to the present disclosure can provide a video service having a quality level or higher even in a situation where network coverage related to the terminal is poor (or a low-speed network environment) or a situation where network resources are limited. Accordingly, the present disclosure can provide an improved user experience. The electronic device, method, and storage medium according to the present disclosure can increase the resource utilization efficiency of a wireless network and lower the operating costs of operators operating the network by reducing the load on the network node. The electronic device, method, and storage medium according to the present disclosure can perform more efficient control by having the network node actively participate in a framework for video services between terminals.

[0165] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0166] As described above, a device of a network node may include a memory that stores instructions. The device may include at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the device to obtain compressed data including feature points of a first frame for a video service between a first terminal and a second terminal. The instructions, when individually or collectively executed by the at least one processor, may cause the device to identify a load associated with a neural network for performing decompression of the compressed data within the network node. The instructions, when individually or collectively executed by the at least one processor, may cause the device to transmit the compressed data to the second terminal to enable the second terminal to perform decompression of the compressed data in response to the load exceeding a reference load. The above instructions, when individually or collectively executed by the at least one processor, may cause the device to generate a second frame for the video service by decompressing the compressed data based on the neural network according to the load being less than the reference load, and to transmit the second frame to the second terminal.

[0167] According to one embodiment, the feature points of the compressed data of the first frame may include feature points for a specified object of the first frame.

[0168] In one embodiment, the designated object may include a user's face. Each of the feature points of the compressed data may include a vector value indicating a change from each of the other feature points of the designated object in a third frame prior to the first frame. The number of feature points of the compressed data may be determined based on the load of the network node.

[0169] According to one embodiment, the load may include at least one of resources of the at least one processor used to perform the decompression based on the neural network, resources of the memory used to perform the decompression based on the neural network, or resources of a physical channel allocated for the video service.

[0170] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the device to obtain battery information of the second terminal connected to the network node. The instructions, when individually or collectively executed by the at least one processor, may cause the device to identify a remaining battery capacity of the second terminal based on the battery information of the second terminal. The instructions, when individually or collectively executed by the at least one processor, may cause the device to transmit the compressed data to the second terminal to authorize the second terminal to decompress the compressed data, based on the load exceeding the reference load and the remaining battery capacity exceeding the reference battery capacity. The instructions, when individually or collectively executed by the at least one processor, may cause the device to generate the second frame for the video service by decompressing the compressed data based on the neural network according to the load being less than the reference load or the remaining battery amount being less than the reference battery amount, and to transmit the second frame to the second terminal.

[0171] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the device to obtain channel information of a channel between the second terminal connected to the network node and the network node. The instructions, when individually or collectively executed by the at least one processor, may cause the device to identify a value indicative of a quality of the channel based on the channel information. The instructions, when individually or collectively executed by the at least one processor, may cause the device to transmit the compressed data to the second terminal to authorize the second terminal to decompress the compressed data based on the load exceeding the reference load, the remaining battery amount exceeding the reference battery amount, and the value exceeding the reference value. The instructions, when individually or collectively executed by the at least one processor, may cause the device to generate the second frame for the video service by performing decompression of the compressed data based on the neural network according to the load being less than the reference load, the remaining battery amount being less than the reference battery amount, or the value being less than the reference value, and to transmit the second frame to the second terminal.

[0172] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the device to obtain location information of the first terminal together with the compressed data including the feature points of the first frame. The instructions, when individually or collectively executed by the at least one processor, may cause the device to transmit, to the second terminal, the location information of the first terminal together with the compressed data, to authorize the second terminal to decompress the compressed data, in response to the load exceeding the reference load. The instructions, when individually or collectively executed by the at least one processor, may cause the device to identify, in respect to the location information of the first terminal, a background image to be mapped, in response to the load being less than the reference load, perform synthesis of the background image for the second frame, and transmit, to the second terminal, the second frame with the background image synthesized therewith.

[0173] According to one embodiment, the location information may include global positioning system (GPS) coordinates of the first terminal.

[0174] According to one embodiment, the compressed data including the feature points of the first frame can be received from the first terminal connected to the network node.

[0175] According to one embodiment, the compressed data including the feature points of the first frame may be received via another network node connected to the first terminal.

[0176] According to one embodiment, the neural network may include a generative adversarial network (GAN).

[0177] According to one embodiment, the video service may include at least one of a video call service or a video streaming service.

[0178] The method performed by the network node as described above may include an operation of obtaining compressed data including feature points of a first frame for a video service between a first terminal and a second terminal. The method may include an operation of identifying a load associated with a neural network for performing decompression of the compressed data within the network node. The method may include an operation of transmitting the compressed data to a second terminal to enable the second terminal to perform decompression of the compressed data, based on the load exceeding a reference load. The method may include an operation of generating a second frame for the video service by performing decompression of the compressed data based on the neural network, based on the load being less than the reference load, and an operation of transmitting the second frame to the second terminal.

[0179] The non-transitory computer-readable storage medium as described above may store one or more programs comprising instructions that, when individually or collectively executed by at least one processor of a network node, cause the network node to obtain compressed data comprising feature points of a first frame for a video service between a first terminal and a second terminal. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the network node to identify a load associated with a neural network for performing decompression of the compressed data within the network node. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor, cause the network node to transmit the compressed data to the second terminal to enable the second terminal to perform decompression of the compressed data in response to the load exceeding a reference load. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor, cause the network node to generate a second frame for the video service by decompressing the compressed data based on the neural network according to the load being less than the reference load, and to transmit the second frame to the second terminal.

[0180] As described above, the first terminal may include at least one transceiver. The first terminal may include a memory storing instructions and including one or more storage media. The first terminal may include at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to identify a value indicative of a quality of a channel between the first terminal and a network node associated with a video service between the first terminal and a second terminal. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to transmit, to the network node, first compressed data of a frame generated by performing compression on the frame for the video service according to the value exceeding a reference value. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to generate second compressed data including feature points by performing compression on the frame based on a neural network within the first terminal according to the value being less than the reference value, and to transmit the second compressed data to the network node.

[0181] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to determine whether the frame includes a designated object based on the value being less than the reference value. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to generate the second compressed data including the feature points of the designated object by performing compression on the designated object of the frame based on the neural network upon determining that the frame includes the designated object. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to generate the second compressed data including the feature points of the frame by performing compression on the frame based on the neural network upon determining that the frame does not include the designated object.

[0182] According to one embodiment, the feature points of the frame of the second compressed data generated upon determining that the frame does not include the specified object may include feature points for a background image included in the frame.

[0183] In one embodiment, the designated object may include a user's face. Each of the feature points of the designated object may include a vector value indicating a change from each of the other feature points of the designated object in another frame prior to the frame. The number of feature points of the designated object may be determined based on the value.

[0184] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to identify a remaining battery amount of the first terminal. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to transmit, to the network node, the first compressed data of the frame generated by performing compression on the frame for the video service according to the remaining battery amount exceeding a reference battery amount and the value exceeding the reference value. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to generate, to the network node, the second compressed data including the feature points, by performing compression on the frame based on the neural network in the first terminal according to the remaining battery amount being less than the reference battery amount or the value being less than the reference value.

[0185] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to receive, from the network node, information about a load related to the video service of the network node. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to transmit, to the network node, the first compressed data of the frame generated by performing compression on the frame for the video service according to the load exceeding a reference load, the remaining battery amount exceeding the reference battery amount, and the value exceeding the reference value. The instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to generate the second compressed data including the feature points by performing compression on the frame based on the neural network in the first terminal according to the load being less than the reference load, the remaining battery amount being less than the reference battery amount, or the value being less than the reference value, and to transmit the second compressed data to the network node.

[0186] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the first terminal to transmit, to the network node, location information of the first terminal together with the second compressed data including the feature points. The location information of the first terminal may be used to identify a background image that is mapped with respect to the location information of the first terminal, to be synthesized into a frame decompressed from the second compressed data.

[0187] According to one embodiment, the neural network of the first terminal may include a convolution neural network (CNN).

[0188] The method performed by the first terminal as described above may include an operation of identifying a value representing the quality of a channel between the first terminal and a network node related to a video service between the first terminal and the second terminal. The method may include an operation of transmitting, to the network node, first compressed data of a frame generated by performing compression on a frame for the video service according to the value exceeding a reference value. The method may include an operation of generating second compressed data including feature points by performing compression on the frame based on a neural network in the first terminal according to the value being less than the reference value, and an operation of transmitting the second compressed data to the network node.

[0189] The non-transitory computer-readable storage medium as described above may store one or more storage media comprising instructions that, when individually or collectively executed by at least one processor of a first terminal including at least one transceiver, cause the first terminal to identify a value indicative of a quality of a channel between the first terminal and a network node associated with a video service between the first terminal and a second terminal. The non-transitory computer-readable storage medium may store one or more storage media comprising instructions that, when individually or collectively executed by the at least one processor, cause the first terminal to transmit, to the network node, first compressed data of a frame generated by performing compression on a frame for the video service according to the value exceeding a reference value. The non-transitory computer-readable storage medium may store one or more storage media including instructions that, when individually or collectively executed by the at least one processor, cause the first terminal to generate second compressed data including feature points by performing compression on the frame based on a neural network within the first terminal according to the value being less than the reference value, and to transmit the second compressed data to the network node.

[0190] The methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0191] When implemented in software, a computer-readable storage medium storing one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium are configured to be executed by one or more processors in an electronic device. The one or more programs include instructions that cause the electronic device to execute methods according to embodiments described in the claims or specification of the present disclosure. The one or more programs may be provided as a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0192] These programs (software modules, software) may be stored in random access memory, non-volatile memory including flash memory, read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic disc storage devices, compact disc-ROM (CD-ROM), digital versatile discs (DVDs) or other forms of optical storage devices, magnetic cassettes, or may be stored in memories formed by a combination of some or all of these. In addition, each configuration memory may include multiple copies.

[0193] Additionally, the program may be stored on an attachable storage device that is accessible via a communication network, such as the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a storage area network (SAN), or a combination thereof. Such a storage device may be connected to a device implementing an embodiment of the present disclosure via an external port. Additionally, a separate storage device on the communication network may be connected to a device implementing an embodiment of the present disclosure.

[0194] In the specific embodiments of the present disclosure described above, components included in the disclosure are expressed singularly or plurally, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in plural may be composed of singular elements, or components expressed in singular may be composed of plural elements.

[0195] According to embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., modules or programs) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0196] Meanwhile, although the detailed description of the present disclosure has described specific embodiments, it is obvious that various modifications are possible within the scope of the present disclosure.

Claims

In the device of the network node, Memory that stores instructions; and Contains at least one processor, The above instructions, when individually or collectively executed by the at least one processor, cause the device to: Obtain compressed data including feature points of a first frame for a video service between a first terminal and a second terminal; Identifying a load associated with a neural network for performing decompression of the compressed data within the network node; According to the load exceeding the reference load, transmitting the compressed data to the second terminal to enable the second terminal to perform decompression of the compressed data; and For the above loads which are less than the above reference load: Based on the neural network, by performing decompression of the compressed data, a second frame for the video service is generated, and causing the second frame to be transmitted to the second terminal, device. In claim 1, The feature points of the compressed data of the first frame include feature points for a specified object of the first frame. device. In claim 2, The above specified object includes the user's face, Each of the feature points of the compressed data includes a vector value representing a change from each of the other feature points of the designated object in the third frame prior to the first frame, and The number of the above characteristic points of the above compressed data is determined based on the load of the above network node. device. In claim 1, The above load is: Resources of at least one processor used to perform the decompression based on the neural network, The resources of the memory used to perform the decompression based on the neural network, or At least one of the resources of a physical channel allocated for the above video service, device. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the device to: Obtain battery information of the second terminal connected to the network node; Identifying the remaining battery amount of the second terminal based on the battery information of the second terminal; Transmitting the compressed data to the second terminal to authorize the second terminal to decompress the compressed data according to the load exceeding the reference load and the remaining battery amount exceeding the reference battery amount; and Depending on the load being less than the reference load or the remaining battery amount being less than the reference battery amount: Based on the neural network, by performing decompression of the compressed data, the second frame for the video service is generated, and causing the second frame to be transmitted to the second terminal, device. In claim 5, The above instructions, when individually or collectively executed by the at least one processor, cause the device to: Obtain channel information of a channel between the second terminal connected to the network node and the network node; Based on the above channel information, a value representing the quality of the channel is identified; Transmitting the compressed data to the second terminal to authorize the second terminal to decompress the compressed data according to the load exceeding the reference load, the remaining battery amount exceeding the reference battery amount, and the value exceeding the reference value; and Depending on the load being less than the reference load, the remaining battery amount being less than the reference battery amount, or the value being less than the reference value: Based on the neural network, by performing decompression of the compressed data, the second frame for the video service is generated, and causing the second frame to be transmitted to the second terminal, device. In claim 1, The above instructions, when individually or collectively executed by the at least one processor, cause the device to: Acquire location information of the first terminal together with the compressed data including the feature points of the first frame; According to the load exceeding the reference load, transmitting the location information of the first terminal to the second terminal together with the compressed data to authorize the second terminal to perform decompression of the compressed data; and For the above loads which are less than the above reference load: Identifying a background image to be mapped with respect to the location information of the first terminal; Performing synthesis of the background image for the second frame; and Causing the second frame, in which the background image is synthesized, to be transmitted to the second terminal, device. In claim 7, The above location information includes the GPS (global positioning system) coordinates of the first terminal. device. In claim 1, The compressed data including the feature points of the first frame is received from the first terminal connected to the network node. device. In claim 1, The compressed data including the feature points of the first frame is received via another network node connected to the first terminal. device. In claim 1, The above neural network includes a generative adversarial network (GAN). device. In claim 1, The above video service includes at least one of a video call service or a video streaming service. device. In the first terminal, At least one transmitter / receiver; A memory storing instructions and including one or more storage media; and At least one processor comprising a processing circuit, The above instructions, when individually or collectively executed by the at least one processor, cause the first terminal to: Identify a value representing the quality of a channel between the first terminal and the network node related to the video service between the first terminal and the second terminal; transmitting, to the network node, first compressed data of the frame generated by performing compression on the frame for the video service according to the value exceeding the reference value; and According to the above values ​​which are below the above reference values: By performing compression on the frame based on the neural network in the first terminal, second compressed data including feature points is generated, and causing the second compressed data to be transmitted to the network node, Terminal 1. In claim 13, The above instructions, when individually or collectively executed by the at least one processor, cause the first terminal to: Depending on the value being less than the reference value, determining whether the frame contains the specified object; Upon determining that the frame includes the designated object, generating the second compressed data including the feature points of the designated object by performing compression on the designated object of the frame based on the neural network; and When the frame is determined not to contain the specified object, performing compression on the frame based on the neural network to generate the second compressed data including the feature points of the frame, Terminal 1. In claim 14, The feature points of the frame of the second compressed data generated upon determining that the frame does not include the specified object include feature points for a background image included in the frame. Terminal 1.

Images