Method and device for transmitting / receiving media on basis of energy information in wireless communication system

Abstract

The present disclosure relates to a 5G or 6G communication system for supporting a higher data transmission rate.

Description

Method and apparatus for transmitting and receiving media based on energy information in a wireless communication system

[0001] The present disclosure relates to a wireless communication system or a mobile communication system. Specifically, it relates to a method and apparatus for transmitting and receiving media based on energy information in a wireless communication system.

[0002] 5G mobile communication technology defines a wide frequency band to enable fast transmission speeds and new services, and can be implemented not only in frequency bands below 6 GHz ('Sub 6 GHz'), such as 3.5 gigahertz (3.5 GHz), but also in ultra-high frequency bands called millimeter waves (mmWave), such as 28 GHz and 39 GHz ('Above 6 GHz'). In addition, for 6G mobile communication technology, which is referred to as a system beyond 5G, implementation in the terahertz (THX) band (e.g., the 3 terahertz band at 95 GHz) is being considered to achieve transmission speeds 50 times faster and ultra-low latency reduced to one-tenth compared to 5G mobile communication technology.

[0003] In the early stages of 5G mobile communication technology, aiming to satisfy service support and performance requirements for enhanced Mobile BroadBand (eMBB), Ultra-Reliable Low-Latency Communications (URLLC), and massive Machine-Type Communications (mMTC), technologies such as beamforming and Massive MIMO to mitigate path loss and increase transmission distance in ultra-high frequency bands, support for various numerologies (such as the operation of multiple subcarrier spacings) and dynamic operation of slot formats for the efficient utilization of ultra-high frequency resources, initial access techniques to support multi-beam transmission and broadband, definition and operation of Band-Width Parts (BWP), Low Density Parity Check (LDPC) codes for high-volume data transmission, new channel coding methods such as Polar Codes for the reliable transmission of control information, and L2 pre-processing (L2 Standardization has been carried out for pre-processing, network slicing which provides a dedicated network specialized for specific services, and other methods.

[0004] Currently, discussions are underway to improve and enhance the performance of the initial 5G mobile communication technology, taking into account the services that the 5G mobile communication technology was intended to support. Additionally, standardization of the physical layer is in progress for technologies such as V2X (Vehicle-to-Everything), which helps autonomous vehicles make driving decisions and enhance user convenience based on their own location and status information transmitted by the vehicle; NR-U (New Radio Unlicensed), which aims for system operation in unlicensed bands to comply with various regulatory requirements; NR terminal low power consumption technology (UE Power Saving); Non-Terrestrial Network (NTN), which is direct terminal-satellite communication for securing coverage in areas where communication with the terrestrial network is impossible; and positioning.

[0005] In addition, standardization is underway in the field of wireless interface architecture / protocols for technologies such as the Industrial Internet of Things (IIoT) for supporting new services through linkage and convergence with other industries, Integrated Access and Backhaul (IAB) which provides nodes for expanding network service areas by integrating wireless backhaul links and access links, Mobility Enhancement including Conditional Handover and Dual Active Protocol Stack (DAPS) Handover, and 2-step Random Access (2-step RACH for NR) which simplifies random access procedures. Standardization is also underway in the field of system architecture / services for 5G baseline architectures (e.g., Service based Architecture, Service based Interface) for incorporating Network Functions Virtualization (NFV) and Software-Defined Networking (SDN) technologies, and Mobile Edge Computing (MEC), which provides services based on the location of the terminal.

[0006] When such 5G mobile communication systems are commercialized, connected devices, which are increasing explosively, will be connected to communication networks. Accordingly, it is expected that there will be a need to enhance the functionality and performance of 5G mobile communication systems and to integrate the operation of connected devices. To this end, new research is planned to be conducted on 5G performance improvement and complexity reduction, support for AI services, support for metaverse services, and drone communication using eXtended Reality (XR), Artificial Intelligence (AI), and Machine Learning (ML) to efficiently support Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR).

[0007] Furthermore, the advancement of these 5G mobile communication systems encompasses multi-antenna transmission technologies such as new waveforms to guarantee coverage in the terahertz band of 6G mobile communication technology, Full Dimensional MIMO (FD-MIMO), array antennas, and large-scale antennas; metamaterial-based lenses and antennas to improve terahertz band signal coverage; high-dimensional spatial multiplexing technology using OAM (Orbital Angular Momentum); and Reconfigurable Intelligent Surface (RIS) technology; as well as Full Duplex technology for enhancing frequency efficiency and system networks in 6G mobile communication technology; AI-based communication technologies that realize system optimization by utilizing satellites and AI from the design stage and internalizing end-to-end AI support functions; and the realization of services of complexity exceeding the limits of terminal computing capabilities by utilizing ultra-high-performance communication and computing resources. It could serve as a foundation for the development of next-generation distributed computing technologies.

[0008] The present disclosure aims to provide an apparatus and method capable of effectively providing services in a wireless communication system.

[0009] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0010] A method according to one embodiment for solving the above-mentioned problem comprises, in a method for processing a control signal in a wireless communication system, a step of receiving a first control signal transmitted from a base station; a step of processing the received first control signal; a step of generating a second signal based on the processing; and a step of transmitting the generated second control signal to the base station.

[0011] The disclosed embodiments provide an apparatus and a method capable of effectively providing services in a wireless communication system.

[0012] FIG. 1 is a drawing illustrating a cache server, media stored in an original server, a base station, and a terminal according to one embodiment of the present disclosure.

[0013] FIG. 2 is a drawing illustrating a cache server, a terminal simultaneously connected to one or more base stations, and one or more base stations according to one embodiment of the present disclosure.

[0014] FIG. 3 is a diagram illustrating the architecture of a media streaming system for transmitting and receiving media based on energy information according to one embodiment of the present disclosure.

[0015] FIG. 4 is a diagram showing operations performed in the architecture of a media streaming system according to one embodiment of the present disclosure.

[0016] FIG. 5 illustrates the structure of a terminal according to various embodiments of the present disclosure.

[0017] FIG. 6 illustrates the structure of a server according to one embodiment of the present disclosure.

[0018] FIG. 7 illustrates the structure of a network entity according to an embodiment of the present disclosure.

[0019] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0020] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0021] Various embodiments are described in detail below with reference to the accompanying drawings. Furthermore, in describing the embodiments of this disclosure, specific descriptions of related known functions or configurations are omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the embodiments. Additionally, terms used below are defined considering their functions in the embodiments, and these may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0022] For the same reason, some components in the attached drawings have been exaggerated, omitted, or schematically depicted. Additionally, the size of each component does not entirely reflect its actual dimensions.

[0023] The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the disclosure, and the present disclosure is defined only by the scope of the claims.

[0024] At this time, it will be understood that each block of the process flow diagrams and combinations of the flow diagrams can be executed by computer program instructions. Since these computer program instructions can be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create means to perform the functions described in the flow diagram block(s). Since these computer program instructions can also be stored in computer-available or computer-readable memory that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in computer-available or computer-readable memory can also produce a manufactured item containing means of instruction to perform the function described in the flow diagram block(s). Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in the flowchart block(s).

[0025] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). It should also be noted that in some alternative execution examples, the functions mentioned in the blocks may occur out of order. For instance, two blocks described in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to their corresponding functions.

[0026] In this case, the term “part” as used in various embodiments of the present disclosure refers to a software or hardware component, such as an FPGA or ASIC, and the “part” may perform certain roles. However, the “part” is not limited to software or hardware. The “part” may be configured to reside in an addressable storage medium or may be configured to run one or more processors. Accordingly, as an example, the “part” may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “parts” may be combined into a smaller number of components and “parts” or further separated into additional components and “parts.” In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0027] Preferred embodiments of the present disclosure are described in detail below with reference to the attached drawings. It should be noted that identical components in the attached drawings are indicated by the same reference numerals whenever possible. Furthermore, it should be noted that the drawings of the present invention attached below are provided to aid in understanding the present invention, and that the present invention is not limited to the forms or arrangements illustrated in the drawings. Additionally, detailed descriptions of known functions and configurations that may obscure the essence of the present invention will be omitted. It should be noted that in the following description, only the parts necessary for understanding the operation according to various embodiments of the present invention are described, and descriptions of other parts will be omitted so as not to distract from the essence of the present invention. Furthermore, the present disclosure describes various embodiments using terms used in some communication standards (e.g., 3GPP (3rd Generation Partnership Project)), but this is merely an example for illustrative purposes. Various embodiments of the present disclosure can be easily modified and applied to other communication systems.

[0028] In the present disclosure, each of the phrases such as “A / B”, “A and / or B”, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order). In this disclosure, network technology may refer to standard specifications defined by the International Telecommunication Union (ITU) or 3GPP (e.g., TS 23.501, TS 23.502, TS 23.503, etc.), and the components included in the network structure of FIG. 1 described below may refer to physical entities, or to software performing individual functions or hardware combined with software. Reference numerals denoted by Nx, such as N1, N2, N3,... in the drawings, represent known interfaces between NFs in a 5G core network (CN).

[0029] Based on the discussion described above, the present disclosure may provide a method and apparatus for determining energy information required to play media or effectively managing the operating mode of a terminal when transmitting and receiving media based on energy information in a mobile communication system.

[0030] Regarding the energy consumption and performance management of electronic devices, including mobile devices, various power modes such as high-performance mode, optimization mode, and power-saving mode may be considered based on user preferences or system standards. For example, high-performance mode may be used when the remaining battery level of the electronic device is above a certain threshold, optimization mode may be used to maintain a balance between the device's performance and battery life under specific conditions, and power-saving mode may be used when the remaining battery level drops below a critical threshold. The power modes of the electronic device can be managed by software algorithms that adjust the amount of energy allocated to hardware components such as processors and GPUs, and control resources such as CPU and memory allocated to background processes and networking. Furthermore, electronic devices may utilize machine learning models to predict user behavior and dynamically optimize resource allocation based on usage patterns.

[0031] Media consumption is one of the primary uses of electronic devices, including mobile devices, and can be identified as a primary use of energy consumption for mobile devices. Along with the current battery level, the remaining usage time of the mobile device can be predicted based on the trend of energy reduction per unit time resulting from the operation of current applications. However, the predicted remaining usage time may not be able to predict the amount of energy the electronic device will consume when running media before selecting and running the media; it may not be possible to determine whether the media can be run to completion with the remaining energy of the electronic device; and it may not reflect changes in the electronic device's energy consumption that vary depending on whether the complexity of the remaining segments of the running media becomes more or less complex.

[0032] Accordingly, the present disclosure may consider a method for switching the performance mode of a terminal in advance based on the amount of energy expected to be consumed for current and future media consumption. Furthermore, the present disclosure may consider a method for selectively adjusting energy consumption by switching the media to be consumed to a media with lower energy consumption. For example, a method may consider determining that the currently selected media can be played in high-performance mode but is difficult to play in optimization mode. For example, a method may consider determining that the current high-performance mode is performance beyond what is necessary because the currently selected media can be played sufficiently in low-power mode, and that switching to low-power mode allows the mobile device to be used for a longer period. For example, a method may consider knowing what media can be selected in low-power mode and selecting from among the media that can operate in low-power mode.

[0033] [Media Energy Index]

[0034] In the present disclosure, the term "Media Energy Index" may refer to a value of the amount of energy required for the creation and consumption of media, or a value of an indicator that can infer the amount of energy. The Media Energy Index (MEIX; Media Energy IndeX) may be composed of pairs of energy items (or indicators) and item values. The Media Energy Index may be at least one of those derived at the media creation stage, derived at the media consumption stage, derived from an analysis of the media, or derived at the media transmission stage.

[0035] The media energy index can be defined as a numerical value representing a certain amount required for the playback of a specific segment of media. This numerical value required for the playback of a specific segment of media can be derived from measured power consumption, the number of CPU instructions, memory access frequency (e.g., number of times), memory read / write volume (e.g., byte size), encoding / decoding operation processing time (e.g., seconds), complexity improvement rate / size improvement rate, etc.

[0036] Media playback is not merely the decoding of an encoded stream; it can be a complex result involving multiple interplay of factors, such as network access for stream reception, decoder cache buffer read / write operations, output buffer memory read / write operations, screen rendering, display device energy consumption, and the management of other OS resources. Therefore, the amount of energy consumed during media playback, and / or a certain value, can be used as an indicator by measuring decoder components—such as the number of CPU instructions, memory access frequency, memory read / write volume, and operation processing time—to determine the algorithmic complexity of the decoder alone. Alternatively, it can be used as an indicator of the overall system's power consumption, such as through a power consumption meter connected externally to the terminal. As the number of factors included in the energy consumption calculation increases, reproducibility through repeated measurements decreases, and the value may become irregular and subject to change depending on system hardware configuration and temperature variations. Consequently, energy indices can be calculated for each component (e.g., number of CPU instructions, memory read / write volume), allowing for the provision of various energy indices derived from multiple measurement environments for a single medium. For example, the measurement environment for the energy index may include reference software (e.g., HEVC Reference Model), reference product (e.g., Galaxy S25), reference HW decoder (e.g., SnapDragon 8 Elite), and display usage (e.g., conditions using a 1500 nits brightness display). For example, media can be played under the measurement environment, and the results of measurements taken in segments can be provided as the energy index of the media.

[0037] The range of energy index values ​​(energy_scale) may be at least one of the absolute, relative, or normalized values ​​(normalized, log_normalized) of the measurements produced by the measuring instrument. For example, the absolute value may be the value measured by the measuring instrument. For example, the relative value may be the average of the entire media or a value expressed in the form of + / - after setting the initial value to 0. For example, the normalized value may be a value in which the min and max of the measurements are mapped to 0 and 1, respectively, and the mean or median is mapped to 0.5. For example, the log_normalized value may be a value obtained by taking the logarithm of the measurements and then normalizing them.

[0038] An energy index is a value measuring energy consumption during media playback under specific conditions, and index values ​​of different media may not be directly comparable. Accordingly, if a measurement environment is specified using reference hardware or software, it may be possible to compare media indices that share the same measurement environment identifier. For example, when content A and B share the same measurement environment identifier, it is possible to compare whether the energy consumption of A is higher or lower than that of B.

[0039] [Energy Item]

[0040] The energy item of the media energy index may be listed for the whole and / or part of the media. The part of the media may be referred to as at least one of the following: a temporal interval from one point in time to another, one or more hierarchical layers in scalable coding, or one or more visual views in multi-view coding. For example, in a media with a playback time of 2 hours, n media with different qualities or different energy consumption may be provided for each time interval of every cycle from the start. That is, when the structure of the media is identified, there may be n different media with the same specific condition, such as a 'time interval from the start to 1 minute,' and a media energy index may be listed for each media.

[0041] [Criteria]

[0042] - Comprehensive criteria: When a type of energy item in this disclosure is referred to as a comprehensive criterion, this may mean that the media energy index is a comprehensive result value for conditions involving a mixture of one or more factors. For example, when periodically measuring the remaining battery level of a terminal while playing media, the energy consumption of the terminal may include at least one of the following: the media, the terminal's operating system, radio frequency power for maintaining the terminal's network connection, or background operations of other apps on the terminal.

[0043] In one embodiment, the overall result value may be difficult or impossible to control. Therefore, while the energy consumption for the overall operation can be inferred, the value of the corresponding energy item may no longer be referenced when specific conditions change. For example, values ​​measured in a reference device with a first condition, such as CPU speed and memory size, may not contain information to determine how much influence the CPU factor has in order to be referenced in an actual terminal with a second condition using a different CPU.

[0044] - Single criteria: In this disclosure, when a type of energy item is referred to as a single criterion, it may mean a result value extracted for a single factor.

[0045] In one embodiment, since various factors (e.g., CPU, GPU, memory, display, OS, etc.) are used simultaneously in media consumption, a single factor may be insufficient to be referenced as a representative value; however, by referencing multiple factors that match the situation for each terminal, more personalized energy consumption can be inferred. For example, when the number of CPU operations per time interval is counted according to media playback, a single metric such as 'number of CPU operations' may be insufficient to represent the total power consumption of the system, such as memory usage requirements or display power consumption. Therefore, the media application can examine whether information regarding key metrics related to media playback can be extracted within the terminal or received from a service provider, and infer energy usage based on the available information.

[0046] [Reference Hardware]

[0047] Reference hardware may be a hardware configuration that is expected to be primarily used or has common features when compared to many other devices.

[0048] In one embodiment, if the media energy index is measured in reference hardware, it may have an identification code indicating that it was measured in a reference device and an identifier capable of identifying the reference device.

[0049] In one embodiment, a service provider providing a media energy index may provide the same or similar reference identification codes for different reference hardware having similar energy index characteristics. For example, since Company A's product and Company B's product equipped with the same Application Processor (AP) may have the same media decoding performance, they may be assigned the same identification code 0000 or provide similar reference identification codes such as 0000-A and 0000-B.

[0050] As an example of a reference device, a specific mobile terminal from a specific manufacturer (e.g., the Galaxy S-00 product) may be considered. When there is an identification code matching that terminal in the reference device section of the received media energy index, the terminal may refer to it as being similar or equivalent to the energy consumption of that terminal.

[0051] In one embodiment, for the identification of a terminal, the terminal model name may be written in the form of a string or configured as a pair of an identification code management system and an identifier managed by the system. For example, the identification code management system may be indicated as 3gpp.org / reference_code / imei-tac, and the TAC value of the IMEI may be used as the identifier value. IMEI is a method for identifying a terminal in a 3GPP network, and IMEI is a 15-digit number, of which 8 digits are called TAC (Type Allocation Code) and can represent terminal manufacturer and terminal model information. The terminal can know its own IMEI, and when the TAC written in the reference hardware identification type and value among the media energy index received from a service entity providing the media energy index, such as a network service or media service provider, matches, it determines that the value was measured from the same model and can use that value as is.

[0052] In one embodiment, the terminal identification code may be a single item or a list. For example, the identification code management system may be designated as a TAC, and multiple TACs may be listed in the identifier value. Terminals identified by the corresponding TAC may have similar or equivalent key specifications related to media playback and energy consumption. For example, a terminal may know that it has a similar configuration even if its TAC is not in the list.

[0053] In one embodiment, after receiving the TAC of the first terminal, the media energy index service provider may query the energy index of the second terminal, which has the most similar configuration, and provide the media energy index of the second terminal instead. Hardware specifications such as the type of AP, memory, battery size, and the type of modem chip may be considered in determining the similar configuration.

[0054] [Terminal Components (Reference Hardware Components)]

[0055] Instead of a single terminal model name, common hardware used by multiple manufacturers, such as AP chips, GPU / NPU chips, and modem chips, can be referenced. Since the terminal's OS can provide chipset information—such as CPU, AP, GPU, and NPU—to applications, terminal applications can identify the terminal's hardware configuration based on this chipset information. Terminal applications can determine whether the terminal's hardware components match the received media energy index. Terminal applications can request the media energy index based on the terminal's hardware components.

[0056] In one embodiment, if there is a terminal hardware component that matches the reference hardware, the terminal application can expect to consume an equivalent level of energy at the terminal.

[0057] [Reference Software]

[0058] Reference software refers to the source code of a specific codec used in the standardization and technical development process, where the logic of audio or video encoding or decoding can be described as software logic. For example, it can quantify the number of CPU addition and multiplication operations, and the size of memory read and write operations during the decoding of a certain media. Reference software is characterized by the ability to reproduce exactly the same numerical values ​​for the same input media. In other words, a single metric can be generated.

[0059] In one embodiment, the patterns of CPU computation and memory read / write operations may vary depending on the complexity of each segment and frame of the media, and accordingly, indicators extracted from reference software can infer the expected energy consumption between different media or between different time segments within a single media. Since mobile terminals implement video encoding and decoding in hardware, software indicators such as the number of CPU computation calls may not have a 1:1 or linear relationship with the GPU or hardware implementation. However, they can serve as a basis for estimating how much more energy will be consumed relatively when comparing two media of different options.

[0060] In one embodiment, the metrics that can be extracted from the reference software may include at least one of the following: the number of CPU operation calls, memory read / write size, the number of cache memory calls, the accuracy of motion estimation, the bit rate per segment, the quality measure compared to the original (PSNR; peak signal to noise ratio, SSIM; structural similarity index, MSE; Mean squared error, MAE; Mean absolute error), the compression ratio, the entropy coding (dictionary), whether loop filtering is applied, the difference in quality measure values ​​depending on whether loop filtering is applied (e.g., how much the PSNR increases when loop filtering is applied compared to when loop filtering is not applied), the error rate compared to the original, the encoding time, the decoding time (e.g., the time required to decode one frame, one segment, or the entire segment. The decoding time may differ from the playback time), the number of contexts, or the number of context replacements. In addition, during the operation of the reference software, at least one of the following may be considered as an indicator extractable by the reference software: a measurement of energy consumption for the system on which the reference software is running, the relative remaining battery capacity of the entire media or by section, the amount of current consumed, CPU utilization, GPU utilization, or a measurement of GPU temperature change. However, it is not limited to the examples described above.

[0061] [Reference Profile]

[0062] The codec predefines the parameters of the settings and tools used for coding and can define them as profiles, levels, etc. An application can read the media and identify the profile, level, and additionally applied tools used. For example, if a first media is encoded with a first profile and a first level, all terminals and applications supporting the first level of the first profile can decode the first media. Since the first profile and / or the first level have different setting values ​​and different tool configurations compared to other profiles and / or other levels, the combination of profile, level, and tool can be used as an indicator of the media energy index and can be considered as having a relative energy consumption ratio compared to other combinations of profile, level, and tool.

[0063] [Video Coding]

[0064] In one embodiment, for a video codec, the image is composed of multiple frames, and each frame can be divided into macroblocks of different sizes. Each macroblock can reference a macroblock of the same intra type frame or a different inter type frame. The size of the macroblock varies from 4 pixels x 4 pixels to 16 pixels x 16 pixels, and depending on the codec, the size may become larger, such as 32 pixels x 32 pixels or 64 pixels x 64 pixels. Larger macroblocks contain unique information and may consume more energy to decode them. Therefore, the total number of macroblocks or macroblocks per time interval, and the number and ratio by size and type within the total number, can be used as indicators to determine whether the energy consumption rate will increase or decrease.

[0065] For example, in a daily test sequence, the total number of macroblocks in the first time interval (00:00:00-00:00:01, from 0 seconds to 1 second) is 48,404 and the total number of macroblocks in the second time interval (00:00:02-00:00:03, from 2 seconds to 3 seconds) is 93,741. Although the number of macroblocks is greater in the second time interval, if the number of Intra type 16x16 macroblocks is 6,179 in the first time interval and 2,681 in the second time interval, it can be determined that the complexity of the first time interval is higher.

[0066] For example, when encoding another 15-second sample of a test sequence with a fixed bit rate of 5 Mbps using baseline, main, and high profiles, the sizes of baseline, main, and high can be 7.2 MB, 6.4 MB, and 6.36 MB, respectively. In this case, the decoding times for the baseline, main, and high profiles can be 6.174 seconds, 7.905 seconds, and 8.014 seconds, respectively. That is, in terms of complexity (or energy consumption), high is the most complex, followed by main, and baseline may be less complex compared to high and main. Therefore, assuming that the energy required for transmission is not considered or that the data has already been received, baseline may consume the least energy, followed by the main profile, and high profile may consume the most energy. In addition, when the energy consumption of the main and high profiles is 128% and 129.8% of the baseline, respectively, compared to the baseline, the difference between the main profile and the high profile can be said to be a very small difference within 1-2%.

[0067] For example, the comparison between when cabac, one of the encoding tools, is applied and when it is not applied in the same sequence with the same profile (fixed to main) can be as follows. When cabac is not applied, the data size becomes 6.9MB, and when cabac is applied, the data size becomes 6.4MB. When cabac is applied, the decoding time is 7.26 seconds, and when cabac is not applied, the decoding time is 7.89 seconds. Applying cabac can reduce the data size and bitrate, and increase the decoding energy.

[0068] For example, the following comparisons can be made by adjusting the number of b-frames to 0, 3, and 5, respectively, within the same sequence and the same profile (fixed to main). When the number of b-frames is 0, the data size can be 6.53 MB and the decoding time can be 6.373 seconds. When the number of b-frames is 3, the data size can be 6.4 MB and the decoding time can be 7.856 seconds. When the number of b-frames is 5, the data size can be 6.32 MB and the decoding time can be 8.233 seconds. In other words, depending on the number of b-frames, the data size is reduced to 98% when there are 3 b-frames and 96% when there are 5 b-frames compared to when there are 0 b-frames, but the complexity increases by 123% when there are 3 b-frames and 129% when there are 5 b-frames. If the number of bframes increases, the data size decreases by 2-4% and the decoding energy can increase by 23-29%.

[0069] For example, the comparison of adjusting the number of weighted P-frames to 0, 1, and 2, respectively, in the same profile (fixed to main) of the same sequence can be as follows. When there are 0 weighted P-frames, the data size can be 6.87 MB and the decoding time can be 7.823 seconds. When there is 1 weighted P-frame, the data size can be 6.46 MB and the decoding time can be 7.820 seconds. When there are 2 weighted P-frames, the data size can be 6.40 MB and the decoding time can be 7.887 seconds. Depending on the number of weighted P-frames, the data size is reduced to 94% when there is 1 weighted P-frame and 93% when there are 2 weighted P-frames compared to when there are 0 weighted P-frames; however, the complexity can increase to 99.9% when there is 1 weighted P-frame and 100.8% when there are 2 weighted P-frames. As the number of weighted P-frames increases, the data size decreases by 6-7%, and decoding energy may hardly increase. In other words, data can be reduced without affecting energy.

[0070] For example, a comparison of the same sequence with the same profile (fixed to main) with deblock strengths adjusted to (-3,-3), (0,0), (2,2), and (4,4), respectively, may be as follows. When the deblock strength is (-3,-3), the data size may be 6.38 MB and the decoding time may be 7.714 seconds. When the deblock strength is (0,0), the data size may be 6.40 MB and the decoding time may be 7.984 seconds. When the deblock strength is (2,2), the data size may be 6.46 MB and the decoding time may be 8.001 seconds. When the deblock strength is (4,4), the data size may be 6.69 MB and the decoding time may be 8.037 seconds. Depending on the deblock strength, the data size is reduced to 96% when the deblock strength is (2,2) compared to (4,4), and to 95% when the deblock strength is (0,0) or (-3,-3); however, the complexity can be reduced to 99.5% when the deblock strength is (2,2), 99.3% when the deblock strength is (0,0), and 95% when the deblock strength is (-3,-3). If the strength of the deblocking filter is weakened, the data size can be reduced by 3-4% and the decoding energy by 0-4%. In other words, both energy and data can be reduced together depending on the parameter values.

[0071] For example, the following comparisons can be made by adjusting the number of reference frames to 1, 3, and 5, respectively, in the same profile (fixed to main) of the same sequence. When the number of reference frames is 1, the data size is 6.53 MB and the decoding time can be 7.680 seconds. When the number of reference frames is 3, the data size is 6.406 MB and the decoding time can be 7.849 seconds. When the number of reference frames is 5, the data size is 6.403 MB and the decoding time can be 7.859 seconds. Depending on the number of reference frames, the data size is reduced to 98% when the number of reference frames is 3 and 97.9% when the number of reference frames is 5 compared to when the number of reference frames is 1, and the complexity can increase to 102.2% when the number of reference frames is 3 and 102.3% when the number of reference frames is 5. In other words, if the number of reference frames increases, the data size may decrease by 2% and the decoding energy may increase by 2%.

[0072] According to one embodiment, when comparing two media in whole or by time interval, at least one piece of information among the total number of macroblocks, Intra / Inter type, number by size, or ratio in the encoding result can be checked and used as a measure of energy consumption rate when selecting media of the terminal. The relative difference in energy consumption between the comparison targets is derived from the number of large macroblocks of the Intra type, and in the encoding option, at least one option among profile, cabac, number of bframes, number of weighted P-frames, deblock filter strength, or number of reference frames, and the corresponding decoding time are provided and can be used as a measure of energy consumption rate when selecting media of the terminal.

[0073] According to one embodiment, the media energy index may be expressed as a relationship between the ratio of the increase in energy relative to the ratio of the decrease in the transmission bit rate. The energy increase rate may be substituted with the time required. For example, the difference in decoding time between a first media encoded with a first option and a second media encoded with a second option can be seen as a difference in energy consumed.

[0074] In one embodiment, when the gain of the size of the compressed media file according to each encoding option is denoted as S (size) and the decoding time required is denoted as T (time), the relationship T = e * S is given, and the coefficient e can be called the energy efficiency ratio, i.e., the energy efficiency coefficient.

[0075] When e is -1, the rate of size reduction can match the rate of increase in decoding time. In other words, this encoding option can reduce size more by investing more energy.

[0076] When e is greater than -1, that is, as the gradient increases, the increase in decoding time can become larger. Therefore, it may be an encoding option that requires more energy input for the same magnitude of reduction.

[0077] If e is less than -1 and greater than 0, it may mean that the reduction in size is greater than the increase in decoding time. In other words, it may be an encoding option that can reduce the file size more than the energy input.

[0078] If e is greater than 0, it can be said that the size decreases as much as the decoding time decreases. It can be an encoding option where the size decreases while the decoding time is shortened. This can be an energy-efficient encoding option.

[0079] [display]

[0080] According to one embodiment, there may be bright and dark scene sections regardless of the complexity of the scene within the media. In bright scene sections, more electricity is converted into light energy, resulting in high energy consumption and potentially causing a rise in the terminal's temperature and / or performance degradation. For example, when the conversion efficiency from electricity to light is 50% or less, the remaining 50% or more is wasted as heat energy, causing the mobile terminal's temperature to rise. Consequently, operating at a temperature higher than the optimal temperature may lead to performance degradation of the CPU, GPU memory, etc. In one embodiment, in the case of a system where compensation for performance degradation is applied, additional energy consumption may occur.

[0081] According to one embodiment, the brightness of the entire media and scenes by time interval (e.g., average value, median value, peak value, low value) and the interval time or duration within the interval can be used as indicators to predict energy consumption. With respect to the brightness value of the media, the energy consumption rate may vary depending on the implementation technology of the terminal display (e.g., LCD+LED, OLED, MicroLED, etc.).

[0082] [Network transmission]

[0083] The energy consumed for the transmission of media within a network may be listed in the media energy index. Network connections may include wired connections between network equipment or wireless connections between a base station and a terminal.

[0084] In one embodiment, when a user requests transmission of a media, the media requested for transmission may have already been delivered to a network cache server, etc., by a previous request from another user. For any media requested by the user, the mobile communication network may provide it by considering the costs incurred on the wired connection (e.g., caching time) as energy.

[0085] FIG. 1 is a drawing illustrating a cache server, media stored in an original server, a base station, and a terminal according to one embodiment of the present disclosure.

[0086] In one embodiment, the media may be stored in a cache server or a source server. For example, referring to FIG. 1, when a movie named A is requested, the FHD video media of movie A may be stored in a nearby cache server due to requests from other users, while the 4K video media may not be stored in the cache server. From a network perspective, FHD media can save electricity and time used to copy from the source server compared to 4K media. Additionally, the time required to copy 4K media in comparison to network performance can be calculated and provided as one of the media energy indices. For example, referring to FIG. 1, when FHD is requested, the cost of the wired network connection can be calculated as the transmission time in (2). Also, referring to FIG. 1, when 4K media is requested, the processing cost in the first cache server can be considered together with the transmission time in (1) and (2).

[0087] FIG. 2 is a drawing illustrating a cache server, a terminal simultaneously connected to one or more base stations, and one or more base stations according to one embodiment of the present disclosure.

[0088] According to one embodiment, when a terminal is simultaneously connected to one or more base stations, the cost of the wired / wireless path connecting the cache server, the base station, and the terminal may be included in the media energy index. For example, referring to FIG. 2, the cost of the path transmitted from the cache server to the first base station and then received by the terminal may differ from the cost of the path transmitted from the cache server to the second base station and then received by the terminal. The first base station and the second base station can calculate and compare the amount of radio wave output required for each base station from the wireless connection characteristic information with the terminal. If the first base station is closer than the second base station from the terminal's perspective, it can be determined that the base station capable of reducing the terminal's wireless power is the first base station. From the perspective of the first base station, the wireless connection characteristics can be analyzed as being better compared to the second base station. For example, referring to FIG. 2, the transmission cost from the first base station to the terminal can be calculated from the transmission time of the wired connection of (3) and the radio wave characteristics of the wireless connection of (5). Referring to Figure 2, the transmission cost from the second base station to the terminal can be calculated from the transmission time of the wired connection of (4) and the propagation characteristics of the wireless connection of (6).

[0089] According to one embodiment, at least one of the radio frequency intensity, the number of required beams, the signal-to-noise ratio, the wired path travel time (i.e., wiring distance), or the cache server caching time may be considered as media energy index items from a wired / wireless network path.

[0090] [Usefulness of Information]

[0091] According to one embodiment, the media energy index is an indicator of the energy consumed during the playback and transmission process of a terminal, and can represent the pattern of change in energy consumption due to media playback.

[0092] According to one embodiment, an application of a terminal can determine, prior to the transmission of a media, how much more or less energy a specific media consumes compared to other media among several different media. The application can select and receive the media determined to be more appropriate based on the determined energy consumption ratio. The application can select and receive the media determined to be more appropriate based on the performance mode of the terminal.

[0093] According to one embodiment, an application can check at least one of the currently available energy or energy consumption from the energy status monitor of the terminal. Among the energy consumption, the application can check at least one of the transmission energy consumption, the consumption of media operation, or the consumption of the display device.

[0094] According to one embodiment, the application can derive a relationship by linking the energy index of a selected first media with the energy consumption. The relationship may include at least one of total energy consumption, transmission energy consumption, media operation consumption, or remaining battery capacity. The relationship may be described as an energy consumption model of an n-th degree equation or a learned AI model.

[0095] According to one embodiment, the application can continuously update an energy consumption model along with the playback of media. The energy consumption model can be used to predict the remaining battery level of the terminal when each time interval ends by inputting the media energy indices of the current media for each time interval.

[0096] According to one embodiment, the energy consumption model may be transmitted to an external entity permitted by the terminal. For example, it may be transmitted to a media energy index service provider or a mobile communication network operator. The external entity may reprocess the media energy index using the energy consumption model information, and the external entity may provide it to the terminal as a reference model in subsequent services. In addition to using the value received from the same model as a reference, the terminal and application may use a value generated based on the state of that product as a reference.

[0097] According to one embodiment, the charging efficiency of the battery may decrease due to the accumulation of usage time of the terminal. Due to the accumulation of usage time of the terminal, major components such as the CPU, GPU, and memory may deteriorate, performance may decrease, and energy consumption may continuously increase. The energy consumption model of the terminal may include effects caused by factors in which the terminal has different performance compared to when it was first purchased.

[0098] [Media Energy Index]

[0099] According to one embodiment, the media energy index consists of indicators. The indicator includes an indicator type, an indicator coverage range, and an indicator value.

[0100] According to one embodiment, the indicator type may include at least one of the number of CPU operations, memory read size, memory write size, memory usage, terminal system power consumption, CPU / AP power consumption, GPU power consumption, display power consumption, communication frequency power, intra macroblock ratio, inter macroblock ratio, profile, cabac application status, number of B-frames, number of Weighted P-frames, Deblock strength, number of reference frames, decoding time, bitrate gain, complexity gain, or energy efficiency ratio.

[0101] According to one embodiment, as sub-information of the indicator type, whether the reference HW / SW from which the indicator value was derived is used and usage or model information (e.g., TAC) may be listed for each indicator type.

[0102] According to one embodiment, the scope of the indicator may include at least one of the entire media, time intervals, hierarchy, or view.

[0103] According to one embodiment, the indicator value may be at least one of the energy consumption value, number, or time measured during the indicator coverage range for the indicator type. For example, if the decoding time is the indicator, the decoding time may be listed as the indicator value.

[0104] [Energy Consumption Model]

[0105] According to one embodiment, the energy consumption model may take a media energy index as input and one or more energy indices measured at the terminal as output.

[0106] For example, a media energy index with decoding time as an indicator is provided, and the value of the media energy index may be such that when the decoding time is 1 second for a playback time of 1 minute, the decoding time measured by the terminal is 2 seconds and the energy consumption is 1Wh. A formula that generates an output of 2 seconds for an input of 1 second can be the energy consumption model of the terminal. Then, for 1 minute, the decoding time of the media energy index is 1.5 seconds, the decoding time measured by the terminal is 2.5 seconds, and the energy consumption is 1.5Wh. A formula that outputs 2 seconds for 1 second and 2.5 seconds for 1.5 seconds can be updated as the energy consumption model for the decoding time of the terminal. Additionally, a formula that outputs 1Wh for 1 second and 1.5Wh for 1.5 seconds can be updated as the energy consumption model for the energy consumption of the terminal. To derive an energy consumption model that produces accurate or approximate values ​​for a larger number of values, it may be necessary to use an n-th degree equation or an AI model and continuously learn from more inputs.

[0107] According to one embodiment, the energy consumption model can convert a media energy index into energy consumption at a terminal. The energy consumption model can calculate the amount of energy predicted to be consumed at the terminal based on the media energy index value.

[0108] According to one embodiment, the terminal and the application may continuously measure the energy information and operational performance of the terminal to derive an energy consumption model of the terminal and reflect this. The energy consumption model may have high accuracy as it is repeatedly updated.

[0109] In the example of the present disclosure, the terminal derives an energy consumption model based on the energy index of the media and uses it to predict the amount of energy to be consumed in the future during the playback of the media. For multiple segments of the media, the terminal may refer to the media energy index of each segment, or may refer to the energy index of a certain segment by dividing the energy index given for the entire media by the playback time of the media.

[0110] [Terminal Structure]

[0111] According to one embodiment, the terminal has an energy status monitor (ESM) as a component and can enable an application to determine the energy status of the terminal.

[0112] According to one embodiment, the energy status monitor can check the operation and performance of the main components of the terminal. The energy status monitor can record the operation and performance of the main components of the terminal.

[0113] According to one embodiment, the energy status monitor may provide at least one of the remaining available energy, available battery capacity, or available battery ratio of the current terminal.

[0114] According to one embodiment, the energy status monitor may provide at least one of the current energy consumption / ratio of the terminal, the consumption and ratio in the transmitted energy, the consumption / ratio in the media operation, or the consumption / ratio in the display device.

[0115] According to one embodiment, the energy status monitor may provide indicators related to the media energy index. For example, it may provide at least one of the number of CPU operations, memory usage, power consumption of terminal / CPU / GPU / AP / display / communication frequency, or decoding time for a specified section for a specific media.

[0116] According to one embodiment, when factors such as image quality, bitrate, and media energy index are given for each media, the application can select a media with a lower media energy index value while maintaining image quality or bitrate.

[0117] According to one embodiment, the energy status monitor may provide events and conditions to which an application can subscribe regarding the information provided. For example, when an event occurs in which the energy consumption of a terminal exceeds a specific level, an application subscribed to that event may be called.

[0118] According to one embodiment, a terminal application may subscribe to an event service to monitor the amount of terminal energy or notify the application under specific threshold conditions. An energy status monitor may continuously measure the amount of transmitted energy and notify the application when it matches a condition registered by the application.

[0119] [MPEG-DASH]

[0120] When receiving and playing media, the terminal identifies the types of quality that can be selected in advance and can receive and play only the data corresponding to the selected quality. Dynamic Adaptive Streaming over HTTP (DASH) technology can describe the media structure using Media Presentation Description (MPD). MPD can distinguish the components of media (e.g., movies) (e.g., video, audio, or subtitles) as multiple Representations of different qualities. A group of interchangeable Representations can be called an AdaptationSet, and a single Representation consists of multiple segment files, and a single segment file may not overlap with other segments on the time axis.

[0121] MPD can describe the structure of media, AdaptationSets, Representations, and Segments, and explain quality criteria for Representations. Applications containing a DASH client capable of receiving DASH-formatted media can read the MPD to determine which Representation to receive and play at which media time, and which Segment file to receive and play at the corresponding time of that Representation.

[0122] Media components can be compressed, and the technology used for compression is called codec technology. Codecs are based on standard specifications such as H264 and HEVC, and within a single codec, profiles selected for compatibility and tools that are included or excluded from the profiles can be defined. For example, if a device supports profile A but does not support profile B, media compressed with profile B cannot be played on that device. For instance, if a device supports tool C because it is a mandatory tool but does not support tool D because it is an option related to image quality enhancement, media with both C and D applied can be played using C while ignoring D.

[0123] In interactive, P2P calls, if the preferred specifications of the mutual terminals are negotiated, tools that will not be used are not selected and therefore do not need to be transmitted. Since pre-generated content in on-demand environments is written for a large number of viewers, information unnecessary to some viewers may be transmitted.

[0124] [MPD]

[0125] According to one embodiment of the present disclosure, media energy index information may be provided as additional attribute information under the Representation and Segment of the MPD.

[0126] According to one embodiment, an application or DASH client may determine whether to receive media by reading the attributes of a Representation or Segment and, in addition to existing attributes such as bitrate or quality, based on the energy characteristics of the Representation and Segment when deciding which Representation to select for which component in an AdaptationSet.

[0127] According to one embodiment, the terminal can estimate and adjust the coefficients of the relationship between the terminal's energy indicator and the corresponding media's energy index. Once the terminal's energy indicator and the media's energy index are stabilized, the terminal can estimate the terminal's remaining battery capacity or expected current consumption based on the remaining media playback time.

[0128] According to one embodiment, the terminal may predict in advance a future point in time when the trend of the remaining battery amount due to media playback is expected to correspond to certain conditions, and may select a different Representation to control the arrival of that point in time. For example, when the terminal plays a Representation with ID A, it may calculate that the current remaining battery amount of 50% will decrease to 20% after 30 minutes. Accordingly, the terminal may check the energy index of a Representation with ID B and the energy index of a Representation with ID C, and calculate that they will decrease to 30% and 40%, respectively, after 30 minutes. If it is decided to consume less terminal energy based on the preset value of the DASH client or the judgment of the user, the DASH client may receive and play Representation "C," thereby maintaining the remaining battery amount of the terminal at 40% after 30 minutes.

[0129] [MEIX box]

[0130] According to one embodiment, the media energy index may be included within a segment of the DASH Representation.

[0131] According to one embodiment, a Segment may include SIDX (Segment Index) information indicating the location and size of a sub-segment that is a location where switching is possible within the segment. An application or DASH client may perform a switching to a segment file belonging to a different Representation after receiving a segment file and before playback is completed.

[0132] According to one embodiment, SIDX can be extended by adding media energy index information for segments and subsegments. In one embodiment, MEIX information is disclosed, and media energy index information is provided for the segment and subsegment numbers indicated by SIDX, so that the media energy index for the entire segment or for each subsegment can be identified.

[0133] According to one embodiment, when receiving a segment file, an application or DASH client first receives the header portion of the segment and can refer to media energy index information when determining the reception of the remaining portion within the segment.

[0134] [MPEG-SEI]

[0135] The Network Abstraction Layer (NAL) unit is a data structure that plays an important role in video codecs and is used in various video coding standards such as H.264, H.265 (HEVC), and VP9. As a basic unit that constitutes a video frame, the NAL can be used to efficiently encode and decode video data.

[0136] According to one embodiment, each NAL unit may consist of a NAL unit header, a Redundant Bit String Prefix (RBSP), and a payload. The NAL unit header is located at the beginning of each NAL unit and may contain various information such as the type, importance, and referenceability of the NAL unit. The RBSP is used as a separator between the NAL unit header and the payload, and the payload is the part where actual video data is stored and may contain pixel data or metadata of an encoded video frame.

[0137] According to one embodiment, NAL units are classified into various types, and each type performs a specific function; among the major NAL unit types, at least one of SLICE, IDR (Immediate Decoder Refresh) SLICE, SEI (Supplemental Enhancement Information), VPS (Video Parameter Set), SPS (Sequence Parameter Set), or PPS (Picture Parameter Set) may be considered. A SLICE represents a part of a video frame and contains encoded pixel data, and among these, an IDR SLICE may be a SLICE that allows a decoder to decode independently without referring to a previous frame. SEI may contain additional information that complements the video frame. For example, VPS, SPS, and PPS may be NAL units that define parameters of a video stream.

[0138] According to one embodiment, a media energy index may be inserted within the media. It may be included between NAL units as an SEI and may provide media energy index information for slices to be subsequently unfolded. The MEIU (Media Energy Index Unit) may provide reference numbers of referenced slices or NAL units, as well as information on media energy index indicators and indicator values ​​measured or calculated for the corresponding slices.

[0139] According to one embodiment, the MEIU can describe a difference in the media energy index when some of the slices to be subsequently unfolded are not used. For a plurality of slices described by a single MEIU, it can provide the total media energy index value, a list of slices that do not significantly affect playback even if not played, and the corresponding media energy index. For example, while the MEIU provides a media energy index for 10 subsequent slices, it can provide that Section 1 includes all 10 slices, and Section 2 excludes slices 5 and 10 from the 10 slices. The media decoder can identify slices 5 and 10 as having a specific tool applied through the conventional SEI, and through this, can calculate the expected terminal energy consumption when the tool is not applied.

[0140] [3GPP-5GMS (5G Media Streaming) architecture]

[0141] FIG. 3 is a diagram illustrating the architecture of a media streaming system for transmitting and receiving media based on energy information according to one embodiment of the present disclosure.

[0142] A content provider can provide media to a terminal through 5GMS. A content provider can derive a media energy index according to the present disclosure for the media and store it in the media or in a separate location.

[0143] According to one embodiment, a 5G media streaming (5GMS) system may include a terminal (user equipment, UE), a RAN (radio access network), a UPF (user plane function), a DN (data network), a NEF (network exposure function), a PCF (policy control function), and an application service provider. For example, referring to FIG. 3, the terminal (UE) may include at least one of an energy status monitor, a media-aware application, a MSH (media session handler), or a MAF (media aware function). For example, referring to FIG. 3, the data network (DN) may include a media AS or a media AF. For example, referring to FIG. 3, the application service provider may include a media energy index repository and a per UE media consumption model. The media AF may provide energy-related information of the network through the NEF or PCF, specifically including the cost of the wired / wireless path connecting the cache server, base station, and terminal as illustrated in FIG. 2 in the media energy index. Media AF or media AS may receive an energy index of media from an application service provider and provide it to a terminal. However, the components of the 5GMS system according to the present disclosure are not limited to the configuration shown in FIG. 3.

[0144] According to one embodiment, an Application Function (AF) serves as a point of contact for 3GPP media services between a content provider and a terminal (or application), and may receive a provision message from a content provider regarding the provision of media containing a media energy index. The AF may receive a list of media among the media for which an energy index may be provided, or it may notify the application by identifying only that an energy index may be provided.

[0145] According to one embodiment, the application can determine whether a media energy index can be added during the process of receiving information for service access from AF. The application can identify trends, such as the current value or rate of increase or decrease of various energy-related indicators consumed within the terminal, or can identify the total size and remaining capacity of the battery.

[0146] According to one embodiment, if media energy index information is available during the process of requesting and receiving media, the application may obtain the media energy index information before receiving the media or at the start of receiving the media.

[0147] According to one embodiment, the application can link the trend of its own energy consumption with a media energy index, thereby associating the value represented by the index with the trend of the terminal's own energy consumption as an energy consumption model. For example, if the media energy index pertains to the entire media, it is assumed that the energy required for playback of a specific unit of time within the total playback time is Media Energy Index * (Playback Unit Time / Total Playback Time). By comparing the remaining battery level measured by the terminal during that unit time, a method can be derived to convert the Media Energy Index into a remaining battery level value. Since other functions of the terminal are operating during the application's media playback, the terminal's energy consumption model can be continuously updated.

[0148] According to one embodiment, the application can determine from the derived coefficient whether the media energy index of the currently playing media is relatively high or low. If the application wishes to reduce the energy consumption of the terminal, it can select media with a relatively low media energy index from a list of selectable media. Since a low media energy index does not necessarily mean that playback quality is poor, a better choice can be made based on various conditions. For example, a new video codec may consume less energy than a previous video codec and thus have a lower energy index. For example, within a set of media, there are other media that maintain quality, and the media with a lower media energy index value can be selected from among them, thereby inducing lower terminal energy consumption.

[0149] FIG. 4 is a diagram showing operations performed in the architecture of a media streaming system according to one embodiment of the present disclosure.

[0150] Referring to Fig. 4, in the first step, a media content service provider can provide a media energy index from an energy perspective for the media to be provided.

[0151] According to one embodiment, the media energy index may be derived and provided at or after the time of media creation. The media energy index may be at least one of the result of an analysis of a media stream or the result of measuring the operational performance of a media playback client, depending on the indicator.

[0152] In the second stage, when a media content service provider generates a media energy index for the media, it can notify network entities of this.

[0153] According to one embodiment, in a 5GMS architecture, a media content service provider can notify the 5GMS AF that media and media energy indexes are available.

[0154] According to one embodiment, 5GMS AF provides an API that includes a field that allows a media service provider to indicate whether a media energy index is provided, and can use the API to receive information on the availability of a media energy index by media, the storage location of the media energy index, and the media energy index.

[0155] In the third step, the UE can request service discovery from the AF.

[0156] According to one embodiment, 5GMS AF can convey information related to overall media services in response to a service information request from a terminal application that wishes to receive services, and also notify that information regarding the media energy index is available.

[0157] In the fourth step, the information transmitted from the 5GMS AF to the application may indicate that a media energy index is available for at least one of the media provided by the AF, indicate separately that a media energy index is available for each media, or indicate separately the types of indicators of the media energy index available for each media.

[0158] In step 5, the energy status monitor (ESM) can check the Media Energy Index (MEIX) indicators that can be collected from the terminal.

[0159] According to one embodiment, if the application identifies that a media energy index is provided by a network service or content service provider, it may receive all media energy indices that can be provided by the provider, or communicate with the terminal's energy status monitor (ESM) to identify information that can be monitored at the terminal.

[0160] In step 6, the terminal's energy status monitor (ESM) can transmit to the application a list of indicators and indicator values ​​that are determined to be monitorable at the terminal.

[0161] In step 7, the application compares the list of indicators received from the terminal's energy status monitor and AF, and may request to select all or part of the commonly comparable indicators that are deemed useful as monitoring targets.

[0162] In step 8, the application transmits a list of information that can be monitored at the terminal to AF and can receive a media energy index containing the list of information.

[0163] According to one embodiment, the application receives a list of media energy index indicators from AF and can identify indicators among them that can be monitored by the terminal.

[0164] In step 9, AF communicates with AS to verify the list of metrics received from the application, and if the media energy index is separated by metric, it can receive the method for receiving media energy index information provided for the metrics requested by the application (e.g., URL information).

[0165] In step 10, AF can transmit to the application a method for receiving media energy index information received from AS.

[0166] In step 11, the application can request and receive media and media energy indexes from AS.

[0167] In step 12, the ESM can check the energy information of the terminal.

[0168] The application can check the status prior to media playback from the energy status monitor.

[0169] In step 13, the application can play media.

[0170] In one embodiment, the application can play a portion of the media. The portion of the media may be a section specified by the media energy index. That is, it may be characterized as a section in which the change in terminal energy due to media playback and the media energy index can be compared.

[0171] In step 14, the ESM can check the terminal energy information.

[0172] According to one embodiment, the application can check the status after media playback from the energy status monitor.

[0173] In step 15, the terminal can update the energy consumption model.

[0174] According to one embodiment, the application can determine the amount of change in energy of the terminal by comparing the information obtained in steps 12 and 14, and can obtain a written media energy index value by checking the corresponding section information of the media energy index. The application can use the two values ​​to derive an energy consumption model of the terminal, which is a correlation between the media energy index and the terminal energy consumption.

[0175] In step 16, the terminal can determine whether to switch media.

[0176] According to one embodiment, when selecting media, an application may refer to a media energy index along with indicators such as bitrate and quality, which are conventionally provided media attribute information. A media with a high value of the media energy index may consume more energy than other media that do not.

[0177] According to one embodiment, an application receives the remaining battery level, decrease rate, etc. of a terminal from an energy status monitor and can convert the media energy index of a subsequent playback section of media into the terminal's estimated energy consumption. If the estimated energy consumption required for the subsequent playback section is greater than the terminal's remaining battery level, the terminal may request to switch to media having a lower media energy index or to switch the terminal's performance mode.

[0178] In step 17, if necessary, the application performs switching to media having different media energy index and energy characteristics, and can receive a new media energy index created based on the media if one exists. In one embodiment, in step 10, the UE may receive multiple URLs from the AF to receive media energy index information for each media. In one embodiment, the application may return to step 8 for switching and proceed up to step 15.

[0179] [radio]

[0180] According to one embodiment, radio frequency may be considered among the major energy consumption items of the terminal. The terminal may attempt to connect using more power the further it is from the base station. The media bit rate may serve as a criterion for determining how often and how much data should be requested and received in an environment where the terminal connects to a network base station to receive data. In other words, the size and frequency of data reception requests, whether close to or far from the base station, may be considerations regarding energy consumption.

[0181] According to one embodiment, the application can measure the amount of transmission energy consumed in transmission from an energy status monitor and compare the media energy index by wired / wireless path of the network.

[0182] According to one embodiment, when continuous data transmission occurs, such as in media streaming, the application can monitor not only the amount of data received per unit time but also the amount of energy used for reception (amount of transmission energy) and manage it in conjunction with the amount received. The amount of transmission energy consumed per hour, or the amount of transmission energy consumed per byte, may increase or decrease depending on movement, stopping, or moving away from or closer to the base station.

[0183] According to one embodiment, an application can register an event under the condition that the transmission energy consumption relative to the received data becomes below a certain level. Depending on variables such as the movement or stationary status of the terminal and the distance from the base station, the registered event may occur at the time when the transmission energy consumption decreases.

[0184] According to one embodiment, at a time when it is determined that receiving the same amount of data consumes less energy, the application decides to receive more data than is necessary in advance, increases the size of the media buffer to store the data to be received, and can receive as much as the buffer allows.

[0185] According to one embodiment of the present disclosure, a media energy index may be provided for the media. By using the media energy index, the amount of energy to be consumed during the process of playing the media can be estimated in advance.

[0186] According to one embodiment of the present disclosure, information regarding energy to be consumed during the process of playing the media may be calculated from the media, from a system that plays the media, or from a system that transmits the media.

[0187] According to one embodiment of the present disclosure, information regarding energy to be consumed during the process of playing the media may be information regarding the entire or a portion of the media.

[0188] According to one embodiment of the present disclosure, a terminal and an application may have an energy status monitor (ESM) and measure the energy consumption status of the terminal.

[0189] According to one embodiment of the present disclosure, a terminal and an application can create an energy consumption model from the energy consumption trend for a unit interval equipped with a media energy index and the media energy index.

[0190] According to one embodiment of the present disclosure, an energy consumption model can calculate the media energy index as the amount of energy consumed at the terminal.

[0191] According to one embodiment of the present disclosure, a mobile communication network may provide a media energy index to a terminal or application that intends to receive a media service.

[0192] According to one embodiment of the present disclosure, a terminal and an application can determine whether a media energy index is provided.

[0193] According to one embodiment of the present disclosure, a terminal and an application can receive a media energy index from a network.

[0194] According to one embodiment of the present disclosure, a terminal and an application can determine and execute switching to another media based on a media energy index.

[0195] FIG. 5 illustrates the structure of a terminal (500) according to various embodiments of the present disclosure.

[0196] The configuration exemplified in FIG. 5 can be understood as a configuration of a terminal (500). Terms such as '...part', '...unit' used below refer to a unit that processes at least one function or operation, and this can be implemented as hardware or software, or a combination of hardware and software.

[0197] Referring to FIG. 5, the terminal (500) includes a communication unit (510), a storage unit (520), and a control unit (530).

[0198] The communication unit (510) performs functions for transmitting and receiving signals through a wireless channel. For example, the communication unit (510) performs a conversion function between a baseband signal and a bit sequence according to the physical layer specifications of the system. For example, when transmitting data, the communication unit (510) generates complex symbols by encoding and modulating the transmitted bit sequence. Also, when receiving data, the communication unit (510) restores the received bit sequence by demodulating and decoding the baseband signal. Additionally, the communication unit (510) upconverts the baseband signal into an RF band signal and transmits it through an antenna, and downconverts the RF band signal received through the antenna into a baseband signal. For example, the communication unit (510) may include a transmission filter, a reception filter, an amplifier, a mixer, an oscillator, a DAC, an ADC, etc.

[0199] Additionally, the communication unit (510) may include a plurality of transmission and reception paths. Furthermore, the communication unit (510) may include an antenna unit. The communication unit (510) may include at least one antenna array composed of a plurality of antenna elements. In terms of hardware, the communication unit (510) may be composed of a digital circuit and an analog circuit (e.g., a radio frequency integrated circuit (RFIC)). Here, the digital circuit and the analog circuit may be implemented as a single package. Additionally, the communication unit (510) may include a plurality of RF chains. The communication unit (510) may perform beamforming. The communication unit (510) may apply beamforming weights to a signal to impart directionality according to the settings of the control unit (530) to the signal to be transmitted or received. According to one embodiment, the communication unit (510) may include an RF (radio frequency) block (or RF unit). The RF block may include a first RF circuitry associated with an antenna and a second RF circuitry associated with baseband processing. The first RF circuitry may be referred to as RF-A (antenna). The second RF circuitry may be referred to as RF-B (baseband).

[0200] Additionally, the communication unit (510) can transmit and receive signals. To this end, the communication unit (510) may include at least one transceiver. The communication unit (510) can receive downlink signals. The downlink signal may include a synchronization signal (SS), a reference signal (RS) (e.g., DM (demodulation)-RS, PTRS (phase tracking reference signal)), system information (e.g., MIB, SIB, RMSI (remaining system information), OSI (other system information)), a configuration message, control information, or downlink data. Additionally, the communication unit (510) may transmit an uplink signal. The uplink signal may include a random access related signal (e.g., random access preamble (RAP) (or Msg1 (message 1)), Msg3 (message 3)), a reference signal (e.g., SRS (sounding reference signal), DMRS, PTRS), or a power headroom report (PHR).

[0201] Additionally, the communication unit (510) may include different communication modules to process signals of different frequency bands. Furthermore, the communication unit (510) may include multiple communication modules to support multiple different wireless access technologies. For example, different wireless access technologies may include Bluetooth Low Energy (BLE), Wi-Fi (Wireless Fidelity), WiGig (WiFi Gigabyte), cellular networks (e.g., LTE (Long Term Evolution), NR (new radio), etc. Additionally, different frequency bands may include super high frequency (SHF) bands (e.g., 2.5 GHz, 5 GHz) and millimeter wave bands (e.g., 38 GHz, 60 GHz, etc.). Additionally, the communication unit (510) may use the same type of wireless access technology on different frequency bands (e.g., unlicensed band for LAA (licensed Assisted Access), CBRS (citizens broadband radio service) (e.g., 3.5 GHz)).

[0202] The communication unit (510) transmits and receives signals as described above. Accordingly, all or part of the communication unit (510) may be referred to as a 'transmitter', a 'receiver', or a 'transmitter / receiver'. Additionally, in the following description, transmission and reception performed via a wireless channel are used to mean that processing as described above is performed by the communication unit (510).

[0203] The storage unit (520) stores data such as basic programs, application programs, and setting information for the operation of the terminal (500). The storage unit (520) may be composed of volatile memory, non-volatile memory, or a combination of volatile memory and non-volatile memory. Additionally, the storage unit (520) provides the stored data upon the request of the control unit (530).

[0204] The control unit (530) controls the overall operations of the terminal (500). For example, the control unit (530) transmits and receives signals through the communication unit (510). Additionally, the control unit (530) writes and reads data to and from the storage unit (520). Furthermore, the control unit (530) can perform the functions of the protocol stack required by the communication standard. To this end, the control unit (530) may include at least one processor. The control unit (530) may include at least one processor or microprocessor, or may be part of a processor. Additionally, part of the communication unit (510) and the control unit (530) may be referred to as CP. The control unit (530) may include various modules for performing communication. According to various embodiments, the control unit (530) may control the terminal to perform operations according to various embodiments.

[0205] FIG. 6 illustrates the structure of a server according to one embodiment of the present disclosure.

[0206] Referring to FIG. 6, the server may include a communication unit (610), a storage unit (620), and a control unit (630). In the present disclosure, the control unit (630) may be defined as a circuit or an application-specific integrated circuit or at least one processor.

[0207] At this time, the server may correspond to at least one of a computing server, an application server, or a data network configuration server.

[0208] The communication unit (610) can transmit and receive signals with other network entities. The communication unit (610) can transmit and receive information with, for example, another server through a specific interface.

[0209] The storage unit (620) can store at least one of the information transmitted and received through the communication unit (610) and the information generated through the control unit (630).

[0210] The control unit (630) can control the overall operation of the server according to an embodiment of the present disclosure. For example, the control unit (630) can control the signal flow between each block to perform operations according to the flowchart described above.

[0211] FIG. 7 illustrates the structure of a network entity according to an embodiment of the present disclosure.

[0212] A network entity according to one embodiment of the present disclosure may include a processor (720) that controls the overall operation of the network entity, a transceiver (700) including a transmitter and a receiver, and a memory (710). Of course, it is not limited to the above examples, and the network entity may include more or fewer configurations than the configuration shown in FIG. 7.

[0213] According to one embodiment of the present disclosure, the transmitting and receiving unit (700) can transmit and receive a signal with at least one of other network entities or terminals. The signal transmitted and received with at least one of other network entities or terminals may include control information and data.

[0214] According to one embodiment of the present disclosure, the processor (720) can control a network entity to perform any one of the above-described embodiments. Meanwhile, the processor (720), memory (710), and transceiver (700) do not necessarily have to be implemented as separate modules, and can be implemented as a single component in the form of a single chip. Also, the processor (720) and the transceiver (700) can be electrically connected. Additionally, the processor (720) may be an Application Processor (AP), a Communication Processor (CP), a circuit, an application-specific circuit, or at least one processor.

[0215] According to one embodiment of the present disclosure, the memory (710) may store data such as a basic program, an application program, and configuration information for the operation of a network entity. In particular, the memory (710) provides the stored data upon request from the processor (720). The memory (710) may be composed of a storage medium or a combination of storage media such as ROM, RAM, a hard disk, a CD-ROM, and a DVD. Additionally, the memory (710) may be a plurality of. Furthermore, the processor (720) may perform the aforementioned embodiments based on a program for performing the aforementioned embodiments of the present disclosure stored in the memory (710).

[0216] Meanwhile, the embodiments of the present disclosure disclosed in this specification and drawings are merely specific examples provided to facilitate the explanation of the technical content of the present disclosure and to aid in understanding the present disclosure, and are not intended to limit the scope of the present disclosure. That is, it is obvious to those skilled in the art that other variations based on the technical concept of the present disclosure are possible. Furthermore, each of the above embodiments may be combined and operated together as needed.

Claims

1. A method performed by a terminal in a wireless communication system, A step of receiving first information including a Media Energy Index (MEIX) that can be provided by a server from a network entity; A step of identifying a list of at least one MEIX that is a monitoring target of the terminal based on second information including a media energy index that can be collected from the terminal and the first information; A step of obtaining MEIX information for the list of at least one MEIX and a first media; A step of outputting the first media; A step of updating an energy consumption model based on the output of the first media and MEIX information for the first media; and A method comprising the step of switching from the first medium to the second medium based on the above-mentioned updated energy consumption model.

2. In Paragraph 1, The step of receiving the above-mentioned first information is, A step of transmitting a service discovery request to the above network entity; and A method comprising the step of receiving the first information from the network entity in response to the above service discovery request.

3. In Paragraph 1, A method in which the second information is identified by the energy status monitor (ESM) of the terminal.

4. In Paragraph 1, The step of identifying a list of at least one MEIX that is the monitoring target above is, A method comprising the step of identifying at least one common media energy index of the media energy index included in the first information and the media energy index included in the second information.

5. In Paragraph 1, The step of obtaining MEIX information for the list of at least one MEIX above is, A step of transmitting a list of at least one MEIX to the above network entity; A step of receiving information about a path for obtaining MEIX information from the above network entity; and A method comprising the step of obtaining MEIX information from an application server (AS) according to information about the path.

6. In Paragraph 1, The above energy consumption model includes a correlation between the amount of energy change of the terminal according to the output of the first media and the MEIX information.

7. In Paragraph 1, The step of updating the above energy consumption model is, In the ESM of the terminal, a step of identifying the first energy state of the terminal prior to the output of the first media; In the above ESM, a step of identifying a second energy state of the terminal according to the output of at least a portion of the first media; A step of identifying the amount of energy change of the terminal based on the first energy state and the second energy state; and A method comprising the step of updating the energy consumption model based on media energy index information corresponding to the energy change amount.

8. In a method performed by a network entity in a wireless communication system, A step of receiving a service discovery request from a terminal; A step of transmitting to the terminal, based on the service discovery request, first information including a Media Energy Index (MEIX) that can be provided by a server; A step of receiving at least one MEIX list from the terminal that is a monitoring target of the terminal; A step of receiving information about a path for obtaining MEIX information corresponding to at least one MEIX list from an application server (AS); and A method comprising the step of transmitting information about a path for obtaining MEIX information to the above terminal.

9. In a terminal in a wireless communication system, At least one transceiver; At least one processor communicatively coupled to the above at least one transceiver; and It includes at least one memory that is communicationally coupled to the above at least one processor and stores instructions, and The above instructions are executed individually or in any combination by the above at least one processor, so that the terminal: Receive first information from a network entity, including a Media Energy Index (MEIX) that can be provided by a server, and Based on second information including a media energy index that can be collected from the terminal and the first information, a list of at least one MEIX that is a monitoring target of the terminal is identified, and Obtaining MEIX information for the list of at least one MEIX and the first media, and Output the above-mentioned first media, and Update the energy consumption model based on the output of the first media and MEIX information for the first media, and A terminal that switches from the first media to the second media based on the above-mentioned updated energy consumption model.

10. In paragraph 9, the above commands are the terminal: Send a service discovery request to the above network entity, and A terminal that receives the first information from the network entity in response to the above service discovery request.

11. In paragraph 9, the above commands are the terminal: The above second information is a terminal identified by the energy status monitor (ESM) of the terminal.

12. In paragraph 9, the above commands are the terminal: Transmit a list of at least one MEIX to the above network entity, and From the above network entity, information regarding a path for obtaining the MEIX information is received, and A terminal that obtains MEIX information from an application server (AS) according to information about the path.

13. In Paragraph 9, The above energy consumption model includes a correlation between the amount of energy change of the terminal according to the output of the first media and the MEXI information, for the terminal.

14. In paragraph 9, the above commands are the terminal: In the ESM of the above terminal, the first energy state of the terminal prior to the output of the first media is identified, and In the above ESM, a second energy state of the terminal according to the output of at least a portion of the first media is identified, and Identifying the amount of energy change of the terminal based on the first energy state and the second energy state, and A terminal that updates the energy consumption model based on media energy index information corresponding to the above energy change amount.

15. Regarding network entities in wireless communication systems: At least one processor; and It includes at least one memory that is communicationally coupled to the above at least one processor and stores instructions, and The above instructions are executed individually or in any combination by the above at least one processor, so that the network entity: Upon receiving a service discovery request from the terminal, To the above terminal, based on the service discovery request, first information including a Media Energy Index (MEIX) that can be provided by the server is transmitted, and From the above terminal, at least one MEIX list that is a monitoring target of the above terminal is received, and Receive information about a path for obtaining MEIX information corresponding to at least one MEIX list from an application server (AS), and A network entity that transmits information about a path for obtaining the MEIX information to the above terminal.

Images