Image content transmission method and device using edge computing service

Abstract

The present disclosure relates to an image content transmission method and device using an edge computing service (e.g., a multi-access edge computing (MEC) service), and the image content transmission method according to an embodiment may include: obtaining sensor information, the sensor information including orientation information detected by an electronic device connected to an edge data network; obtaining status information, the status information including network status information between the edge data network and the electronic device; identifying a size and a position of a first partial image based on the sensor information and the status information; generating a first frame by encoding the first partial image based on the identified size and position of the first partial image; and sending the generated first frame to the electronic device.

Description

Technical Field

[0001] The present disclosure relates to an image content transmission method and apparatus using an edge computing service (eg, a multi-access edge computing (MEC) service). Background Art

[0002] Recently, edge computing technology that uses edge servers to transmit data has been discussed. Edge computing technology may include, for example, multi-access edge computing (MEC) or fog computing (FOC). Edge computing technology is a technology that provides data to electronic devices through a separate server (hereinafter referred to as an "edge data network" or "MEC server") provided at a location geographically close to the electronic device (e.g., inside or around a base station). For example, an application requiring low latency in at least one application installed in an electronic device can send and receive data through an edge server set at a geographically close location without passing through a server located in an external data network (DN) (e.g., the Internet).

[0003] Recently, services using edge computing technology (hereinafter referred to as "MEC-based services" or "MEC services") have been discussed, and research and development of electronic devices that support MEC-based services has been conducted. For example, an application of an electronic device can send and receive edge computing-based data to and from an edge server (or an application of an edge server) at the application layer.

[0004] As research and development to support MEC-based services progress, technology for providing high-resolution image content to electronic devices by using MEC is being discussed. Summary of the invention

[0005] Technical issues

[0006] Based on the above discussion, the present disclosure relates to an image content transmission method and device using edge computing services.

[0007] More specifically, the present disclosure provides a method for transmitting image content to an electronic device performed by an edge data network, and an edge data network for providing image content to an electronic device.

[0008] In addition, the present disclosure provides a method for receiving image content from an edge data network performed by an electronic device, and an electronic device for receiving image content from an edge data network.

[0009] Technical Solution

[0010] According to an embodiment, a method for sending image content performed by an edge data network includes: obtaining sensor information, the sensor information including orientation information detected by an electronic device connected to the edge data network; obtaining status information, the status information including network status information between the edge data network and the electronic device; based on the sensor information and the status information, identifying the size and position of a first partial image; generating a first frame by encoding the first partial image based on the identified size and position of the first partial image; and sending the generated first frame to the electronic device.

[0011] Herein, the sensor information may also include rotation speed information, the status information may also include connected device status information, and the method may further include: identifying the bit rate or the number of frames per unit time of the first local image based on the sensor information and the status information, and generating the first frame by encoding the first local image may include generating the first frame by encoding the first local image based on at least one of the size, position, bit rate, and number of frames per unit time of the identified first local image.

[0012] The first partial image may be a partial image in a first virtual reality (VR) image having a specific frame index of a VR sequence including a plurality of frames; and the first partial image may be associated with orientation information indicating a position of the first partial image.

[0013] Identifying at least one of the size, position, bit rate, and number of frames per unit time of the first local image based on the sensor information and the status information may include at least one of the following: in a case where the value of at least one state parameter related to at least one of at least one network state, the state of the edge data network, and the state of the electronic device included in the status information is less than or equal to at least one first threshold, identifying the size and number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold; and in a case where the value of at least one state parameter included in the status information is greater than at least one second threshold, identifying the size and number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold, wherein the size of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be greater than the size of the first local image corresponding to at least one state parameter greater than the second threshold, and the number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be less than the number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold.

[0014] Identifying at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on the sensor information and the state information may include at least one of the following: in a case where the value of at least one state parameter included in the state information and related to at least one of at least one network state, the state of the edge data network, and the state of the electronic device is less than or equal to at least one first threshold, identifying the size of the first partial image, the number of frames per unit time of the first partial image, and the bit rate of the first partial image corresponding to the at least one state parameter less than or equal to the first threshold; and in a case where the value of at least one state parameter included in the state information is greater than at least one second threshold, identifying the size of the first partial image, the number of frames per unit time of the first partial image, and the bit rate of the first partial image corresponding to the at least one state parameter greater than the second threshold. The size of a local image, the number of frames per unit time of the first local image and the bit rate of the first local image, wherein the size of the first local image corresponding to at least one state parameter less than or equal to a first threshold may be larger than the size of the first local image corresponding to at least one state parameter greater than a second threshold, the number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be smaller than the number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold, and the bit rate of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be smaller than the bit rate of the first local image corresponding to at least one state parameter greater than the second threshold.

[0015] Identifying at least one of the size, position, bit rate and number of frames per unit time of the first local image based on the sensor information and the status information may include at least one of the following: in a case where the rotation speed of the rotation speed information included in the sensor information is greater than a first threshold, identifying the size of the first local image corresponding to the rotation speed greater than the first threshold; and in a case where the rotation speed of the rotation speed information is less than or equal to a second threshold, identifying the size of the first local image corresponding to the rotation speed less than or equal to the second threshold, wherein the size of the first local image corresponding to the rotation speed greater than the first threshold may be greater than the size of the first local image corresponding to the rotation speed less than or equal to the second threshold.

[0016] Identifying at least one of the size, position, bit rate and number of frames per unit time of the first local image based on the sensor information and the status information may include at least one of the following: in a case where the rotation speed of the rotation speed information is greater than a first threshold, identifying the size of the first local image corresponding to the rotation speed greater than the first threshold and the number of frames per unit time of the first local image; and in a case where the rotation speed of the rotation speed information is less than or equal to a second threshold, identifying the size of the first local image corresponding to the rotation speed greater than the second threshold and the number of frames per unit time of the first local image, wherein the size of the first local image corresponding to the rotation speed greater than the first threshold may be greater than the size of the first local image corresponding to the rotation speed less than or equal to the second threshold, and the number of frames per unit time of the first local image corresponding to the rotation speed greater than the first threshold may be less than the number of frames per unit time of the first local image corresponding to the rotation speed less than or equal to the second threshold.

[0017] Identifying at least one of the size, position, bit rate and number of frames per unit time of the first local image based on sensor information and status information may include at least one of the following: identifying the size of the first local image corresponding to a rotation speed greater than a first threshold, the number of frames per unit time of the first local image and the bit rate of the first local image; and identifying the size of the first local image corresponding to a rotation speed greater than a second threshold, the number of frames per unit time of the first local image and the bit rate of the first local image, wherein the size of the first local image corresponding to the rotation speed greater than the first threshold may be greater than the size of the first local image corresponding to the rotation speed less than or equal to the second threshold, the number of frames per unit time of the first local image corresponding to the rotation speed greater than the first threshold may be less than the number of frames per unit time of the first local image corresponding to the rotation speed less than or equal to the second threshold, and the bit rate of the first local image corresponding to the rotation speed greater than the first threshold may be less than the bit rate of the first local image corresponding to the rotation speed less than or equal to the second threshold.

[0018] The first partial image may include a user field of view image corresponding to the orientation information and an additional field of view image corresponding to the number of frames per unit time.

[0019] The first partial image may include a user field of view image corresponding to the orientation information, and an additional field of view image corresponding to at least one of a rotation speed and a rotation direction of the rotation speed information.

[0020] The first partial image may include a user field of view image corresponding to the orientation information and an additional field of view image identified based on the number of frames per unit time and at least one of a rotation speed and a rotation direction of the rotation speed information.

[0021] The additional field of view image may include a first additional field of view image in a rotation direction relative to the field of view image and a second additional field of view image in a direction opposite to the rotation direction, and the first additional field of view image may be larger than the second additional field of view image.

[0022] The first partial image may be composed of data units of a preset field of view range, and generating a first frame by encoding the first partial image may include: encoding at least one first data unit of a preset field of view range corresponding to a user field of view image in the first partial image and at least one second data unit of a preset field of view range corresponding to an additional field of view image in the first partial image; and generating a first frame including the encoded first data unit and the encoded second data unit.

[0023] Identifying at least one of the size, position, bit rate and number of frames per unit time of the first local image based on the sensor information and the state information may include at least one of the following: when the value of at least one state parameter related to at least one of at least one network state, the state of the edge data network and the state of the electronic device included in the state information is less than or equal to at least one first threshold, and the rotation speed of the rotation speed information is greater than a third threshold, identifying the number of frames per unit time of the first local image corresponding to the at least one state parameter less than or equal to the first threshold and the bit rate of the first local image, and identifying the size of the first local image corresponding to the rotation speed greater than the third threshold; and when the value of at least one state parameter included in the state information is greater than a second threshold, and the rotation speed of the rotation speed information is less than or equal to a fourth threshold, identifying the number of frames per unit time of the first local image corresponding to the at least one state parameter greater than the second threshold and the bit rate of the first local image. bit rate, and identifying the size of a first local image corresponding to a rotation speed less than or equal to a fourth threshold; wherein the number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be less than the number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold, the bit rate of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be less than the bit rate of the first local image corresponding to at least one state parameter greater than the second threshold, the size of the first local image corresponding to a rotation speed greater than the third threshold may be greater than the size of the first local image corresponding to a rotation speed less than or equal to the fourth threshold, the first local image may include a field of view image corresponding to the orientation information and an additional field of view image corresponding to the rotation speed, and the additional field of view image may be identified based on the obtained field of view, based on the rotation speed, and from a lookup table (LUT) including a preset additional field of view.

[0024] According to an embodiment, an edge data network for sending image content to an electronic device includes: a network interface; a memory storing one or more instructions; a processor configured to execute one or more instructions to: obtain sensor information, the sensor information including orientation information detected by an electronic device connected to the edge data network; obtain status information, the status information including network status information between the edge data network and the electronic device; based on the sensor information and the status information, identify the size and position of a first partial image; generate a first frame by encoding the first partial image based on the identified size and position of the first partial image; and send the generated first frame to the electronic device.

[0025] The sensor information may also include rotation speed information, the status information may also include connected device status information, the processor may be further configured to execute one or more instructions to further identify the bit rate or the number of frames per unit time of the first partial image based on the sensor information and the status information, and generating the first frame by encoding the first partial image may include generating the first frame by encoding the first partial image based on at least one of the size, position, bit rate and number of frames per unit time of the identified first partial image.

[0026] The processor may be further configured to execute one or more instructions to, in identifying at least one of the size, position, bit rate, and number of frames per unit time of the first local image based on the sensor information and the status information, perform at least one of the following: in a case where the value of at least one state parameter related to at least one of at least one network state, the state of the edge data network, and the state of the electronic device included in the status information is less than or equal to at least one first threshold, identify the size and number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold; and in a case where the value of at least one state parameter included in the status information is greater than at least one second threshold, identify the size and number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold, wherein the size of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be greater than the size of the first local image corresponding to at least one state parameter greater than the second threshold, and the number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be less than the number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold.

[0027] The processor may be further configured to execute one or more instructions to, in identifying at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on the sensor information and the state information, perform at least one of the following: in a case where a value of at least one state parameter included in the state information and related to at least one of at least one network state, the state of the edge data network, and the state of the electronic device is less than or equal to at least one first threshold, identify the size, number of frames per unit time, and bit rate of the first partial image corresponding to at least one state parameter less than or equal to the first threshold; and in a case where a value of at least one state parameter included in the state information is greater than at least one second threshold, identify the size, number of frames per unit time, and bit rate of the first partial image corresponding to the state parameter greater than the second threshold. The size of the first local image corresponding to at least one state parameter and the number of frames per unit time of the first local image, wherein the size of the first local image corresponding to at least one state parameter less than or equal to a first threshold may be larger than the size of the first local image corresponding to at least one state parameter greater than a second threshold, the number of frames per unit time of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be smaller than the number of frames per unit time of the first local image corresponding to at least one state parameter greater than the second threshold, and the bit rate corresponding to at least one state parameter less than or equal to the first threshold may be greater than the bit rate of the first local image corresponding to at least one state parameter greater than the second threshold.

[0028] The processor may be further configured to execute one or more instructions to, in identifying at least one of the size, position, bit rate, and number of frames per unit time of the first local image based on the sensor information and the status information, perform at least one of the following: in a case where the rotation speed of the rotation speed information is greater than a first threshold, identify the size of the first local image corresponding to the rotation speed greater than the first threshold; and in a case where the rotation speed of the rotation speed information is less than or equal to a second threshold, identify the size of the first local image corresponding to the rotation speed less than or equal to the second threshold, wherein the size of the first local image corresponding to the rotation speed greater than the first threshold may be larger than the size of the first local image corresponding to the rotation speed less than or equal to the second threshold.

[0029] The processor may be further configured to execute one or more instructions to, in identifying at least one of the size, position, bit rate, and number of frames per unit time of the first local image based on the sensor information and the status information, perform at least one of the following: in a case where the rotation speed of the rotation speed information is greater than a first threshold, identify the size of the first local image corresponding to the rotation speed greater than the first threshold, the number of frames per unit time of the first local image, and the bit rate of the first local image; and in a case where the rotation speed of the rotation speed information is less than or equal to a second threshold, identify the size of the first local image corresponding to the rotation speed greater than the second threshold, the number of frames per unit time of the first local image, and the bit rate of the first local image, wherein the size of the first local image corresponding to the rotation speed greater than the first threshold may be greater than the size of the first local image corresponding to the rotation speed less than or equal to the second threshold, the number of frames per unit time of the first local image corresponding to the rotation speed greater than the first threshold may be less than the number of frames per unit time of the first local image corresponding to the rotation speed less than or equal to the second threshold, and the bit rate corresponding to the rotation speed greater than the first threshold may be greater than the bit rate of the first local image corresponding to the rotation speed less than or equal to the second threshold.

[0030] The first partial image may include a user field of view image corresponding to the orientation information and an additional field of view image corresponding to the number of frames per unit time.

[0031] According to an embodiment, a computer-readable recording medium is provided, on which a program for executing a method on a computer is recorded, the method being executed by an edge data network and comprising: obtaining sensor information, the sensor information comprising orientation information detected by an electronic device connected to the edge data network; obtaining status information, the status information comprising network status information between the edge data network and the electronic device; identifying a size and a position of a first local image based on the sensor information and the status information; generating a first frame by encoding the first local image based on the identified size and position of the first local image; and sending the generated first frame to the electronic device.

[0032] Beneficial Effects

[0033] The disclosed embodiments provide a method for efficiently performing operations between an edge data network and an electronic device based on image content. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a diagram schematically illustrating a multi-access edge computing (MEC) technology in a network environment according to an embodiment of the present disclosure.

[0035] Figure 2The invention shows a method of adaptively streaming an extended field of view (FoV) image including an additional FoV image in consideration of a user viewpoint (sight direction), a state parameter, and a rotation speed, performed by an edge data network 2000 according to an embodiment of the present disclosure.

[0036] Figure 3 is a flowchart illustrating an operation process between an electronic device and an edge data network according to an embodiment of the present disclosure.

[0037] Figure 4 is a diagram schematically illustrating an operation process between an electronic device and an edge data network.

[0038] Figure 5a is a flowchart illustrating a method of streaming image content performed by an edge data network according to an embodiment of the present disclosure.

[0039] Figure 5b is a diagram for describing in detail a method of identifying parameters of a frame generated by encoding a first partial image based on state information, performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0040] Figure 5c is a diagram for describing in detail a method of identifying parameters of a frame generated by encoding a first partial image based on rotation speed information, which is performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0041] Figure 5d is a diagram for describing in detail a method of identifying parameters of a frame generated by encoding a first partial image based on state information and rotation speed information, which is performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0042] Figure 6 is a flowchart illustrating a method for streaming image content obtained from an edge data network and performed by an electronic device according to an embodiment of the present disclosure.

[0043] Figure 7 is a flowchart illustrating an operation process between an electronic device and an edge data network according to an embodiment of the present disclosure.

[0044] Figure 8a is a diagram for describing a process of streaming a first partial image performed by the edge data network 2000 when at least one state parameter is greater than a first threshold according to an embodiment.

[0045] Figure 8b is a diagram for describing a process of streaming a first partial image performed by the edge data network 2000 when at least one state parameter is less than or equal to a first threshold according to an embodiment.

[0046] Figure 8c 2 is a diagram for describing a process of adaptively changing encoding generation frame parameters performed by the edge data network 2000 when at least one state parameter is less than or equal to a first threshold according to an embodiment.

[0047] Figure 8d is a diagram illustrating parameters of an encoding-generated frame being streamed if a state parameter is greater than, less than, or equal to a first threshold according to an embodiment.

[0048] Fig. 9 is a flowchart illustrating an operation process between an electronic device and an edge data network according to an embodiment of the present disclosure.

[0049] Fig.10a is a diagram for describing a process of streaming a first partial image performed by an edge data network 2000 when a rotation speed of an electronic device 1000 worn by a user is less than or equal to a first threshold according to an embodiment.

[0050] Fig.10b is a diagram for describing a process of streaming a first partial image performed by the edge data network 2000 when a rotation speed of an electronic device 1040 worn on a user's head is greater than a first threshold according to an embodiment.

[0051] Fig.11 is a diagram for describing a process of identifying left and right FoVs of a coding generation frame performed by the edge data network 2000 when a rotation speed is equal to a first value (ie, the rotation speed is high) according to an embodiment.

[0052] Fig.12 is a diagram for describing a process of identifying left and right FoVs performed by the edge data network 2000 when a rotation speed is equal to a second value (ie, the rotation speed is low) according to an embodiment.

[0053] Fig.13 is a diagram for describing a process of setting a frame per second (FPS) to a low value and then identifying left and right FoVs, performed by the edge data network 2000 , when a rotation speed is equal to a first value (ie, the rotation speed is high) according to an embodiment.

[0054] Fig.14 is a diagram for describing a process of identifying additional FoV areas based on object detection performed by the edge data network 2000 according to an embodiment.

[0055] Fig.15 is a diagram illustrating a process of transmitting a first partial image in a plurality of data units to an electronic device 1000 , which is performed by an edge data network 2000 according to an embodiment.

[0056] Fig.16 is a diagram schematically illustrating an operation process between an electronic device, an edge data network, and a virtual reality (VR) game interface device.

[0057] Fig.17 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0059] In the description of the embodiments, technical features that are well known in the technical field to which the present disclosure belongs but are not directly related to the present disclosure are not described. The purpose of omitting unnecessary descriptions is not to obscure but to clearly convey the gist of the present disclosure.

[0060] For the same reason, in the accompanying drawings, some elements are enlarged, omitted or schematically shown. In addition, the size of the element does not fully reflect its actual size. In the accompanying drawings, the same or corresponding elements have the same reference numerals.

[0061] The advantages and features of the present disclosure and methods for achieving them will become clear with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein, on the contrary, these embodiments are provided so that the present disclosure will be thorough and complete, and the concept of the present disclosure will be fully conveyed to those of ordinary skill in the art, and the present disclosure will only be limited by the concept of the claims. Throughout the specification, the same reference numerals represent the same elements.

[0062] It is understood herein that each block in the process flow chart and the combination of flow charts can be executed by computer program instructions. These computer program instructions can be loaded into a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the instructions executed by the processor of the computer or other programmable data processing device can generate a device configured to perform the functions described in the flow chart block. These computer program instructions can also be stored in a computer-usable or computer-readable memory, which can guide the computer or other programmable data processing device to implement the function in a specific mode, so that the instructions stored in the computer-usable or computer-readable memory can also generate a product item involving an instruction device configured to perform the functions described in the flow chart block. The computer program instructions can also be loaded into a computer or other programmable data processing device, so that a computer executable process can also be generated by executing a series of operating steps on a computer or other programmable data processing device, so that the instructions executed in the computer or other programmable data processing device provide steps for performing the functions described in the flow chart block.

[0063] In addition, each block can indicate a module, segment or part of a code, including one or more executable instructions for performing a specific logical function. In addition, in several alternative embodiments, the functions described in the blocks can also be out of order. For example, two consecutively displayed blocks can actually be executed substantially simultaneously, or blocks can sometimes be executed in reverse order according to the corresponding functions.

[0064] The term "... unit" used in the embodiment means a component including software or hardware, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and the "... unit" performs certain roles. However, the meaning of "... unit" is not always limited to software or hardware. "... unit" can be configured to be stored in an addressable storage medium or to execute one or more processors. Therefore, for example, "... unit" includes components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays and variables. The functions provided inside the components and "... units" can be combined into a smaller number of components and "... units", or further divided into additional components and "... units". In addition, components and "... units" can be implemented as one or more central processing units (CPUs) in a reproduction device or a secure multimedia card. In addition, in an embodiment, "... unit" can include one or more processors.

[0065] Figure 1 is a diagram schematically illustrating a multi-access edge computing (MEC) technology in a network environment according to an embodiment of the present disclosure.

[0066] refer to Figure 1 , the network environment 100 of the present disclosure may include an electronic device 1000, an edge data network 2000, a cloud server 3000, and an access network (AN) 1100. However, the components included in the network environment 100 are not limited thereto.

[0067] According to an embodiment, each component included in the network environment 100 may be a physical entity unit or a software or module unit configured to perform a separate function.

[0068] According to an embodiment, the electronic device 1000 may refer to a device used by a user. For example, the electronic device 1000 may refer to a terminal, a user equipment (UE), a mobile station, a subscriber station, a long-distance terminal, a wireless terminal, or a user device.

[0069] In addition, the electronic device 1000 may be a terminal for providing content so that a user is immersed in a virtual environment including at least one of virtual reality (VR), augmented reality (AR), or mixed reality (MR). That is, according to an embodiment, the electronic device 1000 may be a head mounted display (HMD) or a virtual reality headset (VRH) for providing content for VR, AR, or MR.

[0070] refer to Figure 1 , the electronic device 1000 may include a first application client (or application client) 122, a second application client 124, and an edge enabler client (or MEC enabling layer (MEL)) 130. The electronic device 1000 may use the MEC service by performing necessary operations using the edge enabler client 130. The edge enabler client 130 is described in detail below.

[0071] According to an embodiment, the electronic device 1000 may execute multiple applications. For example, the electronic device 1000 may execute a first application client 122 and a second application client 124. Multiple applications may require different network services based on at least one of the required data transmission rate, delay (or speed), reliability, the number of electronic devices that have accessed the network, the network access time of the electronic device 1000, or the average data usage. Different network services may include, for example, enhanced mobile broadband (eMBB), ultra-reliable and low-latency communication (URLLC), or massive machine type communication (mMTC).

[0072] The application client of the electronic device 1000 may refer to a default application pre-installed in the electronic device 1000 or an application provided by a third party. That is, the application client of the electronic device 1000 may refer to a client application for a specific application service running in the electronic device 1000. Multiple application clients may run on the electronic device 1000. At least one of the application clients may use the service provided by the edge data network 2000. For example, the application client may be an application installed in and executed by the electronic device 1000, and may provide a function of sending and receiving data via the edge data network 2000. The application client in the electronic device 1000 may refer to application software executed in the electronic device 1000 to use the functions provided by one or more specific edge applications.

[0073] According to an embodiment, multiple applications in the electronic device 1000, namely the first application client 122 and the second application client 124, can send and receive data with the cloud server 3000 based on the required network service type, or send and receive data with the edge data network 2000 based on edge computing. For example, in the case where the first application client 122 does not require low latency, the first application client 122 can send and receive data with the cloud server 3000. As another example, in the case where the second application client 124 requires low latency, the second application client 124 can send and receive data based on MEC with the edge data network 2000.

[0074] According to an embodiment, the application in the electronic device 1000 may be referred to as an application client, a client application (or a client app), or a UE application (or a UE app). For convenience, hereinafter in this disclosure, the application in the electronic device 1000 is referred to as an application client.

[0075] According to an embodiment, the AN 1100 may provide a channel for wireless communication with the electronic device 1000. For example, the AN 1100 may refer to a radio access network (RAN), a base station, an evolved Node B (eNodeB or eNB), a 5th generation (5G) node, a transmission / reception point (TRP), or a 5G NodeB (5GNB).

[0076] According to an embodiment, the edge data network 2000 may refer to a server that the electronic device 1000 accesses to use the MEC service. The edge data network 2000 may be set at a location geographically close to the electronic device 1000, for example, inside or around a base station. According to an embodiment, the edge data network 2000 may send and receive data to and from the electronic device 1000 without passing through an external data network (DN) (e.g., the Internet). In an embodiment, MEC may be referred to as multi-access edge computing or mobile edge computing.

[0077] According to an embodiment, the edge data network 2000 may be referred to as an MEC host, an edge computing server, a mobile edge host, an edge computing platform, an MEC server, etc. For convenience, in the following of this disclosure, the edge data network 2000 is referred to as an MEC server. Figure 1 , the edge data network 2000 may include a first edge application 142, a second edge application 144, and an edge enabling server (or MEC platform (MEP)) 146. The edge enabling server 146 provides MEC services or performs traffic control in the edge data network 2000, and the edge enabling server 146 is described in detail below.

[0078] According to an embodiment, the edge data network 2000 can execute multiple applications. For example, the edge data network 2000 can execute a first edge application 142 and a second edge application 144. According to an embodiment, an edge application may refer to an application provided by a third party in an edge data network that provides MEC services, or may be referred to as an edge application. The edge application can be used to establish a data session with an application client to send and receive data related to the application client. That is, the edge application can establish a data session with the application client. In an embodiment, a data session may refer to a communication path established for an application client in an electronic device 1000 and an edge application in an edge data network 2000 to send and receive data to each other.

[0079] According to an embodiment, the application in the edge data network 2000 may be referred to as an MEC application (or MEC app), an ME (MEC) app, an edge application server, and an edge application. For convenience, in the following of this disclosure, the application in the edge data network 2000 is referred to as an edge application. Although the term "application" is used herein, the edge application may refer to an application server present in the edge data network 2000.

[0080] According to an embodiment, the cloud server 3000 may provide content associated with an application. For example, the cloud server 3000 may be managed by a content provider. According to an embodiment, the cloud server 3000 may send and receive data to and from the electronic device 1000 via an external DN (eg, the Internet).

[0081] although Figure 1 1100 and the edge data network 2000. According to an embodiment, the DN may provide a service (e.g., an Internet service or an Internet Protocol Multimedia Subsystem (IMS) service) to the electronic device 1000 by sending and receiving data (or data packets) to and from the electronic device 1000 via the CN and the AN 1100. For example, the DN may be managed by a communication provider. In an embodiment, the edge data network 2000 may be connected to the AN 1100 or the CN via a DN (e.g., a local DN).

[0082] According to an embodiment of the present disclosure, when the first application client 122 or the second application client 124 is executed by the electronic device 1000, the electronic device 1000 can access the edge data network 2000 via the AN 1100 to send and receive data to execute the application client.

[0083] In the present disclosure, a method for streaming image content between the above-mentioned electronic device 1000, edge data network 2000 and cloud server 3000 may be provided. In more detail, a method for effectively providing VR, AR or MR content to a user based on user interaction information on an image reproduced by the electronic device 1000 is described. In the following, a method for streaming image content based on orientation information, state information and rotation speed information of image content reproduced by the electronic device 1000, which is performed by the edge data network 2000, is described.

[0084] Figure 2 The invention shows a method of adaptively streaming an extended field of view (FoV) image including an additional FoV image in consideration of a user viewpoint (sight direction), a state parameter, and a rotation speed, performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0085] refer to Figure 2 The electronic device 1000 may periodically send information about status information (status parameters) 210, a rotation speed 220 of the electronic device 1000, and a user's line of sight direction (viewpoint) 230 (e.g., orientation information) to the edge data network 2000 according to a request from the edge data network 2000 or according to a change in information value.

[0086] Here, the status information may include at least one of network status information and connection device status information. In addition, the network status information may include status parameter 210.

[0087] The state parameter 210 may include, but is not limited to, the throughput of a processing task such as data transmission and reception through a network connection, which is determined by considering the situation of the network, the situation of the edge data network 2000 , or the situation of the electronic device 1000 .

[0088] Throughput may refer to the amount of data per unit time that can be processed in an end-to-end manner (e.g., data-related tasks performed by the edge data network 2000, data transmission through the network, and processing of related tasks performed by the electronic device 1000). For example, throughput may refer to the size or speed per unit time of data transmitted through the network. Alternatively, throughput may refer to the size of data processed by the edge data network 2000 and the electronic device 1000, or the speed of such data processing. Throughput may be identified based on at least one evaluation metric (e.g., the number of packets, bytes, or bits received per unit time).

[0089] For example, the electronic device 1000 may measure the sum of the data sizes received by the receiving module in a unit time based on the transmission protocol, and send throughput information including the measurement value to the edge data network 2000. However, the present disclosure is not limited thereto, and those skilled in the art may understand that various modules may measure the throughput of relevant data and send information about the measured throughput to the edge data network 2000.

[0090] Meanwhile, the connection device state information may be information required for the operation (e.g., image streaming) of the edge data network 2000 and the electronic device 1000. For example, the connection device state information may include capability information of the electronic device 1000. For example, the capability information may be information indicating the operation-related capability of the electronic device 1000, for example, information about the viewport area size of the electronic device 1000.

[0091] In this case, the orientation information may include an angle value of the gaze of the user of the electronic device 1000 measured by a sensing module of the electronic device 1000. For example, the electronic device 1000 may perform sensing by using a sensing module (e.g., a three-axis tilt sensor) to obtain orientation information (e.g., roll, pitch, and yaw) of the gaze of the user viewing a specific partial image in the VR image, and the electronic device 1000 may send the obtained orientation information to the edge data network 2000, so as to share information about the portion of the VR image that the user of the electronic device is currently viewing with the edge data network 2000.

[0092] The rotation speed information is information about the rotation speed 220 of the head of the user using the electronic device, and may include information about the rotation speed and the rotation direction. For example, the electronic device 1000 may sense the instantaneous rotation speed of the electronic device by using a sensing module (e.g., a three-axis tilt sensor). In this case, the rotation speed may include information about the rotation speed of each angular component, such as the instantaneous angular speed of roll, pitch, and yaw.

[0093] The electronic device 1000 can share information about the current rotation direction and rotation speed of the head of the electronic device user (i.e., predicted gaze information about the point that the user will later look at) with the edge data network 2000 by sending the rotation speed information obtained by performing sensing to the edge data network 2000.

[0094] However, the present disclosure is not limited thereto, and the electronic device 1000 may obtain current orientation information by sensing using a sensing module (eg, a three-axis tilt sensor) and then obtain a rotation speed of the electronic device based on a difference between previously measured orientation information and current orientation information.

[0095] At the same time, when the previous orientation information of the electronic device 1000 is shared with the edge data network 2000, the electronic device 1000 may only send the current orientation information of the electronic device to the edge data network 2000 without sending the rotation speed information separately. The edge data network 2000 may receive the current orientation information of the electronic device and obtain the current rotation speed information of the electronic device based on the difference between the previously received orientation information of the electronic device and the current orientation information.

[0096] The edge data network 2000 may set the bit rate and frames per second (FPS) 240 of the image based on the information about the state parameter 210 and the rotation speed 220, identify the size and position of the user FoV image 250 based on the information about the user's sight direction (viewpoint) 230 (e.g., directional information), identify the size and position of the additional FoV image 255 based on the information about the state parameter and the rotation speed, encode the user FoV image 250 and the additional FoV image 255 based on the image encoding parameters (such as the bit rate and frames per second of the image), and transmit the encoded image to the electronic device 1000. Here, the term "FoV" refers to the degree to which the user's eyes can view a scene (e.g., the FoV can be expressed as an angle, but is not limited thereto). In the case of viewing an image with a larger FoV, the user's eyes can view a larger image area.

[0097] However, the extent to which the user can view an image through the display of the electronic device 1000 may be limited to the size of the viewport area of ​​the electronic device 1000. The size of the viewport area is the size of the FoV with which the image can be reproduced on the display of the electronic device 1000 (i.e., the size of the display area), and the size of the viewport area may be identified based on the capability information of the electronic device 1000. That is, the size of the viewport area is one of the hardware specifications of the display of the electronic device 1000, and the edge data network 2000 may receive the capability information of the electronic device 1000 from the electronic device 1000 via a network connection with the electronic device 1000, and identify the size of the viewport area of ​​the electronic device 1000 based on the capability information.

[0098] For example, when the size of the viewport area of ​​the electronic device 1000 in the horizontal direction is 110° and the size of the viewport area of ​​the electronic device 1000 in the vertical direction is 90°, the electronic device 1000 can send capability information of the electronic device 1000 including information about the size of the viewport area to the edge data network 2000 via a network connection, and the edge data network 2000 can identify that the size of the viewport area in the horizontal direction is 110° and the size of the viewport area in the vertical direction is 90° based on the capability information.

[0099] Therefore, the term “user FoV” (or “default FoV”) may refer to an area that a user may view through a viewport area of ​​the electronic device 1000 .

[0100] In a case where the orientation of the electronic device 1000 changes over time according to the movement of the user's head and thus the area of ​​the image viewable by the user changes, the user cannot view the entire area of ​​the image being viewed using the existing user FoV. Therefore, the term "additional FoV" may refer to a FoV obtained in consideration of this situation (e.g., upper, lower, left and right additional FoVs around the user FoV). This is because, in a case where the electronic device 1000 only decodes the encoded user FoV image, and the latest orientation information of the electronic device 1000 at the reproduction time point is different from the orientation information used for encoding, a portion of the image viewed by the user may not be correctly reconstructed, and therefore, the user may view black edges, thereby affecting the user's sense of immersion.

[0101] The FoV of the first partial image encoded to be displayed on the electronic device 1000 may be an extended FoV including the user FoV (default FoV) and the additional FoV. That is, the first partial image may be an extended FoV image including the user FoV image (default FoV image) and the additional FoV image. That is, through the transmitted additional FoV, the electronic device 1000 may perform decoding to obtain the additional FoV image as well as the user FoV image, identify the FoV image to be reproduced according to the image obtained in the decoding by using the latest orientation information of the electronic device 1000 at the reproduction time point, and reproduce the FoV image, thereby preventing the occurrence of black edges.

[0102] The additional FoV image may be an image that corresponds to a specific FoV in all directions that is set as a default based on the user FoV image. The present disclosure is not limited to this, and the additional FoV image may be an image corresponding to a FoV adaptively identified taking into account the state parameters and rotation speed of the electronic device 1000. By taking into account the state parameters and rotation speed of the electronic device 1000, when the network status (or the status of the connected device) is not good (for example, the throughput measured by the receiving module of the electronic device 1000 is low), or even in the worst case, such as when the rotation speed of the electronic device 1000 is high, it is possible to prevent the reproduction from being interrupted or prevent the user from viewing black edges when reproducing the image.

[0103] Here, the frames per second refers to the number of frames reproduced per second, and as the value increases, the image being reproduced will appear natural, while as the value decreases, the image being reproduced will appear unnatural and lagging.

[0104] The bit rate of an image is one of the parameters indicating the image quality, and may indicate, for example, the target number of bits (or bytes or packets) per second when encoding an image. For the same resolution (or size), an encoded image with a higher bit rate may have higher quality than an encoded image with a lower bit rate. The bit rate may be used as a parameter input to an encoding module to adjust the image quality of an image.

[0105] The resolution of an image refers to the number of pixels of a frame to be encoded, and for the same pixel size, as the resolution increases, the size and quality of the image increase. The resolution of an image is related to the size of the image, so it can be considered that when at least one of the size of the user FoV image and the size of the additional FoV image input to the encoding module is adjusted, the resolution of the image is changed.

[0106] For example, considering the size of the viewport area, the size of the user FoV image can be fixed, and the size of the additional FoV image can be identified based on the information of the state parameter and the rotation speed. Therefore, as the size of the additional FoV image increases, the resolution of the image increases.

[0107] For example, when the value of at least one of the state parameters (in the previous state) is greater than or equal to a preset threshold and (in the current state) is less than a preset threshold, in order to ensure that the frame is rendered (to ensure that the frame is reproduced), the edge data network 2000 can set the bit rate and the number of frames per second of the image to be less than the bit rate and the number of frames per second of the image in the previous state, and send the additional additional FoV image taking into account the changed bit rate and the number of frames per second. As the number of frames per second decreases, the interval between the displayed frames increases, resulting in an increase in the time period between the frames that the electronic device 1000 can move, and therefore, taking into account the time period that the electronic device 1000 can move, the size of the additional FoV image can be set to be larger.

[0108] Therefore, the motion-to-photon (MTP) latency can be continuously guaranteed without affecting the user's immersion.

[0109] MTP delay may refer to a time period during which the edge data network 2000 identifies the user's movement (e.g., movement of the user's gaze) based on sensor information (e.g., orientation information) obtained by a sensing module of the electronic device 1000 and sent to the edge data network 2000, the edge data network 2000 provides frames of a local image in a VR image based on the sensor information, the electronic device 1000 decodes and renders the provided frames of the local image, and then displays the rendered frames on a display.

[0110] That is, when encoding a high-quality image and transmitting the encoded image using a poor network state or a connection device state, the delay increases, and thus the transmission of the image is delayed, thereby affecting the user's immersion. Therefore, in order to reduce the delay, the quality of the image to be encoded can be reduced by reducing the bit rate of the image and reducing the number of frames per second. However, when an additional FoV image having the same size as that in the previous state is received, even if the number of frames per second has changed, the orientation information updated according to the latest orientation information of the electronic device 1000 at the reproduction time point is very different from the previous orientation information, and the probability of viewing a black edge increases. Therefore, the size of the additional FoV image can be set to minimize the black edge while taking the number of frames per second into consideration.

[0111] That is, until a subsequent frame (e.g., a second frame) is received based on the subsequent orientation information, the edge data network 2000 can obtain the latest user FoV image from the first partial image 200 obtained from the first frame based on the latest orientation information obtained from the sensing module. Therefore, even if the MTP delay until the time point when the second frame is received is significantly increased, the edge data network 2000 can prevent the user from seeing the black edge by performing local processing based on the additional FoV image so as to achieve sufficient offset within the time period required to receive the second frame, thereby substantially compensating for the MTP delay.

[0112] Alternatively, for example, when the rotation speed of the electronic device 1000 is greater than a preset threshold, the edge data network 2000 may transmit an additional additional FoV image.

[0113] That is, when the rotation speed of the electronic device 1000 is high, the difference between the orientation information at the reproduction time point and the orientation information used to encode the partial image becomes larger, and when the size of the additional FoV image is insufficient, the user is more likely to see black edges, which will hinder the user's immersion. In this case, the edge data network 2000 can minimize the appearance of black edges by setting the size of the additional FoV image to be very large. However, as the size of the additional FoV image increases, the data size obtained by encoding the additional FoV image increases, so the MTP delay increases and the bandwidth increases.

[0114] In order to reduce MTP delay and bandwidth, the bit rate of the image can be reduced. In addition, the number of frames per second of the image can be reduced. When the number of frames per second is reduced, considering that as the interval between the frames to be displayed increases, the time period in which movement may occur before the reproduction time point will also increase, the size of the additional FoV image can be additionally set to be large to ensure MTP delay. In addition, because the user's subsequent viewpoint can be predicted based on the rotation speed of the electronic device 1000, the possibility of black edges can be minimized by ensuring additional FoV images extending in the rotation direction, and the bandwidth and MTP delay can be ensured by appropriately increasing the data size of the image to be encoded.

[0115] Figure 3 is a flowchart illustrating an operation process between an electronic device and an edge data network according to an embodiment of the present disclosure.

[0116] refer to Figure 3 , in operation S300, the edge data network 2000 may establish a network connection with the electronic device 1000, and may share with the electronic device 1000 connection device status information required for the operation of the edge data network 2000 and the electronic device 1000. In this case, the status information may be shared only initially, periodically, or aperiodically only when a request is received or when the status information changes.

[0117] For example, the edge data network 2000 may receive capability information of the electronic device 1000 from the electronic device 1000. For example, the capability information may be information indicating operation-related capabilities of the electronic device 1000, such as information about the size of a viewport area of ​​the electronic device 1000.

[0118] The present disclosure is not limited thereto, and the edge data network 2000 may receive throughput information from the electronic device 1000. For example, the throughput information may be information about the amount of data received by the receiving module per unit time. Alternatively, the edge data network 2000 may obtain delay information from the electronic device 1000.

[0119] For example, the edge data network 2000 may send certain data according to the request of the electronic device 1000, the electronic device 1000 may receive the data, and at this time, the receiving module of the electronic device 1000 may measure the network delay, and the electronic device 1000 may send information about the network delay to the edge data network 2000.

[0120] In operation S302, the electronic device 1000 may obtain sensor information including at least one of orientation information and rotation speed information. The orientation information may be information indicating an angle component. The rotation speed information may be obtained directly from the sensing module, or may be obtained based on the difference between the orientation information at the current time point obtained from the sensing module and the orientation information at the previous time point. In operation S304, the electronic device 1000 may send sensor information including at least one of the orientation information and the rotation speed information to the edge data network 2000. The electronic device 1000 may periodically send the sensor information to the edge data network 2000, but is not limited thereto, and may send the sensor information to the edge data network 2000 non-periodically. For example, the electronic device 1000 may send the sensor information only when the edge data network 2000 requests it or when the sensor information value changes.

[0121] In operation S306, the edge data network 2000 may identify the user FoV image based on the orientation information. For example, the edge data network 2000 may identify the location of the user FoV image based on the received orientation information.

[0122] In operation S308, the edge data network 2000 may identify at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on at least one of the rotation speed information and the status information, the status information including at least one of the network status information and the connection device status information. In this case, since the reference coordinate position of the first partial image corresponds to the orientation information, and the reference coordinate position of the user FoV image also corresponds to the orientation information, the reference coordinate position of the first partial image (e.g., the coordinate position of the center of the image) may be the same as the reference coordinate position of the user FoV image (e.g., the coordinate position of the center of the image), but is not limited thereto. For example, in a case where the reference coordinate position of the image indicates the upper left position of the image, the reference coordinate position of the first partial image and the reference coordinate position of the user FoV image may be different from each other.

[0123] In the case where the edge data network 2000 does not separately receive the rotation speed information from the electronic device 1000 , the rotation speed information may be obtained based on a difference between currently obtained orientation information and previously obtained orientation information.

[0124] For example, in the case where the value of at least one state parameter related to at least one of the network state and the connection device state included in the state information is less than or equal to the first threshold, the edge data network 2000 can identify the size and number of frames of the first partial image corresponding to the at least one state parameter less than or equal to the first threshold. In this case, the first threshold may be a value set assuming that at least one state parameter before the current time point is greater than the first threshold.

[0125] In the case where the value of at least one state parameter included in the state information is greater than the second threshold, the edge data network 2000 can identify the size of the first partial image corresponding to the at least one state parameter greater than the second threshold and the number of frames per unit time. In this case, the second threshold may be a value set assuming that at least one state parameter before the current time point is less than or equal to the second threshold. The first threshold may be equal to the second threshold, but is not limited thereto.

[0126] In this case, the size of the first partial image corresponding to at least one state parameter less than or equal to the first threshold may be larger than the size of the partial image corresponding to at least one state parameter greater than the second threshold.

[0127] In addition, the number of frames per unit time of the first partial image corresponding to at least one state parameter less than or equal to the first threshold may be smaller than the number of frames per unit time of the first partial image corresponding to at least one state parameter greater than the second threshold.

[0128] In this case, the first partial image may include a user FoV image corresponding to the orientation information and an additional FoV image corresponding to the number of frames per unit time. The user FoV image may be obtained based on information about the size of the viewport area of ​​the electronic device 1000. In addition, the additional FoV image may correspond to the number of frames per unit time, and the size of the first additional FoV image when the number of frames per unit time is a first value (e.g., 60fps) may be smaller than the size of the second additional FoV image when the number of frames per unit time is a second value (e.g., 30fps), the second value being smaller than the first value.

[0129] Meanwhile, the edge data network 2000 may additionally identify a bit rate of the first partial image based on the status information.

[0130] In a case where the value of the at least one state parameter is less than or equal to the first threshold, the edge data network 2000 may identify a bit rate of the first partial image corresponding to the at least one state parameter less than or equal to the first threshold.

[0131] In a case where the value of the at least one state parameter is greater than the second threshold, the edge data network 2000 may identify a bit rate of the first partial image corresponding to the at least one state parameter greater than the second threshold.

[0132] A bit rate of a first partial image corresponding to at least one state parameter less than or equal to a first threshold may be lower than a bit rate of a partial image corresponding to at least one state parameter greater than a second threshold.

[0133] In a case where the rotation speed of the rotation speed information is greater than a first threshold, the edge data network 2000 may identify a size of a first partial image corresponding to a rotation speed greater than the first threshold.

[0134] In addition, in a case where the rotation speed of the rotation speed information is less than or equal to the second threshold, the edge data network 2000 may identify the size of the first partial image corresponding to the rotation speed less than or equal to the second threshold.

[0135] In this case, the size of the first partial image corresponding to the rotation speed greater than the first threshold may be greater than the size of the first partial image corresponding to the rotation speed less than or equal to the second threshold.

[0136] According to an embodiment, the first partial image may include a (user) FoV image corresponding to the orientation information, and an additional FoV image corresponding to at least one of a rotation speed and a rotation direction of the rotation speed information.

[0137] Alternatively, according to an embodiment, the first partial image may include a FoV image corresponding to the orientation information, and an additional FoV image identified based on at least one of the rotation speed and the rotation direction of the rotation speed information and the number of frames per unit time. In this case, the number of frames per unit time may be identified based on the rotation speed, but is not limited thereto.

[0138] Meanwhile, the additional FoV image may include a first additional FoV image in a rotation direction relative to the FoV image and a second additional FoV image in a direction opposite to the rotation direction, and the first additional FoV image may be larger than the second additional FoV image. That is, since the size of the FoV image in the rotation direction relative to the FoV image is recognized as larger than the size of the FoV image in the opposite direction (that is, the FoV images are recognized as being asymmetric to each other), black edges do not appear when the additional FoV is fully ensured.

[0139] Based on the rotation speed information, the edge data network 2000 may identify an additional FoV preset in a lookup table (LUT), and identify an additional FoV image based on the identified FoV.

[0140] According to another embodiment, the edge data network 2000 may further identify the number of frames per unit time and the size of the first partial image according to the rotation speed.

[0141] In a case where the rotation speed of the rotation speed information is greater than a first threshold, the edge data network 2000 may identify a size and a number of frames per unit time of a first partial image corresponding to the rotation speed greater than the first threshold.

[0142] In a case where the rotation speed of the rotation speed information is less than or equal to the second threshold, the edge data network 2000 may identify a size and a number of frames per unit time of a first partial image corresponding to the rotation speed less than or equal to the second threshold.

[0143] In this case, the size of the first partial image corresponding to the rotation speed greater than the first threshold value may be greater than the size of the first partial image corresponding to the rotation speed less than or equal to the second threshold value. In addition, the number of frames per unit time of the first partial image corresponding to the rotation speed greater than the first threshold value may be greater than the number of frames per unit time of the first partial image corresponding to the rotation speed less than or equal to the second threshold value.

[0144] According to another embodiment, the edge data network 2000 may identify a bit rate of the first image according to a rotation speed.

[0145] In a case where the rotation speed is greater than the first threshold, the edge data network 2000 may identify a bit rate of the first image corresponding to the rotation speed greater than the first threshold.

[0146] Furthermore, in a case where the rotation speed of the rotation speed information is less than or equal to the second threshold, the edge data network 2000 may identify a bit rate of the first image corresponding to the rotation speed less than or equal to the second threshold.

[0147] In this case, a bit rate of the first partial image corresponding to a rotation speed greater than a first threshold may be lower than a bit rate of the first partial image corresponding to a rotation speed less than or equal to a second threshold.

[0148] According to an embodiment, the edge data network 2000 may identify at least one of a size, a position, a bit rate, and a number of frames per unit time of the first partial image by integrally considering the state information and the rotation speed information.

[0149] For example, when the value of at least one state parameter included in the state information is less than or equal to the first threshold, and the rotation speed of the rotation speed information is greater than a third threshold, the edge data network 2000 can identify the number of frames per unit time and the bit rate of the first partial image corresponding to the at least one state parameter less than or equal to the first threshold. In addition, the edge data network 2000 can identify the size of the first partial image corresponding to the rotation speed greater than the third threshold.

[0150] When the value of at least one state parameter is greater than the second threshold and the rotation speed of the rotation speed information is less than or equal to the fourth threshold, the edge data network 2000 can identify the number of frames and the bit rate per unit time corresponding to the at least one parameter greater than the second threshold. In addition, the edge data network 2000 can identify the size of the first partial image corresponding to the rotation speed less than or equal to the fourth threshold.

[0151] In this case, the number of frames per unit time and the bit rate of the first local image corresponding to at least one state parameter less than or equal to the first threshold may be smaller than the number of frames per unit time and the bit rate of the first local image corresponding to at least one state parameter greater than the second threshold.

[0152] In this case, the size of the first partial image corresponding to the rotation speed greater than the third threshold may be greater than the size of the first partial image corresponding to the rotation speed less than or equal to the fourth threshold.

[0153] Although it is described above that the edge data network 2000 identifies the bit rate and the number of frames per unit time of the first local image corresponding to the status information, and identifies the size of the first local image corresponding to the rotation speed information, this is only an example and the present disclosure is not limited to this. It can be understood that at least one of the size, position, bit rate and the number of frames per unit time of the first local image corresponding to the status information and the rotation speed information can be identified by considering the status information and the rotation speed information as a whole.

[0154] In operation S310 , the edge data network 2000 may generate a first frame by encoding the first partial image based on at least one of a size, a position, a bit rate, and a number of frames per unit time of the first partial image identified in operation S308 .

[0155] In operation S312 , the edge data network 2000 may transmit the first frame generated in operation S310 to the electronic device 1000 .

[0156] In operation S314, the electronic device 1000 may reconstruct the first partial image by decoding the first frame received from the edge data network 2000. In this case, the reconstructed first partial image may include an additional FoV image.

[0157] In operation S316, the electronic device 1000 may reproduce the reconstructed first partial image. In this case, a partial image of the reconstructed first partial image may be obtained based on the orientation information at the reproduction time point, and the obtained partial image may be reproduced.

[0158] Figure 4is a diagram schematically illustrating an operation process between an electronic device and an edge data network.

[0159] According to an embodiment, the electronic device 1000 may include a sensing module 410, a network interface 420, a processor 430, and a memory 440. However, components in the electronic device are not limited thereto, and the electronic device may include more or less components.

[0160] The electronic device 1000 may decode an image received from the edge data network 2000 or the cloud server 3000, and display the decoded image on a display of the electronic device 1000. In addition, the electronic device 1000 may obtain sensor information including at least one of orientation information and rotation speed information about the reproduced image by using the sensing module 410. The orientation information may include an angle value of the gaze of the user of the electronic device measured by using the sensing module. The rotation speed information may include an instantaneous change amount of the angle value of the gaze of the user of the electronic device measured by using the sensing module.

[0161] The electronic device 1000 transmits sensor information including at least one of the orientation information and the rotation speed information 452 to the edge data network 2000 by using the network interface 420. In this case, the electronic device 1000 may transmit frame index information obtained when the orientation information and the rotation speed information are sensed to the edge data network 2000. However, the present disclosure is not limited thereto, and the reference frame information in the reconstructed frame may be transmitted to the edge data network 2000 before the orientation information and the rotation speed information are sensed. Here, the frame index may be information indicating the encoding / decoding order of the frames, but is not limited thereto, and may be information indicating the rendering order of the frames.

[0162] The processor 430 controls the overall operation of the electronic device 1000 by executing one or more instructions in the memory 440. For example, the processor 430 can control the sensing module 410 and the network interface 420 by executing one or more instructions stored in the memory 440. According to an embodiment, the processor 430 can obtain a first frame generated by encoding the first partial image according to at least one of the size, position, bit rate, and number of frames per unit time of the first partial image, wherein the size, position, bit rate, and number of frames per unit time of the first partial image are identified based on at least one of the state information and the rotation speed information. In this case, the first frame can be an intra-frame coded frame (I frame) or a predictive coded frame (P frame) (or a bidirectional predictive frame (B frame)).

[0163] In addition, according to an embodiment, the processor 430 may receive a frame 454 generated by performing encoding from the edge data network 2000 , and reconstruct the first partial image by decoding the frame 454 .

[0164] The processor 430 may obtain orientation information at a reproduced time point, and may reproduce a portion of the reconstructed first partial image based on the obtained orientation information.

[0165] According to an embodiment, the memory 440 may include a decoder module 442 storing instructions for decoding a frame generated by encoding and then received from the edge data network 2000 , but is not limited thereto.

[0166] According to an embodiment, the edge data network 2000 may include a network interface 460, a processor 470, and a memory 480. However, the components of the edge data network 2000 are not limited thereto, and the edge data network 2000 may include more components, or some components may be omitted therefrom.

[0167] The edge data network 2000 may obtain sensor information including orientation information and rotation speed information from the electronic device 1000 by using the network interface 460 , and transmit a frame generated by the edge data network 2000 through encoding based on the sensor information and the like to the electronic device 1000 .

[0168] Processor 470 controls the overall operation of edge data network 2000 by executing one or more instructions in memory 480 .

[0169] For example, the processor 470 may identify the parameters of the frame based on at least one of the state information and the rotation speed information. In detail, the processor 470 may identify at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on at least one of the state information and the rotation speed information. Figure 4 Description has been made, and thus a detailed description thereof is omitted.

[0170] The processor 470 may generate the first frame by obtaining the first partial image corresponding to the orientation information and encoding the same based on at least one of the size of the first partial image, the bit rate, and the number of frames per unit time.

[0171] The processor 470 may transmit the first frame to the electronic device 1000 .

[0172] According to an embodiment, the memory 480 may include an encoder module 482 and an encoding generation frame parameter identification module 484, wherein the encoder module 482 stores instructions for encoding an image sent by the edge data network 2000 to the electronic device 1000, and the encoding generation frame parameter identification module 484 stores parameters for identifying frames generated by performing encoding, and may store a VR sequence 486 including the entire image data for the VR sequence, but is not limited thereto.

[0173] For example, the edge data network 2000 may store a VR sequence including all frames in a database (DB). The edge data network 2000 may recognize a user FoV image, etc. in the VR sequence stored in the DB by using the orientation information obtained from the electronic device 1000 .

[0174] Figure 5a is a flowchart illustrating a method of streaming image content performed by an edge data network according to an embodiment of the present disclosure.

[0175] refer to Figure 5a In operation S502, the edge data network 2000 may obtain sensor information including orientation information from the electronic device. The sensor information may also include rotation speed information.

[0176] In operation S503, the edge data network 2000 may obtain status information including network status information between the edge data network 2000 and the electronic device 1000. The status information may also include connection device status information.

[0177] In operation S504, the edge data network 2000 may identify the size and position of the first partial image based on the sensor information and the state information. The edge data network 2000 may identify at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on the sensor information and the state information. Figure 3 Described.

[0178] In operation S506 , the edge data network 2000 may generate a first frame by encoding a first partial image corresponding to the orientation information based on at least one of a size, a position, a bit rate, and a number of frames per unit time of the first partial image.

[0179] In operation S508 , the edge data network 2000 may transmit the first frame to the electronic device 1000 .

[0180] Figure 5b is a diagram for describing in detail a method of identifying parameters of a frame generated by encoding a first partial image based on state information, performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0181] refer to Figure 5b , in operation S510, the edge data network 2000 may identify whether a value of at least one state parameter related to at least one of a network state and a connection device state included in the state information is less than or equal to a first threshold.

[0182] In operation S512, in a case where the value of at least one state parameter included in the state information is less than or equal to a first threshold, the edge data network 2000 may identify at least one of the size, the number of frames per unit time, and the bit rate of the first partial image corresponding to the at least one state parameter less than or equal to the first threshold. In this case, the first threshold may be a value set assuming that the value of at least one previous state parameter is greater than the first threshold, and therefore, here, the size of the identified first partial image may be greater than the size of the first partial image of the previous state. In addition, the number of frames per unit time of the identified first partial image may be less than the number of frames per unit time of the first partial image of the previous state. The bit rate of the identified first partial image may be less than the bit rate of the first partial image of the previous state.

[0183] Meanwhile, in operation S514, the edge data network 2000 may identify whether a value of at least one state parameter related to at least one of the network state and the connection device state included in the state information is greater than a second threshold value.

[0184] In operation S516, when the value of the at least one state parameter is greater than the second threshold, the edge data network 2000 may identify at least one of a size, a number of frames per unit time, and a bit rate of a first partial image corresponding to the at least one state parameter greater than the second threshold.

[0185] In this case, the second threshold value may be a value set assuming that the value of at least one previous state parameter is less than or equal to the second threshold value, and therefore, here, the size of the identified first partial image may be smaller than the size of the first partial image of the previous state. In addition, the number of frames per unit time of the identified first partial image may be greater than the number of frames per unit time of the first partial image of the previous state. The bit rate of the identified first partial image may be greater than the bit rate of the first partial image of the previous state.

[0186] In operation S518, when the value of at least one status parameter related to at least one of the network status and the connection device status included in the status information is greater than the first threshold, the edge data network 2000 can keep the size of the first partial image, the number of frames per unit time, and the bit rate of the first partial image equal to those of the previous state.

[0187] In addition, when the value of at least one status parameter related to at least one of the network status and the connection device status included in the status information is less than or equal to the second threshold, the edge data network 2000 can keep the size of the first local image, the number of frames per unit time, and the bit rate of the first local image equal to those of the previous state.

[0188] In this case, the first threshold and the second threshold may be different from each other, but are not limited thereto, and may be equal to each other. In this case, those skilled in the art may understand that operation S518 is not performed.

[0189] At the same time, the above description is for reference Figure 5b Provided, and assuming that as the value of at least one state parameter increases, the state is better, but the present disclosure is not limited thereto, it can be understood that, in the opposite case, the edge data network 2000 can be similar to the reference Figure 5b For example, the status may deteriorate as the value of some parameter, such as bit error rate (BER) or delay, increases.

[0190] Those skilled in the art will appreciate that, in this case, the relationship between each parameter of the encoded generated frame corresponding to at least one state parameter less than or equal to the first threshold and the respective parameters of the frame corresponding to at least one state parameter greater than the second threshold may be opposite to that described above.

[0191] Figure 5c is a diagram for describing in detail a method of identifying parameters of a frame generated by encoding a first partial image based on rotation speed information, which is performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0192] refer to Figure 5c , in operation S520, the edge data network 2000 may identify whether the rotation speed of the rotation speed information is greater than a first threshold.

[0193] In operation S522, when the rotation speed of the rotation speed information is greater than the first threshold, the edge data network 2000 may identify at least one of a size, a number of frames per unit time, and a bit rate of a first partial image corresponding to the rotation speed greater than the first threshold.

[0194] In this case, the first threshold value may be a value set assuming that the previous rotation speed is less than or equal to the first threshold value, and the size of the identified first partial image may be larger than the size of the first partial image of the previous state. In addition, the number of frames per unit time of the identified first partial image may be smaller than the number of frames per unit time of the first partial image of the previous state. The bit rate of the identified first partial image may be smaller than the bit rate of the first partial image of the previous state.

[0195] In operation S524 , the edge data network 2000 may identify whether the rotation speed of the rotation speed information is less than or equal to a second threshold.

[0196] In operation S526, when the rotation speed of the rotation speed information is less than or equal to the second threshold, the edge data network 2000 may identify at least one of the size, the number of frames per unit time, and the bit rate of the first partial image corresponding to the rotation speed less than or equal to the second threshold.

[0197] In this case, the second threshold value may be a value set assuming that the previous rotation speed is greater than the second threshold value, and the size of the identified first partial image may be smaller than the size of the first partial image of the previous state. In addition, the number of frames per unit time of the identified first partial image may be greater than the number of frames per unit time of the first partial image of the previous state. The bit rate of the identified first partial image may be greater than the bit rate of the first partial image of the previous state.

[0198] In operation S528, in a case where the rotation speed of the rotation speed information is less than or equal to the first threshold, the edge data network 2000 may maintain the size, the number of frames per unit time, and the bit rate of the first partial image equal to those of the previous state.

[0199] In addition, in a case where the rotation speed of the rotation speed information is greater than the second threshold, the edge data network 2000 may maintain the size, the number of frames per unit time, and the bit rate of the first partial image equal to those of the previous state.

[0200] In this case, the first threshold and the second threshold may be different from each other, but are not limited thereto, and may be equal to each other. In this case, those skilled in the art may understand that operation S528 is not performed.

[0201] Figure 5d is a diagram for describing in detail a method of identifying parameters of a frame generated by encoding a first partial image based on state information and rotation speed information, which is performed by the edge data network 2000 according to an embodiment of the present disclosure.

[0202] refer to Figure 5d , in operation S530, the edge data network 2000 may identify whether a value of the at least one state parameter is less than or equal to a first threshold, and whether a rotation speed of the rotation speed information is greater than a third threshold.

[0203] In operation S532, when the value of at least one state parameter is less than or equal to a first threshold and the rotation speed of the rotation speed information is greater than a third threshold, the edge data network 2000 can identify at least one of the number of frames per unit time and the bit rate of the first partial image corresponding to at least one state parameter less than or equal to the first threshold and the rotation speed greater than the third threshold.

[0204] In this case, the first threshold value may be a value set assuming that the value of at least one previous state parameter is high, and the third threshold value may be a value set assuming that the previous rotation speed is less than or equal to the third threshold value, so here, the size of the identified first partial image may be larger than the size of the first partial image of the previous state. In addition, the number of frames per unit time of the identified first partial image may be smaller than the number of frames per unit time of the first partial image in the previous state. The bit rate of the identified first partial image may be smaller than the bit rate of the first partial image in the previous state. In this case, the size of the first partial image corresponding to the state parameter may be identified. In addition, the number of frames per unit time and the bit rate of the first partial image corresponding to the rotation speed may be identified. However, the present disclosure is not limited thereto, and those skilled in the art may understand that each identified element (e.g., the number of frames per unit time and the bit rate of the first partial image) may be identified by considering the rotation speed and the state parameter as a whole.

[0205] For example, the edge data network 2000 may assign weights to the state parameters and the rotation speed, and identify the size, the number of frames per unit time, and the bit rate of the first partial image in consideration of the weights.

[0206] In operation S534, the edge data network 2000 may identify whether a value of the at least one state parameter is greater than a second threshold value, and whether a rotation speed of the rotation speed information is less than or equal to a fourth threshold value.

[0207] In operation S536, when the value of at least one state parameter is greater than the second threshold and the rotation speed of the rotation speed information is less than or equal to a fourth threshold, the edge data network 2000 can identify at least one of the number of frames per unit time and the bit rate of the first partial image corresponding to the at least one state parameter greater than the second threshold and the rotation speed less than or equal to the fourth threshold.

[0208] In this case, the second threshold value may be a value set assuming that the value of at least one previous state parameter is less than or equal to the second threshold value, the fourth threshold value may be a value set assuming that the previous rotation speed is greater than the fourth threshold value, and the size of the identified first partial image may be smaller than the size of the first partial image of the previous state. In addition, the number of frames per unit time of the identified first partial image may be greater than the number of frames per unit time of the first partial image of the previous state. The bit rate of the identified first partial image may be greater than the bit rate of the first partial image of the previous state.

[0209] In this case, the size of the first partial image corresponding to the state parameter can be identified. In addition, the number of frames per unit time and the bit rate of the first partial image corresponding to the rotation speed can be identified. However, the present disclosure is not limited thereto, and those skilled in the art can understand that each identified element (e.g., the number of frames per unit time and the bit rate of the first partial image) can be identified by considering the rotation speed and the state parameter as a whole.

[0210] In operation S538, when the recognition result in operations S530 and S534 is "No", the edge data network 2000 may keep the size of the first partial image, the number of frames per unit time, and the bit rate equal to those of the previous state. However, the present disclosure is not limited thereto, and when the value of at least one state parameter is less than or equal to the first threshold, or when the rotation speed is greater than the third threshold, at least one of the size of the first partial image, the number of frames per unit time, and the bit rate corresponding to at least one state parameter less than or equal to the first threshold, and other parameters of the encoded generated frame that are not recognized may be kept equal to those of the previous state. Otherwise, the edge data network 2000 may keep the size of the first partial image, the number of frames per unit time, and the bit rate equal to those of the previous state.

[0211] Meanwhile, those skilled in the art can understand that the first threshold value and the second threshold value can be different, and the third threshold value and the fourth threshold value can be different, but the present disclosure is not limited thereto. And they can be respectively equal to each other.

[0212] Figure 6 is a flowchart illustrating a method for streaming image content obtained from an edge data network and performed by an electronic device according to an embodiment of the present disclosure.

[0213] In operation S602 , the electronic device 1000 may transmit sensor information including at least one of orientation information and rotation speed information to the edge data network 2000 .

[0214] In operation S604, the electronic device 1000 may obtain, from the edge data network 2000, a first frame generated by encoding a first partial image according to at least one of a size, a position, a bit rate, and a number of frames per unit time of the first partial image, wherein the size, the position, the bit rate, and the number of frames per unit time of the first partial image are identified based on at least one of status information and rotation speed information including network status information and device status information.

[0215] In this case, the operation performed by the edge data network 2000 to identify at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on at least one of the status information and the rotation speed information is described above, so its detailed description is omitted.

[0216] In operation S606, the electronic device 1000 may reconstruct the first partial image by decoding the first frame.

[0217] In operation S608, the electronic device 1000 may reproduce the reconstructed first partial image. In this case, the electronic device 1000 may reproduce a portion of the reconstructed first partial image based on the orientation information at a point in time of reproduction.

[0218] Figure 7 is a flowchart illustrating an operation process between an electronic device and an edge data network according to an embodiment of the present disclosure.

[0219] refer to Figure 7 In operation S702, the electronic device 1000 may obtain orientation information. In this case, the electronic device 1000 may additionally obtain rotation speed information, but is not limited thereto.

[0220] In operation S704, the electronic device 1000 may transmit the orientation information to the edge data network 2000. In this case, the electronic device 1000 may additionally transmit the rotation speed information. The orientation information and the rotation speed information may be periodically obtained and transmitted, but are not limited thereto.

[0221] In operation S706, the electronic device 1000 may transmit state information including at least one of a network state and a state of the electronic device 1000 to the edge data network 2000. In this case, the transmitted state information may be information about a throughput measured by a specific module of the electronic device 1000, and, for example, information about a size of data received per unit time measured by a receiving module (such as the number of packets or bits received per unit time) may be transmitted to the edge data network 2000.

[0222] In operation S708, the edge data network 2000 may identify the user FoV image based on the orientation information. For example, the edge data network 2000 may identify the location of the user FoV image based on the received orientation information.

[0223] In operation S710 , the edge data network 2000 may identify at least one of a size, a position, a bit rate, and a number of frames per unit time of the first partial image based on the state information.

[0224] In an embodiment, the edge data network 2000 may determine the bit rate or the number of frames per unit time based on the state information. For example, when at least one state parameter included in the state information is greater than a first threshold, it may be determined to increase the bit rate or the number of frames per unit time. On the contrary, when at least one state parameter included in the state information is less than a second threshold, it may be determined to reduce the bit rate or the number of frames per unit time.

[0225] Thereafter, the edge data network 2000 may determine the size of the first partial image based on the determined number of frames per unit time. In the case where the determined number of frames per unit time is greater than the first threshold, it may be determined that the size of the first partial image is to be increased. Conversely, in the case where the determined number of frames per unit time is less than the second threshold, it may be determined that the size of the first partial image is to be decreased. Figures 11 to 13 The relationship between the number of frames per unit time and the size of the first partial image is described in more detail.

[0226] In another embodiment, the edge data network 2000 may determine at least one of the size, the number of frames per unit time, and the bit rate of the first partial image based on the state information. This will be referred to below. Figures 8a to 8d Describe in more detail.

[0227] Other operations refer to above Figure 3 Description has been made, and thus a detailed description thereof is omitted.

[0228] Figure 8a is a diagram for describing a process of streaming a first partial image performed by the edge data network 2000 when at least one state parameter is greater than a first threshold according to an embodiment.

[0229] refer to Figure 8a , the electronic device 1000 may send information about at least one state parameter to the edge data network 2000, and the edge data network 2000 may receive the information and identify at least one state parameter 810 that is greater than the first threshold. In this case, the state parameter may be information about throughput. The throughput may be information about at least one of the amount of data processed by the edge data network 2000, the amount of data processed by the electronic device 1000, or the amount of processed data sent / received through the network. For example, the electronic device 1000 may measure the amount of data per unit time received by the receiving module through the network, and send information about the measured amount to the edge data network 2000.

[0230] The edge data network 2000 can identify at least one state parameter greater than a first threshold, set a bit rate corresponding to the at least one state parameter greater than the first threshold, encode a first partial image 820 for which an additional FoV is set in consideration of a specific rotation speed, and send the encoded first partial image 820 to the electronic device 1000.

[0231] However, the present disclosure is not limited thereto, and those skilled in the art may appreciate that when initially connected to the electronic device 1000, the edge data network 2000 may set the bit rate and additional FoV to default values ​​to be the same as in the above case regardless of the first threshold.

[0232] Figure 8b is a diagram for describing a process of streaming a first partial image performed by the edge data network 2000 when at least one state parameter is less than or equal to a first threshold according to an embodiment.

[0233] refer to Figure 8b , in the case where at least one state parameter is a state parameter 830 less than or equal to the first threshold, the electronic device 1000 may send information about the at least one state parameter to the edge data network 2000, and the edge data network 2000 may receive the information and identify at least one state parameter 830 less than or equal to the first threshold. The edge data network 2000 may identify at least one state parameter less than or equal to the first threshold, and may set the bit rate, FoV, and FPS corresponding to the at least one state parameter less than or equal to the first threshold.

[0234] In this case, the bit rate may be set to be smaller than the bit rate when at least one state parameter is greater than the first threshold. The FPS may be set to be smaller than the FPS when at least one state parameter is greater than the first threshold. The additional FoV may be set to be wider than when at least one state parameter is greater than the first threshold.

[0235] The edge data network 2000 can identify at least one state parameter that is less than or equal to a first threshold, encode the first partial image 840 by setting a bit rate, FPS, and additional FoV corresponding to the at least one state parameter that is less than or equal to the first threshold, and send the encoded first partial image 840 to the electronic device 1000.

[0236] Figure 8c 2 is a diagram for describing a process of adaptively changing encoding generation frame parameters performed by the edge data network 2000 when at least one state parameter is less than or equal to a first threshold according to an embodiment.

[0237] refer to Figure 8c, in operation S840, in the process of generating the current frame by encoding, the edge data network 2000 may identify whether at least one state parameter is less than or equal to a first threshold. In this case, it is assumed that the previous state parameter is greater than the first threshold, and the bit rate, FPS, and additional FoV corresponding to the previous state parameter greater than the first threshold are set, or it is assumed that the bit rate, FPS, and additional FoV are set to default values ​​after the connection is established with the electronic device 1000. The bit rate, FPS, and additional FoV set to the default value may be the same as the bit rate, FPS, and additional FoV corresponding to at least one state parameter greater than the first threshold.

[0238] In the process of generating the current frame by encoding, when at least one state parameter is less than or equal to the first threshold, the edge data network 2000 can set the bit rate to be less than the previous bit rate. That is, the quality of the frame to be generated may be reduced. Therefore, even if the value of at least one state parameter is reduced, the electronic device 1000 can be prevented from stopping frame reproduction.

[0239] In addition, the edge data network 2000 may set the FPS to be lower than the previous FPS. Network bandwidth may be saved by setting the FPS to be low, and the electronic device 1000 may be prevented from stopping frame reproduction even if the value of the state parameter (e.g., the throughput measured by the receiving module of the sub-device 1000) decreases as the frame transmission interval increases.

[0240] In consideration of the reduced FPS, the edge data network 2000 may set a wider additional FoV. When the FPS is reduced, the frame transmission interval increases, and therefore, by setting the additional FoV to an increased amount of the widened interval, the MTP delay may be ensured and asynchronous time warp (ATW) may be properly implemented. Here, ATW refers to a technique for displaying an image drawn by a rendering engine based on a rendering frame on a display by modifying the image according to the movement of the user.

[0241] For example, the FPS in the encoding parameters of the previous frame (i.e., before the edge data network 2000 performs encoding to generate the current frame) is 60fps and the additional FoV is 0.83° in one direction. During encoding to generate the current frame, at least one state parameter is less than or equal to the first threshold. The FPS in the encoding parameters of the current frame can be 30fps, and the additional FoV can be set to 1.67° in one direction.

[0242] Assuming that a user wearing electronic device 1000 rotates his or her head 5° in 100 milliseconds (an example of a typical degree of rotation), the user can rotate 0.83° in 16.6 milliseconds relative to 60fps, so, for the encoding of the previous frame, the edge data network 2000 can set the additional FoV to 0.83° in one direction. When the FPS is reduced to 30fps, the frame is delayed by 16.6ms compared to the transmission at 60fps, and the rotation angle that the electronic device 1000 can move in the interval increases, so, the edge data network 2000 can set the additional FoV to double, i.e., to 1.67° in one direction.

[0243] That is, the edge data network 2000 can identify the degree of rotation of the electronic device 1000 within a preset time period and set an additional FoV based on the following proportional formula.

[0244] [Proportional]

[0245] Preset time: rotation angle = frame transmission interval: additional FoV

[0246] Figure 8d is a diagram illustrating parameters of an encoding-generated frame being streamed if a state parameter is greater than, less than, or equal to a first threshold according to an embodiment.

[0247] refer to Figure 8d , when the state parameter is greater than the first threshold, the width and height of the encoded generated frame 850 being streamed from the edge data network 2000 can be 2500 pixels (px) and 1400 pixels (px) respectively, the bit rate can be 25Mbps, and the FPS can be 60fps.

[0248] On the contrary, when the state parameter is less than or equal to the first threshold, the width and height of the encoded frame 860 being streamed from the edge data network 2000 may be 2600 pixels and 1500 pixels, respectively. The bit rate may be 15 Mbps, and the FPS may be 30 fps.

[0249] However, the above reference Figure 8d The values ​​of the described parameters are only examples, and in the case where the above-mentioned state parameter is less than or equal to the first threshold value and in the case where the above-mentioned state parameter is greater than the first threshold value, the parameters may have values ​​taking into account the relationship between relative parameters.

[0250] As referenced above Figures 7 to 8dAs described, when the state parameter is less than or equal to the first threshold, the edge data network 2000 can adaptively identify the bit rate, frames per second, and additional FoV, and thus has tolerance for the situation where the value of the state parameter is less than or equal to the first threshold (for example, the amount of data per unit time received by the receiving module is less than or equal to the first threshold), minimizes black edges, and continuously guarantees MTP delay.

[0251] Fig. 9 is a flowchart illustrating an operation process between an electronic device and an edge data network according to an embodiment of the present disclosure.

[0252] refer to Fig. 9 , in operation S902, the electronic device 1000 may obtain orientation information and rotation speed information.

[0253] In operation S904 , the electronic device 1000 may transmit orientation information and rotation speed information to the edge data network 2000 .

[0254] In operation S906, the edge data network 2000 may identify the user FoV image based on the received orientation information. For example, the edge data network 2000 may identify the location of the user FoV image based on the received orientation information.

[0255] In operation S908, the edge data network 2000 may determine at least one of the size, position, bit rate, and number of frames per unit time of the first partial image based on the rotation speed information. In this case, the rotation speed information may be received from the electronic device 1000, but is not limited thereto, and may not be received directly from the electronic device 1000, but may be obtained based on the difference between the currently received orientation information and the previously received orientation information.

[0256] In an embodiment, the edge data network 2000 may determine the size and position of a first partial image including a user FoV image and an additional FoV image surrounding the user FoV image based on the rotation speed information. Thereafter, at least one of the bit rate and the number of frames per unit time of the first partial image may be determined based on the determined size of the first partial image. For example, in the case where the size of the first partial image is greater than a first threshold, it may be determined that the bit rate or the number of frames per unit time is to be reduced. Conversely, in the case where the size of the first partial image is less than a second threshold, it may be determined that the bit rate or the number of frames per unit time is to be increased.

[0257] In another embodiment, the size, position, bit rate, and number of frames per unit time of the first partial image can be determined based on the rotation speed information. This will be referred to below. Fig.10a , Fig.10b , Fig.11 and Fig.12Describe in more detail.

[0258] Other operations refer to above Figure 3 Description has been made, and thus a detailed description thereof is omitted.

[0259] Fig.10a is a diagram for describing a process of streaming a first partial image performed by an edge data network 2000 when a rotation speed of an electronic device 1000 worn by a user is less than or equal to a first threshold according to an embodiment.

[0260] refer to Fig.10a , the electronic device 1000 may send the rotation speed information to the edge data network 2000, and the edge data network 2000 may receive the rotation speed information and identify the rotation speed. For example, the electronic device 1000 may identify whether the rotation speed of the electronic device 1010 worn on the user's head is less than a first threshold (i.e., whether the rotation speed is low or whether the rotation stops).

[0261] In the case where the current rotation speed is less than or equal to the first threshold (or immediately after the connection is established), the edge data network 2000 may set the bit rate and additional FoV 1020 corresponding to a rotation speed equal to or less than the first threshold (or as a default value).

[0262] The edge data network 2000 may encode the first partial image 1030 , whose bit rate and additional FoV 1020 are set to correspond to a rotation speed less than or equal to a first threshold, and transmit the encoded first partial image 1030 to the electronic device 1010 .

[0263] Fig.10b is a diagram for describing a process of streaming a first partial image performed by the edge data network 2000 when a rotation speed of an electronic device 1040 worn on a user's head is greater than a first threshold according to an embodiment.

[0264] refer to Fig.10b , the electronic device 1040 can send the rotation speed information to the edge data network 2000, and the edge data network 2000 can receive the rotation speed information and identify the rotation speed. In this case, the electronic device 1040 can identify whether the rotation speed of the electronic device 1040 worn on the user's head is greater than the first threshold (ie, whether the rotation speed is high).

[0265] In the case where the current rotation speed is greater than the first threshold, the edge data network 2000 may set the bit rate and size of the additional FoVs 1055 and 1060 corresponding to the rotation speed greater than the first threshold. In this case, the rotation direction 1050 is taken into consideration, and the additional FoV 1055 along the rotation direction 1050 may be set to be wider than the additional FoV 1060 in the direction opposite to the rotation direction 1050. The size of the additional FoV 1060 in the opposite direction may be equal to the size of the default additional FoV, but is not limited thereto, and may be greater than the size of the default additional FoV.

[0266] Edge Data Network 2000 can be used compared to the above reference Fig.10a The first partial image 1070 is encoded at a bit rate smaller than the bit rate described above, and the additional FoVs 1055 and 1060 are encoded in the rotation direction (and the direction opposite thereto) than the above reference Fig.10a The described additional FoV is wider, and the encoded first partial image 1070 is transmitted to the electronic device 1000 .

[0267] Fig.11 is a diagram for describing a process of identifying left and right FoVs of a coding generation frame performed by the edge data network 2000 when a rotation speed is equal to a first value (ie, the rotation speed is high) according to an embodiment.

[0268] refer to Fig.11 , the electronic device 1000 may periodically send sensor information including orientation information such as yaw, pitch, or roll and instantaneous rotation speed information such as speed to the edge data network 2000. For example, the electronic device 1000 may send sensor information including orientation information and rotation speed information every 16.6 ms.

[0269] In this case, the total processing time period between the time point when the electronic device 1000 sends the sensor information and the time point when the electronic device 1000 receives the response (including the response of the extended height (ext-Height), the extended width (ext-Width) and the reference orientation information yaw (Yaw), pitch (Pitch) and roll (Roll)) can be defined as a delay L. The delay L is a time period that takes into account the time period required for processing by the edge data network 2000 and the network delay, and can be, for example, 33.3ms.

[0270] Meanwhile, a period of time until the electronic device 1000 receives a subsequent frame may be defined as a frame interval r. For example, in the case where the FPS is 60 fps, the frame interval r may be 16.6 ms.

[0271] The default FoV of the electronic device 1000 may be defined as F. For example, the default FoV F of the electronic device 1000 may be 110°. The rotation speed of the head may be defined as V. For example, the rotation speed V may be 600 degrees / second.

[0272] In this case, the edge data network 2000 may identify left and right FoV regions of interest (ROIs) (default FoV+additional FoV) of a frame to be transmitted according to the following equation.

[0273] [equation]

[0274] ROI=F+(2×(V×(L+r)))

[0275] For example, when F is 110°, V is 600 degrees / second, L is 33.3 ms, and r is 16.6 ms, the ROI can be identified as 170°.

[0276] Fig.12 is a diagram for describing a process of identifying left and right FoVs performed by the edge data network 2000 when a rotation speed is equal to a second value (ie, the rotation speed is low) according to an embodiment.

[0277] refer to Fig.12 ,and Fig.11 Different, the rotation speed can be changed from 600 degrees / second to 200 degrees / second. In this case, the edge data network 2000 can identify the left and right FoV ROIs of the frame to be transmitted according to the above equation, and when other conditions are the same, the ROI can be identified as 130°.

[0278] Fig.13 is a diagram for describing a process of setting a frame per second (FPS) to a low value and then identifying left and right FoVs, performed by the edge data network 2000 , when a rotation speed is equal to a first value (ie, the rotation speed is high) according to an embodiment.

[0279] refer to Fig.13 ,and Fig.11 Differently, the FPS set by the edge data network 2000 can be changed to 30fps.

[0280] In this case, the edge data network 2000 can identify the left and right FoV ROIs of the frame to be transmitted according to the above equation, and when other conditions are the same, the ROI can be identified as 170°.

[0281] Therefore, the edge data network 2000 can identify the number of frames per unit time according to the rotation speed, identify the additional FoV according to the number of frames per unit time, and identify the FoV including the default FoV (user FoV) and the additional FoV. In this case, as the number of frames per unit time decreases, the additional FoV identified taking into account the expected head movement during the frame interval increases.

[0282] Reference above Figure 11 to Figure 13 The process of identifying FoV is described, in which it is assumed that the left and right additional FoVs are equal to each other, but the present disclosure is not limited to this, and those skilled in the art will appreciate that the FoV in the rotation direction and the FoV in the opposite direction can be identified as asymmetric to each other. For example, the additional FoV in the rotation direction can be identified as V×(L+r), while the additional FoV in the direction opposite to the rotation direction is identified as the default additional FoV. Alternatively, those skilled in the art will appreciate that the edge data network 2000 can identify the additional FoV by considering the weight between the rotation direction and the opposite direction in addition to the rotation speed, delay, and frame interval of the head.

[0283] In the following, reference Figures 11 to 13 A process is described in which the edge data network 2000 streams a frame and the electronic device 1000 renders the frame based on the above-described method of identifying the left and right FoVs according to an embodiment.

[0284] The edge data network 2000 may pre-identify a specific range of rotation speeds in the horizontal direction. For example, the edge data network 2000 may pre-identify a range from 0 degrees / second to 50 degrees / second as range 1, a range from 50 degrees / second to 200 degrees / second as range 2, and a range from 200 degrees / second to 600 degrees / second as range 3.

[0285] The electronic device 1000 and the edge data network 2000 may exchange virtual data with each other to measure delays including network delays. For example, the electronic device 1000 may send a request for a default FoV image to the edge data network 2000, the request including orientation information (e.g., yaw, pitch, and roll) and timestamp information (i.e., measurement timestamp) indicating a time point of the orientation information, receive the default FoV image from the edge data network 2000, calculate the difference between the timestamp (current timestamp) indicating a time point at which the default FoV image is received and the measurement timestamp, and measure the delay based on the difference between the timestamps.

[0286] The edge data network 2000 may previously identify the FoV of the maximum speed of each range in consideration of delay, etc. In this case, each range and the FoV corresponding thereto may be stored in the LUT.

[0287] The electronic device 1000 may periodically send the angle (e.g., yaw, pitch, and roll) at which the head is currently facing and their instantaneous rotation speeds (e.g., yaw speed, pitch speed, and roll speed) to the edge data network 2000. For example, the information to be transmitted may be {"Yaw": 0.125, "Pitch": 0.545, "Roll": 0.13, "vYaw" = 45 degrees / second, "vPictch": 20 degrees / second, "vRoll": 5 degrees / second}.

[0288] The edge data network 2000 may perform encoding to generate a frame of FoV corresponding to the previously identified range and send the frame. In this case, the reference angle and height and width of the extended FoV area of ​​the encoded generated frame to be sent may be included in the payload of the transport protocol header of the encoded generated frame to be sent. For example, the information included in the payload may be {"Yaw": 0.125, "Pitch": 0.545, "Roll": 0.13, "extWidth": 110px, "extHeight": 200px}.

[0289] The electronic device 1000 may check the payload of the received data, identify the position, width, and height of the frame, decode the frame, and display a reconstructed frame on the display of the electronic device 1000 based on the decoded frame.

[0290] Fig.14 is a diagram for describing a process of identifying additional FoV areas based on object detection performed by the edge data network 2000 according to an embodiment.

[0291] refer to Fig.14 As described above, the edge data network 2000 may identify the default FoV area 1410 and the additional FoV area 1420 in the VR image 1400. In this case, the edge data network 2000 may perform object detection and detect an object 1455 that is not covered by the additional FoV area 1420. In this case, an additional FoV area 1430 that is wider than the additional FoV area 1420 may be identified to cover the detected object.

[0292] That is, because if an object of interest is on the boundary of the FoV area and is therefore not covered by the FoV area, the user may additionally move his or her head out of the FoV area to identify the object, a wider additional FoV area may be identified based on the detected object.

[0293] Fig.15 is a diagram illustrating a process of transmitting a first partial image in a plurality of data units to an electronic device 1000 , which is performed by an edge data network 2000 according to an embodiment.

[0294] The edge data network 2000 can divide the VR image 1500 along the vertical direction (e.g., longitude) to obtain several data units, select one or more data units ①, ②, and ③ as many as the required FoV from the several data units in the process of obtaining the first partial image, encode the selected one or more data units, and send the encoded data units to the electronic device 1000.

[0295] Fig.16 is a diagram schematically illustrating an operation process between an electronic device, an edge data network, and a virtual reality (VR) game interface device.

[0296] Apart from Figure 4 In addition to the components, Fig.16 Also shown is a VR game interface device 4000. In the field of VR games, unlike the field of VR streaming, user interaction is involved, so in addition to the electronic device 1000, the VR game interface device 4000 can also be used.

[0297] The operation process between electronic devices and edge data networks in the field of VR streaming is referenced above Figure 4 It has been described in detail in the figure and other figures, so it will not be repeated here. The following describes the operation process between electronic devices, edge data networks and VR game interface devices in the VR game field.

[0298] The edge data network 2000 can generate a VR game image (interactive VR image) included in the VR sequence based on the instructions of the VR game processing module 1686.

[0299] In detail, the VR game processing module 1686 may store instructions for generating a first partial image of a VR game image based on the orientation information, the rotation speed information, and the VR game interface device sensing information 1652. The VR game processing module 1686 may store instructions for outputting the VR game interface device control information 1654.

[0300] The electronic device 1000 may send the VR game interface device sensing information 1652 to the edge data network 2000 for VR game processing. In this case, the electronic device 1000 may receive the VR game interface device sensing information from the VR game interface device 4000.

[0301] The VR game interface device 4000 may include a network interface 4020, a sensing module 4010, and an actuation module 4030. The network interface 4020 may be a module for communicating with the outside, and the sensing module 4010 may be a module for detecting the user's actions. The sensing module 4010 may obtain VR game interface device sensing information.

[0302] The actuation module 4030 may be a module that provides various types of outputs (eg, tactile feedback), such as vibration, to the user. The actuation module 4030 may provide various types of outputs to the user based on the VR game interface device sensing control information.

[0303] The VR game interface device 4000 may transmit VR game interface device sensing information to the electronic device 1000. In addition, the VR game interface device 4000 may receive VR game interface device control information from the electronic device 1000.

[0304] As described above, the VR interface device sensing information may be sent to the edge data network 2000 via the electronic device 1000, and the VR interface device sensing information and the VR game interface device control information may be sent to the VR game interface device 4000 via the electronic device 1000, but the present disclosure is not limited thereto, and they may be directly exchanged between the VR game interface device 4000 and the edge data network 2000. In this case, synchronization information for synchronization with the electronic device 1000 may also be sent.

[0305] Although the above reference Fig.16 The operation process between electronic devices, edge data networks and VR game interface devices in the VR game field is described, but the present disclosure is not limited to this. Those skilled in the art will understand that interactive VR can be mainly used in many fields where users can directly intervene in content and perform activities, such as simulation, training, etc.

[0306] Fig.17 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0307] Connect to Figures 1 to 16 The electronic device 110 of the edge data network may correspond to Fig.17For example, in the network environment 100, the electronic device 110 may communicate with the electronic device 1702 (here, the electronic device includes a VR game interface device) via the first network 1798 (e.g., a short-range wireless communication network), or may communicate with the electronic device 1704 or the server 1708 via the second network 1799 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 110 may communicate with the electronic device 1704 via the server 1708. According to an embodiment, the electronic device 110 may include a processor 1720, a memory 1730, an input device 1750, a sound output device 1755, a display device 1760, an audio module 1770, a sensor module 1776, an interface 1777, a haptic module 1779, a camera module 1780, a power management module 1788, a battery 1789, a communication module 1790, a subscriber identification module 1796, or an antenna module 1797. In some embodiments, in the electronic device 110, at least one of the components (e.g., the display device 1760 or the camera module 1780) may be omitted, or one or more other components may be further included. In some embodiments, some components may be implemented as integrated circuits. For example, the sensor module 1776 (e.g., a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented by being embedded in the display device 1760 (e.g., a display).

[0308] The processor 1720 may execute, for example, software (e.g., program 1740) to control at least one other component (e.g., hardware or software component) of the electronic device 110 connected to the processor 1720, and perform various types of data processing or calculation. According to an embodiment, as at least part of the data processing or calculation, the processor 1720 may load a command or data received from another component (e.g., sensor module 1776 or communication module 1790) in the volatile memory 1732, process the command or data stored in the volatile memory 1732, and store the result data in the non-volatile memory 1734.

[0309] According to an embodiment, the processor 1720 may execute the application client, and according to the execution of the application client, the processor 1720 may send a request message for checking the network protocol to be used by the application client to the edge data network 2000. In addition, the processor 1720 may receive a response message indicating the network protocol to be used by the application client from the edge data network 2000. The processor 1720 may update the network protocol to be used by the UE application based on the response message. The processor 1720 may select a network socket corresponding to the updated network protocol. The processor 1720 may receive data generated for the application client from the edge data network 2000 by using the selected network socket.

[0310] According to an embodiment, the processor 1720 may include a main processor 1721 (e.g., a CPU or an application processor (AP)) and an auxiliary processor 1723 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, a communication processor (CP) that may operate independently of the main processor 1721 or together with the main processor 1721. Additionally or alternatively, the auxiliary processor 1723 may be configured to use lower power than the main processor 1721 or to be dedicated to one designated function. The auxiliary processor 1723 may be implemented separately from the main processor 1721 or as part of the main processor 1721.

[0311] When the main processor 1721 is in an inactive state (e.g., a sleep state), the auxiliary processor 1723 may control at least some functions or states related to at least one of the components in the electronic device 110 (e.g., the display device 1760, the sensor module 1776, or the communication module 1790) instead of the main processor, or be used together with the main processor 1721 when the main processor 1721 is in an active state (e.g., executing an application). According to an embodiment, the auxiliary processor 1723 (e.g., an ISP or a CP) may be implemented as a part of another component (e.g., a camera module 1780 or a communication module 1790) that is functionally related to the auxiliary processor 1723.

[0312] The memory 1730 may store a plurality of pieces of data to be used by at least one component (e.g., the processor 1720 or the sensor module 1776) in the electronic device 110. The data may include, for example, software (e.g., program 1740) and input data or output data related to commands associated with the software. The memory 1730 may include a volatile memory 1732 and a non-volatile memory 1734.

[0313] The program 1740 may be stored as software in the memory 1730 and may include, for example, an operating system (OS) 1742, middleware 1744, and applications 1746. In an embodiment, the program 1740 may include Figure 1 The first application client 122 and the second application client 124. In addition, the program 1740 may include Figure 1 The edge enabler client 130 of .

[0314] The input device 1750 may receive a command or data intended for a component (eg, the processor 1720 ) of the electronic device 110 from outside the electronic device 110 .

[0315] The sound output device 1755 may output a sound signal to the outside of the electronic device 110. The sound output device 1755 may include, for example, a speaker. The speaker may be used for general purposes such as reproducing multimedia or recording.

[0316] The display device 1760 may visually provide information to the outside of the electronic device 110 (e.g., a user). The display device 1760 may include, for example, a display, a hologram device, or a projector and a control circuit configured to control the corresponding device. According to an embodiment, the display device 1760 may include a touch circuit configured to detect a touch or a sensor circuit (e.g., a pressure sensor) configured to measure the strength of the force generated by the touch.

[0317] The audio module 1770 may convert sound into an electrical signal and vice versa. According to an embodiment, the audio module 1770 may obtain sound through the input device 1750 or output sound through the sound output device 1755 or an external electronic device (e.g., electronic device 1702 (e.g., speaker or earphone)) directly or wirelessly connected to the electronic device 110.

[0318] The sensor module 1776 may detect an operating state (e.g., power or temperature) or an external environment state (e.g., user state) of the electronic device 110, and generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 1776 may include, for example, a gesture sensor, a gyroscope sensor, a barometer sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor (e.g., an RGB (red-green-blue) sensor), an infrared (IR) sensor, a biosensor, a temperature sensor, a humidity sensor, an illumination sensor, or a sensor related to an autonomous driving vehicle (e.g., an inertial measurement unit (IMU)), a global positioning system (GPS) sensor, a camera, a light imaging detection and ranging (LIDAR) sensor, and a radio detection and ranging (RADAR) sensor).

[0319] The interface 1777 may support one or more designated protocols that may be used to directly or wirelessly connect to an external electronic device (eg, the electronic device 1702 ) of the electronic device 110 .

[0320] The connection end 1778 may include a connector through which the electronic device 110 can be physically connected to an external electronic device (eg, the electronic device 1702 ).

[0321] The haptic module 1779 may convert the electrical signal into a mechanical stimulus (eg, vibration or motion) or an electrical stimulus that a user can sense through a tactile or kinesthetic sense.

[0322] The camera module 1780 may capture a still image or a moving image. According to an embodiment, the camera module 1780 may include one or more lenses, an image sensor, an ISP, or a flash.

[0323] The power management module 1788 may manage power to be supplied to the electronic device 110 .

[0324] The battery 1789 may provide power to at least one component in the electronic device 110 .

[0325] The communication module 1790 can establish a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 110 and an external electronic device (e.g., electronic device 1702, electronic device 1704, or server 1708) and support communication through the established channel. The communication module 1790 may include one or more communication processors that operate independently of the processor 1720 (e.g., AP) and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 1790 may include a wireless communication module 1792 (e.g., a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) and a wired communication module 1794 (e.g., a local area network (LAN) communication module or a power line communication module). A corresponding one of these communication modules may communicate 1799 with an external electronic device via a first network 1798 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These types of communication modules may be integrated into one component (e.g., a single chip) or implemented by multiple separate components (e.g., multiple chips).

[0326] The wireless communication module 1792 may verify or authenticate the electronic device 110 in a communication network such as the first network 1798 or the second network 1799 by using subscriber information (eg, International Mobile Subscriber Identity (IMSI)) stored in the subscriber identification module 1796 .

[0327] The antenna module 1797 may transmit a signal or power to the outside (eg, an external electronic device) or receive a signal or power from the outside.

[0328] At least some of the components may be connected to each other in a peripheral device communication scheme such as a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI) and exchange signals (eg, commands or data) with each other.

[0329] According to an embodiment, a command or data may be transmitted or received between the electronic device 110 and the external electronic device 1704 via the server 1708 connected through the second network 1799. Each of the electronic devices 1702 and 1704 may be the same type as that of the electronic device 110 or a different type.

[0330] According to an embodiment, all or part of the operations performed by the electronic device 110 may be performed by one or more external devices in the electronic devices 1702 and 1704 and the server 1708. For example, when the electronic device 110 should automatically or in response to a request from a user or another device to perform a certain function or service, the electronic device 110 may additionally or alternatively autonomously perform the function or service, requesting one or more of the external electronic devices 1702 and 1704 to perform at least a portion of the function or service. One or more of the external electronic devices 1702 and 1704 that receive the request may perform at least a portion of the requested function or service or an additional function or service related to the request, and transmit the result of the execution to the electronic device 110. The electronic device 110 may process the result as is or additionally and provide the processing result as at least a portion of the response to the request. To this end, for example, cloud computing, distributed computing, or client-server computing may be used.

Claims

1. A method for transmitting image content to an electronic device performed by an edge data network, the method comprising: obtaining sensor information, the sensor information comprising orientation information detected by the electronic device connected to the edge data network; obtaining state information, the state information comprising at least one state parameter, the at least one state parameter being related to at least one of: (i) a network state between the edge data network and the electronic device, (ii) a state of the edge data network, or (iii) a state of the electronic device, wherein the state of the edge data network and the state of the electronic device are related to processing of the image content; Based on the sensor information and at least one state parameter included in the state information, identifying a size of a first partial image and a position of the first partial image; Generate a first frame by encoding the first partial image based on the identified size of the first partial image or the identified position of the first partial image; as well as The generated first frame is sent to the electronic device.

2. The method according to claim 1, wherein: The method further includes: identifying a bit rate of the image content or a number of frames per unit time of the image content based on the sensor information and the state information, and Generating the first frame by encoding the first partial image includes: The first frame is generated by encoding the first partial image based on at least one of the identified size of the first partial image, the identified position of the first partial image, the identified bit rate of the image content, or the identified number of frames per unit time of the image content.

3. The method according to claim 1, wherein: The first partial image is a partial image in a first virtual reality (VR) image, and the first VR image has a specific frame index of a VR sequence including a plurality of frames; and The first partial image is associated with orientation information indicating a position of the first partial image.

4. The method according to claim 2, wherein: Identifying at least one of the size of the first partial image, the position of the first partial image, the bit rate of the first partial image, or the number of frames per unit time of the image content based on the sensor information and the state information comprises at least one of the following: in a first case where the value of the at least one state parameter is less than or equal to at least one first threshold, identifying a first size of the first partial image and a first number of frames per unit time of the image content corresponding to the at least one state parameter being less than or equal to the first threshold, wherein the value of the at least one state parameter is related to at least one of the following items included in the state information: (i) the state of the network state, (ii) the state of the edge data network, or (iii) the state of the electronic device; or in a second case where the value of at least one state parameter in the state information is greater than at least one second threshold, identifying a second size of the image content and a second number of frames per unit time corresponding to the at least one state parameter greater than the second threshold, wherein a first size of the first partial image corresponding to the at least one state parameter that is less than or equal to the first threshold is larger than a second size of the first partial image corresponding to the at least one state parameter that is greater than the second threshold, and A first number of frames per unit time of the image content corresponding to the at least one state parameter less than or equal to the first threshold is smaller than a second number of frames per unit time of the first partial image corresponding to the at least one state parameter greater than the second threshold.

5. The method according to claim 2, wherein: Identifying at least one of the size of the first partial image, the position of the first partial image, the bit rate of the image content, or the number of frames per unit time of the image content based on the sensor information and the state information comprises at least one of the following: in a first case where the value of the at least one state parameter is less than or equal to at least one first threshold, identifying a first size of the first partial image, a first number of frames per unit time of the image content, and a first bit rate of the first partial image corresponding to the at least one state parameter that is less than or equal to the first threshold, wherein the value of the at least one state parameter is related to at least one of the following items included in the state information: the state of the network state, the state of the edge data network, or the state of the electronic device; or in a second case where the value of the at least one state parameter is greater than at least one second threshold, identifying a second size of the first partial image, a second number of frames per unit time of the image content, and a second bit rate of the image content corresponding to the at least one state parameter greater than the second threshold, wherein a first size of the first partial image corresponding to the at least one state parameter that is less than or equal to the first threshold is larger than a second size of the first partial image corresponding to the at least one state parameter that is greater than the second threshold, a first number of frames per unit time of the image content corresponding to the at least one state parameter less than or equal to the first threshold is smaller than a second number of frames per unit time of the image content corresponding to the at least one state parameter greater than the second threshold, and A first bit rate of the first partial image corresponding to the at least one state parameter less than or equal to the first threshold is less than a second bit rate of the image content corresponding to the at least one state parameter greater than the second threshold.

6. The method according to claim 2, wherein: The sensor information also includes rotation speed information, Identifying at least one of the size of the first partial image, the position of the first partial image, the bit rate of the image content, or the number of frames per unit time of the first partial image based on the sensor information and the state information comprises at least one of the following: In a first case where the rotation speed of the rotation speed information in the sensor information is greater than a first threshold, identifying a first size of the first partial image corresponding to the rotation speed greater than the first threshold; or in a second case where the rotation speed of the rotation speed information is less than or equal to a second threshold, identifying a second size of the first partial image corresponding to the rotation speed less than or equal to the second threshold, A first size of the first partial image corresponding to a rotation speed greater than the first threshold is greater than a second size of the first partial image corresponding to a rotation speed less than or equal to the second threshold.

7. The method according to claim 2, The sensor information also includes rotation speed information, in, Identifying at least one of the size of the first partial image, the position of the first partial image, the bit rate of the image content, or the number of frames per unit time of the image content based on the sensor information and the state information comprises at least one of the following: In a first case where the rotation speed of the rotation speed information is greater than a first threshold, identifying a first size of the first partial image and a first number of frames per unit time of the image content corresponding to the rotation speed greater than the first threshold; or in a second case where the rotation speed of the rotation speed information is less than or equal to a second threshold, identifying a second size of the first partial image and a second number of frames per unit time of the image content corresponding to a rotation speed less than or equal to the second threshold, wherein a first size of the first partial image corresponding to a rotation speed greater than the first threshold is greater than a second size of the first partial image corresponding to a rotation speed less than or equal to the second threshold, and A first number of frames per unit time of the image content corresponding to a rotation speed greater than the first threshold is smaller than a second number of frames per unit time of the image content corresponding to a rotation speed less than or equal to the second threshold.

8. The method according to claim 2, The sensor information also includes rotation speed information, in, Identifying at least one of the size of the first partial image, the position of the first partial image, the bit rate of the image content, or the number of frames per unit time of the image content based on the sensor information and the state information comprises at least one of the following: identifying a first size of the first partial image, a first number of frames per unit time of the image content, and a first bit rate of the first partial image corresponding to a rotation speed of the rotation speed information greater than a first threshold; or identifying a second size of the first partial image, a second number of frames per unit time of the image content, and a second bit rate of the image content corresponding to a rotation speed of the rotation speed information that is less than or equal to a second threshold, wherein a first size of the first partial image corresponding to a rotation speed greater than the first threshold is greater than a second size of the first partial image corresponding to a rotation speed less than or equal to the second threshold, a first number of frames per unit time of the image content corresponding to a rotation speed greater than the first threshold is smaller than a second number of frames per unit time of the image content corresponding to a rotation speed less than or equal to the second threshold, and A first bit rate of the image content corresponding to a rotation speed greater than the first threshold is lower than a second bit rate of the image content corresponding to a rotation speed less than or equal to the second threshold.

9. The method according to claim 4, wherein: The additional field of view images correspond to the number of frames per unit time.

10. The method according to claim 6, wherein: The additional field of view image corresponds to the rotation direction of the rotation speed information.

11. The method according to claim 7, wherein: Additional field of view images are further identified based on the number of frames per unit time and the rotation direction of the rotation speed information.

12. The method according to claim 10, wherein: The additional field of view images include a first additional field of view image along a rotation direction relative to the field of view image and a second additional field of view image along a direction opposite to the rotation direction, and The first additional field of view image is larger than the second additional field of view image.

13. The method according to claim 9, wherein: The first partial image includes data units of a preset field of view, and Generating the first frame by encoding the first partial image includes: Encoding a first data unit of at least one preset field of view range corresponding to the user field of view image in the first partial image, and encoding a second data unit of at least one preset field of view range corresponding to the additional field of view image in the first partial image; and The first frame is generated including an encoded first data unit and an encoded second data unit.

14. An edge data network for transmitting image content to an electronic device, the edge data network comprising: Network interface; a memory storing one or more instructions; as well as a processor configured to execute the one or more instructions to: obtaining sensor information, the sensor information comprising orientation information detected by the electronic device connected to the edge data network; obtaining state information, the state information comprising at least one state parameter, the at least one state parameter being related to at least one of: (i) a network state between the edge data network and the electronic device, (ii) a state of the edge data network, or (iii) a state of the electronic device, wherein the state of the edge data network and the state of the electronic device are related to processing of the image content; Based on the sensor information and at least one state parameter included in the state information, identifying a size of a first partial image and a position of the first partial image; Generate a first frame by encoding the first partial image based on the identified size of the first partial image or the identified position of the first partial image; as well as The generated first frame is sent to the electronic device.

15. A computer-readable recording medium having recorded thereon a program for executing a method on a computer, the method being executed by an edge data network and comprising: obtaining sensor information, the sensor information comprising orientation information detected by an electronic device connected to the edge data network; obtaining state information, the state information comprising at least one state parameter, the at least one state parameter being related to at least one of: (i) a network state between the edge data network and the electronic device, (ii) a state of the edge data network, or (iii) a state of the electronic device, wherein the state of the edge data network and the state of the electronic device are related to processing of image content; Based on the sensor information and the state information, identifying a size of a first partial image and a position of the first partial image; Generate a first frame by encoding the first partial image based on the identified size of the first partial image or the identified position of the first partial image; as well as The generated first frame is sent to the electronic device.

Images