Virtual reality video system for performing in parallel caching, computing and communication and method thereof

Abstract

A virtual reality video system that performs caching, computing, and communication in parallel, and a method of operating the same are disclosed. An edge device used in the virtual reality video system includes: a communication unit which is a communication connection channel with a cloud server or a user device; and a data processing unit that predicts an FOV based on data transmitted from the user device, computes (renders) the predicted FOV data transmitted from the cloud server in response to the predicted FOV, and transmits the rendered data to the user device through the communication unit. Here, at least two of caching the predicted FOV data from the cloud server to the edge device, computing the predicted FOV data at the edge device, communication for transmitting the rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel.

Description

Technology Field

[0001] The present invention relates to a virtual reality video system that performs caching, computing, and communication in parallel, and a method of operating the same. Background Technology

[0002] In existing systems, caching was performed only while the user watched video chunks. As a result, there was significant latency in environments streaming large-capacity videos, such as 360-degree videos. Prior art literature

[0003] (Patent Document 0001) KR 10-2502546 B The problem to be solved

[0004] The present invention provides a virtual reality video system that performs caching, computing, and communication in parallel, and a method of operating the same. means of solving the problem

[0005] To achieve the above-mentioned purpose, an edge device used in a virtual reality video system according to an embodiment of the present invention comprises: a communication unit which is a communication connection channel with a cloud server or a user device; and a data processing unit which predicts an FOV based on data transmitted from the user device, computes (renders) the predicted FOV data transmitted from the cloud server in response to the predicted FOV, and transmits the rendered data to the user device through the communication unit. Here, at least two of caching the predicted FOV data from the cloud server to the edge device, computing the predicted FOV data at the edge device, communication for transmitting the rendered data from the edge device to the user device, communication for transmitting the unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel.

[0006] A cloud server used in a virtual reality video system according to an embodiment of the present invention includes a communication unit that is a communication connection channel with an edge device or a user device; and a data processing unit that caches predicted FOV data to the edge device through the communication unit in response to a predicted FOV transmitted from the edge device. Here, at least two of caching of predicted FOV data from the cloud server to the edge device, computing of predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel.

[0007] A user device used in a virtual reality video system according to an embodiment of the present invention includes a communication unit which is a communication connection channel with a cloud server or an edge device; and a data processing unit that computes (renders) data transmitted from the edge device and computes unhit data transmitted from the cloud server. Here, at least two of caching of predicted FOV data from the cloud server to the edge device, computing of predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel.

[0008] A virtual reality video system that predicts and uses FOV according to one embodiment of the present invention includes a mode determining unit that determines a mode according to a streaming environment; and a delay determining unit that determines a delay after determining the mode. Here, the delay varies according to the determined mode.

[0009] A method for operating a virtual reality video system that predicts and uses FOV according to an embodiment of the present invention includes: a step of determining a mode according to a streaming environment; and a step of determining a delay after determining the mode. Herein, the delay varies according to the determined mode. Effects of the invention

[0010] A virtual reality video system and a method of operation thereof that perform caching, computing, and communication in parallel according to the present invention can achieve optimized performance in an environment where large-capacity videos, such as 360-degree videos, are streamed by predicting the FOV and performing caching, communication, and computing in parallel. That is, latency can be minimized. Brief explanation of the drawing

[0013] FIGS. 1 and FIGS. 2 are drawings illustrating the structure and operation of a virtual reality video system according to an embodiment of the present invention. FIGS. 3 to 17 are drawings illustrating the operation and delay determination process of a virtual reality video system according to a mode according to an embodiment of the present invention. Figures 18 and 19 are drawings illustrating experimental results under various scenarios. Specific details for implementing the invention

[0014] As used in this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "composed" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may be excluded, or that additional components or steps may be included. Furthermore, terms such as "...part," "module," etc., as used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

[0016] The present invention relates to a virtual reality video system and a method of operating the same, for example, to a mobile virtual reality video (MVRV) system. In particular, the virtual reality video system can achieve optimized performance in an environment where large-capacity video, such as 360-degree video, is streamed.

[0017] According to one embodiment, the virtual reality video system can perform caching, computing (rendering), and communication in parallel in an environment where video (data) corresponding to the user's field of view (FOV) is downloaded.

[0019] Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings.

[0020] FIGS. 1 and 2 are drawings illustrating the structure and operation of a virtual reality video system according to an embodiment of the present invention, FIGS. 3 to 17 are drawings illustrating the operation and delay determination process of a virtual reality video system according to a mode according to an embodiment of the present invention, and FIGS. 18 and 19 are drawings illustrating experimental results in various scenarios.

[0021] Referring to FIG. 1, the virtual reality video system of the present embodiment may include a cloud server (100), at least one edge device (102), such as an edge computing server (MEC), and a user device (104), such as a head-mounted display (HMD). Hereinafter, for convenience of explanation, the edge device (102) will be referred to as the MEC and the user device (104) will be assumed to be the HMD.

[0022] The cloud server (100) is connected to the MEC (102) and HMD (104) via wired or wireless connection and stores video data.

[0023] Additionally, the cloud server (100) can cache predicted FOV data corresponding to the predicted FOV transmitted from the MEC (102) by transmitting it to the MEC (102) according to the predicted FOV, and can transmit unhit data to the HMD (104).

[0024] MEC (102) can predict the FOV during the video chunk duration, which is the length of individual chunks in streaming video, transmit the predicted FOV to a cloud server (100), cache the predicted FOV data transmitted from the cloud server (100), compute (render) the offloaded data (predicted FOV data), and transmit the rendered data / unrendered data to an HMD (104).

[0025] According to one embodiment, the MEC (102) can predict the FOV based on sensory data that detects the video chunk currently being watched by the user. By predicting the FOV and stringing only the video corresponding to the FOV instead of streaming the entire 360-degree video, bandwidth can be optimized, QoE improved, resource efficiency improved, and latency reduced.

[0026] The HMD (104) can receive FOV data from a cloud server (100) or MEC (102) and render and display the received FOV data.

[0027] Overall, the virtual reality video system can perform proactive caching to cache predicted FOV data, communication between the cloud server (100) and the MEC (102) / between the MEC (102) and the HMD (104) / between the cloud server (100) and the HMD (104), computing (rendering) and caching in the MEC (102) and the HMD (104). This process may be referred to as P3C and is shown in FIG. 2.

[0028] Referring to FIG. 2, during the duration of the video chunk, proactive caching is performed from the cloud server (100) to the MEC (102), the MEC (102) performs proactive computing, proactive communication is performed from the MEC (102) to the HMD (104) for rendered data / unrendered data, and the HMD (104) can perform proactive computing.

[0029] According to one embodiment, the virtual reality video system may determine a mode using various streaming environment parameters as illustrated in FIG. 2, perform a P3C process according to the determined mode during the video chunk duration based on the predicted FOV, and determine a delay based on the actual FOV. Such mode determination, P3C process execution, and delay determination may be performed every cycle. Here, the FOV may be predicted based on the video chunk currently being viewed by the user, i.e., the video chunk of the previous cycle. Such prediction of the FOV is not limited to a specific algorithm and various algorithms may be used.

[0030] According to one embodiment, the virtual reality video system can perform caching, communication, and computing in parallel (together). In existing systems, only caching is performed while a user watches a video chunk, but the virtual reality video system of the present invention can perform caching, communication, and computing in parallel.

[0031] The above delay may include a computing delay for remaining data in the MEC (102), a computing delay for remaining data in the HMD (104), a computing delay for unrendered data from the MEC (102) in the HMD (104), a computing delay for unheated data from the cloud server (100) in the HMD (104), a communication delay from the MEC (102) to the HMD (104) for remaining rendered data / unrendered data, a communication delay for remaining rendered data from the MEC (102) to the HMD (104), a communication delay for rendered data from the MEC (102) to the HMD (104), and a communication delay for unheated data from the cloud server (100) to the HMD (104). Here, the above delay may mean the time delay until data corresponding to the FOV is displayed in the HMD (104).

[0032] These mode determination and delay determination are performed by the mode determination unit and delay determination unit of the MEC (102), respectively, but may also be performed by the cloud server (100). Expandably, the mode determination unit and the delay determination unit may not be positionally limited as components of the virtual reality video system.

[0033] Meanwhile, the delay may vary depending on the mode.

[0035] First, we will examine the method for determining the mode, and then sequentially examine the caching, communication, computing, and delay processes according to the mode. In FIGS. 3, 5, 7, 9, 11, 13, 15, and 17, among the delay time blocks, blocks marked with shadows represent processes that are not performed, while blocks not marked with shadows represent processes that are performed. Additionally, actions with the same color are performed sequentially.

[0037] Mode determination method

[0038] The mode determination unit of the above-mentioned virtual reality video system can determine the mode by considering various streaming environments.

[0039] According to one embodiment, the MEC (102) can determine a mode based on at least one of the following: video chunk duration, the rate of change in data size after MEC offloading, the compression rate of the video chunk, the data size processed per second by the MEC (102), the data size processed per second by the HMD (104), the data transfer rate from the cloud server (100) to the HMD (104), and the data transfer rate from the cloud server (100) to the MEC (102).

[0040] That is, the above mode is determined by considering the current streaming environment of the virtual reality video system, and the virtual reality video system can operate in the determined mode.

[0041] According to one embodiment, there may be eight modes.

[0042] - Mode 1: An environment where all 3 proactive Cs can be performed during the video chunk duration (t0).

[0043] - Mode 2: An environment where computing on the HMD (104) is not completed during the video chunk duration (t0).

[0044] - Mode 3: An environment where communication for the rendered data of the proactive 3C is not completed during the video chunk duration (t0).

[0045] - Mode 4: An environment in which communication for computing and rendering data in the HMD (104) of the proactive 3C is not completed during the video chunk duration (t0).

[0046] - Mode 5: An environment where only computing in the MEC (102) of the proactive 3C is completed during the video chunk duration (t0).

[0047] - Mode 6: An environment where computing and communication for rendered data in the MEC (10) of the proactive 3C are not completed during the video chunk duration (t0).

[0048] - Mode 7: An environment where only communication for the unrendered data of the proactive 3C is completed during the video chunk duration (t0).

[0049] - Mode 8: An environment where all 3 proactive Cs are not completed during the video chunk duration (t0).

[0051] Mode 1

[0052] The above mode determination unit can determine the mode as mode 1 if it is an environment where all proactive 3Cs can be performed during the video chunk duration (t0).

[0053] The above virtual reality video system can perform caching, communication, and computing according to Mode 1 based on the FOV predicted by the MEC (102).

[0054] Since all proactive 3Cs are performed during the video chunk duration (t0), there is no separate delay time for the proactive 3Cs, and therefore, the delay determining unit can determine the delay time (D) as the sum of the transmission time of unheat data from the cloud server (100) to the HMD (104) (communication delay for unheat data, t7) and the time of computing the transmitted unheat data at the HMD (104) (computation delay for unheat data, t8), as shown in FIG. 3, FIG. 4 and Equation 1 below. Of course, if there is no unheat data, the delay time is 0.

[0055]

[0056] Here, B FOV represents the size of the FOV data, ρ represents the FOV hit ratio, and C r represents the compression rate, R3 represents the transmission rate from the cloud server (100) to the HMD (104), and W U represents the data size throughput rate in the HMD (104).

[0058] Mode 2

[0059] The above mode determination unit may determine the mode as Mode 2 in the case where computing in the HMD (104) is not completed during the video chunk duration (t0). In this case, since at least part of the computing in the HMD (104) is not completed, this computing delay is considered as a delay time.

[0060] The above delay determining unit can determine the delay time (D) by considering the computing delay (t2) of the remaining data in the HMD (104), the communication delay (t7) for unheat data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheat data in the HMD (104), as shown in FIGS. 5 and 6. At this time, since the computing delay (t2) of the remaining data in the HMD (104) and the communication delay (t7) for unheat data from the cloud server (100) to the HMD (104) are performed in parallel, the delay time (D) can be determined as the sum of the larger of t2 and t3 and t8, as shown in Equation 2 below.

[0061]

[0063] Mode 3

[0064] The above mode determination unit may determine the mode as mode 3 in the case where communication for the rendered data of the proactive 3C is not completed during the video chunk duration (t0). In this case, since at least some of the rendered data from the MEC (102) to the HMD (104) is not transmitted, this communication delay is considered as a delay time.

[0065] The above delay determining unit can determine the delay time (D) as the sum of the communication delay (t5) for the transmission of residual rendered data from the MEC (102) to the HMD (104), the communication delay (t7) for unheat data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheat data in the HMD (104), as shown in FIGS. 7, FIGS. 8 and Equation 3 below.

[0066]

[0067] Here, h represents the rate of change in data size after rendering in the MEC (102), and R2 represents the transmission rate from the MEC (102) to the HMD (104).

[0069] Mode 4

[0070] The above mode determination unit may determine the mode as Mode 4 in an environment where communication for computing and rendering data in the HMD (104) of the proactive 3C is not completed during the video chunk duration (t0). In this case, since at least part of the computing in the HMD (104) is not completed and at least part of the rendered data from the MEC (102) to the HMD (104) is not transmitted, these computing delays and communication delays are considered as delay times.

[0071] The above delay determining unit can determine the delay time (D) by considering the computing delay (t2) of residual data in the HMD (104), the communication delay (t5) for the transmission of residual rendered data from the MEC (102) to the HMD (104), the communication delay (t7) for unheat data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheat data in the HMD (104), as illustrated in FIGS. 9 and FIGS. 10. At this time, since t2 is executed in parallel while t7 is executed after t5 is executed, the delay time (D) is given by the following Equation 4.

[0072]

[0074] Mode 5

[0075] The above mode determination unit may determine the mode as Mode 5 in an environment where only computing in the MEC (102) of the proactive 3C is completed during the video chunk duration (t0). In this case, since the remaining proactive 3C excluding computing in the MEC (102) is not completed, the delay time of the 3C process that is not completed during the video chunk duration (t0) must be taken into account.

[0076] The above delay determining unit can determine the delay time (D) by considering the computing delay (t2) for residual data in the HMD (104), the communication delay (t3) for residual unrendered data from the MEC (102) to the HMD (104), the computing delay (t4) for unrendered data from the MEC (102) to the HMD (104), the communication delay (t5) for residual rendered data from the MEC (102) to the HMD (104), the communication delay (t7) for unheated data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheated data in the HMD (104), as illustrated in FIGS. 11 and 12. At this time, since t2 and t3 are performed in parallel, and t4 is performed in parallel while t5 and t7 are performed sequentially after t2 and t3, the delay time (D) is given by Equation 5 below.

[0077]

[0079] Mode 6

[0080] The above mode determination unit may determine the mode as Mode 6 in the case where communication for computing and rendered data in the MEC (102) of the proactive 3C is not completed during the video chunk duration (t0). In this case, since communication for computing and rendered data in the MEC (102) is not completed, the delay for the uncompleted process must be taken into account in the delay time.

[0081] The above delay determining unit can determine the delay time (D) by considering the computing delay (t1) in the MEC (102), the communication delay (t5) for residual rendered data from the MEC (102) to the HMD (104), the communication delay (t6) for rendered data from the MEC (102) to the HMD (104), the communication delay (t7) for unheat data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheat data in the HMD (104), as shown in FIGS. 13 and 14. At this time, since t1 and t5 are performed in parallel, the delay time (D) is equal to Equation 6 below.

[0082]

[0084] Mode 7

[0085] The above mode determination unit may determine the mode as Mode 7 in an environment where only communication for unrendered data of the proactive 3C is completed during the video chunk duration (t0). In this case, since the remaining 3C processes excluding communication for unrendered data in the MEC (102) are not completed, the delay for the uncompleted processes must be taken into account in the delay time.

[0086] The above delay determining unit can determine the delay time (D) by considering the computing delay (t1) for residual data in the MEC (102), the computing delay (t2) for residual data in the HMD (104), the communication delay (t5) for residual rendered data from the MEC (102) to the HMD (104), the communication delay (t6) for rendered data from the MEC (102) to the HMD (104), the communication delay (t7) for unheat data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheat data in the HMD (104), as shown in FIGS. 15 and 16. At this time, since t1 and t5 are performed in parallel and t2 is performed in parallel with t1, t5, t6, and t7, the delay time (D) is given by Equation 7 below.

[0087]

[0088] Here, W M represents the data size throughput rate in MEC (102).

[0090] Mode 8

[0091] The above mode determination unit may determine the mode as mode 8 in the case where all proactive 3Cs are not completed during the video chunk duration (t0). In this case, since the entire 3C process is not completed, the delay for the uncompleted process must be taken into account in the delay time.

[0092] The above delay determination unit can determine the delay time (D) by considering the computing delay (t1) in the MEC (102), the computing delay (t2) for residual data in the HMD (104), the communication delay (t3) for residual unrendered data from the MEC (102) to the HMD (104), the computing delay (t4) for unremastered data from the MEC (102) to the HMD (104), the communication delay (t5) for residual rendered data from the MEC (102) to the HMD (104), the communication delay (t6) for rendered data from the MEC (102) to the HMD (104), the communication delay (t7) for unheated data from the cloud server (100) to the HMD (104), and the computing delay (t8) for unheated data in the HMD (104), as shown in FIG. 17.

[0094] Meanwhile, the above modes are examples and can be varied depending on the streaming environment.

[0095] According to one embodiment, the mode determining unit can determine the mode using ModeNet, and the delay determining unit can determine the delay using a Rendering Decision Network (RDN).

[0096] The above ModeNet can optimize the caching rate and pre-rendering rate before the FOV is acquired, and the above RDN can optimize the rendering rate before the FOV is acquired.

[0098] Below, we will examine the operation of individual components in the above-mentioned virtual reality video system.

[0099] The MEC device (102) may include a communication unit; and a data processing unit that predicts the FOV based on data transmitted from the HMD (104), computes (renders) the predicted FOV data transmitted from the cloud server (100) in response to the predicted FOV, and transmits the rendered data to the user device through the communication unit.

[0100] Here, at least two of the following can be performed in parallel: caching of predicted FOV data from the cloud server (100) to the MEC (102), computing of predicted FOV data in the MEC (102), communication for transmitting rendered data from the MEC (102) to the HMD (104), communication for transmitting unrendered data from the MEC (102) to the HMD (104), and data computing of the HMD (104).

[0101] Additionally, the above delay includes a computing delay for residual data in the MEC (102), a communication delay for residual unrendered data from the MEC (102) to the HMD (104), a communication delay for residual rendered data from the MEC (102) to the HMD (104), and a communication delay for rendered data from the MEC (102) to the HMD (104), wherein the delay may vary depending on the mode.

[0102] The cloud server (100) may include a communication unit; and a data processing unit that caches predicted FOV data to the MEC (102) through the communication unit in response to a predicted FOV transmitted from the MEC (102).

[0103] Here, at least two of the following can be performed in parallel: caching of predicted FOV data from the cloud server (100) to the MEC (102), computing of predicted FOV data in the MEC (102), communication for transmitting rendered data from the MEC (102) to the HMD (104), communication for transmitting unrendered data from the MEC (102) to the HMD (104), and data computing of the HMD (104).

[0104] The HMD (104) may include a communication unit; and a data processing unit that computes (renders) data transmitted from the MEC (102) and computes unheated data transmitted from the cloud server (100).

[0105] Here, at least two of the following can be performed in parallel: caching of predicted FOV data from the cloud server (100) to the MEC (102), computing of predicted FOV data in the MEC (102), communication for transmitting rendered data from the MEC (102) to the HMD (104), communication for transmitting unrendered data from the MEC (102) to the HMD (104), and data computing of the HMD (104).

[0106] Additionally, the above delay may include a computing delay for residual data in the HMD (104), a computing delay for unrendered data from the MEC (102) in the HMD (104), a computing delay for unheated data from the cloud server (100) in the HMD (104), a communication delay for residual unrendered data from the MEC (102) to the HMD (104), a communication delay for residual rendered data from the MEC (102) to the HMD (104), a communication delay for rendered data from the MEC (102) to the HMD (104), and a communication delay for unheated data from the cloud server (100) to the HMD (104).

[0108] Looking at the experimental results, as shown in FIGS. 18 and 19, it can be confirmed that the performance of the virtual reality video system of the present invention is the best for all scenarios. That is, it is effective in reducing average latency in all scenarios.

[0110] Meanwhile, the components of the aforementioned embodiments can be easily identified from a process perspective. That is, each component can be identified as a respective process. Furthermore, the processes of the aforementioned embodiments can be easily identified from the perspective of the device components.

[0111] In addition, the technical details described above may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiments, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa. Industrial applicability

[0112] The embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art with ordinary knowledge of the present invention may make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the following claims. Explanation of the symbols

[0113] 100 : Cloud Server 102 : Edge Device (MEC) 104 : User Device (HMD)

Claims

Claim 1 An edge device used in a virtual reality video system comprises: a communication unit which is a communication connection channel with a cloud server or a user device; and a data processing unit which predicts an FOV based on data transmitted from the user device, computes (renders) predicted FOV data transmitted from the cloud server in response to the predicted FOV, and transmits the rendered data to the user device through the communication unit, wherein at least two of caching of the predicted FOV data from the cloud server to the edge device, computing of the predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel. Claim 2 An edge device according to claim 1, wherein caching of predicted FOV data from the cloud server to the edge device, computing of predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed during the video chunk duration after the FOV prediction, and a delay is performed after the video chunk duration, wherein the delay is performed after the actual FOV occurs. Claim 3 An edge device according to claim 2, wherein the delay includes a computing delay for residual data at the edge device, a communication delay for residual unrendered data from the edge device to the user device, a communication delay for residual rendered data from the edge device to the user device, and a communication delay for rendered data from the edge device to the user device, wherein the delay varies according to the mode. Claim 4 A cloud server used in a virtual reality video system comprises: a communication unit which is a communication connection channel with an edge device or a user device; and a data processing unit which caches predicted FOV data to the edge device through the communication unit in response to a predicted FOV transmitted from the edge device, wherein at least two of caching of predicted FOV data from the cloud server to the edge device, computing of predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel. Claim 5 In claim 4, the caching of predicted FOV data from the cloud server to the edge device, the computing of predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing at the user device are performed during the video chunk duration after the FOV prediction, and a delay is performed after the video chunk duration, wherein the delay is performed after the actual FOV occurs. Claim 6 A user device used in a virtual reality video system comprises: a communication unit which is a communication connection channel with a cloud server or an edge device; and a data processing unit which computes (renders) data transmitted from the edge device and computes unhit data transmitted from the cloud server, wherein at least two of caching predicted FOV data from the cloud server to the edge device, computing predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing of the user device are performed in parallel. Claim 7 In claim 6, caching of predicted FOV data from the cloud server to the edge device, computing of predicted FOV data at the edge device, communication for transmitting rendered data from the edge device to the user device, communication for transmitting unrendered data from the edge device to the user device, and data computing at the user device are performed during the video chunk duration after the FOV prediction, and a delay is performed after the video chunk duration, wherein the delay is performed after the actual FOV occurs, and the delay includes a computing delay for residual data at the user device, a computing delay for unrendered data from the edge device at the user device, a computing delay for unheated data, a communication delay for residual unrendered data from the edge device to the user device, a communication delay for residual rendered data from the edge device to the user device, a communication delay for rendered data from the edge device to the user device, and a communication delay for unheated data from the cloud server to the user device. User device. Claim 8 A virtual reality video system that uses a predicted FOV, comprising: a mode determining unit that determines a mode according to a streaming environment; and a delay determining unit that determines a delay after determining the mode, wherein the delay varies according to the determined mode. Claim 9 A virtual reality video system according to claim 8, wherein the mode determining unit determines a mode based on at least two of the following: a video chunk duration, a rate of change in data size after offloading of an edge device, a compression rate of a video chunk, a data size processed per second by the edge device, a data size processed per second by the user device, a data transmission rate from a cloud server to the user device, and a data transmission rate from a cloud server to the edge device. Claim 10 A virtual reality video system according to claim 8, wherein the delay comprises a computing delay for residual data at an edge device, a computing delay for residual data at a user device, a computing delay for unrendered data from the edge device at the user device, a computing delay for unhit data from a cloud server at the user device, a communication delay for residual unrendered data from the edge device to the user device, a communication delay for residual rendered data from the edge device to the user device, a communication delay for rendered data from the edge device to the user device, and a communication delay for unhit data from the cloud server to the user device. Claim 11 A virtual reality video system according to claim 8, wherein the mode comprises: Mode 1, which is applied to an environment where caching, computing, and communication can all be performed during the duration of a video chunk; Mode 2, which is applied to an environment where computing at an edge device is not completed during the duration of the video chunk; Mode 3, which is applied to an environment where communication for rendered data of caching, computing, and communication is not completed during the duration of the video chunk; Mode 4, which is applied to an environment where computing at an edge device and communication for rendered data of caching, computing, and communication are not completed during the duration of the video chunk; Mode 5, which is applied to an environment where only computing at an edge device of caching, computing, and communication is completed during the duration of the video chunk; Mode 6, which is applied to an environment where computing at an edge device and communication for rendered data of caching, computing, and communication are not completed during the duration of the video chunk; Mode 7, which is applied to an environment where only communication for unrendered data of caching, computing, and communication is completed during the duration of the video chunk; and Mode 8, which is applied to an environment where caching, computing, and communication are not completed during the duration of the video chunk. Claim 12 A virtual reality video system according to claim 8, characterized in that the determination of the mode and the determination of the delay are performed on an edge device or a cloud server. Claim 13 A method for operating a virtual reality video system that uses a predicted FOV, comprising: a step of determining a mode according to a streaming environment; and a step of determining a delay after determining the mode, wherein the delay varies according to the determined mode.

Images