Method and device for providing content

Abstract

A method for providing content and an apparatus thereof. The method may include the following steps: displaying a first front view of 360° content based on a first user position configured based on a user's head position and a user's body position, the 360° content being displayed in a 360° space based on the user position information; in response to determining that the user position information indicates that the head position has changed to a changed head position while the body position remains unchanged, displaying a second front view of the 360° content based on the changed head position, the second front view being different from the first front view; and in response to determining that the user position information indicates that the head position has changed to the changed head position and the body position has changed to the changed body position, continuing to display the first front view based on the changed head position and the changed body position.

Description

Technical Field

[0001] The method for providing content and the apparatus thereof according to an embodiment relate to a method and apparatus for providing XR content (e.g., VR content, AR content, MR content, etc.), which are applicable to all technical fields of 5G communication, robotics, autonomous driving, and artificial intelligence (AI). Background Art

[0002] Virtual reality (VR) technology provides real-world objects or backgrounds solely as CG (computer graphics) images. Augmented reality (AR) technology provides CG images superimposed on the user's real-world view, creating a composite view. Mixed reality (MR) technology is a computer graphics technology that provides virtual objects by mixing and combining them with the real world. The aforementioned VR, AR, and MR technologies can all be referred to as extended reality (ER) technologies. Summary of the Invention

[0003] Devices according to embodiments can provide a frontal view of 360° content based on the user's position. However, if the device does not accurately detect the user's position, it may not accurately respond with a frontal view corresponding to the user's desired direction. Furthermore, if excessive user position changes are required to provide 360° content, this may increase fatigue for users consuming the content. Such content may hinder user immersion and result in a low-quality user experience.

[0004] Therefore, the device according to the embodiment can provide a front view of 360° content based on user position information obtained based on at least one of the head position and the body position of the user.

[0005] Accordingly, embodiments of the present disclosure are directed to a method and apparatus for providing contents that substantially obviate one or more problems due to limitations and disadvantages of the related art.

[0006] Additional advantages, objectives and features of the present invention will be described in the disclosure of this document and the accompanying drawings. These aspects can also be understood by those skilled in the art based on the disclosure of this document.

[0007] To achieve these objects and other advantages and in accordance with the purposes of the present invention, as embodied and broadly described herein, a method for providing content according to one embodiment of the present disclosure may include the following steps: displaying a first front view of 360° content based on a first user position configured based on the user's head position and body position, the 360° content including content that can be displayed in a 360° space represented as one or more areas based on user position information; ensuring user position information; if the ensured user position information indicates a change in the head position, displaying a second front view of the 360° content based on a second user position determined based on the changed head position; and if the ensured user position information indicates a change in the user's head position and body position, displaying the first front view based on a third user position determined based on the changed head position and body position.

[0008] In another aspect of the present disclosure, as specifically implemented and broadly described herein, a content providing apparatus (or device) according to another embodiment of the present disclosure may include: one or more sensors that ensure user position information configured based on a user's head position and body position; one or more processors that process 360° content to be displayed based on the ensured user position information, the 360° content including content that can be displayed in a 360° space represented as one or more areas based on the user position information; one or more memories; and a display that displays a first front view of the 360° content based on a first user position configured based on the user's head position and body position, and if the ensured user position information indicates a change in the head position, displays a second front view of the 360° content based on a second user position determined based on the changed head position, and if the ensured user position information indicates a change in the user's head position and body position, displays the first front view based on a third user position determined based on the changed head position and body position.

[0009] Therefore, the embodiments of the present disclosure provide various effects and / or features.

[0010] First, the content providing method and apparatus according to the embodiments can provide more realistic XR content.

[0011] Second, the content providing method and apparatus according to the embodiments may increase user convenience and provide various user experiences.

[0012] Effects obtainable from the present disclosure may not be limited to the above-mentioned effects, and other unmentioned effects may be clearly understood from the following description by those skilled in the art.

[0013] It is to be understood that both the foregoing general description and the following detailed description of the present disclosure are illustrative and provide examples intended to provide further explanation of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention. In the drawings:

[0015] Figure 1 is a diagram illustrating an example of a resource grid for mapping physical signals / channels in a 3GPP-based system according to an embodiment of the present disclosure;

[0016] Figure 2 is a diagram showing one example of a 3GPP signal transmission / reception method according to an embodiment of the present disclosure;

[0017] Figure 3 is a diagram showing an example of an SSB structure according to an embodiment of the present disclosure;

[0018] Figure 4 is a diagram showing an example of a random access process according to an embodiment of the present disclosure;

[0019] Figure 5 is a diagram illustrating one example of UL transmission according to an uplink grant according to an embodiment of the present disclosure;

[0020] Figure 6 is a diagram illustrating an example of the concept of physical channel processing according to an embodiment of the present disclosure;

[0021] Figure 7 is a block diagram illustrating one example of a transmit stage and a receive stage for hybrid beamforming according to an embodiment of the present disclosure;

[0022] Figure 8a is a diagram showing an example of narrowband operation according to an embodiment of the present disclosure, Figure 8b is a diagram illustrating one example of MTC channel repetition with RF retuning according to an embodiment of the present disclosure;

[0023] Figure 9 is a block diagram illustrating an example of a wireless communication system to which the method suggested in this specification is applied according to an embodiment of the present disclosure;

[0024] Figure 10 is a diagram showing an AI device 1000 according to an embodiment;

[0025] Figure 11 is a diagram illustrating an AI server 1120 according to an embodiment;

[0026] Figure 12 is a diagram illustrating an AI system according to an embodiment;

[0027] Figure 13 It shows that according to the embodiment Figure 10 A block diagram of an XR device;

[0028] Figure 14 Specifically illustrates the embodiment of the present disclosure Figure 13 A block diagram of the memory shown;

[0029] Figure 15 is a diagram illustrating a point cloud data processing system according to an embodiment of the present disclosure;

[0030] Figure 16 is a diagram illustrating an XR device 1600 including a learning processor according to an embodiment of the present disclosure;

[0031] Figure 17 The embodiment of the present disclosure is shown in FIG. Figure 16 A diagram of a process by which an XR device 1600 provides XR services is shown;

[0032] Figure 18 is a diagram showing the appearance of an XR device and a robot according to an embodiment of the present disclosure;

[0033] Figure 19 is a flowchart of a process of controlling a robot using a device loaded with XR technology according to an embodiment of the present disclosure;

[0034] Figure 20 is a diagram illustrating a vehicle providing an autonomous driving service according to an embodiment of the present disclosure;

[0035] Figure 21 is a diagram illustrating a process of providing AR / VR services during autonomous driving services according to an embodiment of the present disclosure;

[0036] Figure 22 1 is a diagram illustrating a case where an HMD-type XR device is implemented according to an embodiment;

[0037] Figure 23 FIG1 is a diagram showing a case where an AR glasses-type XR device is implemented according to an embodiment;

[0038] Figure 24 is a diagram showing a coordinate system of a 360° space according to an embodiment;

[0039] Figure 25 is a diagram showing a 360° space according to an embodiment;

[0040] Figure 26 is a block diagram showing an example of a configuration of an apparatus according to an embodiment;

[0041] Figure 27 is a diagram showing an example of a content providing operation of a device according to an embodiment;

[0042] Figure 28 is a diagram illustrating an example of a user gesture according to an embodiment;

[0043] Figure 29 is a diagram showing an example of a 360° content providing process according to an embodiment;

[0044] Figure 30 is a diagram showing an example of a 360° content providing process according to an embodiment;

[0045] Figure 31 is a diagram illustrating a field of view (FOV) according to an embodiment;

[0046] Figure 32 is a diagram illustrating an example of a process of configuring priority information in response to a user input signal;

[0047] Figure 33 is a diagram showing an example of a 360° content providing process according to an embodiment;

[0048] Figure 34 is a diagram showing an example of a 360° content providing process according to an embodiment;

[0049] Figure 35 is a diagram showing an example of a 360° content providing process according to an embodiment;

[0050] Figure 36 is a flowchart illustrating a content providing method according to an embodiment. DETAILED DESCRIPTION

[0051] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals will be used throughout the accompanying drawings to represent the same or similar parts, and redundant descriptions will be avoided. For ease of description only, the terms "module" and "unit" are used interchangeably, and therefore they should not be considered to have different meanings or roles. In addition, detailed descriptions of well-known technologies will not be given when describing the embodiments of the present disclosure, so as not to obscure the subject matter of the embodiments. The accompanying drawings are provided to assist in understanding the embodiments of the present disclosure, rather than to limit the scope of the present disclosure. It will be understood that the present disclosure encompasses various modifications, equivalents and / or alternatives that fall within the scope and spirit of the present disclosure.

[0052] The following embodiments of the present disclosure are intended to specifically implement the present disclosure, rather than to limit the scope of the present disclosure. Contents that can be easily derived from the detailed description and embodiments of the present disclosure by those skilled in the art are to be construed as falling within the scope of the present disclosure.

[0053] Therefore, the above embodiments are to be interpreted in all aspects as illustrative rather than restrictive. The scope of the present disclosure should be determined by the appended claims and their legal equivalents, rather than by the above description, and all changes coming within the meaning and equivalency range of the appended claims are intended to be encompassed therein.

[0054] introduce

[0055] In this disclosure, downlink (DL) refers to communication from a base station (BS) to a user equipment (UE), and uplink (UL) refers to communication from a UE to a BS. On the DL, the transmitter can be part of the BS and the receiver can be part of the UE, while on the UL, the transmitter can be part of the UE and the receiver can be part of the BS. In this disclosure, the UE can be referred to as a first communication device, and the BS can be referred to as a second communication device. The term BS can be replaced by a fixed station, Node B, evolved Node B (eNB), next-generation Node B (gNB), base transceiver system (BTS), access point (AP), network or 5th generation (5G) network node, artificial intelligence (AI) system, roadside unit (RSU), robot, augmented reality / virtual reality (AR / VR) system, etc. The term UE can be replaced by a terminal, mobile station (MS), user terminal (UT), mobile subscriber station (MSS), subscriber station (SS), advanced mobile station (AMS), wireless terminal (WT), device-to-device (D2D) device, vehicle, robot, AI device (or module), AR / VR device (or module), etc.

[0056] The following technologies can be used in various wireless access systems including Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Orthogonal Frequency Division Multiple Access (OFDMA), and Single Carrier FDMA (SC-FDMA).

[0057] For ease of description, the present disclosure is described in the context of a 3rd Generation Partnership Project (3GPP) communication system (e.g., Long Term Evolution-Advanced (LTE-A) and New Radio or New Radio Access Technology (NR)), which should not be construed as limiting the present disclosure. For reference, 3GPP LTE is part of the Evolved Universal Mobile Telecommunications System (E-UMTS) using Evolved UMTS Terrestrial Radio Access (E-UTRA), and LTE-A / LTE-A pro is an evolution of 3GPP LTE. 3GPP NR is an evolution of 3GPP / LTE-A / LTE-A pro.

[0058] In this disclosure, a node refers to a fixed point that can transmit and receive wireless signals by communicating with a UE. Various types of base stations (BSs) can be used as nodes, regardless of their name. For example, a BS, a NB, an eNB, a picocell eNB (PeNB), a home eNB (HeNB), a repeater, and a transponder can be a node. At least one antenna is installed in a node. An antenna can refer to a physical antenna, an antenna port, a virtual antenna, or an antenna group. A node is also referred to as a point.

[0059] In the present disclosure, a cell may refer to a specific geographical area or radio resource where one or more nodes provide communication services. A "cell" as a geographical area may be understood as a coverage area where services can be provided in a carrier, while a "cell" as a radio resource is associated with the frequency size (i.e., bandwidth (BW)) configured in the carrier. Since the range in which a node can send a valid signal (i.e., DL coverage) and the range in which a node can receive a valid signal from a UE (i.e., UL coverage) depend on the carrier carrying the signal, the coverage of a node is associated with the "cell" coverage of the radio resource used by the node. Therefore, in some cases, the term "cell" may mean the service coverage area of a node, a radio resource, or a range in a radio resource where a signal reaches with effective strength.

[0060] In the present disclosure, communication with a specific cell may be equivalent to communication with a BS or node that provides communication services to the specific cell. In addition, the DL / UL signal of a specific cell means the DL / UL signal from / to the BS or node that provides communication services to the specific cell. Specifically, a cell that provides UL / DL communication services to a UE is referred to as a serving cell of the UE. In addition, the channel state / quality of a specific cell refers to the channel state / quality of the channel or communication link established between the UE and the BS or node that provides communication services to the specific cell.

[0061] A "cell" associated with radio resources can be defined as a combination of DL resources and UL resources, i.e., a combination of a DL component carrier (CC) and a UL CC. A cell can be configured with only DL resources, or a combination of both DL and UL resources. When carrier aggregation (CA) is supported, the link between the carrier frequency of the DL resources (or DL CC) and the carrier frequency of the UL resources (or UL CC) can be indicated by system information transmitted in the corresponding cell. The carrier frequency can be the same as or different from the center frequency of each cell or CC. Hereinafter, a cell operating in a primary frequency is referred to as a primary cell (Pcell) or PCC, and a cell operating in a secondary frequency is referred to as a secondary cell (Scell) or SCC. An Scell can be configured after a UE and a base station perform a radio resource control (RRC) connection establishment procedure, thus establishing an RRC connection between the UE and the base station (i.e., the UE is RRC_CONNECTED). An RRC connection can refer to a path through which the UE's RRC can exchange RRC messages with the base station's RRC. An Scell can be configured to provide additional radio resources to the UE. An Scell and a Pcell can form a serving cell set for the UE based on the UE's capabilities. For an RRC_CONNECTED UE that is not configured with CA or does not support CA, there is only one serving cell configured with Pcell.

[0062] Each cell supports a unique radio access technology (RAT). For example, an LTE cell performs transmission / reception based on the LTE RAT, while a 5G cell performs transmission / reception based on the 5G RAT.

[0063] CA aggregates multiple carriers, each of which has a system BW smaller than a target BW, to support broadband. CA differs from OFDMA in that, in the former, DL or UL communication is performed in multiple carrier frequencies, each forming a system BW (or channel BW), and in the latter, DL or UL communication is performed by loading a basic frequency band divided into multiple orthogonal subcarriers in one carrier frequency. For example, in OFDMA or orthogonal frequency division multiplexing (OFDM), one frequency band with a specific system BW is divided into multiple subcarriers with a predetermined subcarrier spacing, information / data is mapped to the multiple subcarriers, and the frequency band mapped with the information / data is transmitted in the carrier frequency of the frequency band by frequency up-conversion. In wireless CA, frequency bands, each of which has a system BW and a carrier frequency, can be used for communication at the same time, and each frequency band used in CA can be divided into multiple subcarriers with a predetermined subcarrier spacing.

[0064] The 3GPP communication standard defines DL physical channels corresponding to resource elements (REs) that convey information from upper layers of the physical layer (e.g., the Medium Access Control (MAC) layer, the Radio Link Control (RLC) layer, the Packet Data Convergence Protocol (PDCP) layer, the Radio Resource Control (RRC) layer, the Service Data Adaptation Protocol (SDAP) layer, and the Non-Access Stratum (NAS) layer), as well as DL physical signals corresponding to REs used in the physical layer but not conveying information from upper layers. For example, the Physical Downlink Shared Channel (PDSCH), the Physical Broadcast Channel (PBCH), the Physical Multicast Channel (PMCH), the Physical Control Format Indicator Channel (PCFICH), and the Physical Downlink Control Channel (PDCCH) are defined as DL physical channels, and Reference Signals (RSs) and Synchronization Signals are defined as DL physical signals. RSs (also known as pilots) are signals with predefined special waveforms known to both the base station and the user equipment terminal. For example, Cell-Specific RSs (CRSs), UE-Specific RSs (UE-RSs), Positioning RSs (PRSs), Channel State Information RSs (CSI-RSs), and Demodulation RSs (DMRSs) are defined as DL RSs. The 3GPP communication standard also defines UL physical channels corresponding to REs that transmit information from upper layers, and UL physical signals corresponding to REs used in the physical layer but not carrying information from upper layers. For example, the physical uplink shared channel (PUSCH), the physical uplink control channel (PUCCH), and the physical random access channel (PRACH) are defined as UL physical channels, and DMRS for UL control / data signals and sounding reference signals (SRS) for UL channel measurement are defined.

[0065] In the present disclosure, physical shared channels (eg, PUSCH and PDSCH) are used to transmit information originating from upper layers (eg, MAC layer, RLC layer, PDCP layer, RRC layer, SDAP layer, and NAS layer) of the physical layer.

[0066] In the present disclosure, RS refers to a signal with a predefined special waveform known to both the base station and the user equipment terminal. For example, in the 3GPP communication system, the cell-common RS, the UE-RS for demodulating the physical channel of a specific UE, the CSI-RS for measuring / estimating the DL channel state, and the DMRS for demodulating the physical channel are defined as DL RSs, and the DMRS for demodulating the UL control / data signal and the SRS for measuring / estimating the UL channel state are defined as UL RSs.

[0067] In the present disclosure, a transport block (TB) is a payload for the physical layer. For example, data provided to the physical layer by an upper layer or MAC layer is basically referred to as a TB. A UE, which is a device including an AR / VR module (e.g., an AR / VR device), can transmit a TB including AR / VR data to a wireless communication network (e.g., a 5G network) on a PUSCH. In addition, the UE can receive a TB including AR / VR data of a 5G network or a TB including a response to the AR / VR data transmitted by the UE from the wireless communication network.

[0068] In the present disclosure, hybrid automatic repeat request (HARQ) is an error control technology. HARQ acknowledgments (HARQ-ACKs) sent on the DL are used for error control of UL data, and HARQ-ACKs sent on the UL are used for error control of DL data. A transmitter performing HARQ operations waits for an ACK after sending data (e.g., a TB or codeword). A receiver performing HARQ operations sends an ACK only when data is successfully received, and sends a negative ACK (NACK) when the received data contains errors. Upon receiving an ACK, the transmitter can send (new) data, and upon receiving a NACK, the transmitter can resend the data.

[0069] In this disclosure, CSI generally refers to information indicating the quality of a radio channel (or link) established between a UE and an antenna port. CSI may include at least one of a channel quality indicator (CQI), a precoding matrix indicator (PMI), a CSI-RS resource indicator (CRI), a synchronization signal block resource indicator (SSBRI), a layer indicator (LI), a rank indicator (RI), or reference signal received power (RSRP).

[0070] In the present disclosure, frequency division multiplexing (FDM) is the transmission / reception of signals / channels / users in different frequency resources, and time division multiplexing (TDM) is the transmission / reception of signals / channels / users in different time resources.

[0071] In the present disclosure, frequency division duplex (FDD) is a communication scheme in which UL communication is performed in a UL carrier and DL communication is performed in a DL carrier linked to the UL carrier, and time division duplex (TDD) is a communication scheme in which UL communication and DL communication are performed in a time-division manner in the same carrier. In the present disclosure, half-duplex is a scheme in which a communication device operates on UL or DL in only one frequency at one point in time, and operates on DL or UL in another frequency at another point in time. For example, when a communication device operates in half-duplex, the communication device communicates in UL and DL frequencies, wherein the communication device performs UL transmission in the UL frequency for a predetermined time, and retunes to the DL frequency and performs DL reception in the DL frequency in a time-division manner for another predetermined time, without using the UL and DL frequencies at the same time.

[0072] Figure 1 is a diagram showing an example resource grid to which physical signals / channels are mapped in a 3GPP system.

[0073] Reference Figure 1 , for each subcarrier spacing configuration and carrier, N is defined size，μ grid *N RB sc Subcarrier×14·2 μ OFDM symbol resource grid. In this paper, N size，μ grid Indicated by RRC signaling from the BS, μ represents the subcarrier spacing Δf given by Δf = 2μ*15[kHz], where μ∈{0,1,2,3,4} in the 5G system.

[0074] In addition to the subcarrier spacing configuration μ, N size，μ grid It can be different between UL and DL. For each subcarrier spacing configuration μ, antenna port p and transmission direction (UL or DL), there is one resource grid. Each element of the resource grid for the subcarrier spacing configuration μ and antenna port p is called an RE uniquely identified by an index pair (k, l), where k is the frequency domain index and l is the position of the symbol in the relative time domain with respect to the reference point. The frequency unit used to map the physical channel to the RE, a resource block (RB) consists of 12 consecutive subcarriers (N RB sc Considering that the UE may not support the wide BW supported by the 5G system for a while, the UE may be configured to operate in a portion of the frequency BW of the cell, referred to as the bandwidth part (BWP).

[0075] For the background technology, terms and abbreviations used in this disclosure, reference may be made to standard specifications published before this disclosure. For example, reference may be made to the following documents.

[0076] 3GPP LTE

[0077] -3GPP TS 36.211: Physical channels and modulation

[0078] -3GPP TS 36.212: Multiplexing and channel coding

[0079] -3GPP TS 36.213: Physical layer procedures

[0080] -3GPP TS 36.214: Physical layer; Measurement

[0081] -3GPP TS 36.300: General Description

[0082] -3GPP TS 36.304: User Equipment (UE) procedures in idle mode

[0083] -3GPP TS 36.314: Layer 2 - Measurement

[0084] -3GPP TS 36.321: Medium Access Control (MAC) Protocol

[0085] -3GPP TS 36.322: Radio Link Control (RLC) protocol

[0086] -3GPP TS 36.323: Packet Data Convergence Protocol (PDCP)

[0087] -3GPP TS 36.331: Radio Resource Control (RRC) Protocol

[0088] -3GPP TS23.303: Proximity-based Services (Prose); Stage 2

[0089] -3GPP TS23.285: Architecture enhancements for V2X services

[0090] -3GPP TS 23.401: General Packet Radio Service (GPRS) enhancements for Evolved Universal Terrestrial Radio Access Network (E-UTRAN) access

[0091] -3GPP TS23.402: Architecture enhancements for non-3GPP access

[0092] -3GPP TS23.286: Application layer support for V2X services; functional architecture and information flow

[0093] -3GPP TS 24.301: Non-Access Stratum (NAS) Protocol for Evolved Packet System (EPS); Stage 3

[0094] 3GPP TS 24.302: Access to the 3GPP Evolved Packet Core (EPC) via non-3GPP access networks; Stage 3

[0095] -3GPP TS 24.334: Proximity Services (ProSe) User Equipment (UE) to ProSe functional protocol aspects; Stage 3

[0096] -3GPP TS24.386: User Equipment (UE) to V2X Control Function; Protocol Aspects; Stage 3

[0097] 3GPP NR (e.g., 5G)

[0098] -3GPP TS 38.211: Physical channels and modulation

[0099] -3GPP TS 38.212: Multiplexing and channel coding

[0100] -3GPP TS 38.213: Physical layer procedures for control

[0101] -3GPP TS 38.214: Physical layer procedures for data

[0102] -3GPP TS 38.215: Physical layer measurements

[0103] -3GPP TS 38.300: NR and NG-RAN general description

[0104] -3GPP TS 38.304: User Equipment (UE) procedures in idle mode and RRC inactive state

[0105] -3GPP TS 38.321: Medium Access Control (MAC) Protocol

[0106] -3GPP TS 38.322: Radio Link Control (RLC) protocol

[0107] -3GPP TS 38.323: Packet Data Convergence Protocol (PDCP)

[0108] -3GPP TS 38.331: Radio Resource Control (RRC) Protocol

[0109] -3GPP TS 37.324: Service Data Adaptation Protocol (SDAP)

[0110] -3GPP TS 37.340: Multiple Connectivity; General Description

[0111] -3GPP TS23.287: Application layer support for V2X services; functional architecture and information flow

[0112] -3GPP TS23.501: System architecture for 5G systems

[0113] -3GPP TS23.502: Procedures for 5G systems

[0114] -3GPP TS23.503: Policy and Charging Control Framework for 5G Systems; Stage 2

[0115] -3GPP TS24.501: Non-Access Stratum (NAS) Protocol for 5G System (5GS); Stage 3

[0116] -3GPP TS 24.502: Access to the 3GPP 5G Core Network (5GCN) via a non-3GPP access network

[0117] -3GPP TS 24.526: User Equipment (UE) Policy for 5G System (5GS); Stage 3

[0118] Figure 2 is a diagram illustrating an example method of transmitting / receiving a 3GPP signal.

[0119] Reference Figure 2 When a UE is powered on or enters a new cell, it performs an initial cell search (S201) that involves acquiring synchronization with the base station (BS). For the initial cell search, the UE receives the primary synchronization channel (P-SCH) and the secondary synchronization channel (S-SCH), acquires synchronization with the BS, and obtains information such as a cell identifier (ID) from the P-SCH and S-SCH. In LTE and NR systems, the P-SCH and S-SCH are referred to as the primary synchronization signal (PSS) and the secondary synchronization signal (SSS), respectively. The initial cell search process will be described in more detail below.

[0120] After the initial cell search, the UE may receive the PBCH from the BS and acquire broadcast information within the cell from the PBCH. During the initial cell search, the UE may check the DL channel status by receiving the DL RS.

[0121] When the initial cell search is completed, the UE may obtain more specific system information by receiving the PDCCH and receiving the PDSCH according to the information carried on the PDCCH ( S202 ).

[0122] When the UE initially accesses the BS or has no radio resources for signal transmission, the UE may perform a random access procedure with the BS (S203 to S206). To this end, the UE may transmit a predetermined sequence as a preamble on the PRACH (S203 and S205) and receive the PDCCH, and receive a random access response (RAR) message in response to the preamble on the PDSCH corresponding to the PDCCH (S204 and S206). If the random access procedure is contention-based, the UE may additionally perform a contention resolution procedure. The random access procedure will be described in more detail below.

[0123] After the above process, the UE may then perform PDCCH / PDSCH reception (S207) and PUSCH / PUCCH transmission (S208) in a general UL / DL signal transmission process. Specifically, the UE receives DCI on the PDCCH.

[0124] The UE monitors the PDCCH candidate set in the monitoring opportunity configured for one or more control element sets (CORESETs) in the serving cell according to the corresponding search space configuration. The PDCCH candidate set to be monitored by the UE is defined from the perspective of the search space set. The search space set can be a common search space set or a UE-specific search space set. A CORESET includes a set of (physical) RBs that last for a duration of one to three OFDM symbols. The network can configure multiple CORESETs for the UE. The UE monitors PDCCH candidates in one or more search space sets. In this article, the monitoring attempts to decode the PDCCH candidates in the search space. When the UE successfully decodes one of the PDCCH candidates in the search space, the UE determines that the PDCCH has been detected from the PDCCH candidate and performs PDSCH reception or PUSCH transmission based on the DCI included in the detected PDCCH.

[0125] The PDCCH can be used to schedule DL transmissions on the PDSCH and UL transmissions on the PUSCH. The DCI in the PDCCH includes: a DL assignment (e.g., a DL grant), which includes at least the modulation and coding format and resource allocation information of the DL shared channel; and a UL grant, which includes the modulation and coding format and resource allocation information of the UL shared channel.

[0126] Initial Access (IA) Process

[0127] Synchronization Signal Block (SSB) transmission and related operations

[0128] Figure 3 is a diagram showing an example SSB structure. A UE can perform cell search, system information acquisition, beam alignment for initial access, DL measurements, etc. based on the SSB. The term SSB is used interchangeably with Synchronization Signal / Physical Broadcast Channel (SS / PBCH).

[0129] Reference Figure 3 , SSB includes PSS, SSS and PBCH. SSB includes four consecutive OFDM symbols, and PSS, PBCH, SSS / PBCH or PBCH are transmitted in each OFDM symbol. PBCH is encoded / decoded based on polar code and modulated / demodulated by quadrature phase shift keying (QPSK). The PBCH in an OFDM symbol includes data REs mapped with complex modulation values of the PBCH and DMRS REs mapped with DMRS for the PBCH. There are three DMRS REs per RB in an OFDM symbol, and three data REs are present between every two DMRS REs.

[0130] Cell Search

[0131] Cell search is the process by which the UE acquires time and frequency synchronization with a cell and detects the cell ID (e.g., physical cell ID (PCI)) of the cell. The PSS is used to detect a cell ID in a cell ID group, and the SSS is used to detect a cell ID group. The PBCH is used for SSB (time) index detection and half-frame detection.

[0132] In the 5G system, there are 336 cell ID groups, each of which includes three cell IDs. Therefore, a total of 1008 cell IDs are available. Information about the cell ID group to which a cell's cell ID belongs is provided or obtained from the cell's SSS, and information about the cell IDs of each of the 336 cells within the cell ID is provided or obtained from the PSS.

[0133] SSBs are sent periodically with an SSB periodicity. The UE assumes a default SSB periodicity of 20 ms during the initial cell search. After cell access, the SSB periodicity can be set by the network (e.g., BS) to one of {5 ms, 10 ms, 20 ms, 40 ms, 80 ms, 160 ms}. An SSB burst set is configured at the beginning of the SSB period. An SSB burst set consists of a 5-ms time window (e.g., half a frame), and an SSB can be sent up to L times within an SSB burst set. The maximum number of SSB transmissions, L, can be given as follows depending on the frequency band of the carrier.

[0134] - For the frequency range up to 3 GHz, L = 4

[0135] - For the frequency range from 3 GHz to 6 GHz, L = 8

[0136] - For the frequency range from 6 GHz to 52.6 GHz, L = 64

[0137] The possible time positions of SSBs in a half-frame are determined by the subcarrier spacing, and the periodicity of the half-frames carrying SSBs is configured by the network. The time positions of SSB candidates are indexed chronologically from 0 to L-1 (SSB index) in the SSB burst set (e.g., half-frame). Other SSBs may be sent in different spatial directions (by different beams across the coverage area of the cell) during the duration of the half-frame. Therefore, in 5G systems, the SSB index (SSBI) can be associated with the BS transmit (Tx) beam.

[0138] The UE can acquire DL synchronization by detecting SSBs. The UE can identify the structure of the SSB burst set based on the detected (time) SSBI and, therefore, the symbol / slot / half-frame boundaries. The system frame number (SFN) information and half-frame indication information can be used to identify the frame / half-frame number to which the detected SSB belongs.

[0139] Specifically, the UE can obtain the 10-bit SFN of the frame carrying the PBCH from the PBCH. Subsequently, the UE can obtain the 1-bit half-frame indication information. For example, when the UE detects a PBCH with the half-frame indication bit set to 0, the UE can determine that the SSB to which the PBCH belongs is in the first half-frame of the frame. When the UE detects a PBCH with the half-frame indication bit set to 1, the UE can determine that the SSB to which the PBCH belongs is in the second half-frame of the frame. Finally, the UE can obtain the SSBI of the SSB to which the PBCH belongs based on the DMRS sequence and the PBCH payload transmitted on the PBCH.

[0140] System Information (SI) Acquisition

[0141] SI is divided into a master information block (MIB) and multiple system information blocks (SIBs). SI other than the MIB is referred to as remaining minimum system information (RMSI). For details, refer to the following.

[0142] -MIB includes information / parameters for monitoring the PDCCH that schedules the PDSCH carrying systemInformationBlock1 (SIB1) and is transmitted by the BS on the PBCH of the SSB. For example, the UE can determine from the MIB whether there is any CORESET for the Type0-PDCCH common search space. The Type0-PDCCH common search space is a PDCCH search space and is used to transmit the PDCCH that schedules the SI message. In the case where the Type0-PDCCH common search space exists, the UE can determine (1) a plurality of contiguous RBs and one or more consecutive symbols included in the CORESET and (ii) the PDCCH timing (e.g., the time domain position at which the PDCCH is to be received) based on the information included in the MIB (e.g., pdcch-ConfigSIB1).

[0143] -SIB1 includes information about the availability and scheduling (e.g., transmission period and SI window size) of the remaining SIBs (hereinafter referred to as SIBx, where x is an integer equal to or greater than 2). For example, SIB1 may indicate whether SIBx is broadcast periodically or on-demand at the request of a user. If SIBx is provided on-demand, SIB1 may include information for the UE to send an SI request. The PDCCH scheduling SIB1 is transmitted in the Type 0-PDCCH common search space, and SIB1 is transmitted on the PDSCH indicated by the PDCCH.

[0144] -SIBx is included in the SI message and sent on the PDSCH. Each SI message is sent within a periodic time window (eg, SI window).

[0145] Random access process

[0146] The random access procedure is used for various purposes. For example, the random access procedure can be used for initial network access, handover, and UE-triggered UL data transmission. The UE can acquire UL synchronization and UL transmission resources during the random access procedure. The random access procedure can be contention-based or contention-free.

[0147] Figure 4 is a diagram illustrating an example random access procedure. Specifically, Figure 4 The contention-based random access procedure is shown.

[0148] First, the UE may send a random access preamble on the PRACH as the first message (Msg1) of the random access procedure. In the present disclosure, the random access procedure and the random access preamble are also referred to as a RACH procedure and a RACH preamble, respectively.

[0149] Multiple preamble formats are defined by one or more RACH OFDM symbols and different cyclic prefixes (CPs) (and / or guard times). The RACH configuration of a cell is included in the system information of the cell and provided to the UE. The RACH configuration includes information about the subcarrier spacing of the PRACH, available preambles, preamble formats, etc. The RACH configuration includes association information between SSBs and RACH (time-frequency) resources, that is, association information between SSBIs and RACH (time-frequency) resources. The SSBIs are associated with the Tx beams of the BS, respectively. The UE sends a RACH preamble in the RACH time-frequency resources associated with the detected or selected SSB. The BS can identify the preferred BS Tx beam of the UE based on the time-frequency resources where the RACH preamble is detected.

[0150] The SSB threshold for RACH resource association can be configured by the network, and RACH preamble transmission (e.g., PRACH transmission) or retransmission is performed based on the SSB for which the RSRP that meets the threshold is measured. For example, the UE can select one of the SSBs that meets the threshold and transmit or retransmit the RACH preamble in the RACH resource associated with the selected SSB.

[0151] Upon receiving the RACH preamble from the UE, the BS sends a RAR message (second message (Msg2)) to the UE. The PDCCH that schedules the PDSCH carrying the RAR message is masked with a cyclic redundancy check (CRC) by the RA radio network temporary identifier (RNTI) (RA-RNTI) and sent. When the UE detects the PDCCH masked by the RA-RNTI, the UE can receive the RAR message on the PDSCH scheduled by the DCI transmitted on the PDCCH. The UE determines whether the RAR information of the transmitted preamble (i.e., Msg1) is included in the RAR message. The UE can determine whether the random access information of the transmitted Msg1 is included by checking whether there is a RACH preamble ID for the transmitted preamble. If the UE fails to receive a response to Msg1, the UE may send the RACH preamble a predetermined number of times or less while performing power ramping. The UE calculates the PRACH transmission power for the preamble retransmission based on the latest path loss and the power ramp counter.

[0152] When the UE's RAR information is received on the PDSCH, the UE can obtain timing advance information, initial UL grant and UE temporary cell RNTI (C-RNTI) for UL synchronization. The timing advance information is used to control the UL signal transmission timing. In order to allow better alignment between the UE's PUSCH / PUCCH transmission and the subframe timing of the network side, the network (e.g., BS) can measure the time difference between PUSCH / PUCCH / SRS reception and the subframe, and send timing advance information based on the measured time difference. The UE can perform UL transmission on the PUSCH as the third message (Msg3) of the RACH process. Msg3 may include an RRC connection request and a UE ID. The network may send a fourth message (Msg4) in response to Msg3, and Msg4 may be regarded as a contention resolution message on the DL. When the UE receives Msg4, the UE may enter the RRC_CONNECTED state.

[0153] The contention-free RACH process can be used for the UE to switch to another cell or BS, or when requested by the BS command. The contention-free RACH process is basically similar to the contention-based RACH process. However, compared with the contention-based RACH process in which the preamble to be used is randomly selected from multiple RACH preambles, the BS allocates the preamble to be used by the UE (called a dedicated RACH preamble) to the UE in the contention-free RACH process. Information about the dedicated RACH preamble can be included in the RRC message (e.g., handover command) or provided to the UE according to the PDCCH command. When the RACH process starts, the UE sends a dedicated RACH preamble to the BS. When the UE receives the RACH process from the BS, the RACH process is completed.

[0154] DL and UL transmit / receive operations

[0155] DL send / receive operations

[0156] DL grants (also called DL assignments) can be classified into (1) dynamic grants and (2) configured grants. Dynamic grants are a data transmission / reception method based on dynamic scheduling of a BS, with the goal of maximizing resource utilization.

[0157] The base station schedules DL transmissions via DCI. The UE receives DCI (called DL grant DCI) from the base station for DL scheduling (e.g., including scheduling information for the PDSCH). For example, the DCI for DL scheduling may include the following information: a bandwidth indicator, frequency domain resource assignments, time domain resource assignments, and a modulation and coding scheme (MCS).

[0158] The UE may determine the modulation order, target code rate, and TB size (TBS) of the PDSCH based on the MCS field in the DCI. The UE may receive the PDSCH in time-frequency resources according to the frequency domain resource assignment and the time domain resource assignment.

[0159] The grant of DL configuration is also called semi-persistent scheduling (SPS). The UE may receive an RRC message including resource configuration for DL data transmission from the BS. In the case of DL SPS, the grant of the actual DL configuration is provided by the PDCCH, and the DL SPS is enabled or disabled by the PDCCH. When configuring DL SPS, the BS provides at least the following parameters to the UE through RRC signaling: a configured scheduling RNTI (CS-RNTI) for enabling, disabling and retransmission; and periodicity. The actual DL grant (e.g., frequency resource assignment) for DL SPS is provided to the UE in the PDCCH addressed to the CS-RNTI through DCI. If a specific field in the DCI of the PDCCH addressed to the CS-RNTI is set to a specific value for scheduling enablement, the SPS associated with the CS-RNTI is enabled. The DCI of the PDCCH addressed to the CS-RNTI includes actual frequency resource allocation information, MCS index, etc. The UE may receive DL data on the PDSCH based on the SPS.

[0160] UL transmit / receive operation

[0161] UL grants can be classified into (1) dynamic grants that dynamically schedule PUSCH through UL grant DCI and (2) configured grants that semi-statically schedule PUSCH through RRC signaling.

[0162] Figure 5 is a diagram illustrating an example UL transmission according to a UL grant. Specifically, Figure 5 (a) shows the UL transmission process based on dynamic grant, Figure 5 (b) shows the UL transmission process based on the configured grant.

[0163] In the case of a UL dynamic grant, the BS sends a DCI including UL scheduling information to the UE. The UE receives DCI for UL scheduling (e.g., including PUSCH scheduling information) on the PDCCH (referred to as UL grant DCI). The DCI for UL scheduling may include, for example, the following information: a BWP indicator, frequency domain resource assignments, time domain resource assignments, and an MCS. In order for the BS to efficiently allocate UL radio resources, the UE may send information about the UL data to be transmitted to the BS, and the BS may allocate UL resources to the UE based on this information. The information about the UL data to be transmitted is called a buffer status report (BSR), and the BSR is related to the amount of UL data stored in the UE's buffer.

[0164] Reference Figure 5 (a) shows the UL transmission procedure for UEs that do not have UL radio resources available for BSR transmission. In the absence of an UL grant available for UL data transmission, the UE cannot send a BSR on the PUSCH. Therefore, the UE should request resources for UL data starting with SR transmission on the PUCCH. In this case, a 5-step UL resource allocation procedure is used.

[0165] Reference Figure 5 (a), in the absence of PUSCH resources for BSR transmission, the UE first sends an SR to the BS for PUSCH resource allocation. When no PUSCH resources are available for the UE despite the occurrence of a buffer status report event, the SR is used by the UE to request PUSCH resources for UL transmission from the BS. In the presence of valid PUCCH resources for SR, the UE sends the SR on the PUCCH, and in the absence of valid PUCCH resources for SR, the UE starts the above-mentioned (contention-based) RACH process. Upon receiving a UL grant from the BS in a UL grant DCI, the UE sends a BSR to the BS in the PUSCH resources allocated by the UL grant. The BS checks the amount of UL data to be sent by the UE based on the BSR and sends a UL grant to the UE in the UL grant DCI. Upon detecting a PDCCH including a UL grant DCI, the UE sends actual UL data to the BS on the PUSCH based on the UL grant included in the UL grant DCI.

[0166] Reference Figure 5(b), in the case of a configured grant, the UE receives an RRC message including resource configuration for UL data transmission from the BS. In the NR system, two types of UL configured grants are defined: type 1 and type 2. In the case of UL configured grant type 1, the actual UL grant (e.g., time resources and frequency resources) is provided by RRC signaling, while in the case of UL configured grant type 2, the actual UL grant is provided by PDCCH and is enabled or disabled by PDCCH. If the configured grant type 1 is configured, the BS provides at least the following parameters to the UE through RRC signaling: CS-RNTI for retransmission; configured periodicity of grant type 1; information about the starting symbol index S and number of symbols L for PUSCH in the time slot; time domain offset indicating the resource offset relative to SFN=0 in the time domain; and MCS index indicating the modulation order, target code rate, and TB size. If the configured grant type 2 is configured, the BS provides at least the following parameters to the UE through RRC signaling: CS-RNTI for activation, deactivation, and retransmission; and configured periodicity of grant type 2. The actual UL grant of the configured grant type 2 is provided to the UE via the DCI of the PDCCH addressed to the CS-RNTI. If a specific field in the DCI of the PDCCH addressed to the CS-RNTI is set to a specific value for scheduling activation, the configured grant type 2 associated with the CS-RNTI is activated. The DCI set to a specific value for scheduling activation in the PDCCH includes actual frequency resource allocation information, MCS index, etc. The UE can perform UL transmission on the PUSCH based on the configured grant of type 1 or type 2.

[0167] Figure 6 is a conceptual diagram illustrating example physical channel processing.

[0168] Figure 6 The various blocks shown may be executed in corresponding modules of the physical layer blocks in the transmitting device. More specifically, Figure 6 The signal processing described in the above may be performed by the processor of the UE described in the present disclosure for UL transmission. Figure 6 The signal processing may be performed in the processor of the BS described in this disclosure for DL transmission. Figure 6, UL physical channel processing may include scrambling, modulation mapping, layer mapping, transform precoding, precoding, RE mapping and SC-FDMA signal generation. The above processes can be performed separately or together in the modules of the transmitting device. As a discrete Fourier transform (DFT), transform precoding is a special way to spread the UL data in a way that reduces the peak-to-average power ratio (PAPR) of the waveform. OFDM that uses CP together with transform precoding for DFT spreading is called DFT-s-OFDM, and OFDM that uses CP without DFT spreading is called CP-OFDM. SC-FDMA signals are generated by DFT-s-OFDM. In the NR system, if transform precoding is enabled for UL, transform precoding can be applied optionally. That is, the NR system supports two options for the UL waveform: one is CP-OFDM and the other is DFT-s-OFDM. The BS provides RRC parameters to the UE so that the UE determines whether to use CP-OFDM or DFT-s-OFDM for the UL transmission waveform. Figure 6 is a conceptual diagram showing UL physical channel processing for DFT-s-OFDM. Figure 6 For DL transmission, CP-OFDM is used for DL waveform transmission.

[0169] Each of the above processes will be described in more detail. For a codeword, the transmitting device may scramble the coded bits of the codeword through a scrambler and then send the scrambled bits on a physical channel. The codeword is obtained by encoding the TB. The scrambled bits are modulated into complex-valued modulation symbols by a modulation mapper. The modulation mapper may modulate the scrambled bits according to a predetermined modulation scheme and arrange the modulated bits into complex-valued modulation symbols representing positions on a signal constellation. Pi / 2-binary phase shift keying (pi / 2-BPSK), m-phase shift keying (m-PSK), m-quadrature amplitude modulation (m-QAM), etc. may be used for modulation of coded data. The complex-valued modulation symbols may be mapped to one or more transmission layers by a layer mapper. The complex-valued modulation symbols on each layer may be precoded by a precoder for transmission through an antenna port. If transform precoding is available for UL transmission, the precoder may perform precoding after the complex-valued modulation symbols have been subjected to transform precoding, such as Figure 6As shown. The precoder can output antenna-specific symbols by processing complex-valued modulation symbols in a multiple-input multiple-output (MIMO) scheme according to multiple Tx antennas, and distribute the antenna-specific symbols to the corresponding RE mapper. The output z of the precoder can be obtained by multiplying the output y of the layer mapper by the N×M precoding matrix W, where N is the number of antenna ports and M is the number of layers. The RE mapper maps the complex-valued modulation symbols of each antenna port to the appropriate RE in the RB allocated for transmission. The RE mapper can map the complex-valued modulation symbols to the appropriate subcarriers and multiplex the mapped symbols according to the user. The SC-FDMA signal generator (CP-OFDM signal generator when transform precoding is disabled in DL transmission or UL transmission) can generate a complex-valued time-domain OFDM symbol signal by modulating the complex-valued modulation symbols according to a specific modulation scheme (e.g., OFDM). The SC-FDMA signal generator can perform an inverse fast Fourier transform (IFFT) on the antenna-specific symbols and insert a CP into the symbols processed by the time-domain IFFT. The OFDM symbols undergo digital-to-analog conversion, frequency up-conversion, etc., and are then transmitted to the receiving device through each Tx antenna. Each SC-FDMA signal generator may include an IFFT module, a CP inserter, a digital-to-analog converter (DAC), an up-converter, etc.

[0170] The signal processing process of the receiving device is performed in the reverse order of the signal processing process of the transmitting device. Figure 6 .

[0171] Now, the PUCCH will be described.

[0172] The PUCCH is used for UCI transmission. The UCI includes an SR requesting UL transmission resources, a CSI indicating the DL channel status measured by the UE based on a DL RS, and / or a HARQ-ACK indicating whether the UE successfully received DL data.

[0173] PUCCH supports multiple formats, and PUCCH formats are classified according to symbol duration, payload size, and multiplexing or non-multiplexing. The following [Table 1] lists example PUCCH formats.

[0174] [Table 1]

[0175]

[0176] The BS configures PUCCH resources for the UE through RRC signaling. For example, to allocate PUCCH resources, the BS may configure multiple PUCCH resource sets for the UE, and the UE may select a specific PUCCH resource set corresponding to the UCI (payload) size (e.g., the number of UCI bits). For example, the UE may select a specific PUCCH resource set based on the number of UCI bits N. UCI Select one of the following PUCCH resource sets.

[0177] - If the number of UCI bits ≤ 2, then PUCCH resource set #0

[0178] - If 2 < the number of UCI bits ≤ N1, then PUCCH resource set #1 ...

[0180] - If N K-2 < the number of UCI bits ≤ N K-1 , then PUCCH resource set #(K - 1)

[0181] In this text, K represents the number of PUCCH resource sets (K > 1), and Ni represents the maximum number of UCI bits supported by PUCCH resource set #i. For example, PUCCH resource set #1 may include resources for PUCCH format 0 to PUCCH format 1, and other PUCCH resource sets may include resources for PUCCH format 2 to PUCCH format 4.

[0182] Subsequently, the BS may send DCI to the UE on the PDCCH, and indicate the PUCCH resources among the PUCCH resources of a specific PUCCH resource set to be used for UCI transmission through the ACK / NACK resource indicator (ARI) in the DCI. The ARI can be used to indicate the PUCCH resources for HARQ-ACK transmission, also known as the PUCCH resource indicator (PRI).

[0183] Enhanced Mobile Broadband (eMBB)

[0184] In the NR system, consider the large-scale MIMO environment where the number of Tx / Rx antennas significantly increases. On the other hand, in the NR system operating at 6 GHz or above, consider beamforming, where the energy is concentrated in a specific direction rather than transmitting the signal omnidirectionally to compensate for the fast propagation attenuation. Therefore, for increased performance, flexible resource allocation, and easy beam control in terms of frequency, hybrid beamforming that combines analog beamforming and digital beamforming according to the position of the applied beamforming weight vector / precoding vector is required.

[0185] Hybrid Beamforming

[0186] Figure 7 is a block diagram showing an example transmitter and receiver for hybrid beamforming.

[0187] In hybrid beamforming, the BS or UE may send the same signal via multiple antennas by using an appropriate phase difference, so as to form a narrow beam by only increasing the energy in a specific direction.

[0188] Beam Management (BM)

[0189] BM is a series of processes for acquiring and maintaining a set of BS (or Transmission and Reception Point (TRP)) beams and / or UE beams that can be used for DL and UL transmission / reception. BM may include the following processes and terms.

[0190] - Beam measurement: The BS or UE measures the characteristics of the received beamformed signal.

[0191] -Beam determination: The BS or UE selects its Tx beam / Rx beam.

[0192] - Beam sweeping: Covering the spatial domain using a Tx beam and / or an Rx beam in a predetermined method within a predetermined time interval.

[0193] - Beam reporting: The UE reports information about beamformed signals based on beam measurements.

[0194] The BM process can be divided into (1) a DL BM process using SSB or CSI-RS and (2) a UL BM process using SRS. In addition, each BM process may include Tx beam sweeping for determining a Tx beam and Rx beam sweeping for determining an Rx beam. The following description will focus on the DL BM process using SSB.

[0195] The DL BM process using SSB may include (1) transmitting beamformed SSB from the BS and (2) beam reporting by the UE. SSB can be used for both Tx beam sweeping and Rx beam sweeping. SSB-based Rx beam sweeping can be performed by attempting SSB reception while changing the Rx beam at the UE.

[0196] When CSI / beam is configured in RRC_CONNECTED state, SSB-based beam reporting can be configured.

[0197] -The UE receives information about the SSB resource set used for the BM from the BS. An SSB resource set can be configured with one or more SSBIs. For each SSB resource set, SSBI 0 to SSBI 63 can be defined.

[0198] -The UE receives a signal from the BS in the SSB resource based on the information about the SSB resource set.

[0199] - When the BS configures SSBRI and RSRP reporting to the UE, the UE reports the (best) SSBRI and the RSRP corresponding to the SSBRI to the BS.

[0200] The BS may determine a BS Tx beam to use in DL transmission to the UE based on a beam report received from the UE.

[0201] Beam Failure Recovery (BFR) Process

[0202] In a beamforming system, radio link failure (RLF) may often occur due to the rotation or movement of the UE or beamforming obstacles. Therefore, BFR is supported to prevent frequent RLF in NR.

[0203] For beam failure detection, the BS configures a beam failure detection RS for the UE. If the number of beam failure indications from the UE's physical layer reaches a threshold configured by RRC signaling within a period configured by RRC signaling from the BS, the UE declares beam failure.

[0204] After detecting beam failure, the UE triggers BFR by initiating a RACH procedure on the PCell and performs BFR by selecting an appropriate beam (if the BS provides dedicated RACH resources for certain beams, the UE first performs a RACH procedure using the dedicated RACH resources for BFR). When the RACH procedure is completed, the UE considers BFR to be complete.

[0205] Ultra-Reliable and Low-Latency Communication (URLLC)

[0206] URLLC transmission defined in NR may mean transmission with (1) relatively small traffic size, (2) relatively low arrival rate, (3) very low delay requirement (e.g., 0.5 ms or 1 ms), (4) relatively short transmission duration (e.g., 2 OFDM symbols), and (5) emergency services / messages.

[0207] Preemption Instructions

[0208] Although eMBB and URLLC services can be scheduled in non-overlapping time / frequency resources, URLLC transmissions can be performed in resources scheduled for ongoing eMBB services. In order to enable a UE receiving a PDSCH to determine that the PDSCH is partially punctured due to another UE's URLLC transmission, a preemption indication may be used. The preemption indication may also be referred to as an interrupt transmission indication.

[0209] Regarding the preemption indication, the UE receives DL preemption RRC information (eg, DownlinkPreemption IE) from the BS through RRC signaling.

[0210] The UE receives DCI format 2_1 from the BS based on the DL preemption RRC information. For example, the UE attempts to detect a PDCCH transmitting preemption indication-related DCI (DCI format 2_1) using the int-RNTI configured by the DL preemption RRC information.

[0211] Upon detecting DCI format 2_1 of the serving cell configured by DL preemption RRC information, the UE may assume that no transmission directed to the UE exists in the RBs and symbols indicated by DCI format 2_1 in the RB set and symbol set during the monitoring interval immediately preceding the monitoring interval to which DCI format 2_1 belongs. For example, considering that the signal in the time-frequency resources indicated by the preemption indication is not a DL transmission scheduled for the UE, the UE decodes data based on the signal received in the remaining resource region.

[0212] Massive Machine Machine Communication (mMTC)

[0213] mMTC is one of the 5G scenarios that supports hyperconnected services that simultaneously communicate with multiple UEs. In this environment, UEs communicate intermittently at very low transmission rates while maintaining low mobility. Therefore, mMTC primarily aims to ensure long-term UE operation at low cost. In this regard, the following describes MTC and Narrowband Internet of Things (NB-IoT), addressed within 3GPP.

[0214] The following description assumes that the transmission time interval (TTI) of a physical channel is a subframe. For example, the minimum time interval between the start of a physical channel transmission and the start of the next physical channel transmission is a subframe. However, in the following description, a subframe may be replaced by a slot, a minislot, or multiple slots.

[0215] Machine-Type Communication (MTC)

[0216] MTC is suitable for applications that do not require high throughput and is suitable for machine-to-machine (M2M) or IoT. MTC is a communication technology adopted by 3GPP to meet the requirements of IoT services.

[0217] Although the following mainly describes features related to enhanced MTC (eMTC), unless otherwise mentioned, the same content applies to MTC, eMTC, and MTC to be applied to 5G (or NR). As used herein, the term MTC is interchangeable with eMTC, LTE-M1 / M2, reduced bandwidth low complexity (BL) / coverage enhancement (CE), non-BL UE (in enhanced coverage), NRMTC, enhanced BL / CE, etc.

[0218] Conventional MTC

[0219] (1) MTC operates only within a specific system BW (or channel BW).

[0220] In conventional LTE or NR systems, MTC can use a predetermined number of RBs within the system frequency band. In NR, the operating frequency band (BW) of MTC can be defined taking into account the frequency range and subcarrier spacing. The specific system or frequency band in which MTC operates is called the MTC narrowband (NB) or MTC subband. In NR, MTC can operate in at least one BWP or a specific frequency band of a BWP.

[0221] Although cells with bandwidths much larger than 1.08 MHz (e.g., 10 MHz) support MTC, the physical channels and signals transmitted / received in MTC are always limited to 1.08 MHz or 6 (LTE) RBs. For example, in LTE systems, narrowband is defined as 6 non-overlapping contiguous physical resource blocks (PRBs) in the frequency domain.

[0222] In MTC, some DL channels and UL channels are restrictively allocated within a narrowband, and one channel does not occupy a plurality of narrowbands in one time unit. Figure 8a is a diagram illustrating an example narrowband operation, Figure 8b is a diagram illustrating example MTC channel repetition with RF retuning.

[0223] The MTC narrowband may be configured for the UE through system information or DCI sent by the BS.

[0224] (2) MTC does not use channels (defined in legacy LTE or NR) that are distributed across the total system bandwidth of legacy LTE or NR. For example, since the legacy LTE PDCCH is distributed across the total system bandwidth, MTC does not use the legacy PDCCH. Instead, MTC uses a new control channel, the MTC PDCCH (MPDCCH). The MPDCCH is transmitted / received in up to six RBs in the frequency domain. In the time domain, the MPDCCH can be transmitted in one or more OFDM symbols of a subframe, starting from the OFDM symbol with the start OFDM symbol index indicated by the RRC parameters from the base station.

[0225] (3) In MTC, PBCH, PRACH, MPDCCH, PDSCH, PUCCH, and PUSCH can be transmitted repeatedly. Even when the signal quality or power is very poor (such as in harsh conditions like a basement), MTC repeated transmissions can make these channels decodable, resulting in an increased cell radius and signal penetration.

[0226] MTC operation modes and levels

[0227] For CE, two operation modes, CE Mode A and CE Mode B, and four different CE levels are used in MTC as listed in [Table 2] below.

[0228] [Table 2]

[0229]

[0230] The MTC operation mode is determined by the BS, and the CE level is determined by the MTC UE.

[0231] MTC protection period

[0232] The location of the narrowband used for MTC can change in each specific time unit (e.g., subframe or time slot). MTC UEs can tune to different frequencies in different time units. Frequency retuning may require a specific time, which is used as a protection period for MTC. No transmission or reception occurs during the protection period.

[0233] MTC signal transmission / reception method

[0234] In addition to the inherent characteristics of MTC, the MTC signal transmission / reception process is similar to Figure 2 The process shown can also be performed for MTC Figure 2 The PSS / SSS used in the initial cell search operation in MTC may be a conventional LTE PSS / SSS.

[0235] After acquiring synchronization with the BS using the PSS / SSS, the MTC UE can obtain broadcast information within the cell by receiving a PBCH signal from the BS. The broadcast information sent on the PBCH is the MIB. In MTC, the reserved bits among the bits of the traditional LTE MIB are used to send scheduling information for the new system information block 1 bandwidth reduction (SIB1-BR). The scheduling information of SIB1-BR may include information about the number of repetitions and TBS of the PDSCH that transmits the SIB1-BR. The frequency resource assignment for the PDSCH that transmits the SIB-BR may be a set of 6 consecutive RBs within a narrowband. The SIB-BR is sent directly on the PDSCH without the need for a control channel associated with the SIB-BR (e.g., PDCCH or MPDCCH).

[0236] After completing the initial cell search, the MTC UE can obtain more specific system information by receiving the MPDCCH and PDSCH based on the MPDCCH information ( S202 ).

[0237] The MTC UE then performs a RACH procedure to complete the connection with the base station (S203 to S206). The basic configuration of the RACH procedure for the MTC UE may be transmitted in the SIB2. Furthermore, SIB2 includes paging-related parameters. In 3GPP systems, a paging occasion (PO) refers to a time unit in which the UE can attempt to receive paging. Paging is a network indication that data is available for transmission to the UE. The MTC UE attempts to receive the MPDCCH based on the P-RNTI in the time unit corresponding to its PO within the narrowband configured for paging (the paging narrowband (PNB)). When the UE successfully decodes the MPDCCH based on the P-RNTI, it can check for its paging message by receiving the PDSCH scheduled by the MPDCCH. If its paging message is available, the UE accesses the network by performing a RACH procedure.

[0238] In MTC, signals and / or messages (Msg1, Msg2, Msg3, and Msg4) may be repeatedly transmitted in a RACH procedure, and different repetition patterns may be set according to CE levels.

[0239] For random access, the BS signals PRACH resources for different CE levels. Different PRACH resources for up to 4 corresponding CE levels can be signaled to the MTC UE. The MTC UE uses a DL RS (e.g., CRS, CSI-RS, or TRS) to measure RSRP and determines one of the CE levels signaled by the BS based on the measurement. The UE selects one of the different PRACH resources (e.g., frequency, time, and preamble resources for PARCH) for random access based on the determined CE level and sends PRACH. The BS can determine the CE level of the UE based on the PRACH resources that the UE has used for PRACH transmission. The BS can determine the CE mode of the UE based on the CE level indicated by the UE through PRACH transmission. The BS can send DCI to the UE in CE mode.

[0240] The BS signals the search space of RAR and contention resolution message for PRACH in system information.

[0241] After the above process, the MTC UE may receive an MPDCCH signal and / or a PDSCH signal during normal UL / DL signal transmission ( S207 ) and transmit a PUSCH signal and / or a PUCCH signal ( S208 ). The MTC UE may transmit UCI to the BS on the PUCCH or PUSCH.

[0242] Once an RRC connection is established for an MTC UE, the MTC UE attempts to receive the MDCCH by monitoring the MPDCCH in the configured search space in order to obtain UL and DL data allocations.

[0243] In conventional LTE, the PDSCH is scheduled by the PDCCH. Specifically, the PDCCH may be transmitted in the first N (N=1, 2, or 3) OFDM symbols of a subframe, and the PDSCH scheduled by the PDCCH is transmitted in the same subframe.

[0244] Compared to conventional LTE, in MTC, the MPDCCH and the PDSCH scheduled by the MPDCCH are transmitted / received in different subframes. For example, the last repeated MPDCCH in subframe #n schedules the PDSCH starting in subframe #n+2. The MPDCCH can be transmitted only once or repeatedly. The maximum number of MPDCCH repetitions is configured for the UE via RRC signaling from the base station. The DCI carried on the MPDCCH provides information on the number of MPDCCH repetitions, allowing the UE to determine when PDSCH transmission begins. For example, if the DCI in the MPDCCH starting in subframe #n includes information indicating that the MPDCCH is repeated 10 times, the MPDCCH may end in subframe #n+9 and the PDSCH may start in subframe #n+11. The DCI carried on the MPDCCH may include information on the number of repetitions of the physical data channel (e.g., PUSCH or PDSCH) scheduled by the DCI. The UE can repeatedly transmit / receive the physical data channel in the time domain based on the information on the number of repetitions of the physical data channel scheduled by the DCI. The PDSCH can be scheduled in the same or different narrowband as the narrowband in which the MPDCCH scheduling the PDSCH is transmitted. When the MPDCCH and PDSCH are in different narrowbands, the MTC UE needs to re-tune to the frequency of the narrowband carrying the PDSCH before decoding the PDSCH. UL scheduling can follow the same timing as in conventional LTE. For example, an MPDCCH ending in subframe #n can schedule a PUSCH transmission starting in subframe #n+4. If the physical channel is transmitted repeatedly, frequency hopping between different MTC subbands is supported through RF re-tuning. For example, if the PDSCH is transmitted repeatedly in 32 subframes, the PDSCH is transmitted in the first 16 subframes in the first MTC subband and the remaining 16 subframes in the second MTC subband. MTC can operate in half-duplex mode.

[0245] Narrowband-IoT (NB-IoT)

[0246] NB-IoT refers to a system that supports low complexity, low power consumption, and efficient use of frequency resources, using a system bandwidth corresponding to one RB in a wireless communication system (e.g., an LTE system or a NR system). NB-IoT can operate in half-duplex mode. NB-IoT can be used as a communication solution for implementing IoT in cellular systems by supporting, for example, MTC devices (or UEs).

[0247] In NB-IoT, each UE perceives one RB as one carrier. Therefore, the RB and carrier mentioned in NB-IoT can be interpreted as having the same meaning.

[0248] Although the following description of the frame structure, physical channels, multi-carrier operation, and general signal transmission / reception regarding NB-IoT will be in the context of a conventional LTE system, this description is also applicable to next-generation systems (e.g., NR systems). Furthermore, the description of NB-IoT can also be applied to MTC services that serve similar technical purposes (e.g., low power, low cost, and coverage enhancement).

[0249] NB-IoT frame structure and physical resources

[0250] Different NB-IoT frame structures can be configured according to the subcarrier spacing. For example, for a subcarrier spacing of 15kHz, the NB-IoT frame structure can be the same as that of a traditional system (e.g., an LTE system). For example, a 10-ms NB-IoT frame may include 10 1-ms NB-IoT subframes, each of which includes two 0.5-ms time slots. Each 0.5-ms NB-IoT time slot may include 7 OFDM symbols. In another example, for a BWP or cell / carrier with a subcarrier spacing of 3.75kHz, a 10-ms NB-IoT frame may include five 2-ms NB-IoT subframes, each of which includes 7 OFDM symbols and one guard period (GP). In addition, the 2-ms NB-IoT subframe can be represented in an NB-IoT time slot or an NB-IoT resource unit (RU). The NB-IoT frame structure is not limited to subcarrier spacings of 15kHz and 3.75kHz, and NB-IoT with other subcarrier spacings (e.g., 30kHz) can also be considered by changing the time / frequency unit.

[0251] In addition to the system BW being limited to a predetermined number of RBs (e.g., one RB, i.e., 180 kHz), the NB-IoT DL physical resources may be configured based on the physical resources of other wireless communication systems (e.g., LTE system or NR system). For example, if NB-IoT DL only supports 15-kHz subcarrier spacing as described above, the NB-IoT DL physical resources may be configured as Figure 1 The resource grid shown is restricted to a resource region of one RB in the frequency domain.

[0252] Similar to NB-IoT DL physical resources, NB-IoT UL resources can also be configured by limiting the system BW to one RB. In NB-IoT, the number of UL subcarriers N UL sc and the time slot duration T slotIt can be given as shown in the following [Table 3]. In the NB-IoT of the LTE system, the duration of a time slot T slot Defined by 7 SC-FDMA symbols in the time domain.

[0253] [Table 3]

[0254] Subcarrier spacing <![CDATA[N UL sc ]]> <![CDATA[T slot ]]> Δf=3.75kHz 48 <![CDATA[6144·T s ]]> Δf=15kHz 12 <![CDATA[15360·T s ]]>

[0255] In NB-IoT, RU is used to map to RE of PUSCH (called NPUSCH) of NB-IoT. RU can be represented by N in the time domain. UL symb *N UL slot SC-FDMA symbols × N in the frequency domain RU sc For example, N is listed in [Table 4] for a cell / carrier with an FDD frame structure and in [Table 5] for a cell / carrier with a TDD frame structure. RU sc and N UL symb .

[0256] [Table 4]

[0257]

[0258] [Table 5]

[0259]

[0260] NB-IoT physical channel

[0261] OFDMA can be adopted for NB-IoT DL based on 15-kHz subcarrier spacing. Since OFDMA provides orthogonality between subcarriers, coexistence with other systems (e.g., LTE system or NR system) can be efficiently supported. The names of the DL physical channels / signals of the NB-IoT system may have an "N (narrowband)" prefix to distinguish them from their counterparts in traditional systems. For example, DL physical channels may be named NPBCH, NPDCCH, NPDSCH, etc., and DL physical signals may be named NPSS, NSSS, narrowband reference signal (NRS), narrowband positioning reference signal (NPRS), narrowband wake-up signal (NWUS), etc. DL channels NPBCH, NPDCCH, NPDSCH, etc. may be repeatedly sent in the NB-IoT system to enhance coverage. In addition, newly defined DCI formats such as DCI format N0, DCI format N1, and DCI format N2 may be used in NB-IoT.

[0262] SC-FDMA can be applied to NB-IoT UL with 15-kHz or 3.75-kHz subcarrier spacing. As described for DL, the names of the physical channels of the NB-IoT system may have an "N (narrowband)" prefix to distinguish them from their counterparts in the traditional system. For example, UL channels may be named NPRACH, NPUSCH, etc., and UL physical signals may be named NDMRS, etc. NPUSCH can be classified into NPUSCH format 1 and NPUSCH format 2. For example, NPUSCH format 1 can be used to send (or transmit) uplink shared channel (UL-SCH), and NPUSCH format 2 can be used for UCI transmission such as HARQ ACK signaling. The UL channel NPRACH in the NB-IoT system can be sent repeatedly to enhance coverage. In this case, the repeated transmissions can undergo frequency hopping.

[0263] Multi-carrier operation in NB-IoT

[0264] NB-IoT can be implemented in multi-carrier mode. Multi-carrier operation may refer to the use of multiple carriers configured for different purposes (e.g., multiple carriers of different types) in the transmission / reception channels and / or signals between the base station and the user equipment terminal.

[0265] In the multi-carrier mode in NB-IoT, carriers can be divided into anchor carriers (e.g., anchor carriers or anchor PRBs) and non-anchor carriers (e.g., non-anchor carriers or non-anchor PRBs).

[0266] From the base station's perspective, the anchor carrier refers to the carrier that carries the NPSS, NSSS, and NPBCH for initial access, as well as the NPDSCH for the system information block (N-SIB). That is, in NB-IoT, the carrier used for initial access is called the anchor carrier, and other carriers are called non-anchor carriers.

[0267] NB-IoT signal transmission / reception processing

[0268] In NB-IoT, in addition to the inherent features of NB-IoT, Figure 2 The process shown is similar to sending / receiving signals. Figure 2 When an NB-IoT UE is powered on or enters a new cell, it performs an initial cell search (S201). For the initial cell search, the NB-IoT UE receives NPSS and NSSS from the BS to synchronize with the BS and obtain information such as the cell ID. Furthermore, the NB-IoT UE receives NPBCH from the BS to obtain broadcast information within the cell.

[0269] When the initial cell search is completed, the NB-IoT UE can obtain more specific system information by receiving the NPDCCH and receiving the NPDSCH corresponding to the NPDCCH (S202). In other words, the BS can send more specific system information to the NB-IoT UE that has completed the initial cell search by sending the NPDCCH and the NPDSCH corresponding to the NPDCCH.

[0270] The NB-IoT UE may then perform a RACH procedure to complete connection establishment with the BS (S203 to S206). To this end, the NB-IoT UE may send a preamble to the BS on the NPRACH (S203). As previously described, the NPRACH may be configured to be sent repeatedly based on frequency hopping for coverage enhancement. In other words, the BS may receive the preamble (repeatedly) from the NB-IoT UE on the NPRACH. The NB-IoT UE may then receive the NPDCCH and the RAR in response to the preamble from the BS on the NPDSCH corresponding to the NPDCCH (S204). In other words, the BS may send the NPDCCH and the RAR in response to the preamble to the NB-IoT UE on the NPDSCH corresponding to the NPDCCH. Subsequently, the NB-IoT UE may use the scheduling information in the RAR to send the NPUSCH to the BS (S205) and perform a contention resolution procedure (S206) by receiving the NPDCCH and the NPDSCH corresponding to the NPDCCH.

[0271] After the above processing, the NB-IoT UE can perform NPDCCH / NPDSCH reception (S207) and NPUSCH transmission (S208) during the general UL / DL signal transmission process. In other words, after the above processing, the BS can perform NPDCCH / NPDSCH transmission and NPUSCH reception with the NB-IoT UE during the general UL / DL signal transmission process.

[0272] In NB-IoT, NPBCH, NPDCCH, and NPDSCH can be transmitted repeatedly for coverage enhancement. In NB-IoT, UL-SCH (e.g., general UL data) and UCI can be transmitted on PUSCH. UL-SCH and UCI can be configured to be sent in different NPUSCH formats (e.g., NPUSCH format 1 and NPUSCH format 2).

[0273] In NB-IoT, UCI can usually be sent on NPUSCH. In addition, the UE can send NPUSCH periodically, aperiodically, or semi-persistently according to the request / instruction of the network (e.g., BS).

[0274] wireless communication equipment

[0275] Figure 9 is a block diagram of an example wireless communication system to which the method proposed in the present disclosure is applicable.

[0276] Reference Figure 9 , a wireless communication system includes a first communication device 910 and / or a second communication device 920. The phrases "A and / or B" and "at least one of A or B" may be interpreted as having the same meaning. The first communication device 910 may be a base station, and the second communication device 920 may be a user equipment terminal (or the first communication device 910 may be a user equipment terminal, and the second communication device 920 may be a base station).

[0277] Each of the first communication device 910 and the second communication device 920 includes a processor 911 or 921, a memory 914 or 924, one or more Tx / Rx RF modules 915 or 925, a Tx processor 912 or 922, an Rx processor 913 or 923, and an antenna 916 or 926. The Tx / Rx module may also be referred to as a transceiver. The processor performs the above-described functions, processes, and / or methods. More specifically, on the DL (communication from the first communication device 910 to the second communication device 920), higher-layer packets from the core network are provided to the processor 911. The processor 911 implements Layer 2 (e.g., L2) functions. On the DL, the processor 911 is responsible for multiplexing between logical channels and transport channels, providing radio resource assignments to the second communication device 920, and signaling to the second communication device 920. The Tx processor 912 performs various signal processing functions of the L1 (e.g., physical layer). The signal processing function facilitates forward error correction (FEC) of the second communication device 920, including encoding and interleaving. After scrambling and modulation, the coded and interleaved signal is modulated into complex-valued modulation symbols. For modulation, BPSK, QPSK, 16QAM, 64QAM, 246QAM, etc. are available depending on the channel. The complex-valued modulation symbols (hereinafter referred to as modulation symbols) are divided into parallel streams. Each stream is mapped to an OFDM subcarrier and multiplexed with a RS in the time and / or frequency domain. By subjecting the mapped signal to an IFFT, a physical channel is generated to carry the time-domain OFDM symbol stream. The OFDM symbol stream is spatially precoded into multiple spatial streams. Each spatial stream can be provided to a different antenna 916 via a separate Tx / Rx module (or transceiver) 915. Each Tx / Rx module 915 can up-convert each spatial stream into an RF carrier for transmission. In the second communication device 920, each Tx / Rx module (or transceiver) 925 receives the signal of the RF carrier via a respective antenna 926. Each Tx / Rx module 925 recovers the RF carrier signal into a baseband signal and provides the baseband signal to the Rx processor 923. The Rx processor 923 performs various L1 (e.g., physical layer) signal processing functions. The Rx processor 923 may perform spatial processing on the information to recover any spatial streams directed to the second communication device 920. If multiple spatial streams are directed to the second communication device 920, the Rx processors may combine the multiple spatial streams into a single OFDMA symbol stream. The Rx processor 923 converts the OFDM symbol stream, which is a time-domain signal, into a frequency-domain signal using an FFT. The frequency-domain signal includes a separate OFDM symbol stream for each subcarrier of the OFDM signal. By determining the most likely signal constellation point transmitted by the first communication device 910, the modulation symbols and RS on each subcarrier are recovered and demodulated. These soft decisions may be based on channel estimation. The soft decisions are decoded and deinterleaved to recover the original data and control signals transmitted by the first communication device 910 on the physical channel. The data and control signals are provided to the processor 921.

[0278] On the UL (communication from the second communication device 920 to the first communication device 910), the first communication device 910 operates in a manner similar to the receiver functionality described for the second communication device 920. Each Tx / Rx module 925 receives signals via an antenna 926. Each Tx / Rx module 925 provides an RF carrier and information to an Rx processor 923. The processor 921 may be associated with a memory 924 that stores program codes and data. The memory 924 may be referred to as a computer-readable medium.

[0279] Artificial Intelligence (AI)

[0280] Artificial intelligence is the field that studies AI or methods for creating AI, while machine learning is the field that defines the various problems addressed in the AI field and studies methods for solving these problems. Machine learning is defined as algorithms that increase the performance of specific operations through consistent experience with these operations.

[0281] An artificial neural network (ANN) is a model used in machine learning and generally refers to a model with problem-solving capabilities. It is composed of artificial neurons (nodes) connected via synapses to form a network. An ANN can be defined by the connection pattern between neurons in different layers, the learning process used to update model parameters, and the activation function used to generate output values.

[0282] An ANN may include an input layer, an output layer, and optionally one or more hidden layers. Each layer includes one or more neurons, and the ANN may include synapses connecting the neurons. In an ANN, each neuron may output a function value of an activation function for inputs of signals, weights, and deflections through the synapses.

[0283] Model parameters are parameters determined by learning and include the weight values of the synaptic connections and deflections of neurons. Hyperparameters are parameters that are set in the machine learning algorithm before learning and include the learning rate, number of repetitions, mini-batch size, and initialization function.

[0284] The goal of ANN learning can be to determine the model parameters that minimize the loss function. The loss function can be used as an indicator to determine the optimal model parameters in the ANN learning process.

[0285] Machine learning can be categorized into supervised learning, unsupervised learning, and reinforcement learning based on the learning method.

[0286] Supervised learning can be a method of training an ANN with labeled training data, where the label refers to the correct answer (or result value) that the ANN should infer based on the training data input to the ANN. Unsupervised learning can be a method of training an ANN without labeled training data. Reinforcement learning can be a learning method that trains an agent defined in a specific environment to select an action or action sequence that maximizes cumulative compensation in each state.

[0287] Machine learning implemented by deep neural networks (DNNs) that include multiple hidden layers within an ANN is also called deep learning, and deep learning is a subset of machine learning. The following description is based on the understanding that machine learning includes deep learning.

[0288] <Robot>

[0289] A robot may refer to a machine that automatically processes or performs a given task through its own capabilities. Specifically, a robot equipped with the function of recognizing the environment and performing operations based on its decisions may be referred to as an intelligent robot.

[0290] Robots can be classified into industrial robots, medical robots, consumer robots, military robots, etc. according to their purpose or application field.

[0291] A robot may be provided with a drive unit including actuators or motors, thereby performing various physical operations such as moving the robot's joints. In addition, a movable robot may include wheels, brakes, propellers, etc. in the drive unit, thereby driving on the ground or flying in the air via the drive unit.

[0292] <Self-driving>

[0293] Self-driving refers to autonomous driving, and a self-driving vehicle refers to a vehicle that drives with no or minimal user control.

[0294] For example, self-driving may include technology for maintaining a lane while driving, technology for automatically adjusting speed (e.g., adaptive cruise control), technology for automatically driving along a predetermined route, and technology for automatically setting a route and driving along the route when a destination is set.

[0295] The vehicle may include a vehicle having only an internal combustion engine, a hybrid vehicle having both an internal combustion engine and an electric motor, and an electric vehicle having only an electric motor, and may include not only cars but also trains, motorcycles, and the like.

[0296] In this paper, a self-driving vehicle can be considered as a robot with self-driving capabilities.

[0297] Extended Reality (XR)

[0298] Extended reality is an umbrella term encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR presents only real-world objects and backgrounds as computer graphics (CG) images, while AR presents virtual CG images overlayed on real-world objects. MR is a computer graphics technology that blends and combines virtual objects into the real world.

[0299] MR is similar to AR in that real objects and virtual objects are displayed together. However, in AR, virtual objects serve as a supplement to real objects, while in MR, virtual objects and real objects are treated equally.

[0300] XR may be applied to a head-mounted display (HMD), a head-up display (HUD), a portable phone, a tablet PC, a laptop computer, a desktop computer, a TV, a digital signage, etc. A device to which XR is applied may be referred to as an XR device.

[0301] Figure 10 An AI device 1000 according to an embodiment of the present disclosure is shown.

[0302] Figure 10 The AI device 1000 shown can be configured as a fixed device or a mobile device, such as a TV, a projector, a portable phone, a smart phone, a desktop computer, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a digital multimedia broadcasting (DMB) receiver, a radio, a washing machine, a refrigerator, a digital signage, a robot, or a vehicle.

[0303] Reference Figure 10 , the AI device 1000 may include a communication unit 1010 , an input unit 1020 , a learning processor 1030 , a sensing unit 1040 , an output unit 1050 , a memory 1070 , and a processor 1080 .

[0304] The communication unit 1010 can send and receive data to and from an external device (e.g., another AI device or an AI server) via wired or wireless communication. For example, the communication unit 1010 can send and receive sensor information, user input, learning models, and control signals to and from the external device.

[0305] The communication schemes used by the communication unit 1010 include Global System for Mobile Communications (GSM), CDMA, LTE, 5G, Wireless Local Area Network (WLAN), Wireless Fidelity (Wi-Fi), Bluetooth TM , Radio Frequency Identification (RFID), Infrared Data Association (IrDA), ZigBee, Near Field Communication (NFC), etc. Specifically, the aforementioned references may also be applied. Figures 1 to 9Describe 5G technology.

[0306] The input unit 1020 can acquire various types of data. The input unit 1020 may include a camera for inputting video signals, a microphone for receiving audio signals, and a user input unit for receiving information from a user. The camera or microphone can be considered a sensor, and thus the signal acquired from the camera or microphone can be referred to as sensed data or sensor information.

[0307] The input unit 1020 can obtain training data for model training and input data to be used to obtain output using the learning model. The input unit 1020 can obtain raw input data. In this case, the processor 1080 or the learning processor 1030 can extract input features by preprocessing the input data.

[0308] The learning processor 1030 can use the training data to train a model composed of an ANN. The trained ANN can be referred to as a learning model. The learning model can be used to infer the result value of new input data other than the training data, and the inferred value can be used as the basis for determining the execution of a specific operation.

[0309] The learning processor 1030 may perform AI processing together with a learning processor of an AI server.

[0310] The learning processor 1030 may include a memory integrated or implemented in the AI device 1000. Alternatively, the learning processor 1030 may be implemented using the memory 1070, an external memory directly connected to the AI device 1000, or a memory maintained in an external device.

[0311] The sensing unit 1040 may acquire at least one of internal information about the AI device 1000 , surrounding environment information about the AI device 1000 , and user information using various sensors.

[0312] The sensors included in the sensing unit 1040 may include a proximity sensor, an illumination sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, a red, green, and blue (RGB) sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, an optical sensor, a microphone, a light detection and ranging (LiDAR), and a radar.

[0313] The output unit 1050 may generate visual, auditory, or tactile output.

[0314] Therefore, the output unit 1050 may include a display unit for outputting visual information, a speaker for outputting auditory information, and a haptic module for outputting tactile information.

[0315] The memory 1070 may store data supporting various functions of the AI device 1000. For example, the memory 1070 may store input data acquired through the input unit 1020, training data, a learning model, a learning history, and the like.

[0316] The processor 1080 may determine at least one executable operation of the AI device 1000 based on information determined or generated by a data analysis algorithm or a machine learning algorithm. The processor 1080 may control the components of the AI device 1000 to perform the determined operation.

[0317] To this end, the processor 1080 may request, search, receive, or utilize data from the learning processor 1030 or the memory 1070. The processor 1080 may control the components of the AI device 1000 to perform a predicted operation or an operation determined to be desirable among at least one executable operation.

[0318] When the determined operation needs to be performed in conjunction with the external device, the processor 1080 may generate a control signal for controlling the external device and transmit the generated control signal to the external device.

[0319] The processor 1080 may obtain intention information about the user input and determine the user's requirement based on the obtained intention information.

[0320] The processor 1080 may obtain intent information corresponding to the user input using at least one of a speech-to-text (STT) engine for converting a voice input into a text string or a natural language processing (NLP) engine for obtaining intent information of a natural language.

[0321] At least one of the STT engine or the NLP engine may be configured as an ANN, at least a portion of which is trained according to a machine learning algorithm. At least one of the STT engine or the NLP engine may be trained by a learning processor, a learning processor of an AI server, or distributed processing of learning processors. For reference, Figure 11 Specific components of the AI server are shown in FIG.

[0322] The processor 1080 may collect historical information including the operation content of the AI device 1000 or user feedback on the operation, and may store the collected historical information in the memory 1070 or the learning processor 1030 or transmit the collected historical information to an external device such as an AI server. The collected historical information may be used to update the learning model.

[0323] The processor 1080 may control at least a portion of the components of the AI device 1000 to drive the application stored in the memory 1070. In addition, the processor 1080 may operate two or more components included in the AI device 1000 in combination to drive the application.

[0324] Figure 11 An AI server 1120 according to an embodiment of the present disclosure is shown.

[0325] Reference Figure 11 , AI server 1120 may refer to a device that trains an ANN using a machine learning algorithm or uses a trained ANN. AI server 1120 may include multiple servers that perform distributed processing or may be defined as a 5G network. AI server 1120 may be included as part of AI device 1100 and perform at least a portion of AI processing.

[0326] The AI server 1120 may include a communication unit 1121 , a memory 1123 , a learning processor 1122 , a processor 1126 , and the like.

[0327] The communication unit 1121 may transmit and receive data to and from an external device such as the AI device 1100 .

[0328] The memory 1123 may include a model storage unit 1124. The model storage unit 1124 may store a model (or ANN 1125) that has been trained or is being trained by the learning processor 1122.

[0329] The learning processor 1122 may train the ANN 1125 through the training data. The learning model may be used while being loaded on the AI server 1120 of the ANN or an external device such as the AI device 1110.

[0330] The learning model may be implemented in hardware, software, or a combination of hardware and software. If all or part of the learning model is implemented in software, one or more instructions of the learning model may be stored in memory 1123.

[0331] The processor 1126 may use the learned model to infer result values for new input data and may generate responses or control commands based on the inferred result values.

[0332] Figure 12 An AI system according to an embodiment of the present disclosure is shown.

[0333] Reference Figure 12 In the AI system, at least one of an AI server 1260, a robot 1210, a self-driving vehicle 1220, an XR device 1230, a smartphone 1240, or a home appliance 1250 is connected to a cloud network 1200. The robot 1210, the self-driving vehicle 1220, the XR device 1230, the smartphone 1240, or the home appliance 1250 to which AI is applied may be referred to as an AI device.

[0334] The cloud network 1200 may refer to a network that forms part of or exists within a cloud computing infrastructure. The cloud network 1200 may be configured using a 3G network, a 4G or LTE network, or a 5G network.

[0335] That is, the devices 1210 to 1260 included in the AI system may be interconnected via the cloud network 1200. Specifically, each of the devices 1210 to 1260 may communicate with each other directly or through a BS.

[0336] The AI server 1260 may include a server that performs AI processing and a server that performs calculations on big data.

[0337] The AI server 1260 can be connected to at least one of the AI devices included in the AI system, i.e., at least one of the robot 1210, the self-driving vehicle 1220, the XR device 1230, the smart phone 1240, or the home appliance 1250, via the cloud network 1200, and can assist at least a portion of AI processing of the connected AI devices 1210 to 1250.

[0338] The AI server 1260 may train the ANN according to a machine learning algorithm on behalf of the AI devices 1210 to 1250 , and may directly store the learning model or transmit the learning model to the AI devices 1210 to 1250 .

[0339] The AI server 1260 may receive input data from the AI devices 1210 to 1250 , infer result values of the received input data using a learning model, generate responses or control commands based on the inferred result values, and send the responses or control commands to the AI devices 1210 to 1250 .

[0340] Alternatively, the AI devices 1210 to 1250 may directly use the learning model to infer result values of the input data, and generate a response or control command based on the inference result.

[0341] Hereinafter, various embodiments of the AI devices 1210 to 1250 to which the above-described technology is applied will be described. Figure 12 The AI devices 1210 to 1250 shown can be considered as Figure 10 A specific embodiment of the AI device 1000 is shown.

[0342] <AI+XR>

[0343] The AI-applied XR device 1230 may be configured as an HMD, an HUD provided in a vehicle, a TV, a portable phone, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot, a mobile robot, and the like.

[0344] The XR device 1230 can obtain information about the surrounding space or real objects by analyzing 3D point cloud data or image data acquired from various sensors or external devices, and thus generate position data and attribute data of 3D points, and can render XR objects to be output. For example, the XR device 1230 can output an XR object corresponding to the identified object, including additional information about the identified object.

[0345] The XR device 1230 can use a learning model composed of at least one ANN to perform the above operations. For example, the XR device 1230 can use the learning model to identify real objects from 3D point cloud data or image data, and can provide information corresponding to the identified real objects. The learning model can be directly trained by the XR device 1230 or by an external device such as the AI server 1260.

[0346] Although the XR device 1230 can operate by directly using the learning model to generate results, the XR device 1230 can also operate by sending sensor information to an external device such as the AI server 1260 and receiving the results.

[0347] <AI + Robot + XR>

[0348] The robot 1210 applied with AI and XR can be implemented as a guiding robot, a delivery robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, a drone, etc.

[0349] The robot 1210 applied with XR can refer to a robot to be controlled / interacted within an XR image. In this case, the robot 1210 can be distinguished from the XR device 1230 and communicate with the XR device 1230.

[0350] When the robot 1210 to be controlled / interacted within the XR image obtains sensor information from sensors each including a camera, the robot 1210 or the XR device 1230 can generate an XR image based on the sensor information, and the XR device 1230 can output the generated XR image. The robot 1210 can operate based on the control signal received through the XR device 1230 or based on the interaction of the user.

[0351] For example, the user can check the XR image corresponding to the view of the robot 1210 in remote communication through an external device such as the XR device 1210, adjust the self-driving route of the robot 1210 through interaction, control the operation or driving of the robot 1210, or check information about the environmental objects around the robot 1210.

[0352] <AI + Self-driving + XR>

[0353] The self-driving vehicle 1220 to which AI and XR are applied may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, and the like.

[0354] The XR-enabled self-driving vehicle 1220 may refer to a self-driving vehicle equipped with a device for providing XR images or a self-driving vehicle to be controlled or interacted with within the XR images. Specifically, the self-driving vehicle 1220 to be controlled or interacted with within the XR images may be distinguished from the XR device 1230 and may communicate with the XR device 1230.

[0355] Self-driving vehicle 1220, equipped with a device for providing XR images, can acquire sensor information from sensors, each including a camera, and output generated XR images based on the acquired sensor information. For example, self-driving vehicle 1220 may include a head-up display (HUD) to output XR images, thereby providing passengers with XR objects corresponding to real objects or objects on a screen.

[0356] When an XR object is output to the HUD, at least a portion of the XR object may be output to be superimposed on the actual object being viewed by the passenger. When an XR object is output to a display provided in the self-driving vehicle 1220, at least a portion of the XR object may be output to be superimposed on an object within the display. For example, the self-driving vehicle 1220 may output an XR object corresponding to an object such as a lane, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, a building, or the like.

[0357] When the self-driving vehicle 1220 to be controlled / interacted within the XR image acquires sensor information from sensors each including a camera, the self-driving vehicle 1220 or the XR device 1230 may generate an XR image based on the sensor information, and the XR device 1230 may output the generated XR image. The self-driving vehicle 1220 may operate based on a control signal received through an external device such as the XR device 1230 or based on user interaction.

[0358] The VR, AR, and MR technologies disclosed herein are applicable to various devices, including, for example, HMDs, HUDs attached to vehicles, portable phones, tablet PCs, laptops, desktop computers, TVs, and signage. VR, AR, and MR technologies may also be applicable to devices equipped with flexible or rollable displays.

[0359] The above-mentioned VR, AR, and MR technologies may be implemented based on CG and be distinguished by the ratio of CG images in images viewed by a user.

[0360] That is, VR only provides real objects or backgrounds in CG images, while AR superimposes virtual CG images on images of real objects.

[0361] MR and AR are similar in that virtual objects are mixed and combined with the real world. However, in AR, real objects and virtual objects created as CG images are distinguished from each other and used to supplement real objects, while in MR, virtual objects and real objects are treated equally. More specifically, for example, holographic services are an example of MR representation.

[0362] Recently, VR, AR, and MR are collectively referred to as XR without distinguishing each other. Therefore, the embodiments of the present disclosure are applicable to all VR, AR, MR, and XR.

[0363] For example, wired / wireless communication, input interfaces, output interfaces, and computing devices can be used as hardware (HW)-related element technologies applied to VR, AR, MR, and XR. In addition, tracking and matching, voice recognition, interaction and user interface, location-based services, search, and AI can be used as software (SW)-related element technologies.

[0364] Specifically, embodiments of the present disclosure are directed to solving at least one of communication with another device, efficient memory usage, reduced data throughput due to inconvenient user experience / user interface (UX / UI), video, sound, motion sickness, or other issues.

[0365] Figure 13 13 is a block diagram illustrating an XR device according to an embodiment of the present disclosure. The XR device 1300 includes a camera 1310, a display 1320, a sensor 1330, a processor 1340, a memory 1350, and a communication module 1360. Obviously, when necessary, one or more modules may be deleted or modified, and one or more modules may be added to the module without departing from the scope and spirit of the present disclosure.

[0366] The communication module 1360 can communicate with an external device or server by wire or wirelessly. The communication module 1360 can use, for example, Wi-Fi, Bluetooth, etc. for short-range wireless communication, and use, for example, 3GPP communication standards for long-range wireless communication. LTE is a technology after 3GPP TS 36.xxx version 8. Specifically, LTE after 3GPP TS 36.xxx version 10 is called LTE-A, and LTE after 3GPP TS 36.xxx version 13 is called LTE-Apro. 3GPP 5G refers to the technology after TS 36.xxx version 15 and the technology after TS 38.xxx version 15. Specifically, the technology after TS 38.xxx version 15 is called 3GPP NR, and the technology after TS 36.xxx version 15 is called enhanced LTE. "xxx" represents the technical specification number. LTE / NR can be collectively referred to as the 3GPP system.

[0367] The camera 1310 can capture the surrounding environment of the XR device 1300 and convert the captured image into an electrical signal. The image captured by the camera 1310 and converted into an electrical signal can be stored in the memory 1350 and then displayed on the display 1320 by the processor 1340. In addition, the image can be displayed on the display 1320 by the processor 1340 without being stored in the memory 1350. In addition, the camera 1310 can have a field of view (FoV). For example, the FoV is an area in which real objects around the camera 1310 can be detected. The camera 1310 can only detect real objects within the FoV. When a real object is within the FoV of the camera 1310, the XR device 1300 can display an AR object corresponding to the real object. In addition, the camera 1310 can detect the angle between the camera 1310 and the real object.

[0368] Sensor 1330 may include at least one sensor. For example, sensor 1330 may include a sensing device such as a gravity sensor, a geomagnetic sensor, a motion sensor, a gyroscope sensor, an acceleration sensor, a tilt sensor, a brightness sensor, an altitude sensor, an olfactory sensor, a temperature sensor, a depth sensor, a pressure sensor, a bending sensor, an audio sensor, a video sensor, a global positioning system (GPS) sensor, and a touch sensor. Furthermore, while display 1320 may be a fixed type, display 1320 may be configured as a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an electroluminescent display (ELD), or a micro-LED (M-LED) display to provide flexibility. Herein, sensor 1330 is designed to detect the curvature of display 1320 configured as the aforementioned LCD, OLED, ELD, or M-LED display.

[0369] The memory 1350 is provided with a function of storing all or part of the result values obtained by wired / wireless communication with an external device or service and a function of storing images captured by the camera 1310. In particular, considering the trend of increasing communication data traffic (for example, in a 5G communication environment), efficient memory management is desired. In this regard, reference will be made to Figure 14 Give a description.

[0370] Figure 14 yes Figure 13 Detailed block diagram of memory 1350 is shown in FIG. Figure 14 , a swap-out process between a random access memory (RAM) and a flash memory according to an embodiment of the present disclosure will be described.

[0371] When swapping out AR / VR page data from the RAM 1410 to the flash memory 1420 , the controller 1430 may swap out only one of two or more AR / VR page data of the same content among the AR / VR page data to be swapped out to the flash memory 1420 .

[0372] That is, the controller 1430 may calculate an identifier (e.g., a hash function) that identifies each content of the AR / VR page data to be swapped out, and determine that two or more AR / VR page data having the same identifier among the calculated identifiers contain the same content. Thus, the problem of shortening the lifespan of the AR / VR device including the flash memory 1420 and the lifespan of the flash memory 1420 due to unnecessary AR / VR page data being stored in the flash memory 1420 can be overcome.

[0373] The operations of the controller 1430 may be implemented in software or hardware without departing from the scope of the present disclosure. More specifically, Figure 14 The memory shown is included in an HMD, a vehicle, a portable phone, a tablet PC, a laptop computer, a desktop computer, a TV, a sign, etc., and performs an exchange function.

[0374] The device according to an embodiment of the present disclosure may process 3D point cloud data to provide users with various services such as VR, AR, MR, XR, and self-driving.

[0375] The sensor that collects 3D point cloud data can be any of LiDAR, red, green, and blue depth (RGB-D), and 3D laser scanners. The sensor can be installed inside or outside of HMDs, vehicles, mobile phones, tablet PCs, laptops, desktop computers, TVs, signage, etc.

[0376] Figure 15 A point cloud data processing system is shown.

[0377] Reference Figure 15 The point cloud processing system 1500 includes: a sending device that acquires, encodes, and sends point cloud data; and a receiving device that acquires point cloud data by receiving and decoding video data. Figure 15As shown, point cloud data according to an embodiment of the present disclosure can be acquired by capturing, synthesizing or generating point cloud data (S1510). During acquisition, data of the 3D position (x, y, z) / attributes (color, reflectivity, transparency, etc.) of the point can be generated (for example, a polygon file format or a standard triangle format (PLY) file). For a video of multiple frames, one or more files can be acquired. Metadata related to point cloud data (for example, metadata related to capture) can be generated during capture. A sending device or encoder according to an embodiment of the present disclosure can encode point cloud data through video-based point cloud compression (V-PCC) or geometry-based point cloud compression (G-PCC) and output one or more video streams (S1520). V-PCC is a scheme for compressing point cloud data based on a 2D video codec such as High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), and G-PCC is a scheme for encoding point cloud data into two streams: geometry and attributes. The geometry stream can be generated by reconstructing and encoding position information about the points, and the attribute stream can be generated by reconstructing and encoding attribute information (e.g., color) related to each point. In V-PCC, although compatible with 2D video, more data is required to restore the data processed by V-PCC (e.g., geometry video, attribute video, occupancy map video, and auxiliary information) compared to G-PCC, resulting in a long delay in providing services. One or more output bitstreams can be encapsulated in the form of a file (e.g., a file format such as ISOBMFF) along with related metadata and sent via a network or through a digital storage medium (S1530).

[0378] The device or processor according to an embodiment of the present disclosure may decapsulate the received video data to obtain one or more bitstreams and related metadata, and decode the obtained V-PCC or G-PCC bitstream to restore 3D point cloud data (S1540). A renderer may render the decoded point cloud data and provide content suitable for VR / AR / MR / services to the user on a display (S1550).

[0379] like Figure 15 As shown, the device or processor according to an embodiment of the present disclosure may perform decoding processing or feedback processing (S1560) to transmit various feedback information obtained during rendering / display to the transmitting device. The feedback information according to the embodiment of the present disclosure may include head orientation information, viewport information indicating the area the user is viewing, and the like. Since the user interacts with the service (or content) provider through feedback processing, the device according to the embodiment of the present disclosure may use the above-mentioned V-PCC or G-PCC scheme to provide a higher data processing speed, or may allow clear video construction and provide various services in consideration of the user's high convenience.

[0380] Figure 16 is a block diagram of an XR device 1600 including a learning processor. Figure 13 In comparison, only the learning processor 1670 is added, so redundant description is avoided, as other components can be referred to. Figure 13 .

[0381] Reference Figure 16 , the XR device 1600 may be loaded with a learning model. The learning model may be implemented in hardware, software, or a combination of hardware and software. If all or part of the learning model is implemented in software, one or more instructions forming the learning model may be stored in memory 1650.

[0382] According to an embodiment of the present disclosure, the learning processor 1670 can be communicatively coupled to the processor 1640 and repeatedly train a model including an artificial neural network (ANN) using training data. An ANN is an information processing system in which multiple neurons are linked in layers, thereby modeling the operating principles of biological neurons and the connections between neurons. An ANN is a statistical learning algorithm in machine learning and cognitive science that is inspired by neural networks (specifically, the brain in the central nervous system of animals). Machine learning is a field of AI that gives computers the ability to learn without explicit programming. Machine learning is the technology of studying and constructing systems and algorithms for learning, predicting, and improving their capabilities based on empirical data. Therefore, according to an embodiment of the present disclosure, the learning processor 1670 can infer result values from new input data by determining optimized model parameters of the ANN. Therefore, the learning processor 1670 can analyze a user's device usage patterns based on historical information about the user's device usage. In addition, the learning processor 1670 can be configured to receive, classify, store, and output information for data mining, data analysis, intelligent decision-making, and machine learning algorithms and techniques.

[0383] According to an embodiment of the present disclosure, the processor 1640 may determine or predict at least one executable operation of the device based on data analyzed or generated by the learning processor 1670. In addition, the processor 1640 may request, search, receive or use data of the learning processor 1670, and control the XR device 1600 to perform the predicted operation or the operation determined to be desirable among at least one executable operation. According to an embodiment of the present disclosure, the processor 1640 may perform various functions for implementing intelligent simulation (e.g., knowledge-based systems, reasoning systems, and knowledge acquisition systems). The various functions can be applied to adaptive systems, machine learning systems, and various types of systems including ANNs (e.g., fuzzy logic systems). That is, the processor 1640 may predict the user's device usage pattern based on the usage pattern data analyzed by the learning processor 1670, and control the XR device 1600 to provide a more suitable XR service to the UE. Herein, XR services include at least one of AR services, VR services, or MR services.

[0384] Figure 17 Shown by Figure 16 The illustrated XR device 1600 of the present disclosure provides processing for XR services.

[0385] According to an embodiment of the present disclosure, the processor 1670 may store device usage history information about the user in the memory 1650 (S1710). The device usage history information may include information about the name, category, and content of content provided to the user, information about the time of device usage, information about the location of device usage, time information, and information about the use of applications installed in the device.

[0386] According to an embodiment of the present disclosure, the learning processor 1670 may acquire device usage pattern information about the user by analyzing device usage history information (S1720). For example, when the XR device 1600 provides specific content A to the user, the learning processor 1670 may learn information about the device usage pattern of the user using the corresponding terminal by combining specific information about content A (e.g., information about the age of users who typically use content A, information about the content of content A, and information about content similar to content A) with information about the time, location, and number of times the user using the corresponding terminal consumed content A.

[0387] According to an embodiment of the present disclosure, the processor 1640 may obtain user device pattern information generated based on the information learned by the learning processor 1670, and generate device usage pattern prediction information (S1730). In addition, when the user is not using the device 1600, if the processor 1640 determines that the user is in a place where the user frequently uses the device 1600 or it is almost the time when the user usually uses the device 1600, the processor 1640 may instruct the device 1600 to operate. In this case, the device according to an embodiment of the present disclosure may provide AR content based on the user pattern prediction information (S1740).

[0388] When the user is using the device 1600, the processor 1640 may check information about the content currently provided to the user and generate device usage pattern prediction information about the user with respect to the content (for example, when the user requests other related content or additional data related to the current content). In addition, the processor 1640 may provide AR content based on the device usage pattern prediction information by instructing the device 1600 to operate (S1740). The AR content according to an embodiment of the present disclosure may include advertisements, navigation information, danger information, etc.

[0389] Figure 18 Shows the appearance of the XR device and the robot.

[0390] The component modules of the XR device 1800 according to embodiments of the present disclosure are previously described with reference to previous drawings, and thus a redundant description is not provided herein.

[0391] Figure 18 The appearance of the robot 1810 shown is only an example, and according to the present disclosure, the robot 1810 can be implemented to have various appearances. For example, Figure 18 The robot 1810 shown may be a drone, a cleaner, a cooker, a wearable robot, etc. Specifically, the various components of the robot 1810 may be arranged in different positions such as up, down, left, right, back, or front according to the shape of the robot 1810 .

[0392] The robot 1810 may be provided with various sensors on its exterior to identify surrounding objects. In addition, in order to provide specific information to the user, the robot 1810 may be provided with an interface unit 1811 on its top surface or rear surface 1812.

[0393] In order to sense the movement of the robot 1810 and surrounding objects and control the robot 1810, a robot control module 1850 is installed inside the robot 1810. The robot control module 1850 can be implemented as a software module or a hardware chip in which the software module is implemented. The robot control module 1850 may include a deep learner 1851, a sensing information processor 1852, a movement path generator 1853, and a communication module 1854.

[0394] The sensing information processor 1852 collects and processes information sensed by various types of sensors (e.g., LiDAR sensor, IR sensor, ultrasonic sensor, depth sensor, image sensor, and microphone) arranged in the robot 1810.

[0395] The deep learner 1851 may receive information processed by the sensing information processor 1852 or accumulated information stored during the movement of the robot 1810 and output results for the robot 1810 to determine surrounding conditions, process information, or generate a movement path.

[0396] The movement path generator 1853 may calculate a movement path of the robot 1810 using data calculated by the deep learner 8151 or data processed by the sensing information processor 1852 .

[0397] Since each of the XR device 1800 and the robot 1810 is provided with a communication module, the XR device 1800 and the robot 1810 can transmit and receive data via short-range wireless communication such as Wi-Fi or Bluetooth or 5G long-range wireless communication. Figure 19 Techniques for controlling a robot 1810 using an XR device 1800 are described.

[0398] Figure 19 is a flowchart illustrating a process of controlling a robot using an XR device.

[0399] The XR device and the robot are communicatively connected to the 5G network (S1901). Obviously, without departing from the scope of this disclosure, the XR device and the robot can send and receive data through any other short-range or long-range communication technology.

[0400] The robot captures images / videos of the robot's surroundings using at least one camera mounted inside or outside the robot (S1902) and transmits the captured images / videos to the XR device (S1903). The XR device displays the captured images / videos (S1904) and transmits commands for controlling the robot to the robot (S1905). Without departing from the scope of the present disclosure, the commands may be manually input by a user of the XR device or automatically generated by AI.

[0401] The robot executes the function corresponding to the command received in step S1905 (S1906) and sends the result value to the XR device (S1907). The result value can be a general indicator indicating whether the data has been successfully processed, the currently captured image, or specific data considering the XR device. The specific data is designed to change, for example, depending on the state of the XR device. If the display of the XR device is off, a command for turning on the display of the XR device is included in the result value in step S1907. Therefore, when an emergency occurs around the robot, a notification message can be sent even if the display of the remote XR device is off.

[0402] The AR / VR content is displayed according to the result value received in step S1907 ( S1908 ).

[0403] According to another embodiment of the present disclosure, the XR device may display location information about the robot using a GPS module attached to the robot.

[0404] Reference Figure 13 The XR device 1300 described above can be connected to a vehicle providing a self-driving service in a manner that allows wired or wireless communication, or can be installed in a vehicle providing a self-driving service. Therefore, various services including AR / VR can be provided even in a vehicle providing a self-driving service.

[0405] Figure 20 A vehicle providing a self-driving service is shown.

[0406] According to an embodiment of the present disclosure, the vehicle 2010 may include a car, a train, and a motorcycle as a means of transportation traveling on roads or railways. According to an embodiment of the present disclosure, the vehicle 2010 may include all internal combustion engine vehicles provided with an engine as a power source, hybrid vehicles provided with an engine and an electric motor as power sources, and electric vehicles provided with an electric motor as a power source.

[0407] According to an embodiment of the present disclosure, the vehicle 2010 may include the following components to control the operation of the vehicle 2010: a user interface device, an object detection device, a communication device, a driving control device, a main electronic control unit (ECU), a drive control device, a self-driving device, a sensing unit, and a position data generation device.

[0408] Each of the user interface device, object detection device, communication device, driving control device, main ECU, drive control device, self-driving device, sensing unit and position data generation device may generate an electric signal and be implemented as an electronic device that exchanges the electric signal.

[0409] The user interface device can receive user input and provide information generated from vehicle 2010 to the user in the form of a UI or UX. The user interface device may include an input / output (I / O) device and a user monitoring device. The object detection device can detect the presence of objects outside vehicle 2010 and generate information about the objects. The object detection device may include, for example, at least one of a camera, LiDAR, an IR sensor, or an ultrasonic sensor. The camera can generate information about objects outside vehicle 2010. The camera may include one or more lenses, one or more image sensors, and one or more processors to generate object information. The camera can use various image processing algorithms to obtain information about the location, distance, or relative speed of an object. Furthermore, the camera may be installed in vehicle 2010. The camera can secure a FoV position to capture images of the environment surrounding vehicle 2010 and can be used to provide AR / VR-based services. The LiDAR can generate information about objects outside vehicle 2010. The LiDAR may include an optical transmitter, an optical receiver, and at least one processor electrically coupled to the optical transmitter and receiver to process received signals and generate data about the objects based on the processed signals.

[0410] The communication device can exchange signals with devices external to vehicle 2010 (e.g., infrastructure such as a server or broadcast station, another vehicle, or a terminal). The driving control device is a device that receives user input for driving. In manual mode, vehicle 2010 can travel based on signals provided by the driving control device. The driving control device may include a steering input device (e.g., a steering wheel), an acceleration input device (e.g., an accelerator pedal), and a braking input device (e.g., a brake pedal).

[0411] The sensing unit may sense the state of vehicle 2010 and generate state information. The position data generating device may generate position data of vehicle 2010. The position data generating device may include at least one of a GPS or a differential global positioning system (DGPS). The position data generating device may generate position data of vehicle 2010 based on a signal generated from at least one of the GPS or DGPS. The main ECU may provide overall control of at least one electronic device provided in vehicle 2010, and the drive control device may electrically control the vehicle drive device in vehicle 2010.

[0412] The self-driving device can generate a route for the self-driving service based on data acquired from the object detection device, the sensing unit, the position data generation device, and the like. The self-driving device can generate a travel plan for traveling along the generated route and, based on the travel plan, generate a signal for controlling the movement of the vehicle. The signal generated by the self-driving device is transmitted to the drive control device, which can then control the vehicle drive device in the vehicle 2010.

[0413] like Figure 20 As shown, a vehicle 2010 providing a self-driving service is connected to an XR device 2000 in a manner that allows wired / wireless communication. The XR device 2000 may include a processor 2001 and a memory 2002. Although not shown, Figure 20 The XR device 2000 may also include the previously referenced Figure 13 Components of the XR device 1300 are described.

[0414] If the XR device 2000 is connected to the vehicle 2010 in a manner that allows wired / wireless communication, the XR device 2000 can receive / process AR / VR service-related content data that can be provided in conjunction with the self-driving service and transmit the received / processed AR / VR service-related content data to the vehicle 2010. Furthermore, when the XR device 2000 is mounted on the vehicle 2010, the XR device 2000 can receive / process AR / VR service-related content data based on user input signals received via the user interface device and provide the received / processed AR / VR service-related content data to the user. In this case, the processor 2001 can receive / process AR / VR service-related content data based on data obtained from the object detection device, the sensing unit, the position data generation device, the self-driving device, and the like. According to embodiments of the present disclosure, the AR / VR service-related content data may include entertainment content unrelated to the self-driving service, weather information, and the like, as well as information related to the self-driving service (e.g., driving information, path information for the self-driving service, driving maneuver information, vehicle status information, and object information).

[0415] Figure 21 A process for providing AR / VR services during self-driving services is shown.

[0416] According to an embodiment of the present disclosure, the vehicle or the user interface device may receive a user input signal (S2110). According to an embodiment of the present disclosure, the user input signal may include a signal indicating a self-driving service. According to an embodiment of the present disclosure, the self-driving service may include a full self-driving service and a general self-driving service. The full self-driving service means that the vehicle completely drives itself to the destination without manual driving by the user, while the general self-driving service means that the vehicle drives to the destination through a combination of manual driving by the user and self-driving.

[0417] It may be determined whether the user input signal according to an embodiment of the present disclosure corresponds to a fully self-driving service (S2120). When it is determined that the user input signal corresponds to a fully self-driving service, the vehicle according to an embodiment of the present disclosure may provide the fully self-driving service (S2120). Since the fully self-driving service does not require user manipulation, the vehicle according to an embodiment of the present disclosure may provide VR service-related content to the user through the vehicle window, the rearview mirror of the vehicle, the HMD or the smart phone (S2130). The VR service-related content according to an embodiment of the present disclosure may be content related to fully self-driving (e.g., navigation information, driving information, and external object information), and may also be content unrelated to fully self-driving according to user selection (e.g., weather information, distance images, natural images, and voice call images).

[0418] If it is determined that the user input signal does not correspond to a fully autonomous driving service, the vehicle according to an embodiment of the present disclosure may provide a general autonomous driving service (S2140). Since the user's FoV should be ensured for manual driving in the general autonomous driving service, the vehicle according to an embodiment of the present disclosure may provide the user with AR service-related content through a vehicle window, a rearview mirror of the vehicle, an HMD, or a smartphone (S2140).

[0419] According to an embodiment of the present disclosure, AR service-related content may be content related to fully autonomous driving (e.g., navigation information, driving information, and external object information), and may also be content unrelated to autonomous driving (e.g., weather information, distance images, natural images, and voice call images) according to user selection.

[0420] Although the present disclosure is applicable to all fields of 5G communication, robotics, self-driving, and AI as described above, a description of the present disclosure as applied to XR devices will be given below with reference to the following drawings.

[0421] Figure 22 is a conceptual diagram illustrating an example method of implementing an XR device using an HMD type according to an embodiment of the present disclosure. Figure 22 The HMD type shown is implemented.

[0422] Figure 22 The HMD-type XR device 100a shown may include a communication unit 110, a control unit 120, a memory unit 130, an input / output (I / O) unit 140a, a sensor unit 140b, a power supply unit 140c, etc. Specifically, the communication unit 110 embedded in the XR device 10a may communicate with the mobile terminal 100b via wired or wireless communication.

[0423] Figure 23is a conceptual diagram illustrating an example method of implementing an XR device using AR glasses according to an embodiment of the present disclosure. Figure 23 The AR glasses type implementation shown.

[0424] Reference Figure 23 , the AR glasses may include a frame, a control unit 200 and an optical display unit 300.

[0425] Although the framework can be Figure 23 The frame is shown formed into a glasses shape worn on the face of the user 10, but the scope or spirit of the present disclosure is not limited thereto, and it should be noted that the frame may also be formed into a goggle shape worn in close contact with the face of the user 10.

[0426] The frame may include a front frame 110 and first and second side frames.

[0427] The front frame 110 may include at least one opening and may extend in a first horizontal direction (e.g., an X-axis direction). The first and second side frames may extend in a second horizontal direction (e.g., a Y-axis direction) perpendicular to the front frame 110 and may extend parallel to each other.

[0428] The control unit 200 may generate an image to be viewed by the user 10, or may generate a result image formed from a series of images. The control unit 200 may include an image source configured to create and generate an image, a plurality of lenses configured to diffuse and converge light generated from the image source, and the like. The image generated by the control unit 200 may be transmitted to the optical display unit 300 via a guide lens P200 disposed between the control unit 200 and the optical display unit 300.

[0429] The control unit 200 may be fixed to any one of the first and second side frames. For example, the control unit 200 may be fixed to the inside or outside of any one of the side frames, or may be embedded in any one of the side frames and integrated therewith.

[0430] The optical display unit 300 may be formed of a translucent material so that the optical display unit 300 may display an image created by the control unit 200 for easy recognition by the user 10 and may allow the user to view the external environment through the opening.

[0431] The optical display unit 300 may be inserted into the opening included in the front frame 110 and fixed, or may be located at the rear surface of the opening (interposed between the opening and the user 10) so that the optical display unit 300 can be fixed to the front frame 110. For example, the optical display unit 300 may be located at the rear surface of the opening and, as an example, may be fixed to the front frame 110.

[0432] Reference Figure 23In the XR device shown, when an image is incident on the incident area S1 of the optical display unit 300 through the control unit 200, the image light can be transmitted through the optical display unit 300 to the emission area S2 of the optical display unit 300, and the image created by the control unit 200 can be displayed for recognition by the user 10.

[0433] Therefore, the user 10 can view the external environment through the opening of the frame 110 while viewing the image created by the control unit 200 .

[0434] In addition, the following shows Figure 24 The XR device shown in FIG. 1 is a display device. The embodiment of the present invention will be described using it as an example. However, of course, the XR device according to the embodiment of the present invention can be used Figures 1 to 23 It is implemented by the XR device shown.

[0435] The multimedia device (or device) according to the embodiment may include a device equipped with a display function, which is similar to the device described in the embodiment of the present invention. Figures 1 to 9 The described user equipment (UE) corresponds to, but is not limited to, an XR device and can additionally perform 5G communications.

[0436] According to the device of the embodiment (for example, Figures 1 to 23 The device described in ( ) can provide content (e.g., VR content, AR content, MR content, XR content, etc.). The content according to the embodiment includes 360° content that can be displayed in a 360° space. 360° content is content that can be displayed in a virtual 360° space represented as one or more areas based on user location information. Therefore, the 360° content according to the embodiment may be represented as a set of content corresponding to one or more areas. The 360° space according to the embodiment may be represented as a 2D or 3D coordinate system, and each area may be represented as a coordinate of the coordinate system. Therefore, the 360° content according to the embodiment may include signaling information related to one or more areas. The 360° content according to the embodiment can be used to provide navigation services (e.g., car navigation, bicycle navigation, etc.), sports services (e.g., golf), and the embodiment is not limited thereto.

[0437] The device according to the embodiment obtains user position information by tracking the user position and provides 360° content on one or more areas based on signaling information according to the user position information. The user position information according to the embodiment is information about the user's posture in the 360° space and is determined based on the user's head position information (e.g., head posture (position and / or angle) information) and / or the user's body position information (e.g., body posture (position and / or angle) information). In addition, the user position information according to the embodiment can be represented as coordinates of a coordinate system representing a 360° space. Therefore, the device can provide the user with content corresponding to the area corresponding to the current user position based on the user position information. Therefore, the content provided to the user according to the user position will be referred to as a front view. The size of the front view according to the embodiment corresponds to the size of the field of view (FOV). Therefore, the device provides instant content according to the user's changed position, thereby providing a more realistic XR environment.

[0438] Figure 24 is a diagram showing a coordinate system of a 360° space according to an embodiment.

[0439] Reference Figure 24 , the coordinate system 2400 includes a 2D coordinate system generated by setting the user's position as the center point 2410. The 360° space according to the embodiment may be represented as a 3D coordinate system as well as a 2D coordinate system, but the present embodiment is not limited thereto. The above-mentioned device may display a front view to the user located at the center point 2410. The colored area 2420 in the coordinate system 2400 indicates the FOV (e.g., 120°) of the user at the initial user position (e.g., the first user position) located at the center point 2410. That is, when the user located at the center point 2410 looks in the direction of the arrow 2415 corresponding to the vertical axis of the coordinate system 2400, the FOV may be represented as 60° to the left and right with the vertical axis as the center in the coordinate system 2400. Therefore, the device according to the embodiment displays a front view (e.g., a first front view) of the content of the area corresponding to the direction of the arrow 2415.

[0440] While the body is fixed, the person's head can be rotated up to 90° to the left and right. The dotted area 2430 in the coordinate system 2400 indicates the FOV at the user position (e.g., the second user position) when the user at the center point 2410 rotates his head up to 90° in the direction of the arrow 2435 shown on the right side of the figure while the body is fixed. Although the position of the user's head or body changes, the size of the FOV does not change. Therefore, as shown in the figure, the size of the FOV according to the changed user position is equal to the size of the FOV according to the user position before the change. However, the FOV according to the changed user position is represented as +30° to +150° in the coordinate system 2400. Therefore, the device according to the embodiment can display a front view (e.g., a second front view) of the content of the area corresponding to the direction of the arrow 2435. The oblique line area 2440 in the coordinate system 2400 indicates the overlapping area between the colored area 2420 and the dot area 2430. Therefore, if the user's head position at center point 2410 gradually changes until it reaches 90° (i.e., the maximum head rotation angle), the size of diagonal area 2440 gradually decreases. Diagonal area 2440 shown in the figure is the minimum overlapping area, represented as +30° to +60° in coordinate system 2400. Therefore, if the user's head position changes, the device can provide 360° content corresponding to the area corresponding to the changed head position.

[0441] Figure 25 is a diagram illustrating a 360° space according to an embodiment.

[0442] Reference Figure 25 , the 3D ring 2500 shown on the left indicates a 360° space that can display 360° content with the user's location set as the center point. As shown in the figure, the 3D space can be represented as at least four areas. The four areas according to the embodiment can be based on Figure 24 The FOV size and the maximum change in head position are configured as described in [1].

[0443] The first region 2501 shown in the figure is located at the center point (eg, Figure 24 The area corresponding to the FOV at the position of the user (e.g., the first user position) (e.g., the center point 2410 shown in the figure). Therefore, the size of the first area 2501 corresponds to the size of the FOV (e.g., 120°). The device can provide a first front view of the content belonging to the first area based on the position of the user. The second area 2505 shown in the figure is an area set according to the maximum change value of the user's head position. As shown in FIG. Figure 24 As described above, since the head of a person can be rotated to the left and right by a maximum of 90° while the body is fixed, the maximum change value of the head position according to the embodiment corresponds to 90°. Therefore, the size of the second area 2505 corresponds to the size of the maximum change value. Figure 24 As described above, if a user at a center point (e.g., center point 2410) turns his head by a maximum of 90°, the FOV may be represented as +30° to +150° in the coordinate system (e.g., coordinate system 2400). The device according to the embodiment may display a second front view corresponding to the FOV according to the changed head position. Figure 24 As depicted, region 2430 of the FOV corresponding to the second front view overlaps region 2420 of the FOV corresponding to the first front view. Therefore, the second front view may include content corresponding to a portion of first region 2501 and all of second region 2505. As described above, the second front view may include content corresponding to a portion of first region 2501 and a portion of second region 2505 depending on the change in head position.

[0444] As shown in the figure, the third area has the same size as the second area 2505, and the fourth area has the same size as the first area 2501. Therefore, the sum of the angles corresponding to the four areas shown in the figure is 360°.

[0445] Figure 25 The right portion of the is a diagram showing a 3D space 2510 represented by an expanded 3D ring 2500, according to an embodiment. As described above, while the user's body is fixed, the user's head can be rotated up to 90° to the left and right. Therefore, if the user's head is rotated up to 90° in the direction of left arrow 2511, the device can provide a front view of the content corresponding to a portion of the first area and the entire third area. If the user's head is rotated up to 90° in the direction of right arrow 2515, the device can provide a front view of the content corresponding to a portion of the first area and the entire third area.

[0446] Figure 26 is a block diagram showing an example of the configuration of a device according to an embodiment.

[0447] Reference Figure 26 , device 2600 is provided with reference Figure 24 and Figure 25 An example of a content providing device for 360° content is described, and reference may be made to Figures 1 to 25 Describe the functionality / operation. In addition, Figure 26 The device 2600 shown may also include a device for performing a reference Figures 1 to 25 Describe the function / mode of operation etc.

[0448] The device 2600 according to an embodiment may include one or more sensors 2610, one or more processors 2620, a display 2632, and one or more memories 2640. The components of the device 2600 may be connected to each other to allow mutual communication.

[0449] According to an embodiment, one or more sensors 2610 can ensure user position information. As described above, according to an embodiment, the user position can be set based on the user's head position and body position. One or more sensors 2610 can periodically detect the user's head position and body position. One or more sensors 2610 can send the ensured user position information to one or more processors 2620. Although one or more sensors 2610 are depicted as a single block, they can be implemented as a collection or group of physical sensors. One or more sensors 2610 may include a camera sensor, an IR sensor, etc. According to an embodiment, one or more sensors 2610 can ensure user input signals (e.g., device touch input signals, user gestures for executing displayed virtual menus, etc.) that control the device or execute the content provided by the device, as well as the user's head position and / or body position information.

[0450] According to an embodiment, the one or more processors 2620 may process the 360° content to be displayed based on the ensured user location information. As described above, the 360° content is content that can be displayed in a 360° space represented as one or more areas based on the user location information. Although the one or more processors 2620 are depicted as a single block, they may be implemented as one or more physical chips. The one or more processors 2620 may process the 360° content to be displayed based on the first user location indicated by the user location information (e.g., Figure 24 a first user position) to provide a first front view (e.g., Figure 24 The first front view according to the embodiment is content corresponding to the first area.

[0451] If the user position information indicates that the user's head position changes and the user's body position is fixed, the one or more processors 2620 may determine the second user position (e.g., Figure 24 ) to process the second front view (e.g., Figure 24 ). Figure 24 and Figure 25 As described above, the second area (e.g., second area 2505) or the third area can be set based on the maximum change value of the head position (e.g., 90°). Therefore, depending on the change value of the head position, the second front view according to the embodiment may be content corresponding to a portion of the first area and a portion of the second area or a portion of the third area, or content corresponding to a portion of the first area and the entire second area or the entire third area.

[0452] If the user position information indicates that the user's head position and the user's body position have changed, the one or more processors 2620 may process the first front view to be displayed based on a third user position determined based on the changed head position and body position. That is, if the head position and the body position change together, the one or more processors 2620 may process the first front view displayed based on the first user position to be continuously displayed.

[0453] According to an embodiment, the display 2632 displays 360° content based on the user's position. That is, the display 2632 can display a front view provided based on the user's position. As described above, the display 2632 can display a first front view provided based on a first user position. The display 2632 can also display a second front view provided based on a second user position determined based on a changed head position. In addition, the display 2632 can accurately display the first front view provided at the first user position based on a third user position. Although the display 2632 is depicted as a single block, it can be implemented as one or more physical displays.

[0454] According to an embodiment, one or more memories 2640 may store one or more programs, which include instructions for executing and controlling at least one operation of one or more sensors 2610, one or more processors 2620, and display 2632. According to an embodiment, one or more programs may include instructions for executing one or more algorithms. According to an embodiment, one or more memories 2640 may include high-speed random access memory and non-volatile memory (e.g., one or more magnetic disk storage devices, flash memory devices, non-volatile solid-state memory devices, etc.). The one or more processors 2620 may allow or execute a set of software programs and / or instructions stored in one or more memories to perform various functions of the device 2600 and process data.

[0455] Figure 26 One or more components of the device 2600 shown may be implemented using hardware, software, firmware, or a combination thereof, including one or more processors or integrated circuits, but the above embodiments are not limited thereto.

[0456] Figure 27 is a diagram illustrating an example of a content providing operation of a device according to an embodiment.

[0457] Figure 27 Show Figure 26 The left part of the figure shows an example of a process of providing 360° content based on user location information by the device described in the embodiment (e.g., device 2600). Figure 24 First user location, Figure 262700 of providing a first front view of the first user position, etc. Figure 26 and Figure 27 As described, the first user position can be set based on both the user's head position and the user's body position. Figures 24 to 26 As depicted, the first front view corresponds to the FOV 2732 at the first user position and is content corresponding to the first area.

[0458] Figure 27 The middle portion of the diagram shows a process 2710 in which the apparatus according to an embodiment provides a second front view according to a second user position based on the user position information. If the user position information indicates that the body position of the user at the first user position is fixed (for example, the body position does not change) and only the head position changes, the apparatus according to an embodiment determines the second user position based on the changed head position and provides a second front view based on the second user position. The process 2710 shown in the figure shows a case where the user's head position changes by a maximum change value. The change value of the head position is equal to Figures 24 to 26 The same as described in , and its description will be omitted. Therefore, as reference Figures 24 to 26 As described, the second front view corresponds to the FOV 2740 at the second user position. Since the user's FOV does not change, the FOV 2740 at the second user position is equal to the FOV 2732 at the first user position. In addition, the second front view is content corresponding to a portion of the first area and the entire second area.

[0459] Figure 27 The bottom portion of the figure shows a process 2720 in which a device according to an embodiment provides a first front view based on a third user position based on user position information. If the user position information indicates that both the user's head position and the user's body position have changed from the first user position, the device according to an embodiment determines a third user position based on the changed head and body positions and provides the first front view based on the third user position. Specifically, if both the head and body positions change, the device according to an embodiment may continue to provide the first front view regardless of the user's position. Furthermore, since the user's FOV does not change, the FOV 2750 at the third user position is equal to the FOV 3732 at the first user position or the FOV 2740 at the second user position. Furthermore, the first front view is content corresponding to the first area. That is, if the user's head and body positions change together (e.g., simultaneously), the device according to an embodiment determines the user position determined based on the changed head and body positions as a reference position and provides 360° content.

[0460] Figure 28 is a diagram illustrating an example of a user gesture according to an embodiment.

[0461] As reference Figures 24 to 27 As described, the apparatus according to the embodiment ensures user position information and provides 360-degree content according to the user position determined based on the ensured user position information. The user position according to the embodiment is determined based on the user's head position and body position.

[0462] Figure 28 The left portion shown indicates a first user position 2800 (e.g., Figures 24 to 27 The device according to the embodiment may set the first user position 2800 as the center point of the 360° space for providing 360° content (for example, Figure 24 The device according to the embodiment may provide a first front view (e.g., Figures 24 to 27 1st front view described in FIG).

[0463] Figure 28 The second portion on the left side indicates a second user location 2810 (eg, Figures 24 to 27 As described above, if the user's body position is fixed and only the user's head position changes, the second user position 2810 may be determined based on the changed head position. The device may provide a second front view (e.g., Figures 24 to 27 ).

[0464] Figure 28 The third part in the 2820 indicates the third user position 2820. After the head position is changed, the third user position 2820 according to the embodiment may be determined based on the changed body position. In this case, the device according to the embodiment may provide a third front view of the content of the area corresponding to the third user position 3820 based on the third user position 3820. The third view according to the embodiment may be Figure 24 and Figure 25 2501, a portion of the second area 2505, and a portion of the fourth area described in the preceding text. In addition, the third front view may be content corresponding to a portion of the second area 2505 and a portion or all of the fourth area according to the third user position. Figure 27 Unlike the description of FIG, the device does not provide the first front view, but provides a third front view of the content of the area corresponding to the third position. If the user's body position and head position change together, the device according to the embodiment may perform Figure 27 The process 2710 described in for providing a first front view based on a third user position.

[0465] Figure 28The upper portion of FIG indicates a fourth user position 2830 determined when the body position and head position of the user at the third user position 2820 are changed. The fourth user position 2830 according to an embodiment is equal to the first user position 2800 as a reference position. Therefore, the device can provide a first front view (e.g., Figures 24 to 27 1st front view described in FIG).

[0466] Figure 29 is a diagram illustrating an example of a 360° content providing process according to an embodiment.

[0467] As reference Figures 24 to 27 As described, 360° content according to an embodiment is content that can be displayed in a 360° space represented as one or more areas. Therefore, a device (e.g., Figure 26 The device 2600 may provide the user with a front view of the content of the area corresponding to the user location based on the user location information. The front view according to the embodiment may correspond to a single area (e.g., Figures 24 to 27 ) or a portion or all of one or more regions (e.g., Figures 24 to 27 ). The 360° content according to an embodiment may include one or more virtual objects. The one or more virtual objects according to an embodiment may indicate device-related information (e.g., battery information, network information, etc.), time information, applications that can be provided to the user, etc., and may be represented as images, text, etc. The 360° content according to an embodiment may include signaling information related to one or more virtual objects. The signaling information according to an embodiment may include flag information indicating whether the virtual object is a fixed object that should always be displayed. If the corresponding virtual object is a fixed object, the flag information according to the embodiment has a default value. Therefore, if the flag information according to the embodiment indicates that the virtual object is a fixed object, the device (e.g., device 2600) may always display content including the virtual object regardless of the user location information.

[0468] Figure 29 A portion of FIG. 2900 illustrates a process 2900 in which an apparatus according to an embodiment provides a first front view to a user. Since the user position according to an embodiment corresponds to a first area (e.g., Figures 24 to 28), the first front view may include one or more virtual objects corresponding to the first area. As shown in the figure, since the user's position does not correspond to the second area, the device does not provide content corresponding to the second area (e.g., the oblique line area). Signaling information or preset flag information included in the 360° content according to the embodiment may respectively indicate whether the one or more virtual objects corresponding to the first area are fixed objects. The first front view 2900 according to the embodiment includes a first virtual object 2901, a second virtual object 2902, and a third virtual object 2903.

[0469] Figure 29 Another part of FIG. 29 shows a process 2910 in which the apparatus according to an embodiment provides the user with a second front view. If the user position determined according to the change in the user's head position corresponds to the second area (eg, Figures 24 to 28 ), the second front view according to an embodiment may include one or more virtual objects corresponding to the second area. However, since the first to third virtual objects 2901 to 2903 are fixed objects, they should always be displayed despite changes in the user's position. Therefore, the second front view according to an embodiment includes the first to third virtual objects 2901 to 2903.

[0470] Figure 30 is a diagram illustrating an example of a 360° content providing process according to an embodiment.

[0471] Figure 30 Show Figure 29 For example, refer to Figure 29 As described, according to an embodiment, 360° content may include one or more virtual objects corresponding to a first area and one or more virtual objects corresponding to a second area. If the user's position corresponds to the first area, the device (e.g., device 2600) may provide a front view including one or more virtual objects corresponding to the first area. When the user's position changes and corresponds to the second area, the device may provide a front view including one or more virtual objects corresponding to the second area.

[0472] Figure 30The left portion shows a process 3000 in which a device according to an embodiment provides a first front view corresponding to a first area. The first front view according to an embodiment may include a first virtual object 3001 as a fixed object. As shown in the figure, since the user's position does not correspond to the second area, the device does not provide content including a virtual object corresponding to the second area (e.g., the oblique line area). If the user's position corresponds to the second area, the device according to an embodiment may provide a second front view corresponding to the second area. As described above, since the first virtual object 3001 as a fixed object should always be displayed regardless of changes in the user's position, the second front view should include the first virtual object 3001. If at least one virtual object 3005 exists corresponding to the second area, the device may display a second front view including the first virtual object 3001 that completely or partially overlaps with the corresponding virtual object. Such content may disrupt the user experience.

[0473] Figure 30 The right portion of FIG3 shows a process 3010 in which the apparatus according to an embodiment provides a second front view corresponding to the second area. For a virtual object 3005 that completely or partially overlaps with the first virtual object 3001, the apparatus according to an embodiment (e.g., one or more processors 2610) may adjust the depth information or image size of the virtual object 3005 so that the corresponding virtual object 3005 can be displayed behind the first virtual object 3001. Therefore, the second front view may include the first virtual object 3001 located in the front and the virtual object 3005 processed to be located behind the first virtual object 3001.

[0474] In the case where 360° content is used to provide a bicycle navigation service, the 360° content may include one or more fixed objects that are always displayed. That is, regardless of the user's location, the device according to the embodiment may always display the fixed objects.

[0475] The fixed object according to an embodiment may include a virtual object indicating user status information (e.g., user's heart rate, user's breathing rate, user's blood sugar level, etc.). The information indicating the user status according to an embodiment may include information advising the user to drink water, information to adjust the breathing rate, etc.

[0476] According to an embodiment, a fixed object may include a virtual object indicating bicycle-related information. According to an embodiment, the bicycle-related information may include information about the bicycle status (e.g., average speed, current speed, riding time, helmet wearing, tire pressure, replacement time, brake health, etc.).

[0477] According to an embodiment, a fixed object may include a virtual object indicating information related to bicycle riding. According to an embodiment, information related to bicycle riding may include a mini map (indicating current location check, toilets, shelters, convenience stores, etc. on a fixed map), destination information, remaining distance (time) to the destination, traffic signal information, information about application execution, weather recommendation riding wear information, information that can be ensured by a bicycle rearview mirror, etc. According to an embodiment, a fixed object may include a virtual object indicating information unrelated to bicycle riding. For example, information unrelated to bicycle riding may include manipulation information for photography, etc. The fixed objects of the embodiment are not limited to the above examples.

[0478] The 360° content according to the embodiment may include one or more virtual objects set as non-fixed objects. That is, as described above, if the user position corresponds to a specific area, the device according to the embodiment may display a virtual object corresponding to the corresponding area.

[0479] Unfixed objects according to an embodiment may include virtual objects indicating pedestrian detection, information about surrounding buildings, information about surrounding vehicles (e.g., speed information, vehicle travel prediction information, abnormality detection information, etc.), parking information, currently playing music information, image reproduction, information for launching other additional applications, etc. Abnormality detection information according to an embodiment may include information about obstacles to bicycle riding (e.g., insects, etc.).

[0480] The unfixed objects according to the embodiment may include virtual objects indicating wind direction information, information related to the user's posture, user-customized seat height information, driving assistance information according to terrain (e.g., information predicting a fall on a terrain where a fall is likely), etc. The unfixed objects according to the embodiment are not limited to the above examples.

[0481] When the 360° content according to the embodiment is used to provide golf services, the 360° content may include one or more fixed objects that are always displayed. That is, regardless of the user's location, the device according to the embodiment may always display the fixed objects.

[0482] Fixed objects according to an embodiment may include virtual objects indicating information about other players, score information, weather information, golf course related information (e.g., map, distance to hole, slope information, etc.), information related to score, game progress time, etc. Fixed objects according to an embodiment are not limited to the above examples.

[0483] The 360° content according to the embodiment may include one or more virtual objects set as non-fixed objects. That is, as described above, if the user position corresponds to a specific area, the device according to the embodiment may display a virtual object corresponding to the corresponding area.

[0484] Unfixed objects according to embodiments may include virtual objects indicating club recommendation information based on the position of the ongoing game, wind direction information, ball-related information (e.g., ball direction prediction information, information about the ball's flight direction, ball position prediction information, etc.), and user posture-related information (e.g., the angle between the two feet, ball-to-user distance information, user eye correction information, etc.). Unfixed objects according to embodiments may also include virtual objects indicating information for assisting in ball-hitting power control and information related to a virtual driving range (e.g., a virtual field, ball, wind direction, etc.). Unfixed objects according to embodiments are not limited to the above examples.

[0485] Figure 31 is a diagram illustrating a field of view (FOV) according to an embodiment.

[0486] Figure 31 An example 3100 is shown indicating the FOV recognized by each of the left and right eyes in a monocular situation rather than a binocular situation. Figure 31 , the solid line indicates the FOV recognized by the left eye (e.g., 155°), and the dotted line indicates the FOV recognized by the right eye (e.g., 155°). Figure 31 The angle 3110 shown indicates the FOV (e.g., 120°) recognized by the user's two eyes. Therefore, the FOV in the monocular case is larger than that in the binocular case. The oblique line portion 3120 in the figure indicates the difference between the monocular FOV and the binocular FOV.

[0487] According to an embodiment, a device (e.g., one or more sensors 2610) may receive a user input signal for configuring priority information for each virtual object. If a virtual object completely or partially overlaps with at least one other virtual object, such as Figure 30 As shown, the priority information according to the embodiment can be used to determine the display setting order. The priority information according to the embodiment can be referred to as priority flag information or flag information. The device according to the embodiment (e.g., one or more processors 2620) can set a default value for the priority information. The device according to the embodiment can configure and change the priority information for each virtual object according to the user input signal, and then display the content including the virtual object according to the changed priority information.

[0488] Figure 32 is a diagram illustrating an example of a process of configuring priority information in response to a user input signal.

[0489] The device according to an embodiment (e.g., device 2600) may receive a user input signal for selecting the first virtual object 3210 to configure priority information of the first virtual object 3210 corresponding to the first area. The user input signal for selecting the first virtual object 3210 according to an embodiment may include a signal transmitted from a remote controller, a user touch gesture input to the device, a user gesture ensured from one or more sensors of the device (e.g., one or more sensors 2610), a user posture, etc. The user input signal for selecting the first virtual object 3210 according to an embodiment is not limited to the above examples.

[0490] Based on a user input signal selecting the first virtual object 3210, the device according to an embodiment may display priority information 3220 of the selected first virtual object 3210. The priority information 3220 according to an embodiment may be displayed in the form of a user-selectable menu, item, etc. As shown in the figure, the priority information 3220 according to an embodiment may include other values in addition to the preset default value.

[0491] As described above, the default value according to an embodiment indicates that the corresponding virtual object should always be set to the front. Therefore, when providing the second front view corresponding to the second area in response to a change in the user's position, if the priority information of the first virtual object corresponding to the first area has the default value, the device according to an embodiment provides the second front view including the first virtual object that should be set to the front.

[0492] According to an embodiment, a flag value of "1" means that the relative position of the first virtual object 3210 needs to be maintained. While the device according to an embodiment displays the front view corresponding to the first area, if the user's position corresponds to the second area, the device according to an embodiment may display a second front view corresponding to the second area. If the priority information of the first virtual object 3210 has a value of "1", the second front view may include the first virtual object 3210. If there are one or more objects corresponding to the second area and at least one virtual object completely or partially overlaps with the first virtual object 3210, the device according to an embodiment may compare the priority information value of the first virtual object 3210 with the priority information value of the corresponding virtual object.

[0493] If the priority information value of the virtual object that completely or partially overlaps with the first virtual object 3210 is 2 or 3, the device according to the embodiment (for example, one or more processors 2620) can provide a second front view in which the first virtual object 3210 is set in the front by changing the depth information of the corresponding virtual object.

[0494] If the priority information value of the virtual object that completely or partially overlaps with the first virtual object 3210 is 1, the device according to the embodiment (e.g., one or more processors 2620) may process the first virtual object 3210 to be set at another position. For example, the device according to the embodiment may provide a second front view including the first virtual object 3210, which is set in front of the virtual object whose priority information value is set to 2 or 3 among the virtual objects corresponding to the second area.

[0495] The flag value "2" according to an embodiment indicates that the relative position of the first virtual object 3210 is not maintained. Therefore, the device according to an embodiment can provide a second front view including the first virtual object 3210 whose size and the like are adjusted so as not to overlap with one or more virtual objects corresponding to the second area.

[0496] The flag value "3" according to an embodiment indicates that the first virtual object 3210 is an unfixed object and is not provided in other front views except the first front view corresponding to the first area. Therefore, the device according to an embodiment can provide a second front view that does not include the first virtual object 3210.

[0497] The device according to an embodiment may receive a user input signal for setting a flag value that appears in the priority information 3220. The user input signal for setting a flag value according to an embodiment may include a signal transmitted from a remote controller, a user touch gesture input to the device, a user gesture secured from one or more sensors of the device (e.g., one or more sensors 2610), a user posture, etc. The user input signal for setting a flag value according to an embodiment is not limited to the above examples.

[0498] Figure 33 is a diagram illustrating an example of a 360° content providing process according to an embodiment.

[0499] Figure 33 Showing an apparatus according to an embodiment (eg, Figure 26 The device 2600) is based on Figure 32 to provide 360° content processing using the priority information described in [1].

[0500] Figure 33The left portion of FIG. 3 shows an example of a process 3300 in which a device (e.g., device 2600) according to an embodiment provides a first front view of a first virtual object 3301 having a priority information value set to 1. The 360° content according to an embodiment may include one or more virtual objects corresponding to a second area (e.g., a second virtual object 3302, a third virtual object 3303, and a fourth virtual object 3304). According to an embodiment, the priority information value of the second virtual object 3302 is 1, the priority information value of the third virtual object 3303 is 3, and the priority information value of the fourth virtual object 3304 is 2. Since the user's location does not correspond to the second area, the second front view corresponding to the second area (indicated by a dotted line) is not displayed.

[0501] Figure 33 The right portion of the figure shows an example of a process 3320 in which, if the user position corresponds to the second area, the device according to the embodiment provides a second front view including the first virtual object 3301. As described above, since the priority information value of the first virtual object 3301 is 1, the relative position of the first virtual object 3301 should be maintained. However, if the relative position of the first virtual object 3301 is maintained, the first virtual object 3301 and the second virtual object 3302 inevitably overlap with each other in whole or in part. Therefore, the device according to the embodiment (e.g., one or more processors 2620) may adjust the depth information of the third virtual object 3303 so that the first virtual object 3301 is set before the third virtual object 3303 having the priority information value of 3. That is, as shown in the figure, the second front view includes the first virtual object 3301 set before the third virtual object 3303.

[0502] Figure 34 is a diagram illustrating an example of a 360° content providing process according to an embodiment.

[0503] As Figure 33 For example, Figure 34 Showing an apparatus according to an embodiment (eg, Figure 26 The device 2600) is based on Figure 32 to provide 360° content processing using the priority information described in [1].

[0504] Figure 34The left portion of FIG. 3 illustrates an example of a process 3400 in which a device (e.g., device 2600) according to an embodiment provides a first front view of a first virtual object 3401 including a priority information value set to 1. The 360° content according to an embodiment may include one or more virtual objects (e.g., second virtual object 3402) corresponding to a second area. According to an embodiment, the priority information value of second virtual object 3402 is 1. Since the user's location does not correspond to the second area, the second front view corresponding to the second area (indicated by a dotted line) is not displayed.

[0505] Figure 34 The right portion of the figure shows an example of a process 3410 in which, if the user position corresponds to the second area, the device according to the embodiment provides a second front view including the first virtual object 3401. As described above, since the priority information value of the first virtual object 3401 is 1, the relative position of the first virtual object 3401 should be maintained. However, if the relative position of the first virtual object 3401 is maintained, the first virtual object 3401 and the second virtual object 3402 inevitably overlap with each other in whole or in part. Therefore, the device according to the embodiment (e.g., one or more processors 2620) may adjust the depth information of the second virtual object 3402 so that the first virtual object 3401 is set in front of the second virtual object 3402. That is, as shown in the figure, the second front view includes the first virtual object 3401 set in front of the second virtual object 3402.

[0506] Figure 35 is a diagram illustrating an example of a 360° content providing process according to an embodiment.

[0507] As Figure 33 and Figure 34 For example, Figure 35 Showing an apparatus according to an embodiment (eg, Figure 26 The device 2600) is based on Figure 32 to provide 360° content processing using the priority information described in [1].

[0508] Figure 35 The left portion of FIG. 3 illustrates an example of a process 3500 in which a device (e.g., device 2600) according to an embodiment provides a first front view of a first virtual object 3501 including a priority information value set to 1. The 360° content according to an embodiment may include one or more virtual objects (e.g., second virtual object 3502) corresponding to a second area. According to an embodiment, the priority information value of second virtual object 3502 is 2. Since the user's location does not correspond to the second area, the second front view corresponding to the second area (indicated by a slash) is not displayed.

[0509] Figure 35The right portion of FIG shows an example of a process 3510 in which, if the user's position corresponds to the second area, the device according to an embodiment provides a second front view including the first virtual object 3501. As described above, since the priority information value of the first virtual object 3501 is 2, the relative position of the first virtual object 3501 is not maintained. Therefore, the device according to an embodiment (e.g., one or more processors 2620) can adjust the size of the first virtual object 3501 so that the first virtual object 3501 does not overlap with the second virtual object 3502. That is, as shown in the figure, the second front view includes the first virtual object 3501 positioned in the available space below the second virtual object 3502.

[0510] Figure 36 is a flowchart illustrating a content providing method according to an embodiment.

[0511] Figure 36 It is an instruction Figures 24 to 35 Flowchart 3600 of a content providing method of a device (e.g., device 2600) described in.

[0512] Reference Figure 36 , a device according to an embodiment (e.g., device 2600) displays a first front view of the 360° content based on a first user position

[3610] . Figures 24 to 35 As described, according to an embodiment, a first user position is configured based on the user's head and body position, and 360° content is content that can be displayed in a 360° space represented as one or more regions based on the user position information. According to an embodiment, the one or more regions include a first region and a second region adjacent to the first region. The first front view is content corresponding to the first region, and the size of the first region corresponds to the size of the field of view (FOV) of the user at the first user position. The size of the second region corresponds to the maximum change in head position. The maximum change is 90° (i.e., the maximum angle the user's head can rotate from the head position indicated by the first position). According to an embodiment, the size of the second front view corresponds to the size of the FOV of the user at the second user position, and depending on the changed head position, the second front view is content corresponding to a portion of the first region and a portion of the second region, or a portion of the first region and all of the second region. According to an embodiment, the 360° content includes one or more virtual objects and signaling information related to the one or more virtual objects. The one or more virtual objects may include a first virtual object corresponding to the first region and a second virtual object corresponding to the second region. The signaling information includes first flag information indicating whether the first virtual object is a fixed object that is always displayed and second flag information indicating whether the second virtual object is a fixed object that can be always displayed (for example, Figure 29 and Figure 30If the first virtual object is a fixed object, the first flag information has a default value. Figures 24 to 35 The same or similar operations and / or methods as those described in Figures 24 to 35 The details will be omitted due to redundancy of the previous description.

[0513] The device according to the embodiment (e.g., one or more sensors 2610) ensures user position information

[3620] . The device according to the embodiment (e.g., one or more sensors 2610) can detect a user input signal for configuring first priority information of the first virtual object. The first priority information is information for determining display settings of the first virtual object and at least one virtual object, and is related to Figures 32 to 35 The device according to the embodiment performs the same priority information as described in Figures 24 to 35 The same or similar operations and / or methods as those described in Figures 24 to 35 The details will be omitted due to redundancy of the previous description.

[0514] If the ensured user position information indicates a change in the head position, the device according to the embodiment (e.g., display 2632) displays a second front view of the 360° content based on a second user position determined based on the changed head position. If the ensured user position information indicates a change in the head position and body position of the user, the device displays a first front view based on a third user position determined based on the changed head position and body position

[3630] . The device according to the embodiment (e.g., display 2632) may display a second front view including at least one of the first virtual object and the second virtual object based on the first priority information and the second priority information of the second virtual object. When the first virtual object and the second virtual object completely or partially overlap each other, if the first priority information has a default value, the device according to the embodiment (e.g., display 2632) may display a second front view including the first virtual object set in front of the second virtual object. The device according to the embodiment performs the same Figures 24 to 35 The same or similar operations and / or methods as those described in Figures 24 to 35 The details will be omitted due to redundancy of the previous description.

[0515] Figures 1 to 36The elements of the apparatus according to the embodiment described in the embodiment may be implemented by hardware, software, firmware, or a combination thereof including one or more processors connected to a memory. The elements of the apparatus according to the embodiment may be implemented as a single chip, such as a single hardware circuit. In addition, the elements of the apparatus according to the embodiment may be implemented as separate chips, respectively. At least one element of the apparatus according to the embodiment may be implemented as one or more processors capable of executing one or more programs, and the one or more programs may include a processor that allows or executes a program. Figures 24 to 36 Instructions for one or more operations among the previous operations / methods of the device described in.

[0516] The executable instructions for executing the operation of the apparatus according to the embodiment may be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or a transient CRM or other computer program product configured to be executed by one or more processors. The memory according to the embodiment may be used to conceptually include non-volatile memory, flash memory, PROM, etc., as well as volatile memory (e.g., RAM, etc.).

[0517] It will be understood that although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally used only to distinguish one element from another. For example, a first image may be referred to as a second image. Similarly, a second image may be referred to as a first image. Such variations should be interpreted as not departing from the scope of the various embodiments described above. Although both the first image and the second image refer to images, they are not to be construed as the same image unless the context clearly indicates otherwise.

[0518] It will be apparent to those skilled in the art that various modifications and variations can be made.

[0519] It should be considered broadly within its scope as defined in the appended claims and all changes and modifications that come within the metes and bounds of the claims or equivalents of those metes and bounds are therefore intended to be covered by the appended claims.

Claims

1. A method for providing content, the method comprising the following steps: displaying a first front view of 360° content based on a first user position configured based on a head position of a user and a body position of the user, the 360° content being displayed in a 360° space represented as one or more regions based on the user position information; in response to determining that the user position information indicates that the head position has changed to a changed head position while the body position has remained unchanged, displaying a second front view of the 360° content based on a second user position determined according to the changed head position, the second front view being different from the first front view; as well as in response to determining that the user position information indicates that the head position is changed to the changed head position and the body position is changed to the changed body position, continuing to display the first front view based on a third user position determined based on the changed head position and the changed body position, wherein the one or more areas include a first area and a second area adjacent to the first area, wherein the 360° content includes one or more virtual objects and signaling information related to the one or more virtual objects, and wherein the one or more virtual objects include a first virtual object corresponding to the first area and a second virtual object corresponding to the second area; wherein the step of displaying the first front view of the 360° content based on the first user position comprises detecting a user input signal for configuring first priority information of the first virtual object, and The first priority information is used to determine a display setting order between the first virtual object and the second virtual object when the first virtual object completely or partially overlaps with the second virtual object.

2. The method according to claim 1, wherein The first front view includes first content corresponding to the first area, wherein the size of the first area corresponds to the size of the field of view FOV of the user located at the first user position, and The size of the second area corresponds to the maximum change value of the head position.

3. The method according to claim 2, wherein: The maximum change value includes a maximum angle of 90° that the user's head is rotated away from the head position indicated by the first user position.

4. The method according to claim 2, wherein: The size of the second front view corresponds to the size of the FOV of the user at the second user position, and According to the changed head position, the second front view includes second content corresponding to a portion of the first area and a portion of the second area or a portion of the first area and the entire second area.

5. The method according to claim 1, wherein The signaling information includes first flag information and second flag information, wherein the first flag information indicates whether the first virtual object is a fixed type object that is always displayed regardless of the changed head position or a non-fixed type object that disappears based on the user's head movement, and the second flag information indicates whether the second virtual object is the fixed type object or the non-fixed type object.

6. The method according to claim 5, wherein: When the first virtual object is the fixed-type object, the first flag information has a default value.

7. The method according to claim 1, wherein The displaying of the second front view includes displaying at least one of the first virtual object and the second virtual object based on the first priority information and second priority information of the second virtual object.

8. The method according to claim 7, further comprising the steps of: When the first virtual object and the second virtual object overlap each other in the second front view and the first priority information has a default value or a higher priority than the second priority information, the first virtual object is displayed in front of the second virtual object.

9. An apparatus for providing content, the apparatus comprising: at least one sensor configured to sense user position information configured based on a head position of a user and a body position of the user; a display configured to display at least a portion of the 360° content in a 360° space represented as one or more regions based on the user position information; as well as at least one processor configured to: displaying a first front view of the 360° content based on a first user position configured based on the head position of the user and the body position of the user, in response to determining that the user position information indicates that the head position has changed to a changed head position while the body position has remained unchanged, displaying a second front view of the 360° content based on a second user position determined according to the changed head position, the second front view being different from the first front view, and in response to determining that the user position information indicates that the head position is changed to the changed head position and the body position is changed to the changed body position, continuing to display the first front view based on a third user position determined based on the changed head position and the changed body position, wherein the one or more areas include a first area and a second area adjacent to the first area, wherein the 360° content includes one or more virtual objects and signaling information related to the one or more virtual objects, and wherein the one or more virtual objects include a first virtual object corresponding to the first area and a second virtual object corresponding to the second area; wherein the at least one sensor is further configured to detect a user input signal for configuring first priority information of the first virtual object, and The first priority information is used to determine a display setting order between the first virtual object and the second virtual object when the first virtual object completely or partially overlaps with the second virtual object.

10. The device according to claim 9, wherein The first front view includes first content corresponding to the first area, wherein the size of the first area corresponds to the size of the field of view FOV of the user located at the first user position, and The size of the second area corresponds to the maximum change value of the head position.

11. The device according to claim 10, wherein The maximum change value includes a maximum angle of 90° that the user's head is rotated away from the head position indicated by the first user position.

12. The device according to claim 10, wherein The size of the second front view corresponds to the size of the FOV of the user at the second user position, and According to the changed head position, the second front view includes second content corresponding to a portion of the first area and a portion of the second area or a portion of the first area and the entire second area.

13. The device according to claim 9, wherein The signaling information includes first flag information and second flag information, wherein the first flag information indicates whether the first virtual object is a fixed type object that is always displayed regardless of the changed head position or a non-fixed type object that disappears based on the user's head movement, and the second flag information indicates whether the second virtual object is the fixed type object or the non-fixed type object.

14. The device according to claim 13, wherein When the first virtual object is the fixed-type object, the first flag information has a default value.

15. The device according to claim 9, wherein The at least one processor is further configured to: comparing the first priority information with the second priority information to generate a comparison result, and At least one of the first virtual object or the second virtual object is displayed in the second front view based on the comparison result.

16. The device according to claim 15, wherein The at least one processor is further configured to: When the first virtual object and the second virtual object overlap each other in the second front view and the first priority information has a default value or a higher priority than the second priority information, the first virtual object is displayed in front of the second virtual object.

Images