Extended reality device and operation method thereof

Abstract

An extended reality device according to an embodiment may sequentially perform: generating a texture map including metadata of text; obtaining a first composite image by compositing the texture map with a first background layer for a first background; obtaining a second composite image by reprojecting the first composite image; restoring the text included in the second composite image on the basis of the metadata; and rendering the restored text and a second background included in the second composite image.

Description

Extended reality device and method of operation thereof

[0001] The present disclosure relates to an extended reality device that virtually displays a virtual image on a real object, and more particularly, to an extended reality device and its operating method for restoring text after a reprojection process.

[0002] Extended reality is a technology that overlays virtual images onto the physical environment or real world objects of the real world.

[0003] Augmented reality devices typically allow the user to view a scene through a see-through display positioned close to the user's eyes while worn. The scene includes one or more real-world objects within the physical environment or space directly visible to the user. Augmented reality devices transmit virtual images to the user's eyes through the see-through display, allowing the user to simultaneously view both real-world objects and virtual images.

[0004] When the movement of an AR device is recognized, a new virtual image needs to be output based on the movement of the AR device. During the output process, the background and / or text included in the new image may become distorted. In particular, if text is distorted, this can significantly impair user experience.

[0005] An extended reality device according to one embodiment may include a memory storing at least one command and at least one processor executing at least one command stored in the memory. The at least one processor may generate a texture map including metadata of text by executing the at least one command. The at least one processor may obtain a first synthesized image by synthesizing a first background layer for a first background and the texture map by executing the at least one command. The at least one processor may obtain a second synthesized image by reprojecting the first synthesized image by executing the at least one command. The at least one processor may restore the text included in the second synthesized image based on the metadata by executing the at least one command. The at least one processor may render the restored text and the second background included in the second synthesized image by executing the at least one command.

[0006] A method of operating an extended reality device according to one embodiment may include a step of generating a texture map including metadata of text. A method of operating an extended reality device according to one embodiment may include a step of obtaining a first synthesized image by synthesizing a first background layer for a first background and the texture map. A method of operating an extended reality device according to one embodiment may include a step of obtaining a second synthesized image by re-projecting the first synthesized image. A method of operating an extended reality device according to one embodiment may include a step of restoring the text included in the second synthesized image based on the metadata. A method of operating an extended reality device according to one embodiment may include a step of rendering the restored text and the second background included in the second synthesized image.

[0007] FIG. 1 is a diagram illustrating an example of text distortion when a new virtual image is generated in an extended reality device according to one embodiment.

[0008] FIG. 2 is a block diagram illustrating components of an extended reality device according to one embodiment.

[0009] FIG. 3 is a diagram illustrating a method for an extended reality device according to one embodiment to generate a texture map.

[0010] FIG. 4 is a diagram illustrating a method for an extended reality device according to one embodiment to obtain a first synthetic image.

[0011] FIG. 5A is a diagram illustrating a method for an extended reality device to restore text according to one embodiment.

[0012] FIG. 5b is a diagram illustrating a method for an extended reality device according to one embodiment to separate a second background layer and a foreground layer.

[0013] FIG. 6 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0014] FIG. 7 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0015] FIG. 8 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0016] FIG. 9 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0017] FIG. 10 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0018] FIG. 11 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0019] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0020] The terms used in this invention have been selected from widely used, current terms, taking into account the functions of the invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0021] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0022] Additionally, the description 'at least one of A, B, and C' means that it can be any one of 'A', 'B', 'C', 'A and B', 'A and C', 'B and C', and 'A, B, and C'.

[0023] It should be understood that the combinations of blocks and sequence diagrams in each flowchart can be performed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory or may be divided and stored in multiple different memories.

[0024] All functions or operations described in this document may be performed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing, and may include circuitry such as an Application Processor (AP), a Communication Processor (CP), a Graphical Processing Unit (GPU), a Neural Processing Unit (NPU), a Microprocessor Unit (MPU), a System on Chip (SoC), or an Integrated Chip (IC).

[0025] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0026] In this disclosure, 'Extended Reality' means displaying a virtual image together within the physical environment space of the real world or displaying a real object and a virtual image together.

[0027] In the present disclosure, an 'extended reality device' is a device capable of expressing extended reality, and may be, for example, a head-mounted display apparatus (HMD) worn on the head or face of a user, as well as an augmented reality glasses in the shape of glasses, an augmented reality helmet, etc.

[0028] In the embodiments of this specification, the term “user” may mean a person who controls a system, function, or operation.

[0029] FIG. 1 is a diagram illustrating an example of text distortion when a new virtual image is generated in an extended reality device according to one embodiment.

[0030] Augmented reality devices can visually present virtual images to users through waveguides or displays.

[0031] A virtual image may refer to a frame received from a user device connected to an extended reality device.

[0032] A user device may mean a device capable of operating at least one content by running at least one application.

[0033] For example, the user device may include, but is not limited to, at least one of the user's smartphone, cell phone, tablet personal computer, or PC.

[0034] The virtual image may include a background and / or text.

[0035] As the user changes their posture, if the movement of the AR device is recognized, the AR device can render a new virtual image corresponding to the user's changed posture.

[0036] A change in posture by the user may include, but is not limited to, at least one of: the user shaking his head from side to side, or the user shaking his head forward and backward.

[0037] A new virtual image may mean a virtual image that needs to be newly generated based on the movement of the augmented reality device.

[0038] The augmented reality device may reproject previously rendered virtual images when rendering new virtual images corresponding to the user's changed posture in order to reduce network traffic between the augmented reality device and the user device connected to the augmented reality device, efficiently use memory included in the augmented reality device, and reduce power consumption of the augmented reality device.

[0039] Reprojection may mean not rendering the current virtual image, but instead re-rendering the previous virtual image.

[0040] For example, if the augmented reality device recognizes the movement of the augmented reality device through at least one sensor, the augmented reality device can re-render the previous virtual image based on the direction and position information of the movement.

[0041] The augmented reality device can generate a new virtual image in an area that does not overlap with a previously rendered virtual image and render it (110) together with the previous virtual image.

[0042] When an extended reality device re-projects a previous virtual image containing text and a background, the text (120) and background (130) may be re-projected together, which may cause distortion in the text.

[0043] The distortion occurring in the text may include, but is not limited to, at least one of the following: the text appears blurry or the left and right widths of the text appear different.

[0044] Because text provides information to users, there is a problem that the user's experience with augmented reality devices is degraded when there is distortion in the background.

[0045] Hereinafter, the present disclosure will describe a method or device for providing text more clearly to a user by restoring (150) text included in a second composite image (140) including text and a background generated by reprojecting a first composite image including text and a background using an extended reality device according to one embodiment.

[0046] FIG. 2 is a block diagram illustrating components of an extended reality device according to one embodiment.

[0047] The extended reality device (200) may include a memory (220) that stores at least one command, and at least one processor (210) that executes at least one command stored in the memory (220).

[0048] The memory (220) may store various data, programs, or applications for driving and controlling the extended reality device (200) according to one embodiment. The memory (220) may include, for example, a non-volatile memory including at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a ROM (Read-Only Memory), and an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0049] The memory (220) may store instructions, data structures, and program codes that can be read by the processor (210). In the following embodiments, the processor (210) may be implemented by executing instructions or codes of a program stored in the memory (220).

[0050] The processor (210) is a configuration that controls a series of processes to allow the extended reality device (200) to operate according to the embodiments described below, and may be composed of one or more processors.

[0051] The processor (210) may be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. One or more processors included in the processor (210) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The processor (210) may be composed of at least one of, for example, a Central Processing Unit (CPU), a microprocessor, a Graphic Processing Unit (GPU), Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and Field Programmable Gate Arrays (FPGAs), but is not limited thereto.

[0052] The processor (210) can write data to the memory (220), read data stored in the memory (220), and process data according to predefined operation rules, particularly by executing a program or at least one command stored in the memory (220).

[0053] According to one embodiment, the processor (210) can visually provide a virtual image to a user through a wave guide or a display by executing at least one command stored in the memory (220).

[0054] A virtual image may refer to a frame received from a user device connected to an extended reality device.

[0055] The virtual image may include a background and / or text.

[0056] Background can mean the area in a virtual image excluding text.

[0057] A virtual image may be composed of multiple pixels.

[0058] At least one of the text or background included in the virtual image can be expressed by multiple pixels.

[0059] Each of the plurality of pixels may include at least one of a background layer or a text layer.

[0060] At least one pixel among the plurality of pixels may include at least one of a background layer or a text layer.

[0061] A background layer may refer to a layer that contains information for representing the background included in a virtual image for a single pixel.

[0062] Information for expressing the background may include color information of the background included in the virtual image and transparency information of the background.

[0063] The background layer can be configured as 32 bits.

[0064] The background layer can contain an R channel, a G channel, a B channel, and an alpha channel.

[0065] The R channel, G channel, and B channel can contain color information of the background included in the virtual image.

[0066] The R channel can contain red background information. The R channel can consist of 8 bits.

[0067] The G channel can contain green background information. The G channel can consist of 8 bits.

[0068] The B channel can contain blue background information. The B channel can consist of 8 bits.

[0069] An alpha channel can contain background transparency information. An alpha channel can consist of 8 bits.

[0070] In the following embodiments, the R channel, G channel, B channel, and alpha channel included in the background layer may be expressed as a background R channel, a background G channel, a background B channel, and a background alpha channel, respectively.

[0071] A text layer may refer to a layer that contains information for representing text contained in a virtual image in one pixel.

[0072] Information for expressing text may include color information and transparency information of the text included in the virtual image.

[0073] Text layers can be configured as 32 bits.

[0074] Text layers can contain R, G, B, and alpha channels.

[0075] The R channel, G channel, and B channel can contain color information of text included in the virtual image.

[0076] The R channel can contain red information of the text. The R channel can consist of 8 bits.

[0077] The G channel can contain green information for text. The G channel can consist of 8 bits.

[0078] The B channel can contain blue information for text. The B channel can consist of 8 bits.

[0079] An alpha channel can contain transparency information for text. An alpha channel can consist of 8 bits.

[0080] In the following embodiments, the R channel, G channel, B channel, and alpha channel included in the text layer may be expressed as a text R channel, a text G channel, a text B channel, and a text alpha channel, respectively.

[0081] A processor (210) according to one embodiment can generate a texture map including metadata of text by executing at least one command stored in a memory (220).

[0082] Metadata can contain edge information of the text.

[0083] Edge information can refer to information indicating the boundaries of text.

[0084] A specific method of generating a texture map by having a processor (210) execute at least one command stored in a memory (220) according to one embodiment will be described with reference to FIG. 3.

[0085] FIG. 3 is a diagram illustrating a method for an extended reality device according to one embodiment to generate a texture map.

[0086] At least one pixel among the plurality of pixels included in the virtual image may include a text layer (310).

[0087] A text layer (310) may mean a layer in which one pixel contains information for expressing text included in a virtual image.

[0088] Information for expressing text may include color information and transparency information of the text included in the virtual image.

[0089] The text layer (310) can be composed of 32 bits.

[0090] The text layer (310) may include an R channel, a G channel, a B channel (312), and an alpha channel (314).

[0091] A processor according to one embodiment can generate metadata (324) by executing at least one instruction stored in memory (220).

[0092] Metadata (324) can be composed of 8 bits.

[0093] Metadata (324) may include at least one of edge information of text, color information of text, or font information of text.

[0094] The color information of the text can include information about the R channel, G channel, and B channel of the text layer.

[0095] According to one embodiment, the processor (210) can generate metadata (324) of a total of 8 bits by executing at least one command stored in the memory (220), thereby allocating 4 bits for edge information of the text, 3 bits for color information of the text, and 1 bit for font information of the text. However, the present invention is not limited thereto.

[0096] According to one embodiment, the processor (210) can generate metadata (324) of a total of 8 bits by executing at least one command stored in the memory (220), thereby allocating 3 bits for edge information of the text, 3 bits for color information of the text, and 2 bits for font information of the text. However, the present invention is not limited thereto.

[0097] Metadata (324) may further include separation information for separating the background layer and the foreground layer.

[0098] A foreground layer may mean a layer excluding a background layer among at least one layer contained in one pixel among multiple pixels contained in a virtual image.

[0099] A foreground layer can contain a text layer.

[0100] According to one embodiment, the processor (210) may generate metadata (324) of a total of 8 bits by executing at least one command stored in the memory (220), allocating 3 bits for edge information of the text, 3 bits for color information of the text, 1 bit for font information of the text, and 1 bit for separation information. However, the present invention is not limited thereto.

[0101] According to one embodiment, the processor (210) can generate metadata (324) of a total of 8 bits by executing at least one command stored in the memory (220), allocating 3 bits for edge information of the text, 3 bits for color information of the text, and 2 bits for separation information. However, the present invention is not limited thereto.

[0102] According to one embodiment, the processor (210) can generate metadata (324) of a total of 8 bits by executing at least one command stored in the memory (220) and allocating 4 bits to edge information of the text, 3 bits to color information of the text, and 1 bit to separation information. However, the present invention is not limited thereto.

[0103] According to one embodiment, the processor (210) can replace an alpha channel (314) included in a text layer (310) with metadata (324) by executing at least one command stored in a memory (220).

[0104] According to one embodiment, a processor (210) can generate a texture map (320) in which an alpha channel (314) included in a text layer (310) is replaced with metadata (324) by executing at least one command stored in a memory (220).

[0105] According to one embodiment, a processor (210) can generate a texture map (320) including an R channel, a G channel, a B channel (312), and metadata (324) by executing at least one command stored in a memory (220).

[0106] The texture map (320) may be composed of 32 bits. The R channel, G channel, B channel (312) and metadata (324) included in the texture map (320) may each be composed of 8 bits.

[0107] Referring again to FIG. 2, a processor (210) according to one embodiment can obtain a first synthesized image by synthesizing a first background layer and a texture map for a first background by executing at least one command stored in a memory (220).

[0108] The first background may refer to the background of the virtual image before reprojecting the first composite image.

[0109] The first background may mean an area of ​​a virtual image prior to reprojecting the first composite image, excluding text included in the virtual image.

[0110] According to one embodiment, a processor (210) can synthesize a first background layer and a texture map for a first background by executing at least one command stored in a memory (220).

[0111] A specific method of obtaining a first synthesized image by synthesizing a first background layer and a texture map for a first background by executing at least one command stored in a memory (220) according to an embodiment of the present invention will be described with reference to FIG. 4.

[0112] FIG. 4 is a diagram illustrating a method for an extended reality device according to one embodiment to obtain a first synthetic image.

[0113] A virtual image may be composed of multiple pixels.

[0114] At least one pixel forming an area containing text in a virtual image may include a first background layer (410) and a texture map (320).

[0115] The first background layer (410) may mean a layer that includes information for expressing a background included in a virtual image in which one pixel is included.

[0116] Information for expressing the background may include color information of the background included in the virtual image and transparency information of the background.

[0117] The first background layer (410) can be composed of 32 bits.

[0118] The first background layer (410) may include a background R channel, a background G channel, a background B channel (412), and a background alpha channel (414).

[0119] The background R channel, background G channel, and background B channel (412) may include color information of the first background included in the virtual image. The background R channel, background G channel, and background B channel (412) may each be composed of 8 bits.

[0120] The background alpha channel (414) may include transparency information of the first background. The first background alpha channel may consist of 8 bits.

[0121] A processor (210) according to one embodiment can generate a texture map (320) including a text R channel, a text G channel, a text B channel (312), and metadata (324) by executing at least one instruction stored in a memory (220).

[0122] The text R channel, text G channel, and text B channel (412) may include color information of text included in a virtual image. The text R channel, text G channel, and text B channel (412) may each be composed of 8 bits.

[0123] According to one embodiment, the processor (210) can replace an alpha channel included in a first background layer (410) with metadata (324) included in a texture map (320) by executing at least one command stored in a memory (220).

[0124] According to one embodiment, the processor (210) can generate a first composite layer (430) by executing at least one command stored in the memory (220) to replace an alpha channel (414) included in a first background layer (410) with metadata (324) included in a texture map (320).

[0125] The first composite layer (430) may include a background R channel, a background G channel, a background B channel (412) included in the first background layer, and metadata (324) included in the texture map.

[0126] The first synthesis layer can consist of 32 bits.

[0127] The first synthetic image may mean a synthetic image in which at least one pixel among a plurality of pixels included in a virtual image is expressed by the first synthetic layer.

[0128] The first synthetic image may include an image synthesized from at least one synthetic image stored in the memory (220).

[0129] According to one embodiment, the processor (210) can generate a composite image by synthesizing each of the upper region, the middle region, and the lower region of the virtual image into one image when the virtual image is composed of an upper region, a middle region, and a lower region by executing at least one command stored in the memory (220).

[0130] If a virtual image consists of a top area, a middle area, and a bottom area, this may mean, for example, an address bar at the top of the virtual image, content in the middle of the virtual image, and content control tabs at the bottom of the virtual image. However, this is not limited thereto.

[0131] According to one embodiment, the processor (210) can generate a composite image by synthesizing an address window located at the top of a virtual image, content located at the middle of a virtual image, and a content control tab located at the bottom of a virtual image into one image by executing at least one command stored in a memory (220).

[0132] According to one embodiment, the processor (210) can store a synthetic image generated for each of the upper region, middle region, and lower region of the virtual image in the memory (220) by executing at least one command stored in the memory (220).

[0133] The first synthetic image may mean an image synthesized from a synthetic image for the upper area of ​​the virtual image, a synthetic image for the middle area of ​​the virtual image, and a synthetic image for the lower area of ​​the virtual image.

[0134] Referring again to FIG. 2, a processor (210) according to one embodiment can re-project a first composite image to obtain a second composite image by executing at least one command stored in a memory (220).

[0135] Reprojection may mean not rendering the current virtual image, but instead re-rendering the previous virtual image.

[0136] The previous virtual image may include the first background and text described above.

[0137] A virtual image may refer to a frame received from a user device connected to an extended reality device via wired or wireless means.

[0138] A user device may mean a device capable of operating at least one content by running at least one application.

[0139] For example, the user device may include, but is not limited to, at least one of the user's smartphone, cell phone, tablet personal computer, or PC.

[0140] An extended reality device (200) according to one embodiment may further include at least one sensor (not shown).

[0141] The sensor may include, but is not limited to, at least one of a depth sensor capable of obtaining depth information about an actual space included in the FOV (field of view) of the camera, and an IMU (Inertial Measurement Unit) sensor for tracking the movement of the extended reality device according to one embodiment.

[0142] According to one embodiment, a processor (210) can recognize movement of an extended reality device (200) through at least one sensor by executing at least one command stored in a memory (220) to obtain a sensing value for at least one of a position or direction of the extended reality device (200).

[0143] The sensing value may mean a measurement of at least one of the position or orientation of the extended reality device that changes according to the movement of the extended reality device.

[0144] According to one embodiment, a processor (210) can recognize movement of an extended reality device (200) through at least one sensor by executing at least one command stored in a memory (220).

[0145] According to one embodiment, the processor (210) can obtain information on the direction in which the extended reality device (200) moves through at least one sensor when the movement of the extended reality device (200) is recognized by executing at least one command stored in the memory (220).

[0146] According to one embodiment, the processor (210) can obtain location information of the extended reality device (200) through at least one sensor when movement of the extended reality device (200) is recognized by executing at least one command stored in the memory (220).

[0147] According to one embodiment, the processor (210) can obtain a second synthetic image by reprojecting the first synthetic image based on a sensing value obtained through at least one sensor by executing at least one command stored in the memory (220).

[0148] According to one embodiment, the processor (210) can obtain a second synthetic image by reprojecting the first synthetic image as the movement of the extended reality device (200) is recognized through at least one sensor by executing at least one command stored in the memory (220).

[0149] According to one embodiment, the processor (210) can obtain a second synthetic image by reprojecting the first synthetic image based on information about the direction in which the extended reality device (200) moves or information about the location of the extended reality device (200) by executing at least one command stored in the memory (220).

[0150] The second composite layer may include metadata included in the background R channel, background G channel, background B channel, and texture map included in the second composite image.

[0151] The second synthesis layer can consist of 32 bits.

[0152] According to one embodiment, the processor (210) may re-render a previous virtual image based on information about the direction in which the extended reality device moves when movement of the extended reality device is recognized through at least one sensor by executing at least one command stored in the memory (220).

[0153] According to one embodiment, the processor (210) can generate a new virtual image in an area that does not overlap with a previously rendered virtual image by executing at least one command stored in the memory (220), and render the new virtual image together with the previous virtual image.

[0154] A new virtual image may mean a virtual image that needs to be newly created according to the movement of the extended reality device (200).

[0155] Here, the movement of the extended reality device (200) may mean the movement of the extended reality device (200) recognized based on a sensing value acquired through at least one sensor.

[0156] A second composite image may mean an image that is a composite of a re-rendered previous virtual image and a new virtual image.

[0157] For example, the previous virtual image may mean an image expressed at 90° to the left and 90° to the right, assuming that the front of the extended reality device (200) is 0° before the movement of the extended reality device (200) is recognized.

[0158] According to one embodiment, the processor (210) may, by executing at least one command stored in the memory (220), recognizing that the direction of the extended reality device (200) has changed 1° to the right through at least one sensor, re-render an image expressed at 89° to the left and 90° to the right in the previous virtual image before the movement of the extended reality device (200) is recognized.

[0159] According to one embodiment, the processor (210) can generate a new virtual image in an area expressed from 90° to 91° to the right when the front of the extended reality device (200) is assumed to be 0° before the movement of the extended reality device (200) is recognized by executing at least one command stored in the memory (220).

[0160] Here, the area located 90˚ to 91˚ to the right of the previous virtual image may refer to an area that does not overlap with the previously rendered virtual image.

[0161] According to one embodiment, the processor (210) can generate a new virtual image in an area that does not overlap with a previously rendered virtual image by executing at least one command stored in the memory (220), and render the new virtual image together with the previous virtual image.

[0162] For example, the previous virtual image may mean an image expressed at 90° upward and 90° downward when the front of the extended reality device (200) is assumed to be 0° before the movement of the extended reality device (200) is recognized.

[0163] According to one embodiment, the processor (210) executes at least one command stored in the memory (220), and when it recognizes that the direction of the extended reality device (200) has changed 1° downward through at least one sensor, it can re-render the image expressed at 89° above and 90° below in the previous virtual image, assuming that the front of the extended reality device (200) is 0° before the movement of the extended reality device (200) is recognized.

[0164] According to one embodiment, the processor (210) can generate a new virtual image in an area expressed from 90˚ to 91˚ downward when the front of the extended reality device (200) is assumed to be 0˚ before the movement of the extended reality device (200) is recognized by executing at least one command stored in the memory (220).

[0165] Here, the area located 90˚ to 91˚ from the bottom of the previous virtual image may refer to an area that does not overlap with the previously rendered virtual image.

[0166] According to one embodiment, the processor (210) can generate a new virtual image in an area that does not overlap with a previously rendered virtual image by executing at least one command stored in the memory (220), and render the new virtual image together with the previous virtual image.

[0167] An extended reality device (200) according to one embodiment may further include a communication interface (not shown).

[0168] The communication interface may perform communication with an external device or server via at least one wired or wireless communication network by the processor. According to one embodiment, the communication interface may include at least one short-range communication module that performs communication according to a communication standard such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC (Near Field Communication) / RFID (Radio-Frequency Identification), Wi-Fi Direct, UWB (Ultra-Wide Band), or ZIGBEE, and a long-range communication module that performs communication with a server for supporting long-range communication according to a long-range communication standard. The long-range communication module may perform communication via a communication network according to a 3G, 4G, and / or 5G communication standard, or a network for Internet communication.

[0169] A virtual image may refer to a frame received from a user device connected to an extended reality device via wired or wireless means.

[0170] According to one embodiment, a user device connected to an extended reality device via wired or wireless communication can transmit and receive data with a server. The data may include, but is not limited to, virtual images.

[0171] The server may include a server, server system, server-based device, etc. that transmits and receives data to and processes data with the extended reality device and / or user device through a communication interface.

[0172] A virtual image may refer to a frame transmitted to a server by a user device connected to an augmented reality device, either wired or wirelessly.

[0173] An extended reality device according to one embodiment may be wirelessly connected to a server.

[0174] According to one embodiment, a processor (210) can receive a virtual image transmitted from a user device to a server by executing at least one command stored in a memory (220).

[0175] According to one embodiment, the processor (210) may receive a frame transmitted from a user device to the server by executing at least one command stored in the memory (220). The virtual image may include at least one content received by the extended reality device according to one embodiment from the user device via a communication interface.

[0176] The virtual image may include at least one piece of content transmitted to a server by a user device connected to the augmented reality device, either wired or wirelessly.

[0177] The virtual image may include at least one content received by the extended reality device from a server and / or a user device via a communication interface according to one embodiment.

[0178] According to one embodiment, a processor (210) can receive change information of at least one content output from a user device connected to an extended reality device wired or wirelessly through a communication interface by executing at least one command stored in a memory (220).

[0179] The user device can transmit change information of at least one content output from the user device to the server.

[0180] According to one embodiment, the processor (210) can receive change information of at least one content transmitted from a user device to the server by executing at least one command stored in the memory (220).

[0181] The change information of at least one content may include, but is not limited to, at least one of information indicating that the location of at least one content has changed or information indicating that the content of at least one content has changed.

[0182] Information indicating that the location of at least one content has changed may include, but is not limited to, information on the changed location of at least one content output from the user device by the user performing at least one of drag and drop or swap through a user interface (UI) included in the user device.

[0183] The change information indicating that the content of at least one content has changed may include, but is not limited to, change information of at least one content output from a user device when the user scrolls through the user interface.

[0184] According to one embodiment, the processor (210) can obtain a second composite image by reprojecting the first composite image based on change information of at least one content by executing at least one command stored in the memory (220).

[0185] FIG. 5A is a diagram illustrating a method for an extended reality device to restore text according to one embodiment.

[0186] According to one embodiment, the processor (210) can restore text (550) included in the second synthetic image (510) based on metadata by executing at least one command stored in the memory (220).

[0187] The second composite layer may include metadata included in the background R channel, background G channel, background B channel, and texture map of the second background included in the second composite image (510).

[0188] The second synthesis layer can consist of 32 bits.

[0189] The metadata may further include separation information for separating the second background layer and the foreground layer for the second background (520) included in the second composite layer.

[0190] Metadata can consist of 8 bits.

[0191] The second background (520) may mean the background of the second composite image (510) obtained by reprojecting the first composite image.

[0192] The second background (520) may mean an area excluding text in the second composite image (510) obtained by reprojecting the first composite image.

[0193] The second background layer may mean a layer in which one pixel contains information for expressing the second background (520).

[0194] Information for expressing the second background (520) may include color information of the second background.

[0195] The second background layer may include a background R channel, a background G channel, a background B channel, and an alpha channel of the second background.

[0196] An alpha channel can contain background transparency information. An alpha channel can consist of 8 bits.

[0197] According to one embodiment, the processor (210) can separate the second background layer and the foreground layer based on the separation information by executing at least one command stored in the memory (220).

[0198] A specific method for separating the second background layer and the foreground layer based on separation information will be described with reference to Fig. 5b.

[0199] FIG. 5b is a diagram illustrating a method for an extended reality device according to one embodiment to separate a second background layer and a foreground layer.

[0200] The second composite layer (440) may include a background R channel, a background G channel, a background B channel (442) of the second background included in the second composite image, and metadata (324) included in the texture map.

[0201] The second synthesis layer can consist of 32 bits.

[0202] The background R channel, background G channel, background B channel (442) of the second background included in the second composite layer (440) and the metadata (324) included in the texture map may each be composed of 8 bits.

[0203] The metadata (324) may further include separation information for separating the second background layer (450) and the foreground layer (560) for the second background included in the second composite layer (440).

[0204] Metadata (324) according to one embodiment may be composed of 3 bits for edge information of text, 3 bits for color information of text, 1 bit for font information of text, and 1 bit for separation information.

[0205] According to one embodiment, a processor can separate a second background layer (450) and a foreground layer (560) included in a second composite layer (440) based on separation information by executing at least one instruction stored in a memory.

[0206] Separation information can be expressed as binary numbers of 0 or 1, but is not limited thereto.

[0207] For example, the processor can separate any layer included in the second composite layer (440) into a foreground layer (560) by executing at least one instruction stored in the memory, if the separation information for that layer indicates 0.

[0208] For example, the processor can separate any layer included in the second composite layer (440) into a second background layer (450) by executing at least one instruction stored in the memory, if the separation information for that layer indicates 1.

[0209] The second background layer (450) may mean a layer in which one pixel includes information for expressing the second background.

[0210] The second background layer (450) may include information for one pixel to express the second background.

[0211] Information for expressing the second background may mean the background R channel, background G channel, background B channel (442), and alpha channel (454) of the second background.

[0212] An alpha channel can contain transparency information for a secondary background. The alpha channel can consist of 8 bits.

[0213] The foreground layer (560) may contain information for one pixel to represent the foreground.

[0214] The foreground layer (560) may mean a layer excluding the second background layer among at least one layer included in one pixel among a plurality of pixels included in the second composite image.

[0215] The foreground layer (560) may include a text layer.

[0216] The foreground layer (560) may include a foreground R channel, a foreground G channel, a foreground B channel (562), and an alpha channel (314).

[0217] The alpha channel (314) may contain transparency information of the text. The alpha channel may consist of 8 bits.

[0218] Referring again to FIG. 5A, the processor (210) according to one embodiment can separate the second background layer and the foreground layer based on the separation information by executing at least one command stored in the memory (220).

[0219] A foreground layer can contain information for a single pixel to represent the foreground.

[0220] The foreground layer may mean a layer excluding the second background layer among at least one layer included in one pixel among a plurality of pixels included in the second composite image (510).

[0221] A foreground layer can contain a text layer.

[0222] A synthetic image corresponding to the foreground layer may mean the foreground (530).

[0223] The foreground (530) may mean a virtual image expressed by the foreground R channel, foreground G channel, foreground B channel, and alpha channel included in the foreground layer.

[0224] The foreground (530) may contain text.

[0225] A processor according to one embodiment can restore text included in a synthetic image corresponding to a foreground layer based on edge information by executing at least one command stored in a memory.

[0226] Edge information can refer to information indicating the boundaries of text.

[0227] Edge information may include distance information from the center of all pixels included in the foreground (530) to at least one pixel located at the center of at least one text.

[0228] Text can consist of at least one pixel.

[0229] At least one pixel located at the center of the text may mean at least one pixel located inside the text and at the center of the text based on the border of the text among at least one pixel composing the text.

[0230] A processor according to one embodiment can obtain boundary information of text based on edge information by executing at least one instruction stored in a memory.

[0231] A processor according to one embodiment can obtain boundary information of a text based on edge information and a threshold by executing at least one instruction stored in a memory.

[0232] According to one embodiment, a processor may, by executing at least one instruction stored in a memory, divide a value between 0 and 1 so as to correspond to a distance from the center of all pixels included in the foreground to at least one pixel located at the center of the text.

[0233] According to one embodiment, a processor may map distances from the centers of all pixels included in the foreground to at least one pixel located at the center of the text to values ​​between 0 and 1, respectively, by executing at least one instruction stored in memory.

[0234] For example, the processor may map a distance from the center of the first pixel to at least one pixel located at the center of at least one of the first text or the second text to a value between 0 and 1 by executing at least one instruction stored in the memory.

[0235] For example, the processor may map a distance from the center of the second pixel to at least one pixel located at the center of at least one of the first text and the second text to a value between 0 and 1 by executing at least one instruction stored in the memory.

[0236] The first pixel and the second pixel may refer to at least one pixel included in the foreground.

[0237] The first text and the second text may refer to at least one of the texts included in the foreground.

[0238] According to one embodiment, the processor may execute at least one instruction stored in a memory to map the distance from the center of the first pixel to at least one pixel located at the center of the first text to 0.6 and to map the distance from the center of the second pixel to at least one pixel located at the center of the first text to 0.4, if the distance from the center of the first pixel to at least one pixel located at the center of the first text is shorter than the distance from the center of the second pixel to at least one pixel located at the center of the first text. However, the present invention is not limited thereto.

[0239] In one embodiment, the processor may execute at least one instruction stored in a memory to map the distance from the center of the first pixel to at least one pixel located at the center of the second text to 0.3, and to map the distance from the center of the second pixel to at least one pixel located at the center of the second text to 0.8, if the distance from the center of the first pixel to at least one pixel located at the center of the second text is longer than the distance from the center of the second pixel to at least one pixel located at the center of the second text. However, the present invention is not limited thereto.

[0240] A processor according to one embodiment can set a threshold for a value mapped between 0 and 1 by executing at least one instruction stored in memory.

[0241] A threshold may refer to a setting value for identifying pixels located inside or outside the boundary of the text.

[0242] The threshold can be changed depending on the user's settings.

[0243] In one embodiment, the processor can identify at least one pixel mapped between 0 and 1 with a value less than a threshold value as a pixel located outside the border of the text (540) by executing at least one instruction stored in the memory.

[0244] For example, the processor may map a distance from the center of the first pixel to at least one pixel located at the center of the first text to a value between 0 and 1 by executing at least one instruction stored in memory.

[0245] The first pixel may mean one of at least one pixel included in the foreground.

[0246] The first text may refer to at least one of the texts included in the foreground.

[0247] A processor according to one embodiment may set a threshold value of 0.5 as a setting value for identifying pixels located inside or outside a boundary of text by executing at least one instruction stored in a memory.

[0248] According to one embodiment, a processor may identify a first pixel as a pixel located outside the boundary (540) of the first text if a distance from the center of the first pixel to at least one pixel located at the center of the first text is mapped to 0.3 by executing at least one instruction stored in a memory.

[0249] Accordingly, a processor according to one embodiment can identify a pixel located outside the border (540) of text by executing at least one instruction stored in memory.

[0250] A processor according to one embodiment can identify pixels located outside the border of text as a second background (540) by executing at least one instruction stored in memory.

[0251] According to one embodiment, a processor can identify pixels located outside the border of text as an area (540) excluding text in a second synthetic image by executing at least one instruction stored in a memory.

[0252] In one embodiment, a processor can identify at least one pixel mapped between 0 and 1 with a value greater than a threshold value as a pixel located within a border of text by executing at least one instruction stored in a memory.

[0253] For example, the processor may map a distance from the center of the second pixel to at least one pixel located at the center of the second text to a value between 0 and 1 by executing at least one instruction stored in memory.

[0254] The second pixel may mean at least one pixel included in the foreground.

[0255] The second text may refer to at least one of the texts included in the foreground.

[0256] A processor according to one embodiment may set a threshold value of 0.5 as a setting value for identifying pixels located inside or outside a boundary of text by executing at least one instruction stored in a memory.

[0257] In one embodiment, the processor can identify the second pixel as a pixel located within the boundary of the second text if a distance from the center of the second pixel to at least one pixel located at the center of the second text is mapped to 0.7 by executing at least one instruction stored in the memory.

[0258] Accordingly, a processor according to one embodiment can identify a pixel located within a border of text by executing at least one instruction stored in a memory.

[0259] According to one embodiment, a processor can obtain border information of text by identifying pixels located inside or outside a border of text by executing at least one instruction stored in a memory.

[0260] According to one embodiment, a processor can restore text (550) included in a synthetic image corresponding to a foreground layer based on boundary information by executing at least one command stored in a memory.

[0261] A synthetic image corresponding to the foreground layer may mean the foreground (530).

[0262] According to one embodiment, a processor can restore text (550) by executing at least one command stored in a memory, thereby identifying pixels located within the boundary of the text in a synthetic image corresponding to a foreground layer based on boundary information as pixels constituting the text.

[0263] According to one embodiment, a processor can restore text (550) by executing at least one instruction stored in a memory, thereby identifying pixels located outside the boundary (540) of the text in a synthetic image corresponding to a foreground layer based on boundary information as pixels that do not constitute text.

[0264] According to one embodiment, a processor can restore text (550) by executing at least one instruction stored in a memory, thereby identifying pixels located outside the boundary (540) of the text in a synthetic image corresponding to a foreground layer based on boundary information as pixels that do not constitute text.

[0265] According to one embodiment, a processor can restore at least one text among text having a color included in a first synthetic image or text having a font included in a first synthetic image, based on metadata, by executing at least one command stored in a memory.

[0266] Metadata may include color information including information about the text R channel, text G channel, and text B channel contained in the text layer.

[0267] According to one embodiment, a processor can generate metadata of a total of 8 bits by allocating 4 bits to edge information of text, 3 bits to color information of text, and 1 bit to separation information by executing at least one instruction stored in memory. However, the present invention is not limited thereto.

[0268] A processor according to one embodiment can restore text including a color of text included in a first synthetic image based on color information of the text included in metadata by executing at least one command stored in a memory.

[0269] Metadata may include font information of text included in the first synthetic image.

[0270] According to one embodiment, a processor can generate metadata of a total of 8 bits by allocating 4 bits to edge information of text, 3 bits to font information of text, and 1 bit to separation information by executing at least one instruction stored in memory. However, the present invention is not limited thereto.

[0271] According to one embodiment, a processor can restore text with a font applied to text included in a first synthetic image based on font information of text included in metadata by executing at least one command stored in a memory.

[0272] The metadata may include color information including information about the text R channel, text G channel, and text B channel included in the text layer, and font information of the text included in the first composite image.

[0273] According to one embodiment, a processor may generate metadata of a total of 8 bits by executing at least one instruction stored in a memory, by allocating 3 bits for edge information of text, 3 bits for color information of text, 1 bit for font information of text, and 1 bit for separation information. However, the present invention is not limited thereto.

[0274] According to one embodiment, a processor can restore text having a color included in a first synthetic image and a font applied to the text included in the first synthetic image based on color information and font information included in metadata by executing at least one command stored in a memory.

[0275] A processor according to one embodiment can render the restored text and the second background included in the second synthetic image by executing at least one instruction stored in the memory.

[0276] An extended reality device according to one embodiment may further include an image output unit including a wave guide (not shown) or a display (not shown).

[0277] A processor according to one embodiment can output rendered text and a second background to a wave guide or a display by executing at least one instruction stored in memory.

[0278] FIG. 6 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0279] Augmented reality devices can provide users with visual representations of virtual images through waveguides or displays.

[0280] A virtual image may refer to a frame received from a user device connected to an extended reality device.

[0281] The virtual image may include a background and / or text.

[0282] Background can mean the area in a virtual image excluding text.

[0283] A virtual image may be composed of multiple pixels.

[0284] At least one of the text or background included in the virtual image can be expressed by multiple pixels.

[0285] Each of the plurality of pixels may include at least one of a background layer or a text layer.

[0286] At least one pixel among the plurality of pixels may include at least one of a background layer or a text layer.

[0287] A background layer may refer to a layer that contains information for representing the background included in a virtual image for a single pixel.

[0288] Information for expressing the background may include color information of the background included in the virtual image and transparency information of the background.

[0289] The background layer can be configured as 32 bits.

[0290] The background layer can contain an R channel, a G channel, a B channel, and an alpha channel.

[0291] The R channel, G channel, and B channel can contain color information of the background included in the virtual image.

[0292] The R channel can contain red background information. The R channel can consist of 8 bits.

[0293] The G channel can contain green background information. The G channel can consist of 8 bits.

[0294] The B channel can contain blue background information. The B channel can consist of 8 bits.

[0295] An alpha channel can contain background transparency information. An alpha channel can consist of 8 bits.

[0296] In the following embodiments, the R channel, G channel, B channel, and alpha channel included in the background layer may be expressed as a background R channel, a background G channel, a background B channel, and a background alpha channel, respectively.

[0297] A text layer may refer to a layer that contains information for representing text contained in a virtual image in one pixel.

[0298] Information for expressing text may include color information and transparency information of the text included in the virtual image.

[0299] Text layers can be configured as 32 bits.

[0300] Text layers can contain R, G, B, and alpha channels.

[0301] The R channel, G channel, and B channel can contain color information of text included in the virtual image.

[0302] The R channel can contain red information of the text. The R channel can consist of 8 bits.

[0303] The G channel can contain green information for text. The G channel can consist of 8 bits.

[0304] The B channel can contain blue information for text. The B channel can consist of 8 bits.

[0305] An alpha channel can contain transparency information for text. An alpha channel can consist of 8 bits.

[0306] In the following embodiments, the R channel, G channel, B channel, and alpha channel included in the text layer may be expressed as a text R channel, a text G channel, a text B channel, and a text alpha channel, respectively.

[0307] At step 610, the augmented reality device may generate a texture map containing metadata of the text.

[0308] Metadata can contain edge information of the text.

[0309] Edge information can refer to information indicating the boundaries of text.

[0310] At least one pixel among the plurality of pixels included in the virtual image may include a text layer.

[0311] A text layer may refer to a layer that contains information for representing text contained in a virtual image in one pixel.

[0312] Information for expressing text may include color information and transparency information of the text included in the virtual image.

[0313] Text layers can be configured as 32 bits.

[0314] A text layer can contain an R channel, a G channel, a B channel, and an alpha channel.

[0315] An extended reality device according to one embodiment can generate metadata.

[0316] Metadata can consist of 8 bits.

[0317] Metadata may include at least one of edge information of text, color information of text, or font information of text.

[0318] The color information of the text can include information about the R channel, G channel, and B channel of the text layer.

[0319] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 4 bits for edge information of text, 3 bits for color information of text, and 1 bit for font information of text. However, the present invention is not limited thereto.

[0320] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 3 bits for edge information of text, 3 bits for color information of text, and 2 bits for font information of text. However, the present invention is not limited thereto.

[0321] The metadata may further include separation information to separate the background layer and the foreground layer.

[0322] A foreground layer may mean a layer excluding a background layer among at least one layer contained in one pixel among multiple pixels contained in a virtual image.

[0323] A foreground layer can contain a text layer.

[0324] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 3 bits to edge information of text, 3 bits to color information of text, 1 bit to font information of text, and 1 bit to separation information. However, the present invention is not limited thereto.

[0325] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 3 bits to edge information of text, 3 bits to color information of text, and 2 bits to separation information. However, the present invention is not limited thereto.

[0326] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 4 bits to edge information of text, 3 bits to color information of text, and 1 bit to separation information. However, the present invention is not limited thereto.

[0327] An extended reality device according to one embodiment may replace an alpha channel included in a text layer with metadata.

[0328] An extended reality device according to one embodiment can generate a texture map in which an alpha channel included in a text layer is replaced with metadata.

[0329] An extended reality device according to one embodiment can generate a texture map including an R channel, a G channel, a B channel, and metadata.

[0330] A texture map can consist of 32 bits. The R channel, G channel, B channel, and metadata contained in the texture map can each consist of 8 bits.

[0331] In step 620, the extended reality device can obtain a first synthesized image by synthesizing a first background layer and a texture map for the first background.

[0332] The first background may refer to the background of the virtual image before reprojecting the first composite image.

[0333] The first background may mean an area of ​​a virtual image prior to reprojecting the first composite image, excluding text included in the virtual image.

[0334] An extended reality device according to one embodiment can synthesize a first background layer and a texture map for a first background.

[0335] A virtual image may be composed of multiple pixels.

[0336] At least one pixel forming an area containing text in a virtual image may include a first background layer and a texture map.

[0337] The first background layer may refer to a layer containing information for representing the background included in a virtual image for one pixel.

[0338] Information for expressing the background may include color information of the background included in the virtual image and transparency information of the background.

[0339] The first background layer can consist of 32 bits.

[0340] The first background layer may include a background R channel, a background G channel, a background B channel, and a background alpha channel.

[0341] The background R channel, background G channel, and background B channel may include color information of the first background included in the virtual image. The background R channel, background G channel, and background B channel may each be composed of 8 bits.

[0342] The background alpha channel may contain transparency information of the first background. The first background alpha channel may consist of 8 bits.

[0343] An extended reality device according to one embodiment can generate a texture map including a text R channel, a text G channel, a text B channel, and metadata.

[0344] The text R channel, text G channel, and text B channel may contain color information of text included in a virtual image. The text R channel, text G channel, and text B channel may each consist of 8 bits.

[0345] An extended reality device according to one embodiment may replace an alpha channel included in a first background layer with metadata included in a texture map.

[0346] An extended reality device according to one embodiment can generate a first composite layer by replacing an alpha channel included in a first background layer with a texture map.

[0347] The first composite layer may include metadata included in the background R channel, background G channel, background B channel, and texture map included in the first background layer.

[0348] The first synthesis layer can consist of 32 bits.

[0349] The first synthetic image may mean a synthetic image in which at least one pixel among a plurality of pixels included in a virtual image is expressed by the first synthetic layer.

[0350] The first composite image may include an image synthesized from at least one composite image stored in memory.

[0351] An extended reality device according to one embodiment can generate a composite image by synthesizing each of the upper region, the middle region, and the lower region of the virtual image into a single image when the virtual image is composed of an upper region, a middle region, and a lower region.

[0352] If a virtual image consists of a top area, a middle area, and a bottom area, this may mean, for example, an address bar at the top of the virtual image, content in the middle of the virtual image, and content control tabs at the bottom of the virtual image. However, this is not limited thereto.

[0353] An extended reality device according to one embodiment can generate a composite image by synthesizing an address window located at the top of a virtual image, content located at the middle of the virtual image, and content control tabs located at the bottom of the virtual image into one image.

[0354] An extended reality device according to one embodiment can store in memory a synthetic image generated for each of an upper region, a middle region, and a lower region of a virtual image.

[0355] The first synthetic image may mean an image synthesized from a synthetic image for the upper area of ​​the virtual image, a synthetic image for the middle area of ​​the virtual image, and a synthetic image for the lower area of ​​the virtual image.

[0356] At step 630, the extended reality device can re-project the first synthetic image to obtain a second synthetic image.

[0357] Reprojection may mean not rendering the current virtual image, but instead re-rendering the previous virtual image.

[0358] The previous virtual image may include the first background and text described above.

[0359] A virtual image may refer to a frame received from a user device connected to an extended reality device via wired or wireless means.

[0360] A user device may mean a device capable of operating at least one content by running at least one application.

[0361] For example, the user device may include, but is not limited to, at least one of the user's smartphone, cell phone, tablet personal computer, or PC.

[0362] A specific method of obtaining a second synthetic image by reprojecting a first synthetic image by an extended reality device according to one embodiment will be described with reference to FIGS. 7 and 8.

[0363] FIG. 7 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0364] In step 632, the extended reality device can receive change information of at least one content output from a user device connected to the extended reality device.

[0365] An extended reality device according to one embodiment can receive change information of at least one content output from a user device connected to the extended reality device through a communication interface.

[0366] A communication interface may perform communication with an external device or server through at least one wired or wireless communication network by a processor included in an extended reality device according to one embodiment. The communication interface according to one embodiment may include at least one short-range communication module that performs communication according to a communication standard such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC (Near Field Communication) / RFID (Radio-Frequency Identification), Wi-Fi Direct, UWB (Ultra-Wide Band), or ZIGBEE, and a long-range communication module that performs communication with a server for supporting long-range communication according to a long-range communication standard. The long-range communication module may perform communication through a communication network according to a 3G, 4G, and / or 5G communication standard, or a network for Internet communication.

[0367] A virtual image may refer to a frame received from a user device connected wired or wirelessly to an extended reality device according to one embodiment.

[0368] The virtual image may include at least one piece of content received by the extended reality device from a user device via a communication interface according to one embodiment.

[0369] An extended reality device according to one embodiment can receive change information of at least one content output from a user device connected to the extended reality device by wire or wirelessly through a communication interface.

[0370] The change information of at least one content may include, but is not limited to, at least one of information indicating that the location of at least one content has changed or information indicating that the content of at least one content has changed.

[0371] Information indicating that the position of at least one content has changed may include, but is not limited to, information on the changed position of at least one content output from a user device by the user performing at least one of drag and drop or swap through a user interface (UI).

[0372] The change information indicating that the content of at least one content has changed may include, but is not limited to, change information of at least one content output from a user device when the user scrolls through the user interface.

[0373] In step 634, the extended reality device can obtain a second synthetic image by reprojecting the first synthetic image based on change information of at least one content.

[0374] FIG. 8 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0375] In step 636, the extended reality device can recognize the movement of the extended reality device through at least one sensor to obtain a sensing value for at least one of the position or direction of the extended reality device.

[0376] An extended reality device according to one embodiment may include at least one sensor that recognizes movement of the extended reality device and obtains a sensing value for at least one of a position or orientation of the extended reality device.

[0377] The sensor may include, but is not limited to, at least one of a depth sensor capable of obtaining depth information about an actual space included in the FOV (field of view) of the camera, and an IMU (Inertial Measurement Unit) sensor for tracking the movement of the extended reality device according to one embodiment.

[0378] An extended reality device according to one embodiment can recognize movement of the extended reality device through at least one sensor and obtain a sensing value for at least one of a position or direction of the extended reality device.

[0379] The sensing value may mean a measurement of at least one of the position or orientation of the extended reality device that changes according to the movement of the extended reality device.

[0380] An extended reality device according to one embodiment can obtain information on the direction in which the extended reality device moves through at least one sensor when the movement of the extended reality device is recognized.

[0381] An extended reality device according to one embodiment can obtain location information of the extended reality device through at least one sensor when movement of the extended reality device is recognized.

[0382] In step 638, the extended reality device can obtain a second synthetic image by reprojecting the first synthetic image based on sensing values ​​obtained through at least one sensor.

[0383] An extended reality device according to one embodiment can obtain a second synthetic image by reprojecting a first synthetic image as movement of the extended reality device is recognized through at least one sensor.

[0384] An extended reality device according to one embodiment can obtain a second synthetic image by reprojecting a first synthetic image based on information about the direction in which the extended reality device moves or information about the location of the extended reality device.

[0385] The second composite layer may include metadata included in the background R channel, background G channel, background B channel, and texture map included in the second composite image.

[0386] The second synthesis layer can consist of 32 bits.

[0387] An extended reality device according to one embodiment may re-render a previous virtual image based on information about the direction in which the extended reality device is moving, when movement of the extended reality device is recognized through at least one sensor.

[0388] An extended reality device according to one embodiment can generate a new virtual image in an area that does not overlap with a previously rendered virtual image and render it together with the previous virtual image.

[0389] A new virtual image may mean a virtual image that needs to be newly generated based on the movement of the augmented reality device.

[0390] A second composite image may mean an image that is a composite of a re-rendered previous virtual image and a new virtual image.

[0391] For example, the previous virtual image may mean an image expressed at 90˚ to the left and 90˚ to the right, assuming that the front of the extended reality device is 0˚ before the movement of the extended reality device is recognized.

[0392] In one embodiment, when the extended reality device recognizes through at least one sensor that the direction of the extended reality device has changed by 1° to the right, the device can re-render an image expressed at 89° to the left and 90° to the right from the previous virtual image before the movement of the extended reality device is recognized.

[0393] An extended reality device according to one embodiment can generate a new virtual image in an area expressed from 90° to 91° to the right, assuming that the front of the extended reality device is 0° before the movement of the extended reality device is recognized.

[0394] Here, the area located 90˚ to 91˚ to the right of the previous virtual image may refer to an area that does not overlap with the previously rendered virtual image.

[0395] An extended reality device according to one embodiment can generate a new virtual image in an area that does not overlap with a previously rendered virtual image and render it together with the previous virtual image.

[0396] For example, the previous virtual image may mean an image expressed at 90˚ upward and 90˚ downward when the front of the augmented reality device is assumed to be 0˚ before the movement of the augmented reality device is recognized.

[0397] In one embodiment, when the extended reality device recognizes that the direction of the extended reality device (200) has changed 1° downward through at least one sensor, and when the front of the extended reality device is assumed to be 0° before the movement of the extended reality device is recognized, the device can re-render an image expressed at 89° from the top and 90° from the bottom in the previous virtual image.

[0398] An extended reality device according to one embodiment can generate a new virtual image in an area expressed from 90˚ to 91˚ downward when the front of the extended reality device is assumed to be 0˚ before the movement of the extended reality device is recognized.

[0399] Here, the area located 90˚ to 91˚ from the bottom of the previous virtual image may refer to an area that does not overlap with the previously rendered virtual image.

[0400] An extended reality device according to one embodiment can generate a new virtual image in an area that does not overlap with a previously rendered virtual image and render it together with the previous virtual image.

[0401] Referring again to FIG. 6, at step 640, the augmented reality device can restore text included in the second synthetic image based on the metadata.

[0402] The second composite layer may include metadata included in the background R channel, background G channel, background B channel, and texture map of the second background included in the second composite image.

[0403] The second synthesis layer can consist of 32 bits.

[0404] The metadata may further include separation information for separating the second background layer and the foreground layer for the second background included in the second composite layer.

[0405] Metadata can consist of 8 bits.

[0406] The second background may refer to the background of the second synthetic image obtained by reprojecting the first synthetic image.

[0407] The second background may refer to an area excluding text in a second synthetic image obtained by reprojecting the first synthetic image.

[0408] A second background layer may refer to a layer in which one pixel contains information for representing a second background.

[0409] Information for expressing the second background may include color information of the second background.

[0410] The second background layer may include a background R channel, a background G channel, a background B channel, and an alpha channel of the second background.

[0411] An alpha channel can contain background transparency information. An alpha channel can consist of 8 bits.

[0412] An extended reality device according to one embodiment can separate a second background layer and a foreground layer based on separation information.

[0413] A specific method for separating a second background layer and a foreground layer based on separation information in an extended reality device according to one embodiment will be described with reference to FIG. 9.

[0414] FIG. 9 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0415] At step 642, the extended reality device can separate the second background layer and the foreground layer based on the separation information.

[0416] The second composite layer may include metadata included in the background R channel, background G channel, background B channel, and texture map of the second background included in the second composite image.

[0417] The second synthesis layer can consist of 32 bits.

[0418] The background R channel, background G channel, background B channel, and metadata included in the texture map of the second background included in the second composite layer may each be composed of 8 bits.

[0419] The metadata may further include separation information for separating the second background layer and the foreground layer for the second background included in the second composite layer.

[0420] According to one embodiment, metadata may be composed of 3 bits for edge information of text, 3 bits for color information of text, 1 bit for font information of text, and 1 bit for separation information.

[0421] An extended reality device according to one embodiment can separate a second background layer and a foreground layer included in a second synthetic layer based on separation information.

[0422] For example, the extended reality device may separate any layer included in the second composite layer as a foreground layer if the separation information for that layer indicates 0.

[0423] For example, the extended reality device may separate any layer included in the second synthetic layer into a second background layer if the separation information for that layer indicates 1.

[0424] A second background layer may refer to a layer in which one pixel contains information for representing a second background.

[0425] The second background layer may contain information for one pixel to represent the second background.

[0426] Information for expressing the second background may mean the background R channel, background G channel, background B channel, and alpha channel of the second background.

[0427] An alpha channel can contain transparency information for a secondary background. The alpha channel can consist of 8 bits.

[0428] A foreground layer can contain information for a single pixel to represent the foreground.

[0429] A foreground layer may mean a layer excluding a second background layer among at least one layer included in one pixel among a plurality of pixels included in a second composite image.

[0430] A foreground layer can contain a text layer.

[0431] A foreground layer can contain a foreground R channel, a foreground G channel, a foreground B channel, and an alpha channel.

[0432] An alpha channel can contain transparency information for text. An alpha channel can consist of 8 bits.

[0433] A synthetic image corresponding to the foreground layer can mean the foreground.

[0434] The foreground may refer to a virtual image expressed by the foreground R channel, foreground G channel, foreground B channel, and alpha channel included in the foreground layer.

[0435] The composite image corresponding to the foreground layer may contain text.

[0436] At step 644, the extended reality device can restore text included in the synthetic image corresponding to the foreground layer based on the edge information.

[0437] A specific method for restoring text included in a synthetic image corresponding to a foreground layer based on edge information by an extended reality device according to one embodiment will be described with reference to FIG. 10.

[0438] FIG. 10 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0439] In step 646, the extended reality device can obtain border information of the text based on the edge information.

[0440] Edge information can refer to information indicating the boundaries of text.

[0441] Edge information may include distance information from the center of all pixels included in the foreground to at least one pixel located at the center of at least one text.

[0442] Text can consist of at least one pixel.

[0443] At least one pixel located at the center of the text may mean at least one pixel located inside the text and at the center of the text based on the border of the text among at least one pixel composing the text.

[0444] An extended reality device according to one embodiment can obtain border information of text based on edge information.

[0445] An extended reality device according to one embodiment can obtain boundary information of text based on edge information and a threshold.

[0446] An extended reality device according to one embodiment may divide a value between 0 and 1 to correspond to a distance from the center of all pixels included in the foreground to at least one pixel located at the center of the text.

[0447] An extended reality device according to one embodiment may map each distance from the center of all pixels included in the foreground to at least one pixel located at the center of the text to a value between 0 and 1.

[0448] For example, the augmented reality device may map the distance from the center of the first pixel to at least one pixel located at the center of at least one of the first text or the second text to a value between 0 and 1.

[0449] For example, the extended reality device may map the distance from the center of the second pixel to at least one pixel located at the center of at least one of the first text and the second text to a value between 0 and 1.

[0450] The first pixel and the second pixel may refer to at least one pixel included in the foreground.

[0451] The first text and the second text may refer to at least one of the texts included in the foreground.

[0452] In one embodiment, the extended reality device may map the distance from the center of the first pixel to at least one pixel located at the center of the first text to 0.6, and map the distance from the center of the second pixel to at least one pixel located at the center of the first text to 0.4, when the distance from the center of the first pixel to at least one pixel located at the center of the first text is shorter than the distance from the center of the second pixel to at least one pixel located at the center of the first text. However, the present invention is not limited thereto.

[0453] In one embodiment, the extended reality device may map the distance from the center of the first pixel to at least one pixel located at the center of the second text to 0.3, and map the distance from the center of the second pixel to at least one pixel located at the center of the second text to 0.8, when the distance from the center of the first pixel to at least one pixel located at the center of the second text is longer than the distance from the center of the second pixel to at least one pixel located at the center of the second text. However, the present invention is not limited thereto.

[0454] An extended reality device according to one embodiment may set a threshold for a value mapped between 0 and 1.

[0455] A threshold may refer to a setting value for identifying pixels located inside or outside the boundary of the text.

[0456] The threshold can be changed depending on the user's settings.

[0457] An extended reality device according to one embodiment can identify at least one pixel mapped between 0 and 1 with a value less than a threshold value as a pixel located outside the border of the text.

[0458] For example, the augmented reality device may map the distance from the center of the first pixel to at least one pixel located at the center of the first text to a value between 0 and 1.

[0459] The first pixel may mean one of at least one pixel included in the foreground.

[0460] The first text may refer to at least one of the texts included in the foreground.

[0461] An extended reality device according to one embodiment may set a threshold value of 0.5 as a setting value for identifying pixels located inside or outside the boundary of a text.

[0462] An extended reality device according to one embodiment may identify a first pixel as a pixel located outside a boundary of the first text if a distance from the center of the first pixel to at least one pixel located at the center of the first text is mapped to 0.3.

[0463] Accordingly, the extended reality device according to one embodiment can identify pixels located outside the border of the text.

[0464] An extended reality device according to one embodiment can identify pixels located outside the border of text as a second background.

[0465] An extended reality device according to one embodiment can identify pixels located outside the border of text as an area excluding the text in the second synthetic image.

[0466] An extended reality device according to one embodiment can identify at least one pixel mapped between 0 and 1 with a value greater than a threshold as a pixel located within a border of text.

[0467] For example, the extended reality device may map the distance from the center of the second pixel to at least one pixel located at the center of the second text to a value between 0 and 1.

[0468] The second pixel may mean at least one pixel included in the foreground.

[0469] The second text may refer to at least one of the texts included in the foreground.

[0470] An extended reality device according to one embodiment may set a threshold value of 0.5 as a setting value for identifying pixels located inside or outside the boundary of a text.

[0471] An extended reality device according to one embodiment may identify a second pixel as a pixel located within a boundary of the second text if a distance from the center of the second pixel to at least one pixel located at the center of the second text is mapped to 0.7.

[0472] Accordingly, the extended reality device according to one embodiment can identify pixels located within the border of text.

[0473] An extended reality device according to one embodiment can obtain border information of text by identifying pixels located inside or outside the border of the text.

[0474] At step 648, the augmented reality device can restore text included in the synthetic image corresponding to the foreground layer based on the boundary information.

[0475] An extended reality device according to one embodiment can restore text by identifying pixels located within the boundary of text in a synthetic image corresponding to a foreground layer based on boundary information as pixels constituting the text.

[0476] An extended reality device according to one embodiment can restore text by identifying pixels located outside the border of text in a synthetic image corresponding to a foreground layer based on border information as pixels that do not constitute text.

[0477] An extended reality device according to one embodiment can restore text by identifying pixels located outside the border of text in a synthetic image corresponding to a foreground layer based on border information as pixels that do not constitute text.

[0478] Referring again to FIG. 6, at step 640, the augmented reality device can restore text included in the second synthetic image based on the metadata.

[0479] An extended reality device according to one embodiment can restore at least one text among text having a color included in a first synthetic image or text having a font applied to text included in the first synthetic image, based on metadata.

[0480] Metadata may include color information including information about the text R channel, text G channel, and text B channel contained in the text layer.

[0481] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 4 bits to edge information of text, 3 bits to color information of text, and 1 bit to separation information. However, the present invention is not limited thereto.

[0482] An extended reality device according to one embodiment can restore text including a color of text included in a first synthetic image based on color information of the text included in metadata.

[0483] Metadata may include font information of text included in the first synthetic image.

[0484] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 4 bits to edge information of text, 3 bits to font information of text, and 1 bit to separation information. However, the present invention is not limited thereto.

[0485] An extended reality device according to one embodiment can restore text with a font applied to the text included in the first synthetic image based on font information of the text included in the metadata.

[0486] The metadata may include color information including information about the text R channel, text G channel, and text B channel included in the text layer, and font information of the text included in the first composite image.

[0487] An extended reality device according to one embodiment can generate metadata of a total of 8 bits by allocating 3 bits to edge information of text, 3 bits to color information of text, 1 bit to font information of text, and 1 bit to separation information. However, the present invention is not limited thereto.

[0488] An extended reality device according to one embodiment can restore text having a color included in a first synthetic image and a font applied to the text included in the first synthetic image based on color information and font information included in metadata.

[0489] At step 650, the augmented reality device can render the restored text and the second background included in the second synthetic image.

[0490] FIG. 11 is a flowchart illustrating an operation method of an extended reality device according to one embodiment.

[0491] The extended reality device may include an image output unit including a waveguide or display.

[0492] At step 660, the augmented reality device may output the rendered text and second background to a waveguide or display.

[0493] An extended reality device according to one embodiment may include a memory storing at least one instruction and at least one processor executing at least one instruction stored in the memory.

[0494] The at least one processor can generate a texture map including metadata of text by executing the at least one command.

[0495] The at least one processor can obtain a first composite image by synthesizing a first background layer and the texture map for the first background by executing the at least one command.

[0496] The at least one processor can obtain a second composite image by reprojecting the first composite image by executing the at least one command.

[0497] The at least one processor can restore the text included in the second synthetic image based on the metadata by executing the at least one command.

[0498] The at least one processor can render the restored text and the second background included in the second synthetic image by executing the at least one command.

[0499] Metadata according to one embodiment may include edge information of the text.

[0500] The at least one processor may replace an alpha channel included in a text layer for the text with the metadata by executing the at least one command.

[0501] An alpha channel included in a text layer according to one embodiment may include transparency information of the text.

[0502] The at least one processor can obtain the first synthetic image by replacing an alpha channel included in the first background layer with the metadata by executing the at least one command.

[0503] An alpha channel included in a first background layer according to one embodiment may include transparency information of the first background.

[0504] According to one embodiment, the metadata may further include separation information for separating the second background layer and the foreground layer for the second background.

[0505] The at least one processor can separate the second background layer and the foreground layer based on the separation information by executing the at least one command.

[0506] The at least one processor can restore the text included in the synthetic image corresponding to the foreground layer based on the edge information by executing the at least one command.

[0507] The at least one processor can obtain boundary information of the text based on the edge information by executing the at least one command.

[0508] The at least one processor can restore the text included in the synthetic image corresponding to the foreground layer based on the boundary information by executing the at least one command.

[0509] An extended reality device according to one embodiment may further include a communication interface.

[0510] The at least one processor can receive change information of at least one content output to a user device connected to the extended reality device through the communication interface by executing the at least one command.

[0511] The at least one processor can obtain the second composite image by reprojecting the first composite image based on change information of the at least one content by executing the at least one command.

[0512] A first synthetic image according to one embodiment may include an image synthesized from at least one synthetic image stored in the memory.

[0513] An extended reality device according to one embodiment may further include at least one sensor.

[0514] An extended reality device according to one embodiment may further include at least one sensor that recognizes movement of the extended reality device and obtains a sensing value for at least one of a position or direction of the extended reality device.

[0515] The at least one processor can recognize the movement of the extended reality device through at least one sensor by executing the at least one command, and obtain a sensing value for at least one of a position or a direction.

[0516] The at least one processor can obtain the second synthetic image by reprojecting the first synthetic image based on the sensing value obtained through the at least one sensor by executing the at least one command.

[0517] The at least one processor can recognize movement of the extended reality device through the at least one sensor by executing the at least one command.

[0518] The at least one processor can obtain the second synthetic image by reprojecting the first synthetic image based on the movement of the extended reality device being recognized by executing the at least one command.

[0519] An extended reality device according to one embodiment may further include an image output unit including a wave guide or a display.

[0520] The at least one processor can output the rendered text and the second background to the wave guide or display by executing the at least one command.

[0521] Metadata according to one embodiment may further include at least one of color information including information on R channels, G channels, and B channels included in the text layer, or font information of the text.

[0522] The at least one processor can restore at least one text among text including the color of the text or text to which the font of the text is applied, based on at least one of the color information or the font information, by executing the at least one command.

[0523] A method of operating an extended reality device according to one embodiment may include generating a texture map including metadata of text.

[0524] A method of operating an extended reality device according to one embodiment may include obtaining a first synthesized image by synthesizing a first background layer for a first background and the texture map.

[0525] A method of operating an extended reality device according to one embodiment may include a step of obtaining a second synthetic image by reprojecting the first synthetic image.

[0526] A method of operating an extended reality device according to one embodiment may include a step of restoring the text included in the second synthetic image based on the metadata.

[0527] A method of operating an extended reality device according to one embodiment may include a step of rendering the restored text and the second background included in the second synthetic image.

[0528] Metadata according to one embodiment may include edge information of the text.

[0529] A method of operating an extended reality device according to one embodiment may include a step of replacing an alpha channel included in a text layer for the text with the metadata.

[0530] An alpha channel included in a text layer according to one embodiment may include transparency information of the text.

[0531] A method of operating an extended reality device according to one embodiment may include obtaining the first synthetic image by replacing an alpha channel included in the first background layer with the metadata.

[0532] An alpha channel included in a first background layer according to one embodiment may include transparency information of the first background.

[0533] According to one embodiment, the metadata may further include separation information for separating the second background layer and the foreground layer for the second background.

[0534] A method of operating an extended reality device according to one embodiment may include a step of separating the second background layer and the foreground layer based on the separation information.

[0535] A method of operating an extended reality device according to one embodiment may include a step of restoring the text included in a synthetic image corresponding to the foreground layer based on the edge information.

[0536] A method of operating an extended reality device according to one embodiment may include a step of obtaining boundary information of the text based on the edge information.

[0537] A method of operating an extended reality device according to one embodiment may include a step of restoring the text included in a synthetic image corresponding to the foreground layer based on the boundary information.

[0538] A method of operating an extended reality device according to one embodiment may include a step of receiving change information of at least one content output to a user device connected to the extended reality device.

[0539] A method of operating an extended reality device according to one embodiment may include a step of obtaining the second synthetic image by re-projecting the first synthetic image based on change information of the at least one content.

[0540] A method of operating an extended reality device according to one embodiment may include a step of recognizing movement of the extended reality device through at least one sensor.

[0541] A method of operating an extended reality device according to one embodiment may include a step of recognizing movement of the extended reality device through at least one sensor and obtaining a sensing value for at least one of a position or direction of the extended reality device.

[0542] A method of operating an extended reality device according to one embodiment may include a step of obtaining the second synthetic image by re-projecting the first synthetic image based on a sensing value obtained through the at least one sensor.

[0543] A method of operating an extended reality device according to one embodiment may include a step of obtaining the second synthetic image by reprojecting the first synthetic image based on recognition of movement of the extended reality device.

[0544] A method of operating an extended reality device according to one embodiment may include outputting the rendered text and the second background to a wave guide or a display.

[0545] Metadata according to one embodiment may further include at least one of color information including information about R channel, G channel and B channel included in the text layer or font information of the text.

[0546] A method of operating an extended reality device according to one embodiment may include a step of restoring at least one of text including the color of the text or text to which the font of the text is applied, based on at least one of the color information or the font information.

[0547] The method of operating an extended reality device according to one embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention or may be those known to and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0548] Additionally, the operating method of the extended reality device according to the disclosed embodiments may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer.

[0549] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable app) distributed electronically by an electronic device manufacturer or through an electronic marketplace (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a storage medium of a manufacturer's server, an electronic marketplace server, or a relay server that temporarily stores the software program.

[0550] In a system comprising a server and a client device, the computer program product may include a storage medium of the server or a storage medium of the client device. Alternatively, if a third device (e.g., a smartphone) exists that is communicatively connected to the server or the client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself that is transmitted from the server to the client device or the third device, or from the third device to the client device.

[0551] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0552] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) may execute a computer program product stored on the server, thereby controlling a client device in communication with the server to perform a method according to the disclosed embodiments.

[0553] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

Claims

1. In the extended reality device (200), At least one processor (210); and A memory (220) for storing at least one command; By having the at least one processor (210) execute the at least one instruction, the extended reality device (200) Create a texture map containing metadata of the text, A first composite image is obtained by synthesizing the first background layer and the texture map for the first background, A second composite image is obtained by reprojecting the first composite image, Restore the text included in the second synthetic image based on the metadata, Rendering the restored text and the second background included in the second synthetic image, An extended reality device, wherein the metadata includes edge information of the text.

2. In the first paragraph, the extended reality device (200) executes the at least one instruction by the at least one processor (210), Replace the alpha channel included in the text layer for the above text with the above metadata, An extended reality device, wherein the alpha channel included in the above text layer contains transparency information of the above text.

3. In the first or second paragraph, the extended reality device (200) executes the at least one instruction by the at least one processor (210), Obtaining the first synthetic image by replacing the alpha channel included in the first background layer with the metadata, An extended reality device, wherein the alpha channel included in the first background layer includes transparency information of the first background.

4. In any one of paragraphs 1 to 3, The above metadata further includes separation information for separating the second background layer and the foreground layer for the second background, By having the at least one processor (210) execute the at least one instruction, the extended reality device (200) Based on the above separation information, the second background layer and the foreground layer are separated, An extended reality device that restores the text included in a synthetic image corresponding to the foreground layer based on the edge information.

5. In any one of paragraphs 1 to 4, further comprising a communication interface; By having the at least one processor (210) execute the at least one instruction, the extended reality device (200) Receive change information of at least one content output to a user device connected to the extended reality device through the communication interface, An extended reality device that obtains the second synthetic image by reprojecting the first synthetic image based on change information of at least one content.

6. In any one of paragraphs 1 to 5, At least one sensor that recognizes the movement of the extended reality device and obtains a sensing value for at least one of the position or direction of the extended reality device; By having the at least one processor (210) execute the at least one instruction, the extended reality device (200) Recognize the movement of the extended reality device through at least one sensor to obtain a sensing value for at least one of a position or a direction, An extended reality device that obtains the second synthetic image by re-projecting the first synthetic image based on sensing values ​​obtained through at least one sensor.

7. In any one of paragraphs 1 to 6, The above metadata further includes at least one of color information including information on the R channel, G channel, and B channel included in the text layer or font information of the text, By having the at least one processor (210) execute the at least one instruction, the extended reality device (200) An extended reality device that restores at least one of text including the color of the text or text to which the font of the text is applied, based on at least one of the color information or the font information.

8. In the method of operating an extended reality device, A step (S610) of generating a texture map including metadata of text; A step (S620) of obtaining a first composite image by synthesizing a first background layer and the texture map for the first background; A step (S630) of obtaining a second composite image by reprojecting the first composite image; A step (S640) of restoring the text included in the second synthetic image based on the metadata; and A step (S650) of rendering the restored text and the second background included in the second synthetic image; A method of operation, wherein the above metadata includes edge information of the text.

9. In the 8th paragraph, the step of generating the texture map (S610) is A step of replacing an alpha channel included in a text layer for the above text with the above metadata; A method of operation in which an alpha channel included in the above text layer includes transparency information of the above text.

10. In the 8th or 9th paragraph, the step of obtaining the first synthetic image (S620) is as follows: A step of obtaining the first synthetic image by replacing the alpha channel included in the first background layer with the metadata; An operating method, wherein the alpha channel included in the first background layer includes transparency information of the first background.

11. In any one of paragraphs 8 to 10, The above metadata further includes separation information for separating the second background layer and the foreground layer for the second background, The step (S640) of restoring the text included in the second synthetic image is as follows: Based on the above separation information, a step (S642) of separating the second background layer and the foreground layer; and An operating method comprising a step (S644) of restoring the text included in the synthetic image corresponding to the foreground layer based on the edge information.

12. In any one of the 8th to 11th clauses, the step of obtaining the second synthetic image (S630) is as follows: A step (S632) of receiving change information of at least one content output to a user device connected to the extended reality device; and An operating method comprising: a step (S634) of obtaining the second synthetic image by re-projecting the first synthetic image based on change information of at least one content; 13. In any one of the 8th to 12th clauses, the step of obtaining the second synthetic image (S630) is as follows: A step (S636) of recognizing the movement of the extended reality device through at least one sensor and obtaining a sensing value for at least one of the position or direction of the extended reality device; and An operating method comprising a step (S638) of obtaining the second synthetic image by re-projecting the first synthetic image based on a sensing value obtained through at least one sensor.

14. In any one of paragraphs 8 to 13, The above metadata further includes at least one of color information including information on the R channel, G channel, and B channel included in the text layer or font information of the text, The step of restoring the above text (S640) is: An operating method comprising: a step of restoring at least one of text including the color of the text or text to which the font of the text is applied, based on at least one of the color information or the font information.

15. A computer-readable recording medium recording a program for executing the method of any one of clauses 8 to 14 on a computer.

Images