Display device and operation method thereof

Abstract

A display device according to one embodiment of the present disclosure can: identify a type of image data; identify, on the basis that the type of image data is a first type, setting information associated with the display of graphic information; on the basis that the setting information includes first information, combine first graphic data corresponding to the first type with the image data so as to generate a first image including a three-dimensional (3D) image and a 3D graphic; generate a first three-dimensional image corresponding to the gaze of a user on the basis of gaze tracking information of the user acquired using at least one sensor and the first image; and display the first three-dimensional image on the display of the display device.

Description

Display device and method of operation thereof

[0001] The present disclosure relates to a display device and a method of operating the same.

[0002] Generally, watching 3D (three-dimensional) video involves the inconvenience of having to wear special glasses every time, and prolonged wear exacerbates eye strain and discomfort. In light of this, glasses-free stereoscopic display technology that enables the viewing of 3D video without glasses is currently being developed. Glasses-free stereoscopic displays can provide different images to each of the user's eyes through the special structure of lenses or screens, thereby allowing the user to experience a sense of depth.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the foregoing is to be claimed as prior art related to this document, nor is it to be used to determine prior art.

[0004] Glasses-free stereoscopic displays enable richer visual expression compared to standard displays that provide flat images, thereby allowing for more effective information delivery. For example, glasses-free stereoscopic displays can provide graphic interfaces (e.g., OSD (on-screen display)) to enhance user experience or facilitate intuitive interaction with the user. While such graphics may need to be displayed alongside 3D images, glasses-free stereoscopic displays do not support displaying 3D graphics alongside 3D images. Additionally, when 2D graphics are displayed alongside 3D images on glasses-free stereoscopic displays, the 2D graphics are split between the user's left and right eyes; consequently, there is a problem of crosstalk occurring at the boundaries of the 2D graphics due to the difference between the left and right eyes.

[0005] One embodiment of the present disclosure may provide a display device and a method of operating the same.

[0006] One embodiment of the present disclosure may provide a display device capable of providing not only planar images but also stereoscopic images, and a method of operating the same. One embodiment of the present disclosure may provide a display device capable of providing glasses-free stereoscopic images and a method of operating the same.

[0007] One embodiment of the present disclosure may provide a display device and a method of operating the same that enable the implementation of 2D graphics or 3D graphics for a glasses-free stereoscopic image display.

[0008] One embodiment of the present disclosure can provide a display device and a method of operating the same that can display 2D graphics or 3D graphics together with 3D images without the occurrence of crosstalk in a glasses-free stereoscopic display.

[0009] A display device according to one embodiment of the present disclosure may include a display, a memory, and at least one processor that executes instructions of a program stored in the memory. The at least one processor may be configured to identify a type of image data, identify setting information associated with the display of graphic information based on the fact that the type of image data is a first type, and, based on the fact that the setting information includes the first information, combine a first graphic data corresponding to the first type with the image data to generate a first image including a three-dimensional image and a 3D graphic, generate a first stereoscopic image corresponding to the user's gaze based on the first image and user's gaze tracking information obtained through at least one sensor, and display the first stereoscopic image on the display.

[0010] According to one embodiment, the display device may further include a transceiver, and the at least one sensor may include a camera connected to the display device through the transceiver.

[0011] According to one embodiment, the at least one processor may be configured to generate the first stereoscopic image through view mapping, which determines R (red), G (green), and B (blue) values ​​for each subpixel of the display based on the first image in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and assigns the determined R, G, and B values ​​to each subpixel.

[0012] According to one embodiment, the display device may further include a transceiver, and the image data may include image data received from a source device through the transceiver, or image data stored in the memory.

[0013] According to one embodiment, the display device may further include a user interface, and the at least one processor may be configured to receive an input instructing the display of the graphic information through the user interface and to identify setting information associated with the display of the graphic information based on the input, and the graphic information may include any one of information associated with a function setting of the display device, information associated with a screen setting or function setting of the display, an OSD (on screen display) menu, or information associated with the execution of an application.

[0014] According to one embodiment, the at least one processor may be configured to generate a second image including a 3D image and a 2D (two-dimensional) graphic by combining second graphic data corresponding to a second type with the image data based on the setting information including second information, generate a second stereoscopic image corresponding to the user's gaze based on the gaze tracking information and the second image, and display the second stereoscopic image on the display.

[0015] According to one embodiment, the second type may indicate a type associated with a side-by-side image in which a plurality of images with no difference in viewpoint are arranged side by side.

[0016] According to one embodiment, the at least one processor may be configured to generate the second stereoscopic image through view mapping, which determines R (red), G (green), and B (blue) values ​​for each subpixel of the display based on the second image in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and assigns the determined R, G, and B values ​​to each subpixel.

[0017] According to one embodiment, the at least one processor may be configured to acquire a third stereoscopic image corresponding to the user's gaze based on the image data, based on the fact that the type of the image data is a third type, combine second graphic data corresponding to a fourth type with the third stereoscopic image to update the third stereoscopic image to include 2D graphics, and display the updated third stereoscopic image on the display.

[0018] According to one embodiment, the display device further includes a transceiver, and the image data is received from a source device through the transceiver, and the third type indicates a type associated with an image generated in response to the user's gaze from the source device, and the fourth type may indicate a type associated with 2D graphics.

[0019] A method of operation of a display device according to one embodiment of the present disclosure may include: identifying a type of image data; identifying setting information associated with the display of graphic information based on the fact that the type of image data is a first type; combining a first graphic data corresponding to the first type with the image data based on the fact that the setting information includes the first information to generate a first image including a 3D (three-dimensional) image and a 3D graphic; generating a first stereoscopic image corresponding to the user's gaze based on the first image and user's gaze tracking information obtained through at least one sensor; and displaying the first stereoscopic image on the display of the display device.

[0020] According to one embodiment, the first type may indicate a type associated with a side-by-side image in which a plurality of images with a viewpoint difference are arranged side by side.

[0021] According to one embodiment, the operation of generating the first stereoscopic image may include determining R (red), G (green), and B (blue) values ​​for each subpixel of the display based on the first image in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and generating the first stereoscopic image through view mapping that assigns the determined R, G, and B values ​​to each subpixel.

[0022] According to one embodiment, the image data may include image data received from a source device or image data stored in the display device.

[0023] According to one embodiment, the operation of identifying setting information associated with the display of the graphic information includes: the operation of receiving an input instructing the display of the graphic information; and the operation of identifying setting information associated with the display of the graphic information based on the input, wherein the graphic information may include any one of information associated with a function setting of the display device, information associated with a screen setting or function setting of the display, an OSD (on screen display) menu, or information associated with the execution of an application.

[0024] According to one embodiment, the method may further include: an operation to generate a second image including a 3D image and a 2D (two-dimensional) graphic by combining second graphic data corresponding to a second type with the image data based on the setting information including second information; an operation to generate a second stereoscopic image corresponding to the user's gaze based on the gaze tracking information obtained through the camera and the second image; and an operation to display the generated second stereoscopic image on the display.

[0025] According to one embodiment, the second type may indicate a type associated with a side-by-side image in which a plurality of images with no difference in viewpoint are arranged side by side.

[0026] According to one embodiment, the operation of generating the second stereoscopic image may include determining R (red), G (green), and B (blue) values ​​for each subpixel of the display based on the second image in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and generating the second stereoscopic image through view mapping that assigns the determined R, G, and B values ​​to each subpixel.

[0027] According to one embodiment, the method may further include: an operation of acquiring a third stereoscopic image corresponding to the user's gaze based on the image data, based on the fact that the type of the image data is a third type; an operation of updating the third stereoscopic image by combining second graphic data corresponding to a fourth type with the third stereoscopic image so that the third stereoscopic image includes 2D graphics; and an operation of displaying the updated third stereoscopic image on the display.

[0028] According to one embodiment, the image data is received from a source device, the third type indicates a type associated with an image generated in response to the user's gaze from the source device, and the fourth type may indicate a type associated with 2D graphics.

[0029] FIG. 1 is a block diagram of a display system according to one embodiment.

[0030] FIG. 2a is a diagram illustrating a display structure in a state where the lenticular function is activated in a display device according to one embodiment.

[0031] FIG. 2b is a drawing for explaining the configuration of a display panel with a lenticular function activated according to one embodiment.

[0032] FIG. 3 is a diagram illustrating the first display operation of a display system according to one embodiment.

[0033] FIG. 4 is a diagram illustrating the second display operation of a display system according to one embodiment.

[0034] FIG. 5 is a diagram illustrating an image displayed by view mapping according to one embodiment.

[0035] FIG. 6 is a drawing illustrating various types of graphics according to one embodiment.

[0036] FIG. 7 is a diagram illustrating the operation of generating a 3D image with added 2D graphics through view mapping according to one embodiment.

[0037] FIG. 8 is a diagram illustrating the operation of generating a 3D image with added 3D graphics through view mapping according to one embodiment.

[0038] FIG. 9 is a schematic diagram illustrating the operation of a display device according to one embodiment generating various stereoscopic images.

[0039] FIG. 10 is a flowchart illustrating the operation of a display device according to one embodiment displaying a first stereoscopic image.

[0040] FIG. 11 is a flowchart illustrating the operation of a display device according to one embodiment displaying a second stereoscopic image.

[0041] FIG. 12 is a flowchart illustrating the operation of a display device according to one embodiment displaying a third stereoscopic image.

[0042] In the following description, the attached drawings are referenced, and specific examples of implementation are illustrated within the drawings. Additionally, other examples may be used and structural modifications may be made without departing from the scope of the various examples.

[0043] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0044] FIG. 1 is a block diagram of a display system according to one embodiment.

[0045] Referring to FIG. 1, the display system (100) may include a source device (110) and a display device (120).

[0046] According to one embodiment, the source device (110) may provide various types of images (e.g., 2D (two-dimensional) images or 3D (three-dimensional) images) to be displayed on the display device (120). For example, the source device (110) may be a server (e.g., a content server, a cloud server, a broadcast server, or a web server), or an electronic device such as a mobile device (e.g., a smartphone or tablet) or a computer device (e.g., a PC (personal computer) or a laptop).

[0047] According to one embodiment, the source device (110) may be connected to the display device (120) via a wired or wireless connection and may transmit video data to the display device (120). For example, the source device (110) may transmit video data to the display device (120) based on the completion of the connection setup with the display device (120) or based on a request from the display device (120).

[0048] According to one embodiment, the source device (110) may be optionally connected to the display device (120). For example, the source device (110) may not be connected to the display device (120) when the display device (120) displays an image stored in the display device (120) or displays an image provided by another electronic device.

[0049] According to one embodiment, the source device (110) may include a transceiver (112), a memory (114), and a processor (116). According to one embodiment, the source device (110) may include additional components (e.g., a user interface) in addition to the illustrated components, or at least one of the illustrated components may be omitted.

[0050] The communication unit (112) can support the establishment of a wired or wireless communication channel between a source device (110) and an external electronic device (e.g., a display device (120)), and the performance of communication through the established wired or wireless communication channel. The communication unit (112) can operate independently of a processor (e.g., an application processor) (116) and may include one or more communication processors that support wired or wireless communication.

[0051] The communication unit (112) may include a wired communication module or a wireless communication module. According to one embodiment, the wired communication module may include a USB (universal serial bus) communication module and an HDMI (high-definition multimedia interface) communication module, and may further include a communication module for connecting to various other types of display devices (e.g., TV or monitor). In relation to the wired communication module, the source device (110) may include a communication interface. The communication interface may support one or more specified protocols that can be used for the source device (110) to be wired to an external electronic device (e.g., a display device (120)). According to one embodiment, the communication interface may include an HDMI interface, a USB interface, or a display port.

[0052] According to one embodiment, the wireless communication module may include a cellular communication module that supports legacy communication, 4G, 5G, or 6G communication, or next-generation communication, and / or a short-range wireless communication module that supports Bluetooth communication, BLE (Bluetooth Low Energy) communication, Wi-Fi (Wireless-Fidelity) communication, Wi-Fi Direct (WFD) communication, or IrDA (Infrared Data Association) communication. These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple distinct components (e.g., multiple chips).

[0053] The memory (114) can store various data or information used by at least one component of the source device (110) (e.g., communication unit (112) and / or processor (116)). For example, the memory (114) can store at least one program for processing and controlling the processor (116) and can store input and / or output data (e.g., image data). The memory (114) can also store at least one artificial intelligence (AI) model and may include volatile memory or non-volatile memory.

[0054] The processor (116) can control the overall operation of the source device (110). The processor (116) can perform operations or data processing regarding the control and / or communication of at least one other component of the source device (110). For example, the processor (116) can be electrically connected to the communication unit (112) and the memory (114), and can control the operations of the source device (110) by executing instructions of a program stored in the memory (114). According to one embodiment, the processor (116) can perform the operations of the source device (110) to be described later.

[0055] According to one embodiment, the processor (116) may correspond to a plurality of processors that divide a plurality of operations among the processors and perform them individually or collectively.

[0056] According to one embodiment, the processor (116) may include a processing circuit that executes instructions of a program stored in memory (114). The processor (116) may include at least one of a CPU (central processing unit), NPU (neural processing unit), GPU (graphics processing unit), MPU (micro processing unit), MCU (micro controller unit), AP (application processor), CP (communication processor), SoC (system on chip), or IC (integrated circuit) sensor hub, supplementary processor, communication processor, ASIC (application specific integrated circuit), or FPGA (field programmable gate arrays), and may have multiple cores.

[0057] According to one embodiment, the display device (120) may be a device that displays various images and information. For example, the display device (120) may display an image provided from an external electronic device (e.g., source device (110)) or an image stored in the display device (120), and may also display various information (e.g., graphic information or text information) based on an external input (e.g., user input).

[0058] According to one embodiment, the display device (120) may include a display (121), a communication unit (122), a user interface (123), a memory (124), and a processor (126). According to one embodiment, the display device (120) may include additional components in addition to the illustrated components, or at least one of the illustrated components may be omitted.

[0059] The display (121) can perform various display operations according to the function of the display device (120). For example, the display (121) can display information in the form of text, images, or graphics. According to one embodiment, the display (121) can display 2D images or 3D images and may include a light field display (LFD) that enables viewing 3D images without glasses. The LFD can provide stereoscopic images by generating a light field expressed as the intensity and direction of light in space through a flat display and an optical element. Unlike a general display, the LFD allows viewing the depth or side of an object depending on the viewing angle, making it possible to create natural stereoscopic images without glasses.

[0060] The communication unit (122) can support the establishment of a wired or wireless communication channel between the display device (120) and an external electronic device (e.g., source device (110)), and the performance of communication through the established wired or wireless communication channel. The communication unit (122) can operate independently of a processor (e.g., application processor) (126) and may include one or more communication processors that support wireless or wired communication.

[0061] According to one embodiment, the communication unit (122) may include a wired communication module or a wireless communication module. According to one embodiment, the wired communication module may include a USB communication module and an HDMI communication module, and may further include a communication module for connecting to various other types of external electronic devices (e.g., PC). In relation to the wired communication module, the display device (120) may include a communication interface. The communication interface may support one or more specified protocols that can be used for the display device (120) to be wired to an external electronic device (e.g., a source device (110)). According to one embodiment, the communication interface may include an HDMI interface, a USB interface, or a display port.

[0062] According to one embodiment, the wireless communication module may include a cellular communication module that supports legacy communication, 4G, 5G, or 6G communication, or next-generation communication, and / or a short-range wireless communication module that supports Bluetooth communication, BLE communication, Wi-Fi communication, WFD communication, or IrDA communication. These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple distinct components (e.g., multiple chips).

[0063] The user interface (123) may include an input interface and / or an output interface for interacting with the user. The input interface may receive commands or data to be used by a component of the display device (120) (e.g., processor (126)) from outside the display device (126) (e.g., user). For example, the input interface may include at least one physically configured button (or key) and / or a microphone capable of receiving the user's speech. The output interface may output information to be transmitted from the display device (120) to an external device or user. For example, the output interface may include a speaker and / or an LED (light emitting diode), and the display (121) may also be used as an output interface. According to one embodiment, if the display (121) is configured as a touchscreen, the display (121) may perform both the functions of the input interface and the output interface.

[0064] The memory (124) can store various data or information used by at least one component of the display device (120) (e.g., display (121), communication unit (122), user interface (123), or processor (126)). For example, the memory (124) can store at least one program for processing and controlling the processor (126), and can store input and / or output data.

[0065] The memory (124) may store image data provided from a source device, image data or graphic information to be displayed on a display device (120), at least one image captured by a camera (130), or eye tracking information based on at least one image. The memory (124) may store at least one AI model and may include volatile memory or non-volatile memory.

[0066] The processor (126) can control the overall operation of the display device (120). The processor (126) can perform operations or data processing regarding the control and / or communication of at least one other component of the display device (120). For example, the processor (126) can be electrically connected to the display (121), the communication unit (122), the user interface (123), and the memory (124), and can control the operations of the display device (120) by executing instructions of a program stored in the memory (124). According to one embodiment, the processor (126) can perform the operations of the display device (120) to be described later.

[0067] According to one embodiment, the processor (126) may correspond to a plurality of processors that divide a plurality of operations among the processors and perform them individually or collectively.

[0068] According to one embodiment, the processor (126) may include a processing circuit that executes instructions of a program stored in memory (124). The processor (126) may include at least one of a CPU, NPU, GPU, MPU, MCU, AP, CP, SoC, or IC sensor hub, auxiliary processor, communication processor, ASIC, or FPGA, and may have a plurality of cores.

[0069] According to one embodiment, a display device (120) may be connected to at least one sensor (140) through a communication unit (122) or a communication interface (e.g., a USB interface) included in the communication unit (122). At least one sensor (140) may detect various states associated with the operation of the display device (120) (e.g., an external environment state or a user state of the display device (120)). According to one embodiment, at least one sensor (140) may track or detect the user's gaze and generate gaze tracking information based on the detection result. At least one sensor (140) may provide the gaze tracking information to the display device (120).

[0070] According to one embodiment, at least one sensor (140) may include a camera (130). The camera (130) may take (or capture) still images and video. According to one embodiment, the camera (130) may be positioned at a location adjacent to the display (121) (e.g., at the top of the display (121)) or at a location where it can photograph the user's face or eyes. The camera (130) may photograph the user's face or eyes at the positioned location and provide at least one image or gaze tracking data for generating user gaze tracking information.

[0071] In the following description, the camera (130) included in at least one sensor (140) is connected to and used with the display device (120) as an example. However, it is not limited thereto, and various sensors (e.g., IR (infrared) sensors, optical sensors, or image sensors) or devices capable of generating or providing eye-tracking information may be used in place of the camera (130). Additionally, the camera (130) may be an external device of the display device (120), but if the display device (120) includes a camera, the camera included in the display device (120) may be used. According to one embodiment, the display device (120) may operate in various display modes. For example, the display device (120) may operate in a 2D mode for displaying 2D images, or in a 3D mode for displaying 3D images (stereoscopic images or glasses-free stereoscopic images). The 2D mode or the 3D mode may be switched and used, and may be operated based on set information or external input (e.g., user input). According to one embodiment, a liquid crystal layer may be included in front of a display panel included in the display (121) for selectively using a lenticular function (e.g., a glasses-free stereoscopic image providing function or a 3D visual effect providing function). The liquid crystal layer may be controlled for operation in 2D mode or 3D mode.

[0072] According to one embodiment, the liquid crystal layer can be controlled so that the lenticular function is disabled. For example, the operation of the liquid crystal layer may be turned off so that the lenticular function is not used and a 2D mode can be performed.

[0073] According to one embodiment, the liquid crystal layer can be controlled to activate a lenticular function. For example, the operation of the liquid crystal layer is turned on so that the lenticular function can be used and a 3D mode can be performed. According to one embodiment, when the lenticular function is activated in the liquid crystal layer, a display structure as shown in FIG. 2a can be used.

[0074] FIG. 2a is a diagram illustrating a display structure in a state where the lenticular function is activated in a display device according to one embodiment.

[0075] Referring to FIG. 2a, when the lenticular function is activated in the display device (120), the liquid crystal layer in front of the display panel (210) can be operated as a lenticular lens array (220). According to one embodiment, the lenticular lens array (220) implemented by the liquid crystal layer can enable the display (121) to provide glasses-free stereoscopic images or 3D images.

[0076] The display panel (210) may include multiple pixels, and each of the multiple pixels may include three subpixels. For example, the three subpixels may include subpixels representing red (R), green (G), and blue (B). The color displayed by a single pixel may be determined by a combination of R, G, and B values. The R, G, and B values ​​may each correspond to R, G, and B brightness values ​​determined within a set range (e.g., a range from 0 to 255).

[0077] The lenticular lens array (220) may include a plurality of lenticular lenses. Each of the plurality of lenticular lenses may be formed in a semi-cylindrical shape and may be arranged in one direction at a predetermined tilt or angle of inclination in front of the display panel (210). With such a configuration, a parallax effect may occur as the user's gaze moves, thereby providing a three-dimensional image.

[0078] FIG. 2b is a drawing for explaining the configuration of a display panel with a lenticular function activated according to one embodiment.

[0079] Referring to FIG. 2b, the display panel (210) may include multiple pixels and may have a size in pixels. For example, the display panel (210) may have a width (w) (230) and a height (h) (234) in pixels.

[0080] According to one embodiment, a single pixel may include three subpixels represented by R, G, and B. The width (w) (230) of the display panel (210) may include a pixel offset (po) (232) for color display in pixel units (or RGB units).

[0081] According to one embodiment, multi-view images may be assigned to subpixels included in the remaining portion of the width (w) (230) of the display panel (210) excluding po (232). For example, odd view images may be arranged in the order of view 1, 3, 5, 7, 9, 13, 15, 17, 19, 21, 24, 25, 27, and even view images may be arranged in the order of view 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28. The width of each lenticular lens included in the lenticular lens array (220) may correspond to a horizontal rendering pitch in pixel (px) (240) based on the number of view.

[0082] Lenticular lenses constituting a lenticular lens array (220) may be placed in front of the subpixels to which image data is assigned. The lenticular lenses are inclined at a set angle of inclination (θ) (236), and a d (238) may be created between po (232) and px (240) by the angle of inclination (θ) (236).

[0083] Each lenticular lens can refract images provided by subpixels in different directions and deliver them to the user's left and right eyes. By providing left-eye and right-eye images with different viewpoints, the lenticular lenses can provide a three-dimensional image that matches the user's gaze.

[0084] According to one embodiment, when the lenticular function is deactivated in the display device (120), the liquid crystal layer is turned off so that the lenticular lens array (220) is not used and a 2D image can be provided. FIG. 3 is a diagram for explaining the first display operation of a display system according to one embodiment.

[0085] Referring to FIG. 3, a camera (130) may be positioned at an adjacent location of the display device (120) (e.g., the top of the display (121)). For example, the camera (130) may be a component of the display device (120) or may represent a camera connected via a set communication method (e.g., USB communication method). According to one embodiment, the camera (130) may output in real time image data of a user’s face or eyes in front of the display (121) as eye tracking data for eye tracking of the user. The eye tracking data output by the camera (130) may be provided to the source device (110). The camera (130) may be connected to the source device (110) via USB and may provide eye tracking data to the source device (110) based on USB communication. However, communication between the camera (130) and the source device (110) is not limited to USB communication and may be performed based on other wired or wireless communication.

[0086] According to one embodiment, the source device (110) may include an eye tracking module (302) and a view mapping module (304). The eye tracking module (302) and the view mapping module (304) may be components corresponding to the processor (116) of the source device (110), or independent components that operate based on the control of the processor (116).

[0087] According to one embodiment, the eye tracking module (302) receives eye tracking data from the camera (130) in real time and can perform eye tracking based on the received eye tracking data. The eye tracking module (302) can detect the location or direction of the user's gaze or the user's gaze by performing eye tracking.

[0088] According to one embodiment, the view mapping module (304) can perform view mapping based on the user's gaze. View mapping for a glasses-type 3D display may include the operation of outputting left and right frames alternately in time through glasses, or the operation of separating the left eye image and the right eye image with a polarizing filter to deliver different images to the left and right eyes, but view mapping for a glasses-free 3D display may include the following operations.

[0089] According to one embodiment, view mapping for a glasses-free 3D display may include an operation to generate a stereoscopic image or a glasses-free stereoscopic image corresponding to the user's gaze. View mapping may be performed at the subpixel level. For example, view mapping may include an operation to determine R, G, and B values ​​based on the ratio of the left eye and the right eye, respectively (hereinafter referred to as the 'L / R ratio'). When R, G, and B values ​​are generated at different ratios for the left eye and the right eye, a stereoscopic image reflecting binocular parallax may be provided.

[0090] According to one embodiment, the lenticular lens is configured in a semi-cylindrical shape and arranged diagonally by a set angle of inclination (e.g., angle of inclination (θ) (236) in FIG. 2b), so that a constant L / R ratio can be set for the subpixels. If the user's head moves around the display screen, the user's gaze changes, and thus the L / R ratio of each subpixel may change. Taking this into account, the view mapping module (304) can perform view mapping based on the set L / R ratio through real-time gaze tracking and provide image data corresponding to the result of the view mapping to the display device (120). The image data output by the view mapping module (304) of the source device (110) can be provided to the display device (120) via HDMI communication. However, HDMI communication is merely one example, and communication between the source device (110) and the display device (120) for transmitting and receiving image data may be performed based on other wired or wireless communication.

[0091] Video data provided to the display device (120) can be input to an SoC (322) included in the main board (320) of the display device (120). The SoC (322) may be included in the processor (126) of the display device (120), and the operation of the SoC (322) may substantially be the operation of the processor (126).

[0092] According to one embodiment, the SoC (322) may include a panel control module (306), an image processing module (308), and a graphics mix module (310). The panel control module (306), the image processing module (308), and the graphics mix module (310) may be composed of a single integrated module.

[0093] According to one embodiment, the panel control module (306) can control the operation mode of the display panel (210) included in the display (121). For example, the operation mode of the display panel (210) may include a 2D mode and a 3D mode. The 2D mode may be a mode in which the liquid crystal layer is turned off, a mode in which the lenticular function is disabled, a mode in which the lenticular lens array (220) cannot be used, a mode in which the operation of the lenticular lens array (220) is turned off, or a mode in which a flat image or a 2D image is displayed. The 3D mode may be a mode in which the liquid crystal layer is turned on, a mode in which the lenticular function is enabled, a mode in which the lenticular lens array can be used, a mode in which the operation of the lenticular lens array (220) is turned on, or a mode in which a stereoscopic image, a 3D image, or a view-mapped image is displayed.

[0094] According to one embodiment, the panel control module (306) can control the operating mode of the display panel (210) by controlling the liquid crystal layer based on a mode control signal received from the source device (110). The source device (110) can transmit a mode control signal instructing the display device (120) to operate in 3D mode based on the performance of eye tracking for view mapping. The mode control signal can be transmitted to the panel control module (306) of the display device (120) based on a set communication method (e.g., USB communication method). Based on the reception of the mode control signal, the panel control module (306) can control the display panel (210) to enter 3D mode.

[0095] According to one embodiment, the image quality processing module (308) can perform image quality processing on a view-mapped image. The view-mapped image may represent an image based on image data received from a source device (110) or an image corresponding to the user's gaze. The image quality processing module (308) can perform various technical processing to achieve clear and natural image quality for the view-mapped image. For example, the image quality processing module (308) can perform color or brightness correction, resolution adjustment, boundary processing, or image quality correction using a filter on the view-mapped image.

[0096] According to one embodiment, the graphic mix module (310) may perform a graphic mix operation to combine or synthesize the image data and graphic data so that the image and graphic information processed for image quality are displayed as a single screen or view. The graphic information may include, for example, any one of information related to the function settings of the display device (120), information related to the screen settings or function settings of the display (121), an OSD (on screen display) menu, or information related to the execution of an application.

[0097] According to one embodiment, the graphic mix module (310) may receive input for displaying graphic information through a user interface (e.g., the user interface (123) of FIG. 1) or perform a graphic mix operation based on set information. The graphic mix module (310) may provide image data generated by the graphic mix operation to a display panel (210).

[0098] According to one embodiment, the display panel (210) can enter a 3D mode based on the control of the panel control module (306), and in the 3D mode, can display a graphically mixed image based on image data provided from the graphic mix module (310). For example, the graphically mixed image may be displayed in a form where graphic information is displayed at a set location on the image with improved quality, or where the image with improved quality and graphic information are overlaid.

[0099] The first display operation described above can provide a stereoscopic image with added graphic information, but there may be some limitations. For example, when image quality processing is performed on a view-mapped image, pixel values ​​of the left-eye image and the right-eye image may be mixed due to operations associated with image quality processing or filters used for image quality processing. When pixel values ​​of the left-eye image and the right-eye image are mixed, a crosstalk phenomenon that interferes with the perception of the stereoscopic image may occur. Therefore, to prevent crosstalk, image quality processing must be performed within a very limited range. In addition, since the first display operation does not support 3D graphics, there is a limitation in that 2D graphics must be used for graphic mixing. Even if graphic mixing using 2D graphics is performed, the 2D graphics may be divided between the left and right eyes, and a crosstalk phenomenon caused by the parallax between the left and right eyes may occur at the boundary of the high-resolution graphics. Therefore, a second display operation may be performed to improve image quality and prevent crosstalk, or to support 2D graphics and 3D graphics. The second display operation will be explained below with reference to FIG. 4.

[0100] FIG. 4 is a diagram illustrating the second display operation of a display system according to one embodiment.

[0101] Referring to FIG. 4, the source device (110) may include an eye tracking module (404) and a view mapping module (406). The eye tracking module (404) and the view mapping module (406) may be configured similarly to the eye tracking module (302) and the view mapping module (304) of FIG. 3. For example, the eye tracking module (404) may receive eye tracking data in real time and track the user's gaze based on the received eye tracking data. The view mapping module (406) may perform view mapping on an image based on the user's gaze and transmit image data associated with the view-mapped image to a display device (120) via HDMI communication (or other wired or wireless communication).

[0102] According to one embodiment, the eye tracking module (404) may receive eye tracking data directly from the camera (130) as in the eye tracking module (302) of FIG. 3, or receive eye tracking data through the display device (120) as shown in FIG. 4. The reception of eye tracking data may be performed in real time and may be carried out based on a communication method (e.g., USB communication method) set between the source device (110) and the display device (120).

[0103] According to one embodiment, the source device (110) may transmit image data for a different type of image other than the view-mapped image to the display device (120). For example, the other type of image may be a non-view-mapped image and may include a side-by-side image (hereinafter referred to as 'side-by-side') or a stereo image associated with a 3D image.

[0104] According to one embodiment, a camera (130) included in or connected to a display device (120) can output video data capturing a user's face or eyes in real time as eye tracking data for tracking the user's gaze. The eye tracking data output by the camera (130) can be input to an SoC (422) included in the main board (320) of the display device (120). The SoC (422) may be included in the processor (126) of the display device (120), and the operation of the SoC (422) may substantially be the operation of the processor (126).

[0105] The SoC (422) illustrated in FIG. 4 may be configured differently from the SoC (322) illustrated in FIG. 3. According to one embodiment, the SoC (422) may include an eye tracking module (402), an image quality processing module (408), a graphics mix control module (410), a view mapping module (412), a 3D mix module (414), a 2D mix module (416), and a panel control module (418). The eye tracking module (402), the image quality processing module (408), the graphics mix control module (410), the view mapping module (412), the 3D mix module (414), the 2D mix module (416), and the panel control module (418) may be configured as a single integrated module.

[0106] According to one embodiment, the eye tracking module (402) can perform eye tracking based on eye tracking data provided from the camera (130). The eye tracking module (402) can perform eye tracking to detect the location or direction of the user's gaze or the user's gaze.

[0107] According to one embodiment, the image quality processing module (408) can perform image quality processing on a view-mapped image (e.g., an image of image data received from a source device (110) or an image of view-mapped from a display device (120)) or an image of input image data (e.g., a side-by-side or stereotype image). The image quality processing module (408) can perform various technical processing to achieve a clear and natural image quality. For example, the image quality processing module (408) can perform color or brightness correction, resolution adjustment, boundary processing, or image quality correction using a filter on the image.

[0108] According to one embodiment, the graphic mix control module (410) can perform a graphic mix operation that combines or synthesizes the image data and the graphic data so that the image and the graphic information are displayed as a single screen or view. The graphic information may include, for example, any one of information related to the function settings of the display device (120), information related to the screen settings or function settings of the display (121), OSD menu information, or information related to the execution of an application. The graphic mix control module (410) can control the 3D mix module (414) or the 2D mix module (416) for the graphic mix operation.

[0109] According to one embodiment, the 3D mix module (414) may perform a graphic mix operation that combines image data of an image with a processed image or an input image (e.g., side-by-side or stereo type image data) with 3D type graphic data (e.g., side-by-side or stereo type graphic data) prior to view mapping. The graphically mixed image data output from the 3D mix module (414) may be input to the view mapping module (412) and used for view mapping.

[0110] According to one embodiment, the 2D mix module (416) can perform a graphic mix operation after view mapping to combine image data of a view-mapped image or a view-mapped bypass image (e.g., a non-3D image or a 2D image) with 2D type graphic data. The graphic mix image data output from the 2D mix module (416) is provided to a display panel (210) and can be used to display a 3D or 2D image containing 2D graphics.

[0111] According to one embodiment, the view mapping module (412) can perform view mapping based on eye tracking information provided by the eye tracking module (402) and graphic mixed image data output from the 3D mix module (414). For example, the view mapping module (412) can provide a stereoscopic image by selecting and displaying a left eye image and a right eye image corresponding to the user's gaze in units of R, G, and B subpixels based on graphic mixed image data (e.g., side-by-side or stereotype image data), or by summing and outputting R, G, and B values ​​determined by their respective L / R ratios.

[0112] According to one embodiment, the panel control module (418) can control the operation mode of the display panel (210) included in the display (121). For example, the operation mode of the display panel (210) may include a 2D mode and a 3D mode. The 2D mode may be a mode in which the liquid crystal layer is turned off, a mode in which the lenticular function is disabled, a mode in which the lenticular lens array (220) cannot be used, a mode in which the operation of the lenticular lens array (220) is turned off, or a mode in which a flat image or a 2D image is displayed. The 3D mode may be a mode in which the liquid crystal layer is turned on, a mode in which the lenticular function is enabled, a mode in which the lenticular lens array (220) can be used, a mode in which the operation of the lenticular lens array (220) is turned on, or a mode in which a stereoscopic image, a 3D image, or a view-mapped image is displayed.

[0113] According to one embodiment, the panel control module (418) can control the operating mode of the display panel (210) by controlling the liquid crystal layer based on a mode control signal received from the source device (110), as in the panel control module (306) of FIG. 3, or control the operating mode of the display panel (210) based on whether the eye tracking operation of the eye tracking module (402) is performed.

[0114] For example, the panel control module (418) can control the display panel (210) to enter 3D mode based on receiving a mode control signal or performing an eye tracking operation of the eye tracking module (402).

[0115] For example, the panel control module (418) can control the display panel (210) to enter 2D mode based on whether a mode control signal is not received or whether the eye tracking operation of the eye tracking module (402) is not performed.

[0116] FIG. 5 is a diagram illustrating an image displayed by view mapping according to one embodiment.

[0117] FIG. 5(a) shows an example of a side-by-side image for generating a 3D image. The side-by-side image may be an image in which multiple (e.g., two) images are arranged side by side. The multiple images may include a first image (510) and a second image (520) arranged left and right. The first image (510) and the second image (520) are images with a difference in viewpoint and may be referred to as a left-eye image and a right-eye image associated with the user's left-eye gaze and right-eye gaze.

[0118] According to one embodiment, in order to display a 3D image based on the user's gaze on a display (121) such as an LFD, view mapping (525) using side-by-side images may be performed. View mapping (525) may be performed by a source device (110) or a display device (120).

[0119] According to one embodiment, view mapping (525) may include the operation of mapping a first image (510) and a second image (520) included in a side-by-side image according to the ratio of the L / R images in subpixel units. The ratio of the L / R images may be a ratio representing how much light is divided and provided to the left eye image (e.g., the first image (510)) and the right eye image (e.g., the second image (520)) by the corresponding subpixel, and may be determined per subpixel (or per R, G, B). By determining the ratio of light to be provided to the left eye and the right eye in subpixel units, a sense of depth or 3D effect of the image may be realized.

[0120] According to one embodiment, when view mapping (525) is performed, a view-mapped image (530) as shown in FIG. 5 (b) can be generated. For example, the view-mapped image (530) may appear as if the first image (510) and the second image (520) included in the side-by-side image are superimposed on one screen and may be perceived as a stereoscopic image by the user's eye.

[0121] FIG. 6 is a drawing illustrating various types of graphics according to one embodiment.

[0122] According to one embodiment, various types of graphics may be used to be displayed together with an image (e.g., a 3D image, a view-mapped image, or a 2D image). For example, the graphics may be side-by-side type graphics associated with a 3D graphic as shown in FIG. 6 (a), or 2D graphics as shown in FIG. 6 (b).

[0123] According to one embodiment, a side-by-side type graphic may include a first graphic (610) and a second graphic (620). The side-by-side type graphic may be view-mapped after being mixed with a side-by-side image and graphic.

[0124] According to one embodiment, if the first graphic (610) and the second graphic (620) are graphics with a viewpoint difference, a 3D graphic can be displayed after view mapping.

[0125] According to one embodiment, if the first graphic (610) and the second graphic (620) are graphics with no difference in viewpoint (or are the same graphics), a 2D graphic can be displayed after view mapping.

[0126] According to one embodiment, FIG. 6 (a) and (b) illustrate that the graphic represents an OSD menu (e.g., a setting menu associated with the screen (screen mode, backlight, contrast, brightness, sharpness, saturation, color, or screen adjustment), sound, network, or system), but this is only one example, and the information that can be represented by the graphic may vary. For example, the graphic may represent information associated with the function settings of the display device (120) or information associated with the execution of an application.

[0127] FIG. 7 is a diagram illustrating the operation of generating a 3D image with added 2D graphics through view mapping according to one embodiment.

[0128] Referring to FIG. 7(a), the display device (120) can perform a 3D mix of side-by-side images and side-by-side graphics. The side-by-side images may include a first image (710) and a second image (720) with a viewpoint difference, and the side-by-side graphics may include a first graphic (712) and a second graphic (722). The first image (710) and the second image (720) may be a left-eye image and a right-eye image with a viewpoint difference, and the first graphic (712) and the second graphic (722) may be the same graphic without a viewpoint difference.

[0129] According to one embodiment, the 3D mix performed by the display device (120) may represent an operation of combining or synthesizing side-by-side images and side-by-side graphics in a 3D position to provide images of different viewpoints to the user's left and right eyes. The 3D mix may be performed to include images and graphics in a single screen or a single view. Based on the 3D mix operation, the side-by-side graphics may be displayed overlaid on the side-by-side images. For example, the first graphic (712) and the second graphic (722) may be displayed overlaid on the first image (710) and the second image (720), respectively. According to one embodiment, the first graphic (712) and the second graphic (722) may be displayed transparently, translucently, or opaquely on the first image (710) and the second image (720), based on transparency indicated by a set alpha value.

[0130] According to one embodiment, the display device (120) can perform view mapping (725) on a 3D mixed image. When view mapping (725) is performed, a view-mapped image as shown in FIG. 7 (b) can be generated.

[0131] Referring to FIG. 7(b), the view-mapped image may include an image (730) based on a first image (710) and a second image (720), and a graphic (732) based on a first graphic (712) and a second graphic (722). The first image (710) and the second image (720), which have a difference in viewpoint, may appear as if they are superimposed on one screen and may be perceived as a stereoscopic image by the user's eye. The first graphic (712) and the second graphic (722), which do not have a difference in viewpoint, may be represented as flat graphics such as 2D graphics.

[0132] FIG. 8 is a diagram illustrating the operation of generating a 3D image with added 3D graphics through view mapping according to one embodiment.

[0133] Referring to FIG. 8(a), the display device (120) can perform a 3D mix of side-by-side images and side-by-side graphics. The side-by-side images may include a first image (810) and a second image (820) with different viewpoints, and the side-by-side graphics may include a first graphic (812) and a second graphic (822) with different viewpoints.

[0134] According to one embodiment, based on the 3D mix operation of the display device (120), the side-by-side graphics may be displayed by overlaying them on the side-by-side image. For example, the first graphic (812) and the second graphic (822) may be displayed by overlaying them on the first image (810) and the second image (820), respectively. According to one embodiment, the first graphic (812) and the second graphic (822) may be displayed transparently, translucently, or opaquely on the first image (810) and the second image (820) based on the transparency indicated by a set alpha value.

[0135] According to one embodiment, the display device (120) can perform view mapping (825) on a 3D mixed image. When view mapping (825) is performed, a view-mapped image as shown in FIG. 8 (b) can be generated.

[0136] Referring to FIG. 8(b), the view-mapped image may include an image (830) based on a first image (810) and a second image (820), and a graphic (832) based on a first graphic (812) and a second graphic (822). The first image (810) and the second image (820), which have different viewpoints, may appear as if they are superimposed on one screen and may be perceived as a stereoscopic image by the user's eye. The first graphic (712) and the second graphic (722), which have different viewpoints, may also appear as if they are superimposed on one screen and may be perceived as a stereoscopic graphic by the user's eye.

[0137] FIG. 9 is a schematic diagram illustrating the operation of a display device according to one embodiment generating various stereoscopic images.

[0138] According to one embodiment, the display device (120) can process various types of image data. The image data may be provided from the source device (110) or stored in the display device (120). The display device (120) can perform operations associated with any one of (a) to (c) of FIG. 9 for each type of image data.

[0139] Referring to FIG. 9(a), the display device (120) can identify a first type of image data associated with a 3D image (e.g., side-by-side image) (910) as image data to be processed. The display device (120) can mix (e.g., 3D mix) (912) the first type of image data with graphic data associated with a 3D graphic (e.g., side-by-side graphic) (914). According to one embodiment, the graphic data associated with the 3D graphic (914) can be mixed (912) with the first type of image data based on setting information associated with graphic information display or user selection. When the 3D image (910) and the 3D graphic (914) are combined by the mix (912), the display device (120) can perform view mapping (916) considering the user's gaze to generate a first stereoscopic image containing the 3D image (910) and the 3D graphic (914) (918).

[0140] Referring to FIG. 9(b), the display device (120) can identify a first type of image data associated with a 3D image (e.g., side-by-side image) (920) as image data to be processed. The display device (120) can mix (e.g., 3D mix) (922) the first type of image data with graphic data associated with a 3D graphic (e.g., left-right identical side-by-side graphic) (924). According to one embodiment, the graphic data associated with the 3D graphic (924) can be mixed (922) with the first type of image data based on setting information associated with graphic information display or user selection. When the 3D image (920) and the 3D graphic (924) are combined by the mix (922), the display device (120) can perform view mapping (926) that takes into account the user's gaze. According to one embodiment, since the graphic data corresponds to a side-by-side graphic containing identical left and right graphics, the display device (120) can generate a 2D graphic rather than a 3D graphic through view mapping (926). Based on this, the display device (120) can generate a second stereoscopic image (928) containing a 3D image (920) and a 2D graphic (924) after view mapping (926).

[0141] Referring to FIG. 9(c), the display device (120) can identify a second type of image data associated with a 3D image (e.g., view-mapped image) (930) as image data to be processed. The display device (120) can mix the second type of image data with graphic data associated with 2D graphics (934) (e.g., 2D mix) (932). According to one embodiment, graphic data associated with 2D graphics (934) can be mixed (932) with the second type of image data based on setting information associated with graphic information display or user selection. When the 3D image (930) and 3D graphics (934) are combined by the mix (932), the display device (120) can output (936) without a separate view mapping operation to generate a third stereoscopic image containing the 3D image (930) and 2D graphics (934) (938).

[0142] Although not illustrated in FIG. 9, the display device (120) may perform the following operations. According to one embodiment, the display device (120) may first perform view mapping on first type image data associated with a 3D image (e.g., side-by-side image). The display device (120) may mix the view-mapped image data with graphic data associated with 2D graphics (e.g., 2D mix). According to one embodiment, the graphic data associated with 2D graphics may be mixed with second type image data based on setting information associated with graphic information display or user selection. When the 3D image and 2D graphics corresponding to the view-mapped image are combined by mixing, the display device (120) may generate a fourth stereoscopic image containing the 3D image and 2D graphics.

[0143] The operation of a display device (120) will be described below with reference to FIGS. 10 to 12. According to one embodiment, the operations illustrated in FIGS. 10 to 12 may be understood to be performed by a processor (126) of the display device (120). The operations illustrated in FIGS. 10 to 12 may be performed in various orders, not limited to the order in which they are illustrated. According to one embodiment, at least some of the operations illustrated in FIGS. 10 to 12 may be omitted, or more operations may be performed than those illustrated in FIGS. 10 to 12.

[0144] FIG. 10 is a flowchart illustrating the operation of a display device according to one embodiment displaying a first stereoscopic image.

[0145] Referring to FIG. 10, in operation 1002, when image data is input, the display device (120) can identify that the type of the image data is a first type. According to one embodiment, the image data may be received from a source device (110) or stored in the memory (124) of the display device (120). According to one embodiment, the first type may indicate a type associated with a side-by-side image in which a plurality of images with a viewpoint difference are arranged side by side, as shown in FIG. 5 (a).

[0146] According to one embodiment, the display device (120) can identify setting information associated with the display of graphic information based on the fact that the type of image data is a first type. For example, the display device (120) receives an input instructing the display of graphic information through a user interface (123) and can identify setting information associated with the display of graphic information based on the received input. According to one embodiment, the graphic information may include any one of information associated with the function setting of the display device (120), information associated with the screen setting or function setting of the display (121), an OSD menu, or information associated with the execution of an application.

[0147] In operation 1004, the display device (120) can identify that the setting information associated with the display of graphic information includes first information. For example, the setting information may be set by a user, and the first information included in the setting information may include information instructing to display the graphic information as 3D graphics.

[0148] In operation 1006, the display device (120) may combine (or 3D mix) first graphic data corresponding to a first type with image data to generate a first image containing 3D images and 3D graphics. According to one embodiment, the first graphic data corresponding to a first type may represent side-by-side graphic data as shown in FIG. 6 (a).

[0149] In operation 1008, the display device (120) can generate a first stereoscopic image corresponding to the user's gaze based on the user's gaze tracking information obtained through the camera (130) and the first image. According to one embodiment, the camera (130) may represent a camera within at least one sensor (140) that is included in the display device (120) or connected to the display device (122). The camera (130) may be positioned at a location adjacent to the display (121) (e.g., at the top of the display (121)), or at a location where it can photograph the user's face or eyes. The camera (130) may photograph the user's face or eyes at the position where it is placed to provide at least one image or gaze tracking data for generating the user's gaze tracking information.

[0150] According to one embodiment, the display device (120) can generate a first stereoscopic image through view mapping in which R, G, and B values ​​per subpixel of the display (121) based on the first image are determined in correspondence with the ratio of the left eye and the right eye based on eye tracking information, and the determined R, G, and B values ​​are assigned per subpixel. According to one embodiment, the first stereoscopic image may include a glasses-free stereoscopic image or a glasses-free 3D image that can be viewed without glasses.

[0151] In operation 1010, the display device (120) can provide a 3D image and a 3D graphic on a single screen by displaying a first stereoscopic image on the display (121). According to one embodiment, the 3D graphic may be displayed overlaid on the 3D image. For example, the 3D graphic may be displayed transparently, translucently, or opaquely based on transparency with an alpha value at a set position on the 3D image.

[0152] FIG. 11 is a flowchart illustrating the operation of a display device according to one embodiment displaying a second stereoscopic image.

[0153] Referring to FIG. 11, in operation 1102, the display device (120) can identify that the type of image data is a first type. According to one embodiment, the display device (120) can identify setting information associated with the display of graphic information based on the fact that the type of image data is a first type.

[0154] In operation 1104, the display device (120) can identify that the setting information associated with the display of graphic information includes second information. For example, the second information included in the setting information may include information instructing to display the graphic information as a 2D graphic.

[0155] In operation 1106, the display device (120) may combine (or 3D mix) second graphic data corresponding to the second type with image data to generate a second image containing 3D images and 2D graphics. According to one embodiment, the second type may indicate a type associated with a side-by-side image in which a plurality of images with no difference in viewpoint are arranged side by side.

[0156] In operation 1108, the display device (120) can generate a second stereoscopic image corresponding to the user's gaze based on the gaze tracking information obtained through the camera (130) and the second image. According to one embodiment, the display device (120) can generate the second stereoscopic image through view mapping in which R, G, and B values ​​per subpixel of the display (121) based on the second image are determined in correspondence with the ratio of the left eye and the right eye based on the gaze tracking information, and the determined R, G, and B values ​​are assigned per subpixel. According to one embodiment, the second stereoscopic image may include a glasses-free stereoscopic image or a glasses-free 3D image that can be viewed without glasses.

[0157] In operation 1110, the display device (120) can provide a 3D image and a 2D graphic on a single screen by displaying a second stereoscopic image on the display (121). According to one embodiment, the 2D graphic may be displayed overlaid on the 3D image. For example, the 2D graphic may be displayed transparently, translucently, or opaquely based on transparency with an alpha value at a set position on the 3D image.

[0158] FIG. 12 is a flowchart illustrating the operation of a display device according to one embodiment displaying a third stereoscopic image.

[0159] Referring to FIG. 12, in operation 1202, the display device (120) can identify that the type of image data is a third type. According to one embodiment, the third type may indicate a type associated with an image (or view-mapped image) generated in response to the user's gaze at the source device (110).

[0160] In operation 1204, the display device (120) can acquire a third stereoscopic image corresponding to the user's gaze based on image data.

[0161] In operation 1206, the display device (120) may combine (or 2D mix) the second graphic data corresponding to the fourth type with the third stereoscopic image to update the third stereoscopic image to include 2D graphics. According to one embodiment, the fourth type may indicate a type associated with 2D graphics. For example, the second graphic data corresponding to the fourth type may represent graphic data of 2D graphics as illustrated in FIG. 6 (b). According to one embodiment, the operation of combining the second graphic data with the third stereoscopic image may be performed based on user input indicating the display of graphic information.

[0162] In operation 1208, the display device (120) can provide a 3D image and a 2D graphic on a single screen by displaying the updated third stereoscopic image on the display (121). According to one embodiment, the 2D graphic may be displayed overlaid on the 3D image. For example, the 2D graphic may be displayed transparently, translucently, or opaquely based on transparency by an alpha value at a set position on the 3D image. According to one embodiment, the updated third stereoscopic image may include a glasses-free stereoscopic image or a glasses-free 3D image that can be viewed without glasses.

[0163] Based on the various embodiments described above, the display device (120) may be able to implement not only 2D graphics but also 3D graphics in a glasses-free stereoscopic display, and even if 2D graphics are displayed together with 3D images, the occurrence of crosstalk may be prevented.

[0164] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0165] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0166] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

In a display device, display; Memory; and It includes at least one processor that executes instructions of a program stored in the memory above, The above-mentioned at least one processor is, Identify the type of image data, and Based on the fact that the type of the above image data is a first type, setting information associated with the display of graphic information is identified, and Based on the fact that the above setting information includes the first information, the first graphic data corresponding to the first type is combined with the image data to generate a 3D (three-dimensional) image and a first image including 3D graphics, and A first stereoscopic image corresponding to the user's gaze is generated based on the user's gaze tracking information obtained through at least one sensor and the first image, and A display device configured to display the first stereoscopic image on the display. In paragraph 1, It further includes a transceiver, A display device comprising at least one sensor including a camera connected to the display device through the communication unit. In paragraph 1, The above-mentioned at least one processor is, A display device configured to generate a first stereoscopic image through view mapping, wherein R(red), G(green), and B(blue) values ​​for each subpixel of the display based on the first image are determined in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and the determined R, G, and B values ​​are assigned to each subpixel. In paragraph 1, It further includes a transceiver, A display device comprising the above image data, which includes image data received from a source device through the communication unit or image data stored in the memory. In paragraph 1, It further includes a user interface, The above-mentioned at least one processor is, It is configured to receive an input instructing the display of the graphic information through the above user interface, and to identify setting information associated with the display of the graphic information based on the input, A display device comprising any one of the above graphic information, information associated with a function setting of the display device, information associated with a screen setting or function setting of the display, an OSD (on screen display) menu, or information associated with an application execution. In paragraph 1, The above-mentioned at least one processor is, Based on the fact that the above setting information includes second information, second graphic data corresponding to the second type is combined with the image data to generate a second image including the 3D image and 2D (two-dimensional) graphics, and Based on the above eye tracking information and the above second image, a second stereoscopic image corresponding to the user's gaze is generated, and A display device configured to display the above-mentioned second stereoscopic image on the display. In paragraph 6, A display device in which the above second type indicates a type associated with a side-by-side image in which a plurality of images without a difference in viewpoint are arranged side by side. In paragraph 6, The above-mentioned at least one processor is, A display device configured to generate a second stereoscopic image through view mapping, wherein R(red), G(green), and B(blue) values ​​for each subpixel of the display based on the second image are determined in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and the determined R, G, and B values ​​are assigned to each subpixel. In paragraph 1, The above-mentioned at least one processor is, Based on the fact that the type of the above image data is a third type, a third stereoscopic image corresponding to the user's gaze is obtained based on the above image data, and Combining second graphic data corresponding to the fourth type with the third stereoscopic image, and updating the third stereoscopic image to include 2D graphics, and A display device configured to display the above-mentioned updated third stereoscopic image on the display. In Paragraph 9, It further includes a transceiver, The above video data is received from a source device through the communication unit, and The above third type indicates a type associated with an image generated in the source device in correspondence with the user's gaze, and The above-mentioned fourth type is a display device indicating a type associated with 2D graphics. In a method of operating a display device, An operation to identify the type of input image data; An operation to identify setting information associated with the display of graphic information based on the fact that the type of the above image data is a first type; Based on the fact that the above setting information includes the first information, the operation of combining the first graphic data corresponding to the first type with the image data to generate a first image including a 3D (three-dimensional) image and 3D graphics; The operation of generating a first stereoscopic image corresponding to the user's gaze based on user gaze tracking information obtained through at least one sensor and the first image; and A method of operating a display device, comprising the operation of displaying the first stereoscopic image on the display of the display device. In Paragraph 11, A method of operation of a display device, wherein the first type above indicates a type associated with a side-by-side image in which a plurality of images with a viewpoint difference are arranged side by side. In Paragraph 11, The operation of generating the first stereoscopic image above is, A method of operation of a display device comprising the operation of determining R(red), G(green), and B(blue) values ​​for each subpixel of the display based on the first image in correspondence with the ratio of the left eye and the right eye based on the eye tracking information, and generating the first stereoscopic image through view mapping that assigns the determined R, G, and B values ​​to each subpixel. In Paragraph 11, A method of operating a display device, wherein the above image data includes image data received from a source device or image data stored in the display device. In Paragraph 11, The operation of identifying setting information associated with the display of the above graphic information is, An operation of receiving an input that directs the display of the above graphic information; and It includes an operation of identifying setting information associated with the display of the graphic information based on the above input, and A method of operating a display device, wherein the above graphic information includes any one of information associated with a function setting of the display device, information associated with a screen setting or function setting of the display, an OSD (on screen display) menu, or information associated with an application execution.

Images