Method for displaying stereoscopic image

Abstract

There is provided a method of displaying a stereoscopic image, the method comprising the steps of: generating a three-dimensional model; mapping each sub-pixel among a plurality of sub-pixels of the display panel to a corresponding viewpoint among the viewpoints to generate a mapping table; generating rendered image data for each of the viewpoints by rendering the three-dimensional model using the mapping table; and generating stereoscopic image data based on the rendered image data for each of the viewpoints.

Description

[0001] This patent application claims priority to Korean Patent Application No. 10-2023-0085164, filed on June 30, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The disclosed embodiments generally relate to a method for displaying a stereoscopic image and a display system for performing the method. More specifically, the disclosed embodiments relate to a method for displaying a stereoscopic image generated by stereoscopic image data and a display system for performing the method. Background Art

[0003] The display device serves as a connection medium between a user and information. Examples of the display device include a liquid crystal display device and an organic light emitting display device.

[0004] A stereoscopic image display device is a display device that creates an illusion of depth in an image. For example, a stereoscopic image display device can provide different images to the left and right eyes of a viewer so that the viewer can view a stereoscopic image through binocular parallax between the left and right eyes.

[0005] An autostereoscopic method is a method for displaying a stereoscopic image without using glasses. Examples of the autostereoscopic method include a lens array method for separating a left-eye image and a right-eye image using a cylindrical lens array and a barrier method for separating a left-eye image and a right-eye image using a barrier. However, generating a stereoscopic image using the autostereoscopic method may take a long time. Summary of the invention

[0006] An object of the disclosure is to provide a method for displaying a stereoscopic image which increases the speed of generating stereoscopic image data.

[0007] Another object of the disclosure is to provide a display system for executing the method of displaying a stereoscopic image.

[0008] According to the disclosed embodiment, the method for displaying a stereoscopic image includes the following steps: generating a three-dimensional model; mapping each sub-pixel among a plurality of sub-pixels of a display panel to a corresponding viewpoint in viewpoints to generate a mapping table; generating rendering image data for each of the viewpoints by rendering the three-dimensional model using the mapping table; and generating stereoscopic image data based on the rendering image data for each of the viewpoints.

[0009] In an embodiment, the three-dimensional model may be rendered with respect to sub-pixels mapped to the same viewpoint among the viewpoints.

[0010] In an embodiment, the three-dimensional model may be rendered with respect to sub-pixels mapped to at least two adjacent viewpoints among the viewpoints.

[0011] In an embodiment, three-dimensional image data may be generated by summing the rendered image data for each of the viewpoints.

[0012] In an embodiment, the step of generating the three-dimensional model comprises generating the three-dimensional model having a first resolution, and the method may further comprise the step of upscaling the rendered image data from the first resolution to a second resolution higher than the first resolution.

[0013] In an embodiment, the second resolution may be a resolution of the display panel.

[0014] In an embodiment, the upscaling step may be performed by a super-resolution algorithm utilizing deep learning.

[0015] In an embodiment, the rendered image data for a moving image may be upscaled by an ultra-high-speed upscaling algorithm, and the rendered image data for a still image may be upscaled by a high-definition upscaling algorithm.

[0016] In an embodiment, the ultra-high speed upscaling algorithm may utilize bicubic interpolation.

[0017] In an embodiment, the high-definition upscaling algorithm may utilize a fast super-resolution convolutional neural network.

[0018] According to the disclosed embodiment, the method for displaying a stereoscopic image includes the following steps: generating a three-dimensional model with a first resolution; generating rendered image data for each of a plurality of viewpoints by rendering the three-dimensional model; upscaling the rendered image data from the first resolution to a second resolution higher than the first resolution; and generating stereoscopic image data by mapping each of the viewpoints of the rendered image data to sub-pixels.

[0019] In an embodiment, the second resolution may be a resolution of a display panel including sub-pixels.

[0020] In an embodiment, the upscaling step may be performed by a super-resolution algorithm utilizing deep learning.

[0021] In an embodiment, the rendered image data for a moving image may be upscaled by an ultra-high-speed upscaling algorithm, and the rendered image data for a still image may be upscaled by a high-definition upscaling algorithm.

[0022] In an embodiment, the ultra-high speed upscaling algorithm may utilize bicubic interpolation.

[0023] In an embodiment, the high-definition upscaling algorithm may utilize a fast super-resolution convolutional neural network.

[0024] A display system according to a disclosed embodiment includes a main processor and a display device. The main processor is configured to generate stereoscopic image data based on image data. The display device displays a stereoscopic image based on the stereoscopic image data. The display device includes a display panel including sub-pixels, a lens superimposed with the sub-pixels, and a display panel driver configured to drive the display panel. The main processor processes the image data to generate a three-dimensional model, maps each of the sub-pixels to a corresponding viewpoint in the viewpoint to generate a mapping table, renders the three-dimensional model using the mapping table to generate rendered image data for each of the viewpoints, and generates stereoscopic image data based on the rendered image data for each of the viewpoints.

[0025] In an embodiment, the three-dimensional model may be rendered with respect to sub-pixels mapped to the same viewpoint among the viewpoints.

[0026] In an embodiment, the three-dimensional model may be rendered with respect to sub-pixels mapped to at least two adjacent viewpoints among the viewpoints.

[0027] In an embodiment, a three-dimensional model having a first resolution is generated, and a display panel driver scales up the rendered image data from the first resolution to a second resolution higher than the first resolution.

[0028] The method of displaying a stereoscopic image according to the disclosed embodiment may improve rendering speed by rendering a three-dimensional model with respect to sub-pixels for each viewpoint.

[0029] The method of displaying a stereoscopic image according to the disclosed embodiment may improve a rendering speed by rendering a three-dimensional model generated by modeling image data at a low resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other features of the disclosure will become more apparent by further describing the disclosed embodiments in detail with reference to the accompanying drawings, in which:

[0031] Figure 1 is a flow chart showing a method of displaying a stereoscopic image according to a disclosed embodiment;

[0032] Figure 2 It shows that according to Figure 1 A block diagram of a display system for a method of displaying a stereoscopic image;

[0033] Figure 3 and Figure 4 is a diagram showing a display device for displaying a stereoscopic image using a lens array method;

[0034] Figure 5 It is shown Figure 2 A block diagram of an example of a host processor;

[0035] Figure 6 It is shown Figure 2 A diagram showing an example of a structure of a sub-pixel;

[0036] Figure 7 It shows that according to Figure 1 FIG. 1 is a diagram showing an example of a mapping table of a method for displaying a stereoscopic image;

[0037] Figure 8 It shows that according to Figure 1 FIG. 1 is a diagram showing an example of a rendering of a method for displaying a stereoscopic image;

[0038] Fig. 9 is a diagram showing an example of rendering according to a method of displaying a stereoscopic image according to a disclosed embodiment;

[0039] Fig.10 is a flowchart illustrating a method of displaying a stereoscopic image according to a disclosed embodiment;

[0040] Fig.11 It shows that according to Fig.10 A block diagram of an example of a main processor of a method for displaying a stereoscopic image;

[0041] Fig.12 It shows that according to Fig.10 FIG. 1 is an enlarged example of a method of displaying a stereoscopic image;

[0042] Fig.13 is a flowchart illustrating a method of displaying a stereoscopic image according to a disclosed embodiment; and

[0043] Fig.14 It shows that according to Fig.13 A block diagram of an example of a host processor of a method of displaying a stereoscopic image. DETAILED DESCRIPTION

[0044] Hereinafter, the embodiments disclosed are described in detail with reference to the accompanying drawings. It should be noted that in the following description, only the parts necessary for understanding the operation disclosed are described, and the description of other parts is omitted to avoid blurring the disclosed subject matter. In addition, the disclosure may be implemented in other forms, and is not limited to the embodiments described herein. However, the embodiments described herein are provided with enough details to enable those of ordinary skill in the art to implement the disclosed technical spirit.

[0045] Throughout the specification, when a part is "connected" to another part, this case includes not only a case where the parts are "directly connected" but also a case where the parts are "indirectly connected" and another element is placed between them. The terms used herein are used to describe specific embodiments and are not intended to limit the disclosure. Here, "and / or" includes all combinations of one or more of the corresponding configurations.

[0046] Spatially relative terms such as "under ...", "on ...", etc. may be used for descriptive purposes to describe the relationship between an element or feature and another element or feature as shown in the drawings. Spatially relative terms are intended to include other directions in use, operation, and / or manufacture in addition to the directions depicted in the drawings. For example, when the device shown in the drawings is turned over, the elements depicted as being "under" other elements or features are positioned in the direction "on" other elements or features. Therefore, in an embodiment, the term "under ..." may include both upper and lower directions. In addition, the device may face other directions (e.g., rotated 90 degrees or in other directions), so the spatially relative terms used herein are interpreted accordingly.

[0047] Figure 1 is a flowchart illustrating a method of displaying a stereoscopic image according to a disclosed embodiment.

[0048] Reference Figure 1 According to an embodiment, the method for displaying a stereoscopic image includes generating a three-dimensional (3D) model (S110). In one embodiment, the three-dimensional model may be generated by converting (i.e., modeling) two-dimensional (2D) image data captured by a camera into three-dimensional image data (i.e., a three-dimensional model). In another embodiment, the three-dimensional model may be generated by converting (i.e., modeling) two-dimensional image data captured by a virtual camera into three-dimensional image data (i.e., a three-dimensional model). In another embodiment, the three-dimensional model may be generated without two-dimensional image data. According to an embodiment, Figure 1 The method further includes mapping each of the plurality of viewpoints of the three-dimensional model to a sub-pixel (S120). For example, the sub-pixel may correspond to a sub-pixel of a display panel that emits light of different colors. According to an embodiment, Figure 1 The method further includes generating rendered image data for each of the viewpoints by rendering the three-dimensional model (S130). Generating the rendered image data may include processing the three-dimensional model using the result of the mapping. According to an embodiment, Figure 1 The method further includes generating stereoscopic image data based on the rendered image data for each of the viewpoints (S140).

[0049] In the following, reference is made to Figures 2 to 8 The specific description is disclosed.

[0050] Figure 2 It shows that according to Figure 1 A block diagram of a display system of a method for displaying a stereoscopic image.

[0051] Reference Figure 2, the display system may include a main processor 1100 and a display device 2000. For example, the main processor 1100 may include one or more of a central processing unit (CPU) and an application processor (AP). The main processor 1100 may also include any one or more of a graphics processing unit (GPU), a communication processor (CP), and an image signal processor (ISP). The main processor 1100 may also include a neural processing unit (NPU). The NPU may be a processor dedicated to processing an artificial intelligence model, and the artificial intelligence model may be generated by machine learning. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recursive deep neural network (BRDNN), a deep Q network, or a combination of two or more of the foregoing, but is not limited to the above examples. Additionally or optionally, in addition to the hardware structure, the artificial intelligence model may also include a software structure. At least two of the above-mentioned processing units and processors may be implemented as an integrated structure (e.g., a single chip), or each may be implemented as an independent structure (e.g., multiple chips).

[0052] The main processor 1100 may generate stereoscopic image data SIMG. For example, when the stereoscopic image data SIMG includes 12 viewpoints, the stereoscopic image data SIMG may be image data that is a combination of 12 two-dimensional image data.

[0053] The display device 2000 may include a display panel 100 and a display panel driver (e.g., a panel driver circuit). The display panel driver may include a drive controller 200 (e.g., a controller circuit), a gate driver 300 (e.g., a first driver circuit), and a data driver 400 (e.g., a second driver circuit). In an embodiment, the drive controller 200 and the data driver 400 may be integrated into one chip.

[0054] The display panel 100 may include a display area DA for displaying an image and a non-display area NDA disposed adjacent to the display area DA. In an embodiment, the gate driver 300 may be installed in the non-display area NDA.

[0055] The display panel 100 may include a plurality of gate lines GL, a plurality of data lines DL, and a plurality of sub-pixels SP electrically connected to the gate lines GL and the data lines DL. The gate lines GL may extend in a first direction D1, and the data lines DL may extend in a second direction D2 crossing the first direction D1.

[0056] The driving controller 200 may receive stereoscopic image data SIMG and an input control signal CONT from the main processor 1100. For example, the stereoscopic image data SIMG may include red image data, green image data, and blue image data. In an embodiment, the stereoscopic image data SIMG may also include white image data. As another example, the stereoscopic image data SIMG may include magenta image data, yellow image data, and cyan image data. The input control signal CONT may include a main clock signal and a data enable signal. The input control signal CONT may also include a vertical synchronization signal and a horizontal synchronization signal.

[0057] The driving controller 200 may generate a first control signal CONT1 , a second control signal CONT2 , and a data signal DATA based on the stereoscopic image data SIMG and the input control signal CONT.

[0058] The driving controller 200 may generate a first control signal CONT1 for controlling the operation of the gate driver 300 based on the input control signal CONT and output the first control signal CONT1 to the gate driver 300. The first control signal CONT1 may include a vertical start signal and a gate clock signal.

[0059] The driving controller 200 may generate a second control signal CONT2 for controlling the operation of the data driver 400 based on the input control signal CONT and output the second control signal CONT2 to the data driver 400. The second control signal CONT2 may include a horizontal start signal and a load signal.

[0060] The driving controller 200 may generate a data signal DATA by receiving the stereoscopic image data SIMG and the input control signal CONT. The driving controller 200 may output the data signal DATA to the data driver 400.

[0061] The gate driver 300 may generate a gate signal for driving the gate line GL in response to the first control signal CONT1 received from the driving controller 200. The gate driver 300 may output the gate signal to the gate line GL. For example, the gate driver 300 may sequentially output the gate signal to the gate line GL.

[0062] The data driver 400 may receive the second control signal CONT2 and the data signal DATA from the driving controller 200. The data driver 400 may generate a data voltage obtained by converting the data signal DATA into an analog voltage. The data driver 400 may output the data voltage to the data line DL.

[0063] Figure 3 and Figure 4FIG. 1 is a diagram showing a display device for displaying a stereoscopic image using a lens array method.

[0064] Reference Figures 2 to 4 , the display device 2000 may display images of a plurality of viewpoints P. For example, different images may be recognized by the viewer according to the viewing direction in which the viewer views the display panel 100 .

[0065] The display device 2000 may include a display panel 100 and a lens LS. The sub-pixels SP may be arranged in a first direction D1 and a second direction D2 intersecting the first direction D1. The lens LS may span a plurality of sub-pixels SP in a third direction D3 intersecting the first direction D1 and the second direction D2. For example, the lens LS may overlap the sub-pixels SP in the third direction D3 or in a plan view.

[0066] The display panel 100 may include sub-pixels SP that emit light to display an image. In an embodiment, each of the sub-pixels SP may output one of light of a first color (e.g., red), light of a second color (e.g., green), and light of a third color (e.g., blue). However, this is an example, and the color of light emitted from the sub-pixel SP is not limited thereto, and light of various colors for full-color realization may be output. The display panel 100 may include an organic light emitting display panel, a liquid crystal display panel, a quantum dot display panel, etc.

[0067] The lens LS may refract light incident from the sub-pixel SP. For example, the lens array including the lens LS may be implemented as a lenticular lens array, a micro lens array, or the like.

[0068] A light field display is a 3D display device that realizes a stereoscopic image by forming a light field represented by the vector distribution (intensity, direction) of light in space using a flat panel display and an optical element (e.g., a lens LS). Since a light field display can realize a more natural stereoscopic image by enabling the depth, side surface, etc. of an object to be viewed, a light field display is a display technology that is expected to be utilized in various applications by merging with augmented reality (AR) technology, etc.

[0069] The light field can be implemented in various ways. For example, the light field can be formed by using a method of creating light fields in various directions using multiple projectors, a method of controlling the direction of light using a diffraction grating, a method of controlling the direction and intensity (brightness) of light according to the combination of each pixel using two or more panels, a method of controlling the direction of light using a pinhole or a barrier, a method of controlling the refraction direction of light through a lens array, etc.

[0070] In the embodiment, a display device displaying a stereoscopic image of a lenticular array method displays a stereoscopic image (three-dimensional image) by forming a light field.

[0071] A series or subset of sub-pixels SP may be assigned to a corresponding lens LS in the lens LS, and light emitted from each of the sub-pixels SP in the series or subset may be refracted by the corresponding lens LS, may travel only in a specific direction, and thus may form a light field represented by the intensity and direction of light. When a viewer views the display device 2000 within the light field formed as described above, the viewer may perceive a three-dimensional effect of the corresponding image.

[0072] Figure 3 and Figure 4 The display device 2000 is shown to display three viewpoints P, but the disclosure is not limited to this number of viewpoints P.

[0073] Figure 5 It is shown Figure 2 An example block diagram of a host processor is shown in Figure 1. Figure 6 It is shown Figure 2 FIG. 1 is an example of a structure of a sub-pixel, Figure 7 It shows that according to Figure 1 A diagram of an example of a mapping table of a method of displaying a stereoscopic image, and Figure 8 It shows that according to Figure 1 FIG. 1 is a diagram of an example of a rendering of a method of displaying a stereoscopic image.

[0074] Reference Figure 5 In an embodiment, the main processor 1100 includes a 3D modeler 1110, a 3D mapper 1120, a 2D renderer 1130, and an image summer 1140. In an embodiment, each of the 3D modeler 1110, the 3D mapper 1120, the 2D renderer 1130, and the image summer 1140 is implemented by a corresponding logic circuit.

[0075] The three-dimensional modeler 1110 can generate a three-dimensional model TDM. For example, the three-dimensional model TDM (e.g., model data) can include coordinate information and depth information of the object. For example, the data of the three-dimensional model TDM can be composed of three-dimensional coordinate values ​​and color and grayscale values ​​at each three-dimensional coordinate value. However, the data format of the three-dimensional model TDM of the present invention is not limited.

[0076] Reference Figures 5 to 7 In an embodiment, the three-dimensional mapper 1120 maps each of the viewpoints P1 to P12 of the three-dimensional model TDM to some of the sub-pixels SP. In an embodiment, each sub-pixel SP of at least a portion of the display area DA is mapped to a corresponding one of the viewpoints P1 to P12. The three-dimensional mapper 1120 may map one of the viewpoints P1 to P12 to each of the sub-pixels SP according to the direction of light emission from each of the sub-pixels SP. In an embodiment, information about the direction of emission may be received from the outside. In an embodiment, the three-dimensional mapper 1120 may map the viewpoints P1 to P12 to each of the sub-pixels SP through the display panel 100 (see Figure 4 ) and lens LS (see Figure 4 ) structure determines the emission direction. In this case, information about the display panel 100 (see Figure 4 ) and lens LS (see Figure 4 ) structure information.

[0077] In an embodiment, the three-dimensional mapper 1120 may receive viewpoints P1 to P12 displayed by each of the sub-pixels, and map the viewpoints P1 to P12 to each of the sub-pixels. For example, the three-dimensional mapper 1120 may generate a mapping table MT by receiving viewpoints P1 to P12 displayed by each of the sub-pixels SP. In another embodiment, the main processor 1100 may receive the mapping table MT from the outside. In this case, the main processor may not include the three-dimensional mapper 1120.

[0078] One pixel unit PU may include a first color sub-pixel R displaying a first color, a second color sub-pixel G displaying a second color, and a third color sub-pixel B displaying a third color. Figure 7 Shown according to Figure 6 The structure of the sub-pixel SP is mapped to the viewpoints P1 to P12 of each of the sub-pixels SP. However, the disclosure is not limited to the structure of the sub-pixel SP. For example, all sub-pixels of the pixel unit PU may be mapped to the same viewpoint among the viewpoints P1 to P12, but the disclosed embodiments are not limited thereto.

[0079] The three-dimensional mapper 1120 may determine a sub-pixel SP on which each of the viewpoints P1 to P12 of the three-dimensional model TDM is displayed. For example, the first color sub-pixel R of the first pixel unit PU1 may display the third viewpoint P3 of the three-dimensional model TDM. For example, the second color sub-pixel G of the first pixel unit PU1 may display the third viewpoint P3 of the three-dimensional model TDM. For example, the third color sub-pixel B of the first pixel unit PU1 may display the third viewpoint P3 of the three-dimensional model TDM.

[0080] like Figure 4 As shown in , light emitted from each of the sub-pixels SP may be refracted in a specific direction (i.e., a specific viewpoint). Therefore, the three-dimensional mapper 1120 may generate a mapping table MT by mapping each of the viewpoints P1 to P12 to some of the sub-pixels SP. For example, the mapping table MT may map a first subset of the sub-pixels SP to a first viewpoint P1, a second subset of the sub-pixels SP that is different from the first subset to a second viewpoint P2, and so on.

[0081] Reference Figure 5 and Figure 8, the two-dimensional renderer 1130 may generate rendered image data RIMG for each of the viewpoints P1 to P12 by rendering the three-dimensional model TDM. For example, the rendering may include processing the three-dimensional model TDM using a mapping table MT. The rendered image data RIMG may be two-dimensional image data for each of the viewpoints P1 to P12 of the three-dimensional model TDM. For example, the display device may display the rendered image data RIMG for the first viewpoint P1 on the sub-pixel SP that emits light refracted to the first viewpoint P1, may display the rendered image data RIMG for the second viewpoint P2 on the sub-pixel SP that emits light refracted to the second viewpoint P2, and so on. For example, the two-dimensional renderer 1130 determines which image data portion of the three-dimensional model TDM is associated with the first viewpoint P1, accesses the mapping table MT to determine which sub-pixels are associated with the first viewpoint P1, and generates first rendered image data for the determined sub-pixel from the determined image data portion so that it can be displayed on the determined sub-pixel later. In Figure 7 In the example shown in , the two-dimensional renderer 1130 determines that the first to third subpixels in the first row, the tenth to twelfth subpixels in the third row, and the fourth to sixth subpixels in the sixth row are associated with the first viewpoint P1; and generates first rendered image data for the determined subpixels from the portion of the three-dimensional model TDM associated with the first viewpoint P1.

[0082] The two-dimensional renderer 1130 may determine which image data portion of the three-dimensional model TDM is associated with any viewpoint based on the three-dimensional coordinates.

[0083] The three-dimensional model TDM may be rendered relative to the sub-pixels SP mapped to the same viewpoint among the viewpoints P1 to P12. For example, the rendered image data RIMG for the first viewpoint P1 may include data (e.g., pixel data) of an image displayed on the sub-pixels SP mapped to the first viewpoint P1. That is, when generating the rendered image data RIMG for the first viewpoint P1, the two-dimensional renderer 1130 may only consider the sub-pixels SP mapped to the first viewpoint P1 instead of all the sub-pixels SP. Therefore, the rendering of the three-dimensional model TDM may reduce the rendering time compared to generating the rendered image data RIMG for each of the viewpoints P1 to P12 relative to all the sub-pixels SP.

[0084] The image summer 1140 may generate stereoscopic image data SIMG based on the rendered image data RIMG for each of the viewpoints P1 to P12. In an embodiment, the stereoscopic image data SIMG may be generated by summing the rendered image data RIMG for each of the viewpoints P1 to P12. For example, the rendered image data RIMG for the viewpoint P1 may include zero grayscale values ​​for sub-pixels of the viewpoints P2 to P12, the rendered image data RIMG for the viewpoint P2 may include zero grayscale values ​​for sub-pixels of the viewpoints P1 and P3 to P12, and the like.

[0085] For example, the rendered image data RIMG for the first viewpoint P1 may include data for an image displayed on a sub-pixel SP mapped to the first viewpoint P1, and the rendered image data RIMG for the second viewpoint P2 may include data for an image displayed on a sub-pixel SP mapped to the second viewpoint P2. The stereoscopic image data SIMG generated by summing the rendered image data RIMG for the first viewpoint P1 to the twelfth viewpoint P12 may include data for images displayed on all sub-pixels SP.

[0086] For example, the two-dimensional renderer 1130 may perform processing at a first time to generate first rendered image data RIMG for a first viewpoint P1, perform processing at a second time to generate second rendered image data RIMG for a second viewpoint P2, ..., perform processing at a twelfth time to generate twelfth rendered image data RIMG for a twelfth viewpoint P12, and the image summer 1140 may sum the first to twelfth rendered image data RIMG to generate stereoscopic image data SIMG.

[0087] Fig. 9 is a diagram illustrating an example of rendering according to a method of displaying a stereoscopic image according to an embodiment of the disclosure.

[0088] Since the method for displaying a stereoscopic image according to this embodiment is different from the method for displaying a stereoscopic image according to the present embodiment except for rendering, Figure 1 The configuration of the method of displaying a stereoscopic image is substantially the same, and thus the same reference numerals and reference numerals are used for the same or similar components and repeated descriptions are omitted.

[0089] Reference Figures 4 to 7 and Fig. 9, the two-dimensional renderer 1130 generates rendering image data RIMG for each of the viewpoints P1 to P12 by rendering the three-dimensional model TDM. The rendering image data RIMG may be two-dimensional image data for each of the viewpoints P1 to P12 of the three-dimensional model TDM. For example, the display device may display the rendering image data RIMG for the first viewpoint P1 on the sub-pixel SP that emits the light refracted to the first viewpoint P1, and may display the rendering image data RIMG for the second viewpoint P2 on the sub-pixel SP that emits the light refracted to the second viewpoint P2.

[0090] The three-dimensional model TDM may be rendered with respect to sub-pixels SP mapped to at least two adjacent viewpoints (eg, a first viewpoint P1 and a second viewpoint P2 ) among the viewpoints P1 to P12 .

[0091] During the manufacturing process, the lens LS may be partially misaligned with the display panel 100. In this case, the mapping table MT may be corrected. For example, Figure 7 It can be the mapping table MT before correction, and Fig. 9 It may be the mapping table MT after correction.

[0092] For example, when the lens LS is partially misaligned, the light emitted from the third color sub-pixel B of the first pixel unit PU1 may be refracted to both the direction corresponding to the second viewpoint P2 and the direction corresponding to the third viewpoint P3. In this case, the mapping table MT for the third color sub-pixel B of the first pixel unit PU1 may be corrected from the third viewpoint P3 to the 2.5th viewpoint P2.5 (or P2 / 3). For example, the sub-pixels in the mapping table MT may be marked as corresponding to two different viewpoints. In addition, the rendered image data RIMG for the third viewpoint P3 may include pixel data of the third color sub-pixel B of the first pixel unit PU1 determined based on the pixel data before correction of the third color sub-pixel B of the first pixel unit PU1 and the pixel data before correction of the third color sub-pixel B of the second pixel unit PU2. Here, the second pixel unit PU2 may be adjacent to the first pixel unit PU1, and the third color sub-pixel B of the second pixel unit PU2 may be mapped to the second viewpoint P2 adjacent to the third viewpoint P3 mapped to the third color sub-pixel B of the first pixel unit PU1 before correction. In addition, the pixel data may be a grayscale value.

[0093] That is, when generating the rendering image data RIMG for the third viewpoint P3, the two-dimensional renderer 1130 may only consider the sub-pixels SP mapped to the third viewpoint P3 and the second viewpoint P2 instead of all the sub-pixels SP. Therefore, the rendering of the three-dimensional model TDM can reduce the rendering time compared to generating the rendering image data RIMG for each of the viewpoints P1 to P12 with respect to all the sub-pixels SP.

[0094] Fig.10 is a flowchart illustrating a method of displaying a stereoscopic image according to a disclosed embodiment.

[0095] Reference Fig.10 , a method for displaying a stereoscopic image includes: generating a three-dimensional model having a first resolution (S210); generating rendered image data for each of a plurality of viewpoints by rendering the three-dimensional model (S220); performing upscaling of the rendered image data from the first resolution to a second resolution higher than the first resolution (S230); and generating stereoscopic image data by mapping each of the viewpoints of the rendered image data to sub-pixels (S240).

[0096] In the following, reference is made to Fig.11 and Fig.12 The specific description is disclosed.

[0097] Fig.11 It shows that according to Fig.10 A block diagram of an example of a main processor of a method of displaying a stereoscopic image, and Fig.12 It shows that according to Fig.10 FIG. 1 is an enlarged example of a method of displaying a stereoscopic image.

[0098] Reference Fig.11 , the main processor 1200 includes a 3D modeler 1210, a 2D renderer 1220, an amplifier 1230, and a 3D mapper 1240. In an embodiment, each of the 3D modeler 1210, the 2D renderer 1220, the amplifier 1230, and the 3D mapper 1240 is implemented by a corresponding logic circuit.

[0099] The 3D modeler 1210 may generate a 3D model TDM. For example, the 3D model TDM may include coordinate information and depth information of an object.

[0100] In an embodiment, the 3D modeler 1210 generates a 3D model TDM having a first resolution R1. In an embodiment, the first resolution R1 is lower than a second resolution R2 of the display panel 100. That is, a process of enlarging the first resolution R1 to the second resolution R2 may be performed to display the 3D model TDM on the display panel 100.

[0101] The two-dimensional renderer 1220 may generate rendered image data RIMG for each of the viewpoints P1 to P12 by rendering the three-dimensional model TDM. In an embodiment, the rendered image data RIMG is two-dimensional image data for each of the viewpoints P1 to P12 of the three-dimensional model TDM. The rendered image data RIMG may be data for an image displayed at each of the viewpoints P1 to P12.

[0102] The two-dimensional renderer 1220 can generate the rendering image data RIMG of the first resolution R1 by rendering the three-dimensional model TDM of the first resolution R1. Therefore, the two-dimensional renderer 1220 can reduce the rendering time compared with the case where the rendering image data RIMG of the second resolution R2 is generated by rendering the three-dimensional model TDM of the second resolution R2 (i.e., the resolution of the display panel).

[0103] The amplifier 1230 may amplify the rendered image data RIMG of the first resolution R1 to a second resolution R2 higher than the first resolution R1. In an embodiment, the second resolution R2 is the resolution of the display panel 100. For example, the amplification may be performed by a super-resolution algorithm using deep learning. However, the disclosure is not limited to such an amplification method.

[0104] In the embodiments, reference Fig.11 and Fig.12 , the rendered image data RIMG_MI for a moving image may be enlarged by an ultra-high-speed enlargement algorithm, and the rendered image data RIMG_SI for a still image may be enlarged by a high-definition enlargement algorithm.

[0105] For example, the second resolution may be twice the first resolution. For example, the second resolution may be three times the first resolution. For example, the second resolution may be four times the first resolution. As the difference between the second resolution and the first resolution increases, the rendering speed may be further increased.

[0106] The amplifier 1230 may include an image divider 1231 (eg, a first logic circuit), a first amplifier 1232 (eg, a second logic circuit), and a second amplifier 1233 (eg, a third logic circuit).

[0107] The image divider 1231 can distinguish whether the rendered image data RIMG is image data for a moving image or image data for a still image. In an embodiment, the image divider 1231 can distinguish between a moving image and a still image by comparing the rendered image data RIMG of a previous frame with the rendered image data RIMG of a current frame. In an embodiment, the image divider 1231 can distinguish between a moving image and a still image by comparing the three-dimensional model TDM. For example, the image divider 1231 may include a comparator for performing the comparison.

[0108] The first amplifier 1232 may amplify the rendered image data RIMG_MI for the moving image by an ultra-high speed up-scaling algorithm. Compared with the high-definition up-scaling algorithm, the ultra-high speed up-scaling algorithm may perform up-scaling at a relatively high speed. For example, the ultra-high speed up-scaling algorithm may utilize bicubic interpolation.

[0109] The second amplifier 1233 may amplify the rendered image data RIMG_SI for the still image by a high-definition upscaling algorithm. Compared with the ultra-high-speed upscaling algorithm, the high-definition upscaling algorithm may upscale with relatively high display quality. For example, the high-definition upscaling algorithm may utilize a fast super-resolution convolutional neural network.

[0110] Reference Fig.11 , the three-dimensional mapper 1240 may generate stereoscopic image data SIMG by mapping each of the viewpoints of the rendered image data RIMG to sub-pixels.

[0111] The three-dimensional mapper 1240 may determine sub-pixels of each of the viewpoints on which the rendered image data RIMG is displayed. In addition, the three-dimensional mapper 1240 may generate stereoscopic image data SIMG by summing the rendered image data RIMG for each of the viewpoints.

[0112] Fig.13 is a flowchart illustrating a method of displaying a stereoscopic image according to a disclosed embodiment.

[0113] Since the method of displaying a stereoscopic image according to the present embodiment is different from the method of displaying a stereoscopic image according to the present embodiment in addition to modeling and enlarging. Figure 1 The configuration of the method of displaying a stereoscopic image is substantially the same, and thus the same reference numerals and reference numerals are used for the same or similar components, and repeated descriptions are omitted.

[0114] Reference Fig.13 According to an embodiment, a method for displaying a stereoscopic image includes: generating a three-dimensional model having a first resolution (S210); mapping each of a plurality of viewpoints of the three-dimensional model to a sub-pixel (S120); generating rendered image data for each of the viewpoints by rendering the three-dimensional model (S130); upscaling the rendered image data from the first resolution to a second resolution higher than the first resolution (S230); and generating stereoscopic image data based on the rendered image data for each of the viewpoints (S140).

[0115] In the following, reference is made to Fig.14 The specific description is disclosed.

[0116] Fig.14 It shows that according to Fig.13 A block diagram of an example of a host processor of a method of displaying a stereoscopic image.

[0117] Reference Fig.14In an embodiment, the main processor 1300 includes a 3D modeler 1310, a 3D mapper 1320, a 2D renderer 1330, an amplifier 1340, and an image summer 1350. In an embodiment, each of the 3D modeler 1310, the 3D mapper 1320, the 2D renderer 1330, the amplifier 1340, and the image summer 1350 is implemented by a logic circuit.

[0118] The three-dimensional modeler 1310 may generate a three-dimensional model TDM. For example, the three-dimensional model TDM may include coordinate information and depth information of an object.

[0119] In an embodiment, the three-dimensional modeler 1310 generates a three-dimensional model TDM having a first resolution R1. In an embodiment, the first resolution R1 is lower than a second resolution R2 of the display panel 100. That is, a process of enlarging the first resolution R1 to the second resolution R2 may be performed to display the three-dimensional model TDM on the display panel 100.

[0120] The amplifier 1340 may upscale the rendered image data RIMG from the first resolution R1 to a second resolution R2 higher than the first resolution R1. In an embodiment, the second resolution R2 is a resolution of the display panel 100. However, the disclosure is not limited to this upscaling method.

[0121] The disclosure may be applied to display devices and electronic devices including display devices. For example, the disclosure may be applied to digital TVs, 3D TVs, mobile phones, smart phones, tablet computers, VR devices, PCs, home electronic devices, notebook computers, PDAs, PMPs, digital cameras, music players, portable game consoles, navigation systems, etc.

[0122] Although various embodiments have been described above, it will be understood that various modifications and changes may be made to the disclosure by those skilled in the art without departing from the spirit and scope of the disclosure as described in the appended claims.

Claims

1. A method for displaying a stereoscopic image, the method comprising the following steps: Generate 3D models; mapping each sub-pixel among a plurality of sub-pixels of the display panel to a corresponding viewpoint among the viewpoints to generate a mapping table; generating rendered image data for each of the viewpoints by rendering the three-dimensional model using the mapping table; as well as Stereoscopic image data is generated based on the rendered image data for each of the viewpoints.

2. The method according to claim 1, wherein: The three-dimensional model is rendered relative to sub-pixels mapped to a same viewpoint among the viewpoints.

3. The method according to claim 1, wherein: The three-dimensional model is rendered relative to sub-pixels mapped to at least two adjacent viewpoints among the viewpoints.

4. The method according to claim 1, wherein: Three-dimensional image data is generated by summing the rendered image data for each of the viewpoints.

5. The method according to claim 1, wherein: The step of generating the three-dimensional model comprises generating the three-dimensional model with a first resolution, and The method further includes the step of upscaling the rendered image data from a first resolution to a second resolution higher than the first resolution.

6. The method according to claim 5, wherein: The second resolution is a resolution of the display panel.

7. The method according to claim 5, wherein: The upscaling step is performed by a super-resolution algorithm that leverages deep learning.

8. The method according to claim 5, wherein: enlarging the rendered image data for the moving image by an ultra-high-speed enlarging algorithm, and The rendered image data for a still image is upscaled by a high-resolution upscaling algorithm.

9. The method according to claim 8, wherein: The ultra-high-speed upscaling algorithm utilizes bicubic interpolation.

10. The method according to claim 8, wherein: The high-resolution upscaling algorithm utilizes a fast super-resolution convolutional neural network.

Images