Mapping function extraction system and method, display device, and computer readable medium
By scanning and comparing the appearance data of multi-channel lenses to extract correction data and applying it to a standard mapping function, the problem of virtual reality image distortion in head-mounted displays is solved, achieving a fast and effective correction effect.
Patent Information
- Application Number
- CN202410278714.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-24
- Filing Date
- 2024-03-12
- Publication Date
- 2024-11-26
AI Technical Summary
In existing technologies, virtual reality images from head-mounted displays are prone to distortion, and the mapping process is time-consuming, making it difficult to quickly and effectively correct the distortion.
Appearance data is acquired by scanning a multi-channel lens, compared with standard appearance data to generate correction data, and applied to a standard mapping function to extract the mapping function, which is used to correct distortion in virtual reality images.
Quickly extract mapping functions, reduce mapping process time, effectively prevent virtual reality image distortion, and improve image display quality.
Smart Images

Figure CN119027319A_ABST
Abstract
Description
[0001] This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0067082, filed on May 24, 2023, which is hereby incorporated by reference in its entirety. Technical Field
[0002] One or more embodiments described herein relate to a mapping function extraction system, a mapping function extraction method, and a display device. Background Art
[0003] Electronic devices including displays have been designed to be worn on the human body. These electronic devices are generally referred to as wearable devices and are used to improve user convenience, portability, and accessibility. A head-mounted display (or head-mounted electronic device (HMD)) is an example of a wearable electronic device. The HMD can provide virtual reality (VR) images by magnifying a source image through a lens. Summary of the invention
[0004] The embodiments provide a mapping function extraction system and a mapping function extraction method, which can quickly extract a mapping function for preventing distortion of an image (such as but not limited to a VR image).
[0005] The embodiment also provides a display device capable of reducing the time of a mapping process and effectively preventing distortion of a VR image by using the above-mentioned mapping function.
[0006] According to one embodiment of the present invention, a mapping function extraction system is provided, which includes: a scanning unit (or scanner) configured to generate appearance data of a multi-channel lens by scanning the multi-channel lens; a comparing unit (or comparator) configured to generate correction data by comparing the appearance data with pre-stored standard appearance data; and an extraction unit (or extractor) configured to generate a mapping function by applying the correction data to a pre-stored standard mapping function.
[0007] The multi-channel lens may include a plurality of sub-lenses through which light emitted from the display member passes.
[0008] The appearance data may include appearance information of a recessed portion formed by a plurality of sub-lenses.
[0009] The standard appearance data may include appearance information of a multi-channel lens having an ideal design.
[0010] The correction data may be derived from the difference between the standard appearance data and the appearance data.
[0011] The standard mapping function may be a function for calculating position information of pixels corresponding to a VR image to be provided through a multi-channel lens having an ideal design.
[0012] The mapping function may be extracted corresponding to the change in the appearance data.
[0013] According to one embodiment of the present invention, a mapping function extraction method is provided, which includes the following steps: generating appearance data of a multi-channel lens by scanning the multi-channel lens; generating correction data by comparing the appearance data with pre-stored standard appearance data; and extracting a mapping function by applying the correction data to a pre-stored standard mapping function.
[0014] The multi-channel lens may include a plurality of sub-lenses through which light emitted from the display member passes.
[0015] The appearance data may include appearance information of a recessed portion formed by a plurality of sub-lenses.
[0016] The standard appearance data may include appearance information of a multi-channel lens having an ideal design.
[0017] The correction data may be derived from the difference between the standard appearance data and the appearance data.
[0018] The standard mapping function may be a function for calculating position information of pixels corresponding to a VR image to be provided through a multi-channel lens having an ideal design.
[0019] The mapping function may be extracted corresponding to the change in the appearance data.
[0020] According to one embodiment of the present invention, a display device is provided, which includes: a display unit (or display), including pixels; a lens unit (or lens arrangement), including at least one multi-channel lens, and the at least one multi-channel lens is configured to provide a VR image by refracting and reflecting an image displayed on the display unit; and a control unit (or controller), which is configured to generate mapping data including position information of pixels corresponding to the VR image based on a pre-stored mapping function and VR image data, wherein the difference between the standard appearance data of the multi-channel lens and the appearance data of the multi-channel lens is applied to the mapping function.
[0021] The mapping function may be extracted corresponding to the change in the appearance data.
[0022] The standard appearance data may include appearance information of a multi-channel lens having an ideal design.
[0023] The appearance data may include appearance information through a scanned multi-channel lens.
[0024] The appearance data may include appearance information of a recessed portion formed by a plurality of sub-lenses included in the multi-channel lens.
[0025] The control unit may provide the mapping data to the display unit. The display unit may display an image based on the mapping data.
[0026] According to one or more embodiments, a computer-readable medium stores instructions, which, when executed by at least one processor, causes the at least one processor to: generate appearance data of a multi-channel lens by scanning the multi-channel lens; generate correction data by comparing the appearance data with pre-stored standard appearance data; and generate a mapping function by applying the correction data to a pre-stored standard mapping function. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Example embodiments will now be described more fully hereinafter with reference to the accompanying drawings; however, they may be implemented in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the example embodiments to those skilled in the art.
[0028] In the accompanying drawings, dimensions may be exaggerated for clarity of illustration. It will be understood that when an element is referred to as being "between" two elements, the element may be the only element between the two elements, or one or more intervening elements may also be present. The same reference numerals refer to the same elements throughout.
[0029] Figure 1 is a block diagram illustrating a mapping function extraction system according to an embodiment of the present disclosure.
[0030] Figure 2 and Figure 3 is a view showing a multi-channel lens according to an embodiment of the present invention.
[0031] Figure 4 is a view showing a mapping process according to an embodiment of the present invention.
[0032] Figure 5 is a flowchart illustrating a mapping function extraction method according to an embodiment of the present invention.
[0033] Figure 6 is a perspective view of a display device according to an embodiment of the present invention.
[0034] Figure 7 is a plan view of a display device according to an embodiment of the present invention.
[0035] Figure 8 is a side view of a portion of a display device according to an embodiment of the present invention.
[0036] Fig. 9 is a block diagram showing a display device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The invention will now be described more fully hereinafter with reference to the accompanying drawings, in which various embodiments are shown. However, the invention may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art.
[0038] When describing the accompanying drawings, the same reference numerals have been used for the same elements. In the accompanying drawings, in order to clearly explain the invention, the size of the structure is larger than the actual size. It will be understood that although the terms "first", "second", etc. can be used to describe various elements here, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the invention, the first element discussed below can be referred to as the second element. Similarly, the second element can also be referred to as the first element.
[0039] The terms used herein are only used for the purpose of describing specific embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, as used herein, "one", "one (kind / person)", "the (described)" and "at least one (kind / person)" do not represent the limitation of quantity, but are intended to include both the singular and the plural. For example, unless the context clearly indicates otherwise, "(one) element" has the same meaning as "at least one element". "At least one (kind / person)" will not be interpreted as limiting "one" or "one (kind / person)". "Or" means "and / or". As used herein, the term "and / or" includes any combination and all combinations of one or more related listed items. It will also be understood that when the term "comprising" and its variations or "including" and its variations are used in this specification, it is explained that there are stated features, regions, wholes, steps, operations, elements and / or components, but it is not excluded that there are or add one or more other features, regions, wholes, steps, operations, elements, components and / or their groups.
[0040] In the following description, when a first part is "connected" to a second part, this includes not only a case where the first part is directly connected to the second part but also a case where a third part is interposed therebetween and they are connected to each other.
[0041] In addition, relative terms such as "lower" or "bottom" and "upper" or "top" may be used herein to describe the relationship of one element to another element as shown in the accompanying drawings. It will be understood that relative terms are intended to cover different orientations of the device in addition to the orientation depicted in the accompanying drawings. For example, if the device in one of the accompanying drawings is turned over, the element described as being on the "lower" side of the other elements will subsequently be oriented on the "upper" side of the other elements. Therefore, depending on the specific orientation of the accompanying drawings, the term "lower" can cover both "lower" and "upper" orientations. Similarly, if the device in one of the accompanying drawings is turned over, the element described as being "below" or "below" the other elements will subsequently be oriented as "above" the other elements. Therefore, the term "below..." or "below..." can cover both "above" and "below" orientations.
[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It will also be understood that terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless explicitly so defined herein.
[0043] Hereinafter, embodiments of the invention will be described in detail with reference to the accompanying drawings.
[0044] Figure 1 is a block diagram illustrating a mapping function extraction system 100 according to an embodiment of the present invention. Figure 2 and Figure 3 is a view showing a multi-channel lens MCL according to an embodiment of the present invention. Figure 4 is a view showing a mapping process according to an embodiment of the present invention.
[0045] Reference Figure 1 , the mapping function extraction system 100 may include a scanning unit (or scanner) 110 , a comparing unit (or comparator) 120 , and an extracting unit (or extractor) 130 .
[0046] The scanning unit 110 may scan the multi-channel lens MCL. To this end, the multi-channel lens MCL may be provided to the scanning unit 110. Figure 2 and Figure 3, the multi-channel lens MCL may include a plurality of sub-lenses. For example, the multi-channel lens MCL may include four sub-lenses SL1, SL2, SL3, and SL4. Each of the four sub-lenses SL1, SL2, SL3, and SL4 may have a shape including a curved surface. However, the number and shape of the sub-lenses included in the multi-channel lens MCL are not limited to four as in the case of the above example.
[0047] The scanning unit 110 may include a 3D scanner for three-dimensionally scanning the multi-channel lens MCL. For example, the 3D scanner may include a contact 3D scanner or a non-contact 3D scanner. The non-contact 3D scanner may include a 3D laser scanner using a predetermined method such as an optical triangulation method, a 3D scanner using a white light method, a 3D scanner using a modulated light method, etc. However, the 3D scanner included in the scanning unit 110 is not limited to the above examples, and different methods may be used in other embodiments.
[0048] The scanning unit 110 may generate appearance data AD of the multi-channel lens MCL by scanning the multi-channel lens MCL. The appearance data AD may include physical information related to the appearance of the multi-channel lens MCL (hereinafter, referred to as appearance information). For example, the appearance data AD may include appearance information about the curvature, surface profile (or surface roughness), volume, etc. of the multi-channel lens MCL. The scanning unit 110 may provide the appearance data AD to the comparison unit 120.
[0049] Reference Figure 2 and Figure 3 In an embodiment, the appearance data AD may include appearance information of a recessed portion RP formed by a plurality of sub-lenses SL1, SL2, SL3, and SL4 included in the multi-channel lens MCL. The recessed portion RP may be a concave portion formed in a region where the plurality of sub-lenses SL1, SL2, SL3, and SL4 intersect or contact each other. The VR image VR (e.g., see Figure 4 ) may be further disturbed by the interference between the plurality of sub-lenses SL1, SL2, SL3 and SL4 at the recessed portion RP. Therefore, in order to effectively prevent the distortion of the VR image VR, the appearance data AD may include appearance information of the recessed portion RP.
[0050] The comparison unit 120 may compare the appearance data AD of the multi-channel lens MCL with predetermined, reference or standard appearance data RAD. To this end, the standard appearance data RAD may be provided to the comparison unit 120. In one embodiment (discussed in more detail below), the standard appearance data RAD may be pre-stored in the comparison unit 120.
[0051] The standard appearance data RAD may include appearance information of a multi-channel lens MCL corresponding to a predetermined design (such as, but not limited to, an ideal design). For example, the standard appearance data RAD may include appearance information of a multi-channel lens MCL manufactured by injection molding. Due to defects or other aspects considered during the manufacturing process of the multi-channel lens MCL, a tolerance may occur between the appearance data AD and the standard appearance data RAD. The tolerance may be converted into distortion in a virtual image and may be corrected using the embodiments described herein.
[0052] The comparison unit 120 may generate the correction data CD by comparing the appearance data AD with the standard appearance data RAD. For example, the comparison unit 120 may process the above-mentioned tolerance occurring between the appearance data AD and the standard appearance data RAD in the form of data. Therefore, the correction data CD may include information about the tolerance occurring between the appearance data AD and the standard appearance data RAD or be based on information about the tolerance occurring between the appearance data AD and the standard appearance data RAD. For example, the correction data CD may be derived from the difference between the standard appearance data RAD and the appearance data AD or be based on the difference between the standard appearance data RAD and the appearance data AD. The comparison unit 120 may provide the correction data CD to the extraction unit 130.
[0053] Reference Figure 4 , displayed in the display component (for example, Figure 6 The image IM on the display 220 in the multi-channel lens MCL can be converted into a virtual reality image VR while passing through the multi-channel lens MCL. In this process, distortion of the VR image VR may occur based on, for example, the above-mentioned tolerance. For example, a pixel located at a coordinate position (x, y) in the image IM may be shifted or realigned to a coordinate position (x', y') in the VR image VR through the multi-channel lens MCL. As a result, distortion of the VR image VR may occur.
[0054] In order to prevent such distortion of the VR image VR, a process of extracting a correction function may be performed. The process may include positioning a pixel at a coordinate position (x, y) in the VR image VR and then changing (or mapping) the position of the pixel in the image IM to a coordinate position (x", y"), which (due to a tolerance) corresponds to the pixel at the coordinate position (x, y) in the VR image VR. The mapping may be performed using an extracted correction function. Thus, the process may be referred to as a mapping process, and the correction function may be referred to as a mapping function.
[0055] The extraction unit 130 may extract the mapping (or correction) function MF by applying the correction data CD to the standard mapping function RMF. For example, the extraction unit 130 may extract the mapping function MF by applying the tolerance between the appearance data AD of the multi-channel lens MCL and the standard appearance data RAD to the standard mapping function RMF. To this end, Figure 1 As shown in , the standard mapping function RMF can be provided to the extraction unit 130. In one embodiment, the standard mapping function RMF can be pre-stored in the extraction unit 130.
[0056] The standard mapping function RMF may be a correction function for calculating position information of pixels corresponding to a VR image to be provided by an ideal design of the multi-channel lens MCL. For example, the standard mapping function RMF may be a correction function used in the above-mentioned mapping process to prevent distortion of the VR image VR due to the multi-channel lens MCL having an ideal design. The standard mapping function RMF may be derived in advance during the initial design process of the multi-channel lens MCL.
[0057] In one embodiment, when the image IM is provided by the actually designed multi-channel lens MCL, the mapping function MF may be a correction function for calculating the position information of the pixel corresponding to the VR image VR. For example, the mapping function MF may be a correction function used in the above-mentioned mapping process to prevent the distortion of the VR image VR due to the actually designed multi-channel lens MCL manufactured by injection molding. The mapping function MF may be extracted corresponding to the change in the appearance data AD due to defects in the manufacturing process of the multi-channel lens MCL, etc.
[0058] As described above, in order to prevent distortion of the VR image VR, a process of extracting a correction function (i.e., a mapping function MF) is performed in advance. In the example, the mapping function MF can be extracted by acquiring the position information of the pixel through the captured image IM and the VR image VR and deriving the realignment characteristics of the pixel in the form of a function according to the characteristics of the multi-channel lens MCL. In the above process, shooting at various viewpoints is performed by considering the gaze direction of the user. As a result, a lot of time may be spent during the process. In addition, the process of deriving the realignment characteristics of the pixel from the position information of the pixel (according to the characteristics of the multi-channel lens MCL) is complicated. Therefore, it will not be easy to extract the mapping function MF.
[0059] On the other hand, in the mapping function extraction system 100 according to the embodiment of the present disclosure, the appearance information of the multi-channel lens MCL can be easily acquired by scanning, and the mapping function MF obtained by considering the appearance information of the multi-channel lens MCL can be quickly extracted based on the pre-derived standard appearance data RAD and the standard mapping function RMF. In addition, since the mapping function MF according to the embodiment of the present disclosure is extracted by reflecting the tolerance between the appearance data AD of the multi-channel lens MCL and the standard appearance data RAD, the distortion of the VR image VR according to the characteristics of the multi-channel lens MCL can be corrected more accurately.
[0060] Figure 5 1 is a flow chart showing a method for extracting a mapping function according to an embodiment of the present invention. Figure 5 , the description of the parts overlapping with the above parts will be omitted or simplified.
[0061] Reference Figure 1 and Figure 5 First, appearance data AD of the multi-channel lens MCL may be generated by scanning the multi-channel lens MCL (S100). Operation S100 may include a process of extracting actual appearance information of the multi-channel lens MCL (eg, manufactured by injection molding) by performing 3D scanning.
[0062] Next, correction data CD may be generated by comparing the appearance data AD with pre-stored standard appearance data RAD (S200). Operation S200 may include a process of deriving a tolerance by comparing actual appearance information of the multi-channel lens MCL (e.g., manufactured by injection molding) with predetermined or reference (e.g., ideal) appearance information of the multi-channel lens MCL according to an initial design value.
[0063] Next, the mapping function MF may be extracted by applying the correction data CD to a pre-stored standard mapping function RMF (S300). Operation S300 may include extracting a mapping function for preventing a VR image VR (eg, Figure 4 ) of the multi-channel lens MCL. This can be performed by reflecting the actual appearance information on a standard mapping function RMF, which has been pre-derived based on the ideal appearance information of the multi-channel lens MCL. As described below, the mapping function MF can then be used to reduce the distortion in the virtual image.
[0064] Figure 6 is a perspective view of a display device 200 according to an embodiment of the present invention. Figure 7 is a plan view of a display device 200 according to an embodiment of the present invention. Figure 8 is a side view of a portion of a display device 200 according to an embodiment of the present invention.
[0065] Reference Figure 6 , the display device 200 may include a head-mounted display device that is worn on the user's head to provide the user with a screen on which an image is displayed. The head-mounted display device may be a see-through type or a see-closed type. In the see-through type, augmented reality (AR) is provided based on an actual external object. In the closed type, virtual reality (VR) is provided to the user through a screen independent of the external object. In the following, a closed type head-mounted display device is illustrated, but the present disclosure is not limited thereto.
[0066] The display device 200 may include a main frame 210, a display unit (or display) 220, a lens unit (or lens arrangement member) 230, and a cover frame 240. The main frame 210 may be worn on the face of the user. The main frame 210 may have a shape corresponding to the shape of the user's head (face). The lens unit 230, the display unit 220, and the cover frame 240 may be installed in the main frame 210.
[0067] The main frame 210 may include a space or structure for accommodating the display unit 220 and the lens unit 230. The main frame 210 may also include a structure (e.g., a strap or a band) to facilitate mounting it on the user's head. The control unit 250 (e.g., see Fig. 9 ), an image processing unit, a lens housing unit, etc. can be further installed in the main frame 210.
[0068] The display unit 220 may display an image. In one embodiment, the display unit 220 may include a front surface 220_FS on which an image is displayed and a rear surface 220_RS opposite to the front surface 220_FS. In the display unit 220, light for providing an image for viewing by a user may be emitted from the front surface 220_FS. A first multi-channel lens 231 and a second multi-channel lens 232 may be disposed on the front surface 220_FS of the display unit 220. A plurality of camera sensors may be disposed on the rear surface 220_RS of the display unit 220. However, the present disclosure is not limited thereto, and a plurality of camera sensors may be disposed in the lens unit 230, the main frame 210, the cover frame 240, and the like.
[0069] The display unit 220 may be fixed to the main frame 210 and may be detachably arranged in one embodiment. The display unit 220 may be configured to be opaque, transparent or translucent, for example, according to the type of the display device 200. The display unit 220 may include an electronic component (e.g., a display module including a display panel) or a display device (such as a mobile terminal) including a display panel. However, the present disclosure is not limited thereto. For example, the display unit 220 may include a micro display.
[0070] The display unit 220 may include a display panel for image display. The display panel may be a light-emitting display panel including a light-emitting element. For example, the display panel may include an organic light-emitting display panel using an organic light-emitting diode including an organic emission layer, a micro light-emitting diode (LED) display panel using a micro LED, a quantum dot light-emitting display panel using a quantum dot light-emitting diode including a quantum dot emission layer, or an inorganic light-emitting display panel using an inorganic light-emitting diode including an inorganic semiconductor.
[0071] The lens unit 230 may allow light emitted from the display unit 220 to pass through the lens unit 230, thereby providing light to the user. The lens unit 230 may have a plurality of channels for allowing light emitted from the display unit 220 to pass through it. The plurality of channels may allow light emitted from the display unit 220 to pass through it along different paths, thereby providing light to the user. The light emitted from the display unit 220 may be incident into each channel to generate an enlarged image focused on the user's eyes.
[0072] The lens unit 230 may include a plurality of lenses corresponding to the focusing optical system. In an embodiment, the lens unit 230 may include a first multi-channel lens 231 corresponding to one eye of the user and a second multi-channel lens 232 corresponding to the other eye of the user. The multi-channel lenses 231 and 232 are illustrated as a focusing optical system disposed between the user's eye and the front surface 220_FS of the display unit 220, but the present disclosure is not limited thereto. For example, the focusing optical system may include various types of lenses (such as a convex lens, a concave lens, a spherical lens, an aspherical lens, a single lens, a compound lens, a standard lens, a narrow-angle lens, a wide-angle lens, a fixed focal length lens, a variable focal length lens, a flat lens, or a combination thereof).
[0073] like Figure 6 As shown in , the first multi-channel lens 231 and the second multi-channel lens 232 may be disposed on the front surface 220_FS of the display unit 220. The first multi-channel lens 231 and the second multi-channel lens 232 may be arranged on the front surface 220_FS of the display unit 220 corresponding to the positions of the left eye and the right eye of the user, respectively. The first multi-channel lens 231 and the second multi-channel lens 232 may be accommodated in the main frame 210.
[0074] The first multi-channel lens 231 and the second multi-channel lens 232 may reflect and / or refract light to provide an image displayed on the display unit 220 to the user in the form of a virtual reality image.
[0075] The cover frame 240 may be disposed on the rear surface 220_RS of the display unit 220 to protect the display unit 220 .
[0076] Reference Figure 7, the display device 200 according to an embodiment of the present disclosure may include a cover frame 240 , a lens unit (or a lens arrangement member) 230 located on the cover frame 240 , and a display unit (or a display) 220 located between the cover frame 240 and the lens unit 230 .
[0077] The display unit 220 may include a first display unit 221 overlapped with a first multi-channel lens 231 and a second display unit 222 overlapped with a second multi-channel lens 232. The lens unit 230 may include a first multi-channel lens 231 corresponding to a left eye of a user and a second multi-channel lens 232 corresponding to a right eye of the user.
[0078] The first multi-channel lens 231 may include two first sub-regions Ra and two second sub-regions Rb divided along the first direction DR1 and the second direction DR2 with respect to the first central portion CL.
[0079] One first subregion Ra and one second subregion Rb of the first multi-channel lens 231 may be positioned along a diagonal line passing through the first central portion CL and forming a predetermined angle (eg, about 45 degrees or another angle) with the first direction DR1 and the second direction DR2 .
[0080] The first multi-channel lens 231 may be divided into four regions (or quadrants) with respect to a cross-shaped virtual line passing through the first central portion CL, and the two first sub-regions Ra and the two second sub-regions Rb may be located in corresponding regions among the four regions.
[0081] The first multi-channel lens 231 may include a plurality of sub-lenses. In an embodiment, the first multi-channel lens 231 may include four sub-lenses SL11, SL12, SL13, and SL14. However, the number of sub-lenses included in the first multi-channel lens 231 is not limited to four as in the case of the above example.
[0082] The first multi-channel lens 231 may have a predetermined (e.g., approximately circular) shape on a plane. A plurality of sub-lenses SL11, SL12, SL13, and SL14 may be arranged on a plane around the center of the circular shape. For example, the first sub-lens SL11, the second sub-lens SL12, the third sub-lens SL13, and the fourth sub-lens SL14 may be arranged at the upper left end, the upper right end, the lower left end, and the lower right end, respectively, relative to the first central portion CL. Therefore, the first sub-lens SL11 and the third sub-lens SL13 may be arranged in the first sub-region Ra, and the second sub-lens SL12 and the fourth sub-lens SL14 may be arranged in the second sub-region Rb. A plurality of sub-lenses SL11, SL12, SL13, and SL14 may be integrally connected to each other and separated from each other.
[0083] The image of the first display unit 221 corresponding to the two first sub-regions Ra and the two second sub-regions Rb may be refracted toward the first center portion CL of the first multi-channel lens 231 while passing through the first multi-channel lens 231 .
[0084] The second multi-channel lens 232 may include two first sub-regions Ra and two second sub-regions Rb divided along the first direction DR1 and the second direction DR2 with respect to the second center portion CR. One first sub-region Ra and one second sub-region Rb of the second multi-channel lens 232 may be positioned along a diagonal line passing through the second center portion CR and forming a predetermined angle (e.g., about 45 degrees) with the first direction DR1 and the second direction DR2.
[0085] The second multi-channel lens 232 may be divided into four regions (or quadrants) with respect to a cross-shaped virtual line passing through the second center portion CR, and the two first sub-regions Ra and the two second sub-regions Rb may be located in corresponding regions among the four regions.
[0086] The second multi-channel lens 232 may include a plurality of sub-lenses. In an embodiment, the second multi-channel lens 232 may include four sub-lenses SL21, SL22, SL23, and SL24. However, the number of sub-lenses included in the second multi-channel lens 232 is not limited to four as in the above example.
[0087] The second multi-channel lens 232 may have a predetermined (e.g., approximately circular) shape on a plane. A plurality of sub-lenses SL21, SL22, SL23, and SL24 may be arranged on a plane around the center of the circular shape. For example, the fifth sub-lens SL21, the sixth sub-lens SL22, the seventh sub-lens SL23, and the eighth sub-lens SL24 may be arranged at the upper left end, the upper right end, the lower left end, and the lower right end, respectively, relative to the second center portion CR. Therefore, the fifth sub-lens SL21 and the seventh sub-lens SL23 may be arranged in the first sub-region Ra, and the sixth sub-lens SL22 and the eighth sub-lens SL24 may be arranged in the second sub-region Rb. A plurality of sub-lenses SL21, SL22, SL23, and SL24 may be integrally connected to each other and separated from each other.
[0088] The image of the second display unit 222 corresponding to the two first sub-regions Ra and the two second sub-regions Rb may be refracted toward the second center portion CR of the second multi-channel lens 232 while passing through the second multi-channel lens 232. Hereinafter, a change in the direction of the image passing through the first sub-region Ra and the second sub-region Rb will be described.
[0089] Reference Figure 7 and Figure 8, the lens unit 230 may include a first boundary portion S1, a second boundary portion S2, and a third boundary portion S3 sequentially positioned from the display unit 220 in a direction facing the user's eye UEE. The first boundary portion S1 may be a flat first surface of the lens unit 230 facing the display unit 220. The third boundary portion S3 may be a second surface of the lens unit 230 facing the user's eye UEE. The second boundary portion S2 may be a surface of a middle portion located between the first boundary portion S1 and the third boundary portion S3 and inclined in the lens unit 230 to form a slope with the first boundary portion S1. At the center portion C of the lens unit 230, the distance between the second boundary portion S2 and the third boundary portion S3 may be narrower than the distance between the first boundary portion S1 and the second boundary portion S2. The distance between the second boundary portion S2 and the third boundary portion S3 may gradually widen and then narrow again as it approaches the edge from the center portion C.
[0090] Shapes of the lens units 230 located in the first sub-region Ra and the second sub-region Rb may be symmetrical to each other with respect to a central portion C of the lens unit 230 .
[0091] The light Lr0 emitted from the display unit 220 may be refracted at the second boundary portion S2 while passing through the first boundary portion S1 of the lens unit 230. The first refracted light Lr1 subjected to a change in direction may be reflected at the third boundary portion S3 of the lens unit 230 in a direction facing toward the center portion of the lens unit 230. Through the lens arrangement, the direction of the first refracted light Lr1 may be changed to be farther away from the user's eye UEE.
[0092] The first reflected light Lr2 may be re-reflected at the second boundary portion S2 of the lens unit 230. As a result, the direction of the first reflected light Lr2 is changed to be closer to the central portion of the lens unit 230 and closer to the user's eye UEE. The second reflected light Lr3 having undergone the direction change may be refracted at the surface of the lens unit 230. The second refracted light Lr4 refracted at the surface of the lens unit 230 may be emitted to face the user's eye UEE.
[0093] In this way, the lens unit 230 has two first sub-regions Ra and two second sub-regions Rb that are symmetrical to each other relative to the central portion C. The lens units 230 located in each first sub-region Ra and each second sub-region Rb have different distances and different thicknesses, so that the distances between the first boundary portion S1, the second boundary portion S2, and the third boundary portion S3 change at different positions. Therefore, the direction of the light Lr0 emitted from the display unit 220 to the lens unit 230 is changed to face the user's eye UEE. The light Lr0 is refracted to form a first refracted light Lr1, the first refracted light Lr1 is reflected to form a first reflected light Lr2, the first reflected light Lr2 is reflected to form a second reflected light Lr3, and the second reflected light Lr3 is refracted to form a second refracted light Lr4, thereby displaying a three-dimensional image (or VR image).
[0094] Fig. 9 2 is a block diagram showing a display device 200 according to an embodiment of the present invention. Fig. 9 , the description of the parts overlapping with the above parts will be omitted or simplified.
[0095] Reference Fig. 9 , the display device 200 may include a display unit (or display) 220 , a lens unit (or lens arrangement member) 230 , and a control unit (or controller) 250 .
[0096] The display unit 220 includes a plurality of pixels that emit light for displaying an image IM. The display unit 220 may display the image IM based on the mapping data MD provided from the control unit 250.
[0097] The lens unit 230 may provide a VR image VR by refracting and reflecting the image IM displayed on the display unit 220. As described above, the lens unit 230 may include multi-channel lenses 231 and 232 (eg, see Figure 6 ).
[0098] The control unit 250 may control the overall operation of the display device 200. The control unit 250 may be implemented as, for example, a dedicated processor including an embedded processor, etc. and / or a general processor including a central processing unit, an application processor, etc. However, the present disclosure is not limited thereto.
[0099] The mapping function MF may be provided to the control unit 250. Alternatively, the mapping function MF may be pre-stored in the control unit 250. The mapping function MF is extracted from the mapping function extraction system 100 described above (eg, see Figure 1 ) is extracted, and may be a correction function for preventing distortion of the VR image VR. For example, the multi-channel lenses 231 and 232 (see Figure 6 ) standard appearance data RAD (see Figure 1 ) and appearance data AD (see Figure 1 ) can be applied to the mapping function MF. The standard appearance data RAD and the appearance data AD are the same as described above.
[0100] The VR image data VRD may be provided to the control unit 250. Alternatively, the VR image data VRD may be pre-stored in the control unit 250. The VR image data VRD may include information on a VR image VR to be provided (eg, a VR image VR without distortion).
[0101] The control unit 250 may generate mapping data MD based on the mapping function MF and the VR image data VRD. The mapping data MD includes position information of pixels corresponding to the VR image VR. For example, the control unit 250 may process the arrangement characteristics of pixels of the image IM to be displayed on the display unit 220 in the form of data based on the mapping function MF and the VR image data VRD to provide the VR image VR. In this way, since the control unit 250 (based on the multi-channel lenses 231 and 232 (see FIG. 2 ) included in the lens unit 230) Figure 6 ) is reflected in advance on the appearance information of the image processing unit 230, and thus it is not necessary to perform a separate process (e.g., image capturing, etc.) for extracting the mapping function MF, so that the time for performing the mapping process can be significantly reduced. In addition, the control unit 250 can effectively prevent the distortion of the VR image VR according to the characteristics of the multi-channel lenses 231 and 232 included in the lens unit 230. The control unit 250 can provide the mapping data MD to the display unit 220.
[0102] According to one or more of the aforementioned embodiments of the present invention, a mapping function for preventing distortion of a VR image can be quickly extracted or determined. According to one or more of the aforementioned embodiments of the present invention, the time for performing a mapping process can be shortened, and distortion of a VR image can be effectively prevented.
[0103] Method described herein, technology and / or operation can be performed by code or instruction to be executed by computer, processor, controller or other signal processing device.Computer, processor, controller or other signal processing device can be element described herein or element except element described herein.Because the algorithm of the basis of formation method (or operation of computer, processor, controller or other signal processing device) is described in detail, the code or instruction for realizing the operation of method embodiment can convert computer, processor, controller or other signal processing device into the special processor for performing the method herein.
[0104] In addition, another embodiment may include a computer-readable medium (e.g., a non-transitory computer-readable medium) for storing the above-mentioned code or instructions. The computer-readable medium may be a volatile or non-volatile memory or other storage device, which may be removably or fixedly coupled to a computer, processor, controller or other signal processing device that will execute the code or instructions for performing the operations of the method embodiments or apparatus embodiments herein.
[0105] For example, according to one embodiment, a computer-readable medium stores instructions that, when executed by at least one processor, cause the at least one processor to: generate appearance data of a multi-channel lens by scanning the multi-channel lens; generate correction data by comparing the appearance data with pre-stored standard appearance data; and generate a mapping function by applying the correction data to a pre-stored standard mapping function. The computer-readable medium may be included in, for example, Fig. 9 In one embodiment, the computer readable medium may be incorporated into a controller as previously described herein.
[0106] The controllers, processors, devices, modules, units, and other signal generation and signal processing features of the embodiments disclosed herein may be implemented, for example, in non-transitory logic that may include hardware, software, or both. When implemented at least partially in hardware, the controllers, processors, devices, modules, units, and other signal generation and signal processing features may be, for example, any of a variety of integrated circuits (including, but not limited to, application specific integrated circuits, field programmable gate arrays, combinations of logic gates, systems on a chip, microprocessors, or another type of processing or control circuitry).
[0107] When implemented at least in part with software, controllers, processors, devices, modules, units and other signal generation and signal processing features may include, for example, a memory or other storage device for storing a code or instruction to be executed by, for example, a computer, processor, microprocessor, controller or other signal processing device. The computer, processor, microprocessor, controller or other signal processing device may be an element described herein or an element in addition to the element described herein. Because the algorithm forming the basis of the method (or the operation of a computer, processor, microprocessor, controller or other signal processing device) is described in detail, the code or instruction for implementing the operation of the method embodiment can convert a computer, processor, controller or other signal processing device into a special processor for performing the method described herein.
[0108] Although the present invention has been specifically described according to the above embodiments, it should be noted that the above embodiments are intended to illustrate the present invention rather than limit the scope of the present invention. It will be understood by those skilled in the art that various modifications can be made within the scope of the technical spirit of the present invention.
[0109] Therefore, the technical protection scope of the present invention is not limited to the detailed description described in the specification, but should be determined by the appended claims. In addition, all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as included in the scope of the present invention. The embodiments can be combined to form additional embodiments.
Claims
1. A mapping function extraction system, the mapping function extraction system include: A scanner configured to generate appearance data of the multi-channel lens by scanning the multi-channel lens; a comparator configured to generate correction data by comparing the appearance data with pre-stored standard appearance data; as well as The extractor is configured to generate a mapping function by applying the correction data to a pre-stored standard mapping function.
2. The mapping function extraction system according to claim 1, in, The multi-channel lens includes a plurality of sub-lenses through which light emitted from a display passes.
3. The mapping function extraction system according to claim 2, in, The appearance data includes appearance information of a recessed portion formed by the plurality of sub-lenses.
4. The mapping function extraction system according to claim 1, in, The standard appearance data includes appearance information of the multi-channel lens having an ideal design.
5. The mapping function extraction system according to claim 1, in, The correction data is derived based on a difference between the standard appearance data and the appearance data.
6. The mapping function extraction system according to claim 1, in, The standard mapping function is used to calculate the position information of pixels corresponding to the virtual reality image to be provided through the ideally designed multi-channel lens.
7. The mapping function extraction system according to claim 1, in, The mapping function is extracted based on a change in the appearance data.
8. A mapping function extraction method, the mapping function extraction method The following steps are involved: Generating appearance data of the multi-channel lens by scanning the multi-channel lens; generating correction data by comparing the appearance data with pre-stored standard appearance data; as well as A mapping function is extracted by applying the correction data to a pre-stored standard mapping function.
9. The mapping function extraction method according to claim 8, in, The multi-channel lens includes a plurality of sub-lenses through which light emitted from a display passes.
10. The mapping function extraction method according to claim 9, in, The appearance data includes appearance information of a recessed portion formed by the plurality of sub-lenses.
11. The mapping function extraction method according to claim 8, in, The standard appearance data includes appearance information of the multi-channel lens having an ideal design.
12. The mapping function extraction method according to claim 8, in, The correction data is derived based on a difference between the standard appearance data and the appearance data.
13. The mapping function extraction method according to claim 8, in, The standard mapping function is used to calculate the position information of pixels corresponding to the virtual reality image to be provided through the ideally designed multi-channel lens.
14. The mapping function extraction method according to claim 8, in, The mapping function is extracted based on a change in the appearance data.
15. A display device, the display device include: Displays, including pixels; a lens arrangement including at least one multi-channel lens configured to provide a virtual reality image by refracting and reflecting an image displayed on the display; as well as A controller is configured to generate mapping data including position information of the pixels corresponding to the virtual reality image based on a pre-stored mapping function and virtual reality image data, wherein a difference between the standard appearance data of the multi-channel lens and the appearance data of the multi-channel lens is applied to the mapping function.
16. The display device according to claim 15, in, The mapping function is extracted based on a change in the appearance data.
17. The display device according to claim 15, in, The standard appearance data includes appearance information of the multi-channel lens having an ideal design.
18. The display device according to claim 15, in, The appearance data includes appearance information of the multi-channel lens through scanning.
19. The display device according to claim 15, in, The appearance data includes appearance information of a recessed portion formed by a plurality of sub-lenses included in the multi-channel lens.
20. The display device according to claim 15, in: The controller is configured to provide the mapping data to the display, and The display is configured to display the image based on the mapping data.
21. A computer readable medium storing instructions which, when executed by at least one processor, cause the at least one processor to: Generating appearance data of the multi-channel lens by scanning the multi-channel lens; generating correction data by comparing the appearance data with pre-stored standard appearance data; and A mapping function is generated by applying the correction data to a pre-stored standard mapping function.
