Image processing method and display system

Abstract

PURPOSE: To provide an image processing method and a display system, which enable provision of display content with realistic gloss variations over a wide range.SOLUTION: An image processing method is provided, comprising: receiving image data having angular profile data corresponding to a plurality of positions; processing the image data by performing at least one of noise filtering, image compensation, image modification, and image discontinuity reduction to generate processed image data; and outputting the processed image data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a data processing method, and more particularly to an image processing method and display system. [Background technology]

[0002] Currently, high dynamic range (HDR) display technology has been developed to provide high-contrast images, allowing both bright and dark details in an image to be clearly displayed simultaneously. However, displays with HDR display technology still have difficulty reproducing the glossy effect that occurs when real light hits an object. Even if a display can show the glossy or realistic effect, the light profile of the image must be further processed or optimized to provide a satisfactory visual effect. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention provides an image processing method and display system that can provide display content with realistic gloss variations over a wide range. [Means for solving the problem]

[0004] The image processing method of the present invention includes the steps of receiving image data having angular profile data corresponding to a plurality of positions, processing the image data by performing at least one of noise filtering, image compensation, image correction, and image discontinuity reduction to generate processed image data, and outputting the processed image data.

[0005] The display system includes a camera, an image processor, and a display module. The camera provides a plurality of captured images. The image processor is coupled to the camera and receives image data for the plurality of captured images and generates processed image data based on the image data. The display module is coupled to the image processor and receives the processed image data. The image data includes angular profile data corresponding to a plurality of positions. The image processor processes the image data by performing at least one of noise filtering, image compensation, image correction, and image discontinuity reduction to generate the processed image data. [Effects of the Invention]

[0006] As described above, the image processing method and display system of the present invention can perform effective image processing on the angle profile data of the image data.

[0007] To facilitate understanding of the above content, several embodiments will be described in detail below with reference to the drawings. [Brief explanation of the drawings]

[0008] The accompanying drawings are included to provide a further understanding of the principles of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0009] [Figure 1] 1 is a schematic diagram of an image processing device according to one embodiment of the present invention. [Figure 2] 1 is a flowchart of an image processing method according to one embodiment of the present invention. [Figure 3A] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 3B] FIG. 1 is a schematic diagram of a minimum light intensity component of angular profile data according to one embodiment of the present invention. [Figure 3C]FIG. 2 is a schematic diagram of the specular component of angular profile data according to one embodiment of the present invention. [Figure 4A] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 4B] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 4C] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 4D] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 4E] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 4F] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 5A] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 5B] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 5C] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 5D] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 5E] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 5F] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 6A] FIG. 10 is a schematic diagram of adjusting a spotlight effect on an object according to one embodiment of the present invention. [Figure 6B] FIG. 10 is a schematic diagram of adjusting a spotlight effect on an object according to one embodiment of the present invention. [Figure 6C] FIG. 10 is a schematic diagram of adjusting a spotlight effect on an object according to one embodiment of the present invention. [Figure 6D]FIG. 10 is a schematic diagram of adjusting a spotlight effect on an object according to one embodiment of the present invention. [Figure 7A] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 7B] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 7C] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 7D] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 7E] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 8] 1 is a schematic diagram of a display system according to one embodiment of the present invention. [Figure 9] 1 is a schematic diagram of a display system according to one embodiment of the present invention. [Figure 10A] 1 is a schematic diagram of an observer looking at a display module according to one embodiment of the present invention; FIG. [Figure 10B] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 10C] 1 is a schematic diagram of an observer looking at a display module according to one embodiment of the present invention; FIG. [Figure 10D] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 10E] 1 is a schematic diagram of an observer looking at a display module according to one embodiment of the present invention; FIG. [Figure 10F] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 11A] 1 is a schematic diagram of an observer looking at a display module according to one embodiment of the present invention; FIG. [Figure 11B] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 11C]1 is a schematic diagram of an observer looking at a display module according to one embodiment of the present invention; FIG. [Figure 11D] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. [Figure 11E] 1 is a schematic diagram of an observer looking at a display module according to one embodiment of the present invention; FIG. [Figure 11F] FIG. 2 is a schematic diagram of angular profile data according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE INVENTION Exemplary embodiments of the invention are now described in detail with reference to the accompanying drawings, in which: Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.

[0011] Certain terms are used throughout the present specification and the appended claims to refer to particular components. As one skilled in the art will appreciate, manufacturers of electronic devices may refer to the same component by different names. This specification does not intend to distinguish between components that have the same function but different names. In the following description and claims, terms such as "comprises" and "containing" are open-ended terms and should be interpreted as "including, but not limited to...".

[0012] The term "coupled" (or "connected") as used throughout this specification (including the appended claims) can refer to any direct or indirect connection means. For example, if a sentence describes a first device being coupled (or connected) to a second device, it should be interpreted as meaning that the first device may be directly connected to the second device, or that the first device may be indirectly connected to the second device through other devices or specific connection means. The terms "first," "second," and similar terms used throughout this specification (including the appended claims) are used merely to denote separate elements or to distinguish between different embodiments or ranges. Therefore, these terms should not be construed as limiting the number of elements or the order in which elements may be arranged. Furthermore, wherever possible, elements / components / steps with the same reference numbers in the drawings and embodiments represent the same or similar parts. Related descriptions of elements / components / steps using the same reference numbers or terms in different embodiments may be cross-referenced.

[0013] FIG. 1 is a schematic diagram of an image processing device according to one embodiment of the present invention. Referring to FIG. 1, the image processing device 100 includes a processing unit 110, an input unit 120, and an output unit 130. The processing unit 110 is coupled to the input unit 120 and the output unit 130. In an embodiment of the present invention, the input unit 120 is coupled to an image source and can read image data 101 including multiple intensity values ​​and angular profiles of emitted light rays (i.e., surface light field data) corresponding to multiple positions (i.e., multiple screen positions (x, y)). The image source may be a personal computer, one or more digital still cameras (DSCs), a video player, or an image server, but the present invention is not limited thereto. The image data 101 may be two-dimensional image data or three-dimensional image data. The image data 101 may be composed of data of multiple captured images of the same object captured by multiple cameras at different viewing angles. Therefore, the image data 101 may include angular profile data corresponding to multiple positions.

[0014] In an embodiment of the present invention, the processing unit 110 may receive image data 101 provided by the input unit 120, and may perform image processing on the image data 101 to generate processed image data 102. The processing unit 110 outputs the processed image data 102 to the output unit 130. In an embodiment of the present invention, the output unit 130 may be coupled to a storage device or a surface light field display device (i.e., a "glitter display"). In one embodiment of the present invention, the output unit 130 may output the processed image data 102 to a surface light field display device. The surface light field display device may display an image with a realistic glossy display effect to observers viewing from different viewing angles based on the processed image data 102.

[0015] In embodiments of the present invention, the processing unit 110 may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), other similar processing circuits, or a combination of these devices. In embodiments of the present invention, the image processing device 100 may further include a storage unit such as a memory. The processing unit 110 may further be coupled to the storage unit. The storage unit may be, for example, a non-volatile memory (NVM). The storage unit may store related programs, modules, data, or algorithms for implementing various embodiments of the present invention, which the processing unit 110 may access and execute to implement the relevant functions and operations described in various embodiments of the present invention.

[0016] In an embodiment of the present invention, the input unit 120 and the output unit 130 may be communication interfaces and have associated communication data conversion circuits. In one embodiment of the present invention, since the image data including the angular profile data may have a large amount of data, at least one of the input unit 120 and the output unit 130 may include a data expansion circuit or a data compression circuit for the angular profile of the light ray in the form of a series of discrete values, coefficients in the frequency domain, or waveform fitting parameters.

[0017] FIG. 2 is a flowchart of an image processing method according to an embodiment of the present invention. Referring to FIGS. 1 and 2, the image processing apparatus 100 may perform the following steps S210 to S230. In step S210, the input unit 120 may receive image data 101 having angular profile data corresponding to a plurality of positions (i.e., a plurality of screen positions (x, y)). The input unit 120 may further provide the image data 101 having the angular profile data to the processing unit 110. In step S220, the processing unit 110 may process the image data 101 by performing at least one of noise filtering, image compensation, image correction, and image discontinuity reduction to generate processed image data 102. In an embodiment of the present invention, the processing unit 110 may process the image data 101 based on specific display requirements or external control requirements. The processing unit 110 may further provide the processed image data 102 to the output unit 130. In step S230, the output unit 130 may output the processed image data 102. In the following embodiment, a method for processing the image data 101 will be described in detail.

[0018] FIG. 3A is a schematic diagram of angular profile data according to one embodiment of the present invention. Referring to FIGS. 1 and 3A, the processing unit 110 can receive image data including angular profile data. As shown in FIG. 3A, taking angular profile data corresponding to one screen position (i.e., (x, y)) as an example (i.e., each screen position can correspond to multiple image data from different viewing angles, and the data format can be (x, y, angle, light intensity value (i.e., grayscale))), different viewing angles for one screen position can correspond to different light intensity values. In FIG. 3A, curve 301 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)). That is, when the display module displays this image data, the viewer can see display content with different gloss effects (or spotlight effects) when viewing one screen position from different viewing angles.

[0019] FIG. 3B is a schematic diagram of a minimum light intensity component of angular profile data according to one embodiment of the present invention. FIG. 3C is a schematic diagram of a specular component of angular profile data according to one embodiment of the present invention. Referring to FIGS. 1 and 3A-3C, image data 101 may include data of multiple images captured at different viewing angles, and processing unit 110 processes the multiple captured images to separate multiple diffuse light component images and multiple specular light component images, which can be used in the image processing steps of subsequent embodiments. It should be noted that, for example, the image gloss in the diffuse light component image does not change at different viewing angles due to ambient light from walls and ceilings, but the image gloss in the specular light component image may change due to, for example, one or more spotlights.

[0020] In an embodiment of the present invention, processing unit 110 can determine a minimum light intensity component corresponding to each position in the multiple captured images, and processing unit 110 can generate multiple scattered light component images based on the minimum light intensity component of each of the multiple captured images. As shown in FIG. 3B , angular profile data of each scattered light component image at each position can be represented as curve 302 in FIG. 3B . Processing unit 110 can then subtract the minimum light intensity component from each of the multiple captured images to generate multiple specular reflected light component images. As shown in FIG. 3C , angular profile data of each specular reflected light component image at each position can be represented as curve 303 in FIG. 3C .

[0021] 4A-4F are schematic diagrams of angular profile data according to one embodiment of the present invention. Referring to FIGS. 1, 4A, and 4B, image data 101 may include data of a cropped light profile image, and processing unit 110 may compensate for missing peak data in the cropped light profile image. As shown in FIG. 4A, curve 401 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)), and some light intensity values ​​within the peak of curve 401 are cropped (e.g., corresponding to overexposure effects in the image). In an embodiment of the present invention, processing unit 110 may compensate for the missing peak data in FIG. 4A (or the missing peak data in the specular component of the angular profile data of curve 303 in FIG. 3C) to generate compensated angular profile data shown as curve 402 in FIG. 4B. Specifically, the processing unit 110 may estimate a profile corresponding to the missing peak data from adjacent screen positions that are not overexposed, or the processing unit 110 may fit a curve corresponding to the missing peak data due to the light distribution scheme, so that the processing unit 110 can restore the missing peak data of the light profile and generate a photograph with a higher dynamic range.

[0022] 1, 4C, and 4D, the image data 101 may include data of a cropped light profile image, and the processing unit 110 may compensate for missing angular profile data outside the viewing angle range of the cropped light profile image. As shown in FIG. 4C, curve 403 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)), and some light intensity values ​​outside the viewing angle range of curve 403 are cropped. In an embodiment of the present invention, the processing unit 110 may compensate for the missing angular profile data of FIG. 4C (or the missing angular profile data in the specular component of the angular profile data of curve 303 in FIG. 3C) to generate compensated angular profile data shown as curve 404 in FIG. 4D. Specifically, the processing unit 110 may estimate profiles from adjacent screen positions with different peak positions, or the processing unit 110 may fit a curve corresponding to the missing angular profile data due to the light distribution scheme. Therefore, the processing unit 110 can extend the angular range of the light profile even if the measured profile is insufficient.

[0023] 1, 4E, and 4F, the light intensity values ​​of the angular profile data of the image data 101 may be in an 8-bit data format. As shown in FIG. 4E, curve 405 represents the angular profile data in the 8-bit data format. The processing unit 110 may perform step compensation sampling on the angular profile data of curve 405 (or the angular profile data of curve 303 in FIG. 3C) to generate compensated angular profile data shown as curve 406 in FIG. 4F. Specifically, the processing unit 110 may perform linear or nonlinear interpolation on the angular profile data to increase the angular step size of the angular profile data. Alternatively, the processing unit 110 may perform curve fitting using a light distribution method or a low-pass filter to increase the angular step size of the angular profile data, for example, from 0 to 255 (8 bits) to 0 to 4095 (12 bits). Therefore, the processing unit 110 may perform noise filtering on the image to reduce noise in dark areas of the image. The processing unit 110 can solve the Mach band problem and present smooth gradation of the image.

[0024] 5A-5F are schematic diagrams of angular profile data according to one embodiment of the present invention. FIGS. 6A-6D are schematic diagrams of adjusting a spotlight effect on an object according to one embodiment of the present invention. Referring to FIGS. 1, 5A, and 5B, processing unit 110 can receive image data having angular profile data. In FIG. 5A, curve 501 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)), where the light intensity values ​​at certain viewing angles may be too strong. Processing unit 110 can modify at least one light intensity value corresponding to at least one specific viewing angle corresponding to each position in the angular profile data. Specifically, processing unit 110 can generate a scattered light profile image as described in the above-described embodiment of FIG. 3B and obtain a minimum light intensity value. Processing unit 110 can generate angular profile data of curve 502 shown in FIG. 5B by subtracting the minimum light intensity value from each light intensity value of curve 501, multiplying the result by an adjustment coefficient, and then adding the minimum light intensity value. The adjustment coefficient may be greater than 0, so that the corresponding specular reflection light at some specific viewing angles can be appropriate and comfortable. After capturing the image, the processing unit 110 can increase or decrease the specular reflection light intensity of the light profile at some specific viewing angles. The processing unit 110 can optimize the image quality to obtain the best image performance.

[0025] For example, referring to FIG. 6A , the initial lighting conditions 610 may include an object 611, which may have an excessively strong gloss effect due to the spotlight 612. The processing unit 110 may adjust the angle profile data of the corresponding image data as in the above-described embodiment to reduce the light intensity value in the angle profile data. Therefore, the adjusted lighting conditions may be as shown in FIG. 6B . In FIG. 6B , the adjusted lighting conditions 620 may include a weakened spotlight 612, which may cause the object 611 to have an appropriate gloss effect. That is, the display content displayed by the display module provides the effect of dimming the spotlight 612.

[0026] 1, 5C, and 5D, processing unit 110 can receive image data having angular profile data corresponding to different colors (e.g., red, green, and blue). Processing unit 110 can extract multiple specular profile images corresponding to the different colors, as described in the above-described embodiment of FIG. 3C, and processing unit 110 can modify at least one light intensity value corresponding to at least one color corresponding to each position in the angular profile data. Specifically, in FIG. 5C, curves 503R, 503G, and 503B represent the specular components of the red, green, and blue angular profile data, respectively. Each curve includes multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)), and curves 503R, 503G, and 503B can correspond to a yellowish display effect. 5D , processing unit 110 can modify the light intensity values ​​of curves 503R, 503G, and 503B into curves 504R, 504G, and 504B, respectively, where curves 504R, 504G, and 504B can correspond to a bluish display effect. That is, processing unit 110 can, for example, adjust the spotlight color while maintaining the scattered light color (i.e., the basic light color) in the image. Furthermore, processing unit 110 can also adjust the specular reflected light and scattered light to obtain similar colors in the image.

[0027] For example, referring to FIG. 6A , initial light conditions 610 may include an object 611, which appears yellowish due to a spotlight 612. The processing unit 110 may adjust the angle profile data corresponding to different colors of the corresponding image data, as in the above-described embodiment. Therefore, the adjusted light conditions may be as shown in FIG. 6C . In FIG. 6C , adjusted light conditions 630 may include the object 611, which appears bluish. That is, the display content displayed by the display module provides the effect of changing the hue of the spotlight 612.

[0028] 1, 5E, and 5F, processing unit 110 may receive image data having angular profile data. Processing unit 110 may modify at least one light intensity value corresponding to each position in the angular profile data. Processing unit 110 may extract a specular component of the angular profile data, as described in the above-described embodiment of FIG. 3C. Processing unit 110 may modify the light intensity values ​​to shift and / or add a light profile, and add the modified specular component image to the scattered component image to generate a processed image. Specifically, in FIG. 5E, curve 505 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)). Processing unit 110 may shift and add a light profile based on curve 505 to generate curves 506 and 507. That is, processing unit 110 may shift a spotlight effect in the captured image or add more spotlight effects in the captured image.

[0029] For example, referring to FIG. 6A , initial light conditions 610 may include an object 611, and the object image 611 has a spotlight effect due to spotlight 612. The processing unit 110 may adjust the angle profile data of the corresponding image data as in the above-described embodiment. Therefore, the adjusted light conditions may be as shown in FIG. 6D . In FIG. 6D , adjusted light conditions 640 may include an object image 611, and the object 611 has more spotlight effects. That is, the display content displayed by the display module simultaneously provides the spotlight effects of spotlight 612 and spotlight 613 on the object 611.

[0030] 7A to 7E are schematic diagrams of angular profile data according to one embodiment of the present invention. Referring to FIGS. 1, 7A, and 7B, image data 101 may include a light profile with angular discontinuities, and processing unit 110 may adjust at least one light intensity value corresponding to each position in the angular profile data to reduce the angular discontinuities. As shown in FIG. 7A, curve 701 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)), and curve 701 forms a light profile with angular discontinuities. In an embodiment of the present invention, processing unit 110 may adjust some light intensity values ​​of curve 701 to reduce the angular discontinuities. After adjusting some light intensity values ​​of curve 701, an intensity value corresponding to an angle at one end of the light intensity distribution of curve 702 becomes equal to another intensity value corresponding to an angle at the other end of the light intensity distribution of curve 702, as shown in FIG. 7B.

[0031] Specifically, processing unit 110 can generate a scattered light profile image and obtain the minimum light intensity value as described in the above-described embodiment of FIG. 3B. Processing unit 110 can generate angle profile data for curve 702 as shown in FIG. 7B by subtracting the minimum light intensity value from each light intensity value of curve 701, multiplying the result by an adjustment coefficient, and then adding the minimum light intensity value. The adjustment coefficient may be between 0 and 1, and gradually becomes 0 from one end angle to the other end angle. Therefore, the light intensity values ​​of curve 702 at one end angle and the other end angle gradually decrease to 0.

[0032] 1, 7C, and 7D, the image data 101 may include a light profile with angular discontinuities, and the processing unit 110 may adjust at least one light intensity value corresponding to each position in the angular profile data to reduce the angular discontinuities in the light profile. As shown in FIG. 7C, a curve 703 represents angular profile data including multiple light intensity values ​​corresponding to different viewing angles at one screen position (i.e., (x, y)), and the curve 703 forms a light profile with angular discontinuities. In an embodiment of the present invention, the processing unit 110 may adjust some light intensity values ​​of the curve 703 to reduce the angular discontinuities in the light profile. After adjusting some light intensity values ​​of the curve 703, an intensity value corresponding to an angle at one end of the light intensity distribution of the curve 704 becomes equal to another intensity value corresponding to an angle at the other end of the light intensity distribution of the curve 704, as shown in FIG. 7D.

[0033] Specifically, processing unit 110 can generate a scattered light profile image and obtain the minimum light intensity value as described in the above-described embodiment of FIG. 3B. Processing unit 110 can generate the angle profile data of curve 704 shown in FIG. 7D by subtracting the minimum light intensity value from each light intensity value of curve 703, multiplying the result by an adjustment factor, and then adding the minimum light intensity value. The adjustment factor may be between 0 and 1, and gradually becomes 1 from one end angle to the other end angle. Therefore, the light intensity value of curve 704 adjacent to one end angle increases to another light intensity value of curve 704 at the other end angle.

[0034] 1 and 7E, in one embodiment, processing unit 110 may replicate the light intensity values ​​of curve 701 in FIG. 7A to augment the data for curves 705 and 706 shown in FIG. 7E, so that processing unit 110 can effectively expand the light profile at other viewing angles of the angular profile data.

[0035] FIG. 8 is a schematic diagram of a display system according to an embodiment of the present invention. Referring to FIG. 8, the display system 800 includes an electronic device 810 and a display module 820. The electronic device 810 includes an image processing device 811 and a camera 812. The image processing device 811 is coupled to the camera 812 and the display module 820. The image processing device 811 can implement the relevant image processing functions and hardware characteristics of the image processing device 100 in the above-described embodiment. In an embodiment of the present invention, the image processing device 811 and the camera 812 are integrated into the electronic device 810. The electronic device 810 is a terminal device or a computer device, etc. In one embodiment of the present invention, the electronic device 810 is a video capture device, and the display module 820 is a surface light field display device (i.e., a glitter display device). The camera 812 is configured to generate image data having angle profile data and provide the image data having angle profile data to the image processing device 811. The image processing unit 811 is configured to process the image data having the angular profile data and generate processed image data to drive the display module 820. The camera 812 may be a monocular image sensor or a multi-system image sensor, or the camera 812 may include a depth sensor (e.g., a ToF sensor) for acquiring three-dimensional shape information of objects on a scene. In one embodiment of the present invention, the electronic device 810 may further include a memory element for storing the image data. The electronic device 810 may further include a fish-eye sensor for acquiring a light map on a background.

[0036] FIG. 9 is a schematic diagram of a display system according to one embodiment of the present invention. Referring to FIG. 9, the display system 900 includes an electronic device 910 and a camera 920. The electronic device 910 includes an image processing unit 911 and a display module 912. The image processing unit 911 is coupled to the display module 912. The image processing unit 911 can implement the relevant image processing functions and hardware characteristics of the image processing unit 100 in the above-described embodiment. In an embodiment of the present invention, the image processing unit 911 and the display module 912 are integrated into the electronic device 910. The display module 912 is a surface light field display device (i.e., a glitter display device). The camera 920 is configured to generate image data having angular profile data and provide the image data having angular profile data to the image processing unit 911. The image processing unit 911 is configured to process the image data having the angular profile data and generate processed image data to drive the display module 912.

[0037] In one embodiment of the present invention, the display module 912 may further include at least one pixel array on a display module, such as a liquid crystal display (LCD) panel, a mini light emitting diode (LED) panel, a micro LED panel, or an organic light emitting diode (OLED) panel. The display module 912 may further include an optical component that modulates light from the pixels in space to different directions (angles), such as a lenticular lens, a liquid crystal gradient-index (LC GRIN) lens, a parallax barrier, or an LC parallax barrier. The display module 912 may further include a display control circuit configured to control the angular profile output of light intensity from each pixel based on angular profile data of the image data.

[0038] Also, in one embodiment of the present invention, the image processing unit 911, the display module 912, and the camera 920 may all be integrated into the electronic device 910. The electronic device 910 may be, for example, a smartphone or a tablet computer.

[0039] 10A, 10C, and 10E are schematic diagrams illustrating an observer looking at a display module according to an embodiment of the present invention. 10B, 10D, and 10F are schematic diagrams illustrating angle profile data according to an embodiment of the present invention. In an embodiment of the present invention, the display system 800 of FIG. 8 and the display system 900 of FIG. 9 may further include an eye-tracking sensor coupled to the image processing device 811 and the image processing device 911, respectively. The eye-tracking sensor may be an RGB camera, an IR camera, or a ToF sensor. The eye-tracking sensor is configured to detect the observer's eye position or head position, and the image processing device of the electronic device may process the image data based on the eye position or head position and adjust the angle of the output light beam of the display module driven by the processed image data.

[0040] Specifically, in FIG. 10A , the display module may include a pixel array 1011 and a lenticular lens 1012. The pixel array 1011 and the lenticular lens 1012 extend in directions D1 and D3, respectively. The pixel array 1011 provides output light rays that pass through the lenticular lens 1012 in direction D2, and the output light rays are provided to a viewer 1020. The directions D1 to D3 are perpendicular to each other. In an embodiment of the present invention, an eye tracking sensor of the electronic device may detect the eye position or head position of the viewer 1020, and an image processor of the electronic device may process image data based on the eye position or head position. As shown in FIG. 10B , the image processor of the electronic device may generate processed image data for driving the pixel array 1011 using angular profile data corresponding to an angle range 1030 based on the eye position or head position of the viewer 1020.

[0041] In one embodiment of the invention, in Figure 10C, when the viewer 1020 moves to view the display module from the side, an eye tracking sensor of the electronic device can detect that the eye or head position of the viewer 1020 has changed. Accordingly, as shown in Figure 10D, the image processor of the electronic device can adjust the angle of the output light beam provided by the pixel array 1011 by modifying it to use angular profile data corresponding to the angle range 1040 to generate processed image data for driving the pixel array 1011.

[0042] In one embodiment of the present invention, in Figure 10E, when the viewer 1020 moves to view the display module from the side of another side, the eye tracking sensor of the electronic device can detect that the eye position or head position of the viewer 1020 has changed. Accordingly, as shown in Figure 10F, the image processor of the electronic device can adjust the angle of the output light beam provided by the pixel array 1011 by modifying it to use angular profile data corresponding to the angle range 1050 to generate processed image data for driving the pixel array 1011.

[0043] Thus, the image processor of the electronic device can effectively process the angular profile data based on the eye or head position of the observer 1020 to generate processed image data for driving the pixel array 1011. The pixel array 1011 can provide display content that varies based on the viewer's viewing angle and has realistic gloss variations over a wide range.

[0044] 11A, 11C, and 11E are schematic diagrams illustrating an observer viewing a display module according to an embodiment of the present invention. FIGS. 11B, 11D, and 11F are schematic diagrams illustrating angle profile data according to an embodiment of the present invention. In an embodiment of the present invention, the display system 800 of FIG. 8 and the display system 900 of FIG. 9 may further include a motion sensor coupled to the image processing device 811 and the image processing device 911, respectively. The motion sensor may be an accelerometer or a gyroscope. The motion sensor is configured to detect the display position or tilt angle of the display module, and the image processing device of the electronic device may process the image data based on the display position or tilt angle to adjust the angle of the output light beam of the display module driven by the processed image data.

[0045] Specifically, in FIG. 11A , the display module may include a pixel array 1111, a lenticular lens 1112, and a motion sensor 1113. The pixel array 1111 and the lenticular lens 1112 extend in directions D1 and D3, respectively. The pixel array 1111 provides output light rays that pass through the lenticular lens 1112 in direction D2, and the output light rays are provided to a viewer 1120. The directions D1 to D3 are perpendicular to each other. In an embodiment of the present invention, the motion sensor 1113 can detect the display position or tilt angle of the display module, and the image processing device of the electronic device can process image data based on the display position or tilt angle of the display module. As shown in FIG. 11B , the image processing device of the electronic device can generate processed image data for driving the pixel array 1111 using angle profile data corresponding to an angle range 1130 based on the display position or tilt angle of the display module.

[0046] In one embodiment of the present invention, in Fig. 11C, when the display module rotates an angle counterclockwise, the motion sensor 1113 can detect that the display position or tilt angle of the display module has changed. Thus, as shown in Fig. 11D, the image processor of the electronic device can adjust the angle of the output light beam provided by the pixel array 1111 by changing to use angular profile data corresponding to the angle range 1140 to generate processed image data for driving the pixel array 1111.

[0047] In one embodiment of the present invention, in Fig. 11E, when the viewer 1120 rotates the angle clockwise, the motion sensor 1113 can detect that the display position or tilt angle of the display module has changed. Thus, as shown in Fig. 11F, the image processor of the electronic device can adjust the angle of the output light beam provided by the pixel array 1111 by modifying it to use angular profile data corresponding to the angle range 1150 to generate processed image data for driving the pixel array 1111.

[0048] Therefore, the image processing unit of the electronic device can effectively process the angle profile data based on the display position or tilt angle of the display module to generate processed image data for driving the pixel array 1111. The pixel array 1111 can change based on the display position or tilt angle of the display module to provide display content with realistic gloss variations over a wide range.

[0049] As described above, the image processing method and display system of the present invention can perform effective image processing functions on image data having angular profile data to generate appropriate angular profile data required for use. The image processing method can effectively drive a display module based on the processed image data to provide display content with realistic gloss variations over a wide range.

[0050] It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments without departing from the scope or spirit of the invention. In view of this, it is intended that the present invention cover modifications and variations that come within the scope of the following claims and their equivalents. [Industrial Applicability]

[0051] The image processing method and display system of the present invention can be applied to a display device. [Explanation of symbols]

[0052] 100, 811, 911 Image processing device 101 Image data 102 Processed image data 110 Processing Unit 120 input units 130 output units 301~303, 401~406, 501, 502, 503R, 503G, 503B, 504R, 504G, 504B, 505~507, 701~706 curve 610 Initial light conditions 611 Objects 612, 613 Spotlight 620, 630, 640 Adjusted light conditions 800, 900 display system 810, 910 Electronic Devices 812, 920 cameras 820, 912 display module 1011, 1111 pixel array 1012 Lenticular Lens 1020, 1120 Observer 1030, 1040, 1050, 1130, 1140, 1150 angle range 1112 Lenticular lens 1113 Motion Sensor D1, D2, D3 direction S210~S230 Step

Claims

1. receiving image data having angular profile data corresponding to a plurality of locations; processing the image data by performing at least one of noise filtering, image compensation, image correction, and image discontinuity reduction to generate processed image data; outputting the processed image data; An image processing method comprising:

2. The image processing method of claim 1 , wherein the angular profile data comprises a plurality of light intensity values ​​corresponding to a plurality of viewing angles.

3. The image data includes a plurality of captured images, and the step of processing the image data further comprises: The image processing method according to claim 1 , further comprising the step of separating a plurality of scattered light component images and a plurality of specular reflected light component images from the plurality of captured images.

4. the step of separating the plurality of scattered light component images and the plurality of specular reflected light component images from the plurality of captured images comprises: determining a minimum light intensity component corresponding to each location within the plurality of captured images; generating the plurality of scattered light component images based on the minimum light intensity component of each of the plurality of captured images; generating a plurality of specular reflected light component images by subtracting the minimum light intensity component from each of the plurality of captured images; The image processing method according to claim 3 , comprising:

5. the image data includes a cropped light profile image, and the step of processing the image data further comprises: The image processing method of claim 1 , further comprising compensating for at least one of missing peak data and missing angular profile data in the cropped light profile image.

6. said step of processing said image data further comprising: The image processing method of claim 1 , further comprising the step of performing step compensation sampling on the angular profile data.

7. said step of processing said image data further comprising: The method of claim 1 , further comprising adjusting at least one light intensity value corresponding to at least one particular viewing angle corresponding to each location in the angular profile data.

8. said step of adjusting at least one light intensity value further comprising:

8. The image processing method of claim 7, further comprising the step of subtracting a corresponding minimum light intensity value from each light intensity value, multiplying the result by a first adjustment factor, and then adding back the corresponding minimum light intensity value, wherein the first adjustment factor is greater than zero.

9. said step of processing said image data further comprising:

2. The method of claim 1, further comprising modifying at least one light intensity value corresponding to at least one color corresponding to each location in the angular profile data.

10. said step of processing said image data further comprising:

2. The method of claim 1, further comprising adjusting at least one light intensity value corresponding to each location in the angular profile data to reduce angular discontinuities in the light profile.

11. said step of adjusting at least one light intensity value further comprising:

11. The image processing method according to claim 10, further comprising a step of subtracting a corresponding minimum light intensity value from each light intensity value, multiplying the result by a second adjustment coefficient, and then adding the corresponding minimum light intensity value, wherein the second adjustment coefficient is between 0 and 1 and gradually becomes 0 from an angle at one end of the light intensity distribution at each position toward an angle at the other end of the light intensity distribution.

12. 11. The image processing method of claim 10, wherein after adjusting the at least one light intensity value corresponding to each of the positions in the angular profile data, a first intensity value corresponding to an angle at one end of the light intensity distribution at each of the positions becomes equal to a second intensity value corresponding to an angle at the other end of the light intensity distribution at each of the positions.

13. said step of processing said image data further comprising: The image processing method of claim 1 , further comprising duplicating the light intensity values ​​of the angular profile data to expand the light profile at other viewing angles in the angular profile data corresponding to each position.

14. said step of processing said image data further comprising: detecting eye or head position; adjusting an angle of an output light beam of a display module driven by the processed image data by processing the image data based on the eye position or the head position; The image processing method according to claim 1 , comprising:

15. said step of processing said image data further comprising: detecting a display position or tilt angle of a display module; processing image data based on the display position or the tilt angle to adjust an angle of an output light beam of a display module driven by the processed image data; The image processing method according to claim 1 , comprising:

16. A camera providing a plurality of captured images; an image processing device coupled to the camera, the image processing device receiving image data of the plurality of captured images and generating processed image data based on the image data; a display module coupled to the image processing device for receiving the processed image data; wherein the image data includes angular profile data corresponding to a plurality of positions, and the image processor processes the image data by performing at least one of noise filtering, image compensation, image correction, and image discontinuity reduction to generate processed image data.

17. further comprising an eye tracking sensor coupled to the image processor; The display system of claim 16, wherein the eye tracking sensor detects eye position or head position, and the image processing device processes the image data based on the eye position or the head position, thereby adjusting an angle of an output light beam of a display module driven by the processed image data.

18. further comprising a motion sensor coupled to the image processor; The display system of claim 16, wherein the motion sensor detects a display position or tilt angle of the display module, and the image processing device processes the image data based on the display position or the tilt angle, thereby adjusting the angle of the output light beam of the display module driven by the processed image data.

19. The image processing device, an input unit coupled to the camera for receiving the image data of the plurality of captured images; a processing unit coupled to the input unit, for processing the image data based on the image data and generating the processed image data; an output unit coupled to the processing unit and the display module for outputting the processed image data to the display module; 20. The display system of claim 16, comprising:

20. 17. The display system of claim 16, wherein the display module is a surface light field display device.

Images