Image processing method and display system

Abstract

The invention provides an image processing method and a display system. The image processing method includes the steps of: receiving image data having angular profile data corresponding to a plurality of positions; processing the image data by performing at least one of noise filtering, image compensation, image modification, and image discontinuity mitigation to produce processed image data; and outputting the processed image data. According to the image processing method and the display system, image processing can be effectively performed on the angle profile data of the image data.

Description

Technical Field

[0001] The present disclosure relates to a data processing method, and in particular, to an image processing method and a display system. Background Art

[0002] Currently, high dynamic range (HDR) display technology has been developed to present high-contrast images, so that light and dark details in the image can be clearly presented at the same time. However, displays using high dynamic range display technology still have difficulty in achieving the glossy effect (kirameki effect) that appears when real light illuminates an object. In addition, even if the display can present a glossy and realistic effect, the light profile of the image still needs to be further processed or optimized to provide a satisfactory visual effect. Summary of the invention

[0003] The image processing method disclosed herein includes the following steps: receiving image data having angular profile data corresponding to a plurality of positions; processing the image data to generate processed image data by performing at least one of noise filtering, image compensation, image modification, and image discontinuity reduction; and outputting the processed image data.

[0004] The display system includes a camera, an image processing device, and a display module. The camera provides a plurality of captured images. The image processing device is coupled to the camera, receives image data of the plurality of captured images, and generates processed image data based on the image data. The display module is coupled to the image processing device, and receives the processed image data. The image data includes angular profile data corresponding to a plurality of positions. The image processing device processes the image data to generate processed image data by performing at least one of noise filtering, image compensation, image modification, and image discontinuity mitigation.

[0005] Based on the above content, the image processing method and the display system according to the present disclosure can effectively perform image processing on the angular profile data of the image data.

[0006] In order to make the foregoing content more understandable, several embodiments with accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.

[0008] Figure 1 is a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure;

[0009] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0010] Figure 3A is a schematic diagram of angular profile data according to an embodiment of the present disclosure;

[0011] Figure 3B is a schematic diagram of a minimum light intensity component of angular profile data according to an embodiment of the present disclosure;

[0012] Figure 3C is a schematic diagram of a specular reflection light component of angular profile data according to an embodiment of the present disclosure;

[0013] 4A to 4F is a schematic diagram of angular profile data according to an embodiment of the present disclosure;

[0014] FIG. 5A to FIG. 5F is a schematic diagram of angular profile data according to an embodiment of the present disclosure;

[0015] FIG. 6A to FIG. 6D is a schematic diagram of adjusting a spotlight effect on an object according to an embodiment of the present disclosure;

[0016] 7A to 7E is a schematic diagram of angular profile data according to an embodiment of the present disclosure;

[0017] Figure 8 is a schematic diagram of a display system according to an embodiment of the present disclosure;

[0018] Fig. 9 is a schematic diagram of a display system according to an embodiment of the present disclosure;

[0019] Fig. 10A , Fig. 10C and Fig. 10E is a schematic diagram of an observer observing a display module according to an embodiment of the present disclosure;

[0020] Fig. 10B , Fig. 10D and Fig.10F is a schematic diagram of angular profile data according to an embodiment of the present disclosure;

[0021] Fig.11A , Fig. 11C and Fig.11E is a schematic diagram of an observer observing a display module according to an embodiment of the present disclosure;

[0022] Fig. 11B , Fig.11D and Fig.11F is a schematic diagram of angular profile data according to an embodiment of the present disclosure.

[0023] [Explanation of Symbols]

[0024] 100, 811, 911: image processing device;

[0025] 101: image data;

[0026] 102: processed image data;

[0027] 110: processing unit;

[0028] 120: input unit;

[0029] 130: output unit;

[0030] 301, 302, 303, 401, 402, 403, 404, 405, 406, 501, 502, 503R, 503G, 503B, 504R, 504G, 504B, 505, 506, 507, 701, 702, 703, 704, 705, 706: curves;

[0031] 610: Initial lighting conditions;

[0032] 611: Object;

[0033] 612, 613: Spotlight;

[0034] 620, 630, 640: adjusted lighting conditions;

[0035] 800, 900: display system;

[0036] 810, 910: electronic devices;

[0037] 812, 920: Camera;

[0038] 820, 912: display module;

[0039] 1011, 1111: pixel array;

[0040] 1012, 1112: biconvex lens;

[0041] 1020, 1120: Observer;

[0042] 1030, 1040, 1050, 1130, 1140, 1150: angle range;

[0043] 1113: motion sensor;

[0044] D1, D2, D3: directions; S210, S220, S230: steps. DETAILED DESCRIPTION

[0045] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the description to refer to the same or like components.

[0046] Certain terms are used throughout the specification and the appended claims of the present disclosure to refer to specific components. It should be understood by those skilled in the art that electronic device manufacturers may refer to the same components by different names. It is not intended herein to distinguish between components that have the same function but have different names. In the following description and claims, words such as "comprise" and "include" are open-ended terms and should be interpreted as "including but not limited to ..."

[0047] The term "coupled (or connected)" used throughout the entire specification (including the attached claims) of the present disclosure may refer to any direct or indirect connection method. For example, if the context states that a first device is coupled (or connected) to a second device, it should be interpreted that the first device can be directly connected to the second device, or the first device can be indirectly connected to the second device via other devices or some connecting members. The terms "first", "second" and similar terms mentioned throughout the entire specification (including the attached claims) of this application are only used to name discrete elements or to identify them in different embodiments or scopes. Therefore, the terms should not be regarded as limiting the upper or lower limit of the number of elements and should not be used to limit the order of arrangement of elements. In addition, the elements / components / steps with the same reference numbers in the drawings and embodiments represent the same or similar parts as much as possible. The same reference numbers or the same terms can be used in different embodiments to refer to the relevant descriptions of the elements / components / steps.

[0048] Figure 1 is a schematic diagram of an image processing device according to an embodiment of the present disclosure. Figure 1, the image processing device 100 includes a processing unit 110, an input unit 120 and an output unit 130. The processing unit 110 is coupled to the input unit 120 and the output unit 130. In an embodiment of the present disclosure, the input unit 120 can be coupled to an image source to read image data 101, and the image data 101 includes multiple intensity values ​​corresponding to multiple positions (i.e., multiple screen positions (x, y)) and angular profiles of emitted light (i.e., surface light field data). The image source can be a personal computer, one or more digital still cameras (DSC), a video player, or an image server, but the present disclosure is not limited thereto. The image data 101 can be two-dimensional image data or three-dimensional image data. The image data 101 can be composed of data of multiple captured images captured by multiple cameras at different observation angles of the same object. Therefore, the image data 101 can include angular profile data corresponding to multiple positions.

[0049] In an embodiment of the present disclosure, the processing unit 110 may receive the image data 101 provided by the input unit 120, and may perform image processing on the image data 101 to generate processed image data 102. The processing unit 110 outputs the processed image data 102 to the output unit 130. In an embodiment of the present disclosure, the output unit 130 may be coupled to a memory storage device or a surface light field display device (i.e., a "kirameki display"). In one embodiment of the present disclosure, the output unit 130 may output the processed image data 102 to the surface light field display device. The surface light field display device may display an image with a realistic gloss display effect for observers observing from different observation angles based on the processed image data 102.

[0050] In an embodiment of the present disclosure, the processing unit 110 may include, for example, a central processing unit (CPU), a graphic processing unit (GPU) or other programmable general or special microprocessors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), other similar processing circuits or a combination of these devices. In an embodiment of the present disclosure, the image processing device 100 may also include a storage unit, such as a memory. The processing unit 110 may be further coupled to the storage unit. The storage unit may be, for example, a non-volatile memory (NVM). The storage unit may store relevant programs, modules, data or algorithms for implementing various embodiments of the present disclosure, and the relevant programs, modules, data or algorithms are accessed and executed by the processing unit 110 to implement the relevant functions and operations described in various embodiments of the present disclosure.

[0051] In an embodiment of the present disclosure, the input unit 120 and the output unit 130 may be communication interfaces and have related communication data conversion circuits. In one embodiment of the present disclosure, since the image data including the angular profile data may have a large amount of data, at least one of the input unit 120 and the output unit 130 may include a data decompression circuit or a data compression circuit for the angular profile of the light in the form of sequential discrete values, coefficients in the frequency domain, or fitting parameters of the waveform.

[0052] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure. Figure 1 and Figure 2, the image processing device 100 may perform the following steps S210 to S230. In step S210, the input unit 120 may receive the image data 101 having the angular profile data corresponding to the multiple positions (i.e., the multiple screen positions (x, y)). The input unit 120 may further provide the image data 101 having the angular profile data to the processing unit 110. In step S220, the processing unit 110 may process the image data 101 to generate processed image data 102 by performing at least one of noise filtering, image compensation, image modification, and image discontinuity mitigation. In one embodiment of the present disclosure, the processing unit 110 may process the image data 101 according to a specific display requirement or an external control requirement. The processing unit 110 may further provide the processed image data 102 to the output unit 130. In step S230, the output unit 130 may output the processed image data 102. The processing method of the image data 101 will be described in detail in the following embodiments.

[0053] Figure 3A is a schematic diagram of angular profile data according to an embodiment of the present disclosure. Figure 1 and Figure 3A , the processing unit 110 may receive image data having angular profile data. Figure 3A As shown in , taking the angular profile data corresponding to a screen position (i.e. (x, y)) as an example (i.e. each screen position may correspond to a plurality of image data from different observation angles, and the data format may be (x, y, angle, light intensity value (i.e. grayscale)), for a screen position, different observation angles may correspond to different light intensity values. Figure 3A In FIG. 3 , curve 301 represents angular profile data, which includes a plurality of light intensity values ​​corresponding to different observation angles at the one screen position (i.e., (x, y)). In other words, if the display module displays this image data, when the observer observes the one screen position from different observation angles, the observer can see display contents with different gloss effects (or spotlight effects).

[0054] Figure 3B is a schematic diagram of a minimum light intensity component of angular profile data according to an embodiment of the present disclosure. Figure 3C FIG. 4 is a schematic diagram of the specular reflection light component of the angular profile data according to an embodiment of the present disclosure. Figure 1 and FIG. 3A to FIG. 3C, the image data 101 may include data of a plurality of captured images captured at different viewing angles, and the processing unit 110 may process the plurality of captured images to separate a plurality of diffuse light component images and a plurality of specular reflection light component images for use in the image processing process of the subsequent embodiments. It should be noted that the image gloss in the diffuse light component images does not change at different viewing angles due to ambient light from, for example, a wall or ceiling, and the image gloss in the specular reflection light component images may change due to, for example, one or more spotlights.

[0055] In an embodiment of the present disclosure, the processing unit 110 may determine the minimum light intensity component corresponding to each position in the plurality of captured images, and the processing unit 110 may generate the plurality of diffuse light component images according to the minimum light intensity component in each of the plurality of captured images. Figure 3B As shown in , the angular profile data of each diffuse light component image at each position can be shown as Figure 3B Then, the processing unit 110 may respectively subtract the minimum light intensity component from each of the plurality of captured images to generate the plurality of specular reflection light component images. Figure 3C In the figure, the angular profile data of each specular reflection light component image at each position can be shown as Figure 3C The curve 303 is shown in FIG.

[0056] 4A to 4F is a schematic diagram of angular profile data according to an embodiment of the present disclosure. Figure 1 , Figure 4A and Figure 4B , the image data 101 may include data of a truncated light section image, and the processing unit 110 may compensate for the peak data lost in the truncated light section image. Figure 4A As shown in FIG. 4 , curve 401 represents angular profile data, which includes a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)), and some light intensity values ​​in the peak of curve 401 are truncated (e.g., corresponding to an overexposure effect in the image). In an embodiment of the present disclosure, the processing unit 110 may compensate for Figure 4A The missing spike data (or Figure 3C The angular profile data of curve 303 in FIG. 3 is used to generate compensated angular profile data, as shown in FIG. Figure 4B Specifically, the processing unit 110 may estimate a profile corresponding to the missing peak data based on a nearby screen position that is not overexposed, or the processing unit 110 may fit a curve corresponding to the missing peak data through a light distribution formula. Therefore, the processing unit 110 may restore the missing peak data of the light profile and generate a picture with a higher dynamic range.

[0057] refer to Figure 1 , Figure 4C and Figure 4D , the image data 101 may include data of a truncated light section image, and the processing unit 110 may compensate for missing angular section data outside the observation angle range in the truncated light section image. Figure 4C As shown in FIG. 4 , curve 403 represents angular profile data, which includes a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)), and some light intensity values ​​outside the viewing angle range in curve 403 are omitted. In an embodiment of the present disclosure, the processing unit 110 may compensate for Figure 4C Missing angular profile data (or Figure 3C The angular profile data missing from the specular reflection light component of the angular profile data of curve 303 in FIG. 3 is used to generate compensated angular profile data, as shown in FIG. Figure 4D Specifically, the processing unit 110 may estimate the profile according to the nearby screen positions with different peak positions, or the processing unit 110 may fit a curve corresponding to the missing angular profile data through the light distribution formula. Therefore, even if the measured profile is insufficient, the processing unit 110 may still extend the angular range of the light profile.

[0058] refer to Figure 1 , Figure 4E and Figure 4F , the light intensity value of the angular profile data of the image data 101 may be in an 8-bit data format. Figure 4E As shown in FIG. 4 , curve 405 represents the angular profile data in an 8-bit data format. The processing unit 110 may process the angular profile data (or Figure 3C 303 in FIG. 3 ) performs sampling step compensation to generate compensated angular profile data, as shown in FIG. Figure 4F 406 in FIG. 4 . Specifically, the processing unit 110 may perform linear or nonlinear interpolation on the angular profile data to increase the angular step size of the angular profile data. Alternatively, the processing unit 110 may perform curve fitting through a light distribution formula or a low-pass filter to increase the angular step size of the angular profile data, for example, from 0 to 255 (8 bits) to 0 to 4095 (12 bits). Therefore, the processing unit 110 may perform noise filtering on the image to reduce the noise in the dark part of the image. The processing unit 110 may solve the Mach band issue and show the smooth gradation of the photo.

[0059] FIG. 5A to FIG. 5F is a schematic diagram of angular profile data according to an embodiment of the present disclosure. FIG. 6A to FIG. 6D is a schematic diagram of adjusting the spotlight effect on an object according to an embodiment of the present disclosure. Figure 1 , Figure 5A and Figure 5B , the processing unit 110 may receive image data having angular profile data. Figure 5A , curve 501 represents angular profile data, the angular profile data including a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)), and at a certain viewing angle, curve 501 may have an excessive light intensity value. Processing unit 110 may modify at least one light intensity value corresponding to at least one specific viewing angle corresponding to each position in the angular profile data. Specifically, processing unit 110 may generate Figure 3B The processing unit 110 may subtract the minimum light intensity value from each light intensity value of the curve 501, multiply by the adjustment factor, and then add the minimum light intensity value back to generate Figure 5B The angular profile data of the curve 502 shown in FIG. The adjustment factor may be greater than 0. Therefore, the corresponding specular reflection light at a certain viewing angle may become appropriate and comfortable. The processing unit 110 may increase or decrease the specular reflection light intensity of the light profile at a certain viewing angle after shooting. The processing unit 110 may optimize the image quality and obtain the best image performance.

[0060] For example, refer to Fig. 6A , the initial lighting condition 610 may include an object 611, and the object 611 has an overly strong gloss effect caused by a spotlight 612. The processing unit 110 may adjust the angular profile data of the corresponding image data to reduce the light intensity value in the angular profile data as in the above embodiment. Therefore, the adjusted lighting condition may be shown as Figure 6B .exist Figure 6B In the example of FIG. 6 , the adjusted lighting condition 620 may include a weakened spotlight 612, and the object 611 has an appropriate gloss effect. That is, the display content displayed by the display module presents an effect in which the spotlight 612 is dimmed.

[0061] refer to Figure 1 , Figure 5C and Figure 5D , the processing unit 110 may receive image data having angular profile data corresponding to different colors (eg, red, green, and blue). The processing unit 110 may perform the above Figure 3C In the embodiment of the invention, a plurality of specular reflection light profile images corresponding to different colors are extracted, and the processing unit 110 may modify at least one light intensity value corresponding to at least one color corresponding to each position in the angular profile data. Specifically, in Figure 5CIn FIG. 1 , curves 503R, 503G, and 503B respectively represent the red, green, and blue specular reflection light components in the angular profile data. Each curve includes a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)), and curves 503R, 503G, and 503B may correspond to a yellowish display effect. The processing unit 110 may modify the light intensity values ​​of curves 503R, 503G, and 503B to Figure 5D , and the curves 504R, 504G, and 504B shown in FIG. 5 correspond to a bluish display effect. That is, the processing unit 110 may adjust the spotlight color of the image, for example, while maintaining the diffuse light color (i.e., the base light color). In addition, the processing unit 110 may also make adjustments to obtain a similar color between the specular reflected light and the diffuse light in the image.

[0062] For example, refer to Fig. 6A , the initial lighting condition 610 may include an object 611, and the object 611 appears yellowish due to a spotlight 612. The processing unit 110 may adjust the angular profile data corresponding to different colors in the corresponding image data, as in the above embodiment. Therefore, the adjusted lighting condition may be shown as Figure 6C .exist Figure 6C In the example, the adjusted lighting condition 630 may include the object 611, and the object 611 appears to be bluish. That is, the display content displayed by the display module presents the effect of changing the hue of the spotlight 612.

[0063] refer to Figure 1 , Figure 5E and Fig. 5F , the processing unit 110 may receive image data having angular profile data. The processing unit 110 may modify at least one light intensity value corresponding to each position in the angular profile data. The processing unit 110 may extract the specular reflection light component of the angular profile data, such as Figure 3C The processing unit 110 may modify the light intensity value to shift and / or add the light profile, and add the modified specular reflection light component image to the diffuse light component image to generate a processed image. Specifically, in Figure 5E In FIG. 5 , curve 505 represents angular profile data, which includes a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)). The processing unit 110 may shift and add light profiles according to the curve 505 to generate curves 506 and 507. That is, the processing unit 110 may shift the spotlight effect in the captured image or add more spotlight effects in the captured image.

[0064] For example, refer to Fig. 6A, the initial lighting condition 610 may include an object 611, and the object 611 has a spotlight effect caused by a spotlight 612. The processing unit 110 may adjust the angular profile data of the corresponding image data as in the above embodiment. Therefore, the adjusted lighting condition may be shown as Fig.6D .exist Fig.6D In the example, the adjusted lighting condition 640 may include the object 611, and the object 611 has more spotlight effects. That is, the display content displayed by the display module simultaneously presents the spotlight effects of the spotlight 612 and the spotlight 613 on the object 611.

[0065] 7A to 7E is a schematic diagram of angular profile data according to an embodiment of the present disclosure. Figure 1 , Fig. 7A and Figure 7B , the image data 101 may include a light profile with an angular discontinuity, and the processing unit 110 may adjust at least one light intensity value corresponding to each position in the angular profile data to reduce the light profile with an angular discontinuity. Fig. 7A As shown in FIG. 7 , curve 701 represents angular profile data, and the angular profile data includes a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)), and curve 701 forms a light profile with discontinuous angles. In an embodiment of the present disclosure, the processing unit 110 may adjust some light intensity values ​​of curve 701 to reduce the light profile with discontinuous angles. After adjusting some light intensity values ​​of curve 701, as shown in FIG. Figure 7B As shown in , the intensity value corresponding to one end point angle of the light intensity distribution of the curve 702 is equal to another intensity value corresponding to another end point angle of the light intensity distribution of the curve 702 .

[0066] Specifically, the processing unit 110 may generate Figure 3B The processing unit 110 may subtract the minimum light intensity value from each light intensity value of the curve 701, multiply by the adjustment factor, and then add the minimum light intensity value back to generate Figure 7B The angular profile data of the curve 702 shown in FIG. The adjustment factor may be between 0 and 1, and gradually becomes 0 from one end angle to the other end angle. Therefore, the light intensity values ​​of the curve 702 in the one end angle and the other end angle gradually decrease to 0.

[0067] refer to Figure 1 , Figure 7C and Fig.7D , the image data 101 may include a light profile with an angular discontinuity, and the processing unit 110 may adjust at least one light intensity value corresponding to each position in the angular profile data to reduce the light profile with an angular discontinuity. Figure 7CAs shown in FIG. 7 , curve 703 represents angular profile data, and the angular profile data includes a plurality of light intensity values ​​corresponding to different viewing angles at the one screen position (i.e., (x, y)), and curve 703 forms a light profile with discontinuous angles. In an embodiment of the present disclosure, the processing unit 110 may adjust some light intensity values ​​of curve 703 to reduce the light profile with discontinuous angles. After adjusting some light intensity values ​​of curve 703, as shown in FIG. Fig.7D As shown in , the intensity value corresponding to one end point angle of the light intensity distribution of the curve 704 is equal to another intensity value corresponding to another end point angle of the light intensity distribution of the curve 704 .

[0068] Specifically, the processing unit 110 may generate Figure 3B The processing unit 110 may subtract the minimum light intensity value from each light intensity value of the curve 703, multiply by the adjustment factor, and then add the minimum light intensity value back to generate Fig.7D The adjustment factor may be between 0 and 1 and gradually becomes 1 from one end angle to another end angle. Therefore, the light intensity value near the one end angle in the curve 704 increases to another light intensity value at the other end angle in the curve 704.

[0069] refer to Figure 1 and Fig. 7E In one embodiment, the processing unit 110 may also copy Fig. 7A The light intensity value of the curve 701 increases Fig. 7E The data of curve 705 and curve 706 are shown in FIG. Therefore, the processing unit 110 can effectively expand the light profile at other observation angles in the angular profile data.

[0070] Figure 8 is a schematic diagram of a display system according to an embodiment of the present disclosure. Figure 8, the display system 800 includes an electronic device 810 and a display module 820. The electronic device 810 includes an image processing device 811 and a camera 812. The image processing device 811 is coupled to the camera 812 and the display module 820. The image processing device 811 can implement the relevant image processing functions and hardware features of the image processing device 100 in the above embodiments. In an embodiment of the present disclosure, the image processing device 811 and the camera 812 are integrated into the electronic device 810. The electronic device 810 is a terminal device or a computer device, etc. In one embodiment of the present disclosure, the electronic device 810 is a video shooting device, and the display module 820 is a surface light field display device (i.e., a kirameki display device). The camera 812 is configured to generate image data having angular profile data, and provide the image data having angular profile data to the image processing device 811. The image processing device 811 is configured to process the image data having angular profile data, and generate processed image data to drive the display module 820. The camera 812 may be a monocular image sensor or a multi-eye image sensor, or the camera 812 may further include a depth sensor (e.g., a time of flight (ToF) sensor) for obtaining three-dimensional form information of objects in the scene. In one embodiment of the present disclosure, the electronic device 810 may further include a memory device for storing image data. The electronic device 810 may further include a fisheye sensor for obtaining a light map in the scene.

[0071] Fig. 9 is a schematic diagram of a display system according to an embodiment of the present disclosure. Fig. 9 , the display system 900 includes an electronic device 910 and a camera 920. The electronic device 910 includes an image processing device 911 and a display module 912. The image processing device 911 is coupled to the display module 912. The image processing device 911 can implement the relevant image processing functions and hardware features of the image processing device 100 in the above embodiments. In an embodiment of the present disclosure, the image processing device 911 and the display module 912 are integrated into the electronic device 910. The display module 912 is a surface light field display device (i.e., a kirameki display device). The camera 920 is configured to generate image data having angular profile data, and provide the image data having angular profile data to the image processing device 911. The image processing device 911 is configured to process the image data having angular profile data, and generate processed image data to drive the display module 912.

[0072] In one embodiment of the present disclosure, the display module 912 may also include at least one pixel array located on a display module (e.g., a liquid crystal display (LCD) panel, a micro light emitting diode (LED) panel, a micro LED panel, an organic light emitting diode panel, etc.). The display module 912 may also include an optical component that modulates light from a pixel to each different direction (angle) in space, such as a double convex lens, a liquid crystal gradient index (LC GRIN) lens, a parallax grating, or an LC parallax grating. The display module 912 may also include a display control circuit, and the display control circuit is configured to control the angular profile output of the light intensity from each pixel based on the angular profile data of the image data.

[0073] In addition, in one embodiment of the present disclosure, the image processing device 911, the display module 912 and the camera 920 may also be all integrated into the electronic device 910. The electronic device 910 may be, for example, a smart phone or a tablet computer.

[0074] Fig. 10A , Fig. 10C and Fig. 10E is a schematic diagram of an observer observing a display module according to an embodiment of the present disclosure. Fig. 10B , Fig. 10D and Fig.10F is a schematic diagram of angular profile data according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 8 Display system 800 and Fig. 9 The display system 900 may further include an eye tracking sensor coupled to the image processing device 811 and the image processing device 911, respectively, and the eye tracking sensor may be a red / green / blue (RGB) camera, an infrared radiation (IR) camera, or a ToF sensor. The eye tracking sensor is configured to detect the eye position or head position of the observer, and the image processing device of the electronic device may process the image data according to the eye position or the head position to adjust the angle of the emitted light of the display module driven by the processed image data.

[0075] Specifically, in Fig. 10AIn the embodiment of the present disclosure, the display module may include a pixel array 1011 and a lenticular lens 1012. The pixel array 1011 and the lenticular lens 1012 extend in a direction D1 and a direction D3, respectively. The pixel array 1011 may provide an emission light toward a direction D2 through the lenticular lens 1012, and provide the emission light to an observer 1020. Directions D1 to D3 are perpendicular to each other. In an embodiment of the present disclosure, an eye tracking sensor of an electronic device may detect an eye position or a head position of an observer 1020, and an image processing device of the electronic device may process image data according to the eye position or the head position. Fig. 10B As shown in , the image processing device of the electronic device can use the angular profile data corresponding to the angle range 1030 to generate processed image data for driving the pixel array 1011 according to the eye position or the head position of the observer 1020.

[0076] In one embodiment of the present disclosure, Fig. 10C In the embodiment of the present invention, if the observer 1020 moves to observe the display module from the side, the eye tracking sensor of the electronic device can detect that the eye position or head position of the observer 1020 has changed. Fig. 10D As shown in , the image processing device of the electronic device can be changed to use the angular profile data corresponding to the angle range 1040 to generate processed image data for driving the pixel array 1011 to adjust the angle of the emission light provided by the pixel array 1011.

[0077] In one embodiment of the present disclosure, Fig. 10E In the embodiment, if the observer 1020 moves to observe the display module from the other side, the eye tracking sensor of the electronic device may detect that the eye position or head position of the observer 1020 has changed. Fig.10F As shown in , the image processing device of the electronic device can be changed to use the angular profile data corresponding to the angle range 1050 to generate processed image data for driving the pixel array 1011 to adjust the angle of the emission light provided by the pixel array 1011.

[0078] Therefore, the image processing device of the electronic device can effectively process the angular profile data according to the eye position or head position of the observer 1020 to generate processed image data for driving the pixel array 1011. The pixel array 1011 can provide display content with realistic gloss changes within a wide range that change based on the observation angle of the observer.

[0079] Fig.11A , Fig. 11C and Fig.11E is a schematic diagram of an observer observing a display module according to an embodiment of the present disclosure. Fig. 11B , Fig.11D and Fig.11F is a schematic diagram of angular profile data according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 8 Display system 800 and Fig. 9 The display system 900 may further include a motion sensor coupled to the image processing device 811 and the image processing device 911, respectively, and the motion sensor may be an accelerometer or a gyroscope. The motion sensor is configured to detect the display position or tilt angle of the display module, and the image processing device of the electronic device may process the image data according to the display position or tilt angle to adjust the angle of the emitted light of the display module driven by the processed image data.

[0080] Specifically, in Fig.11A In the embodiment of the present disclosure, the display module may include a pixel array 1111, a lenticular lens 1112, and a motion sensor 1113. The pixel array 1111 and the lenticular lens 1112 extend in a direction D1 and a direction D3, respectively. The pixel array 1111 may provide an emission light toward a direction D2 through the lenticular lens 1112, and provide the emission light to an observer 1120. Directions D1 to D3 are perpendicular to each other. In an embodiment of the present disclosure, the motion sensor 1113 may detect a display position or a tilt angle of the display module, and an image processing device of the electronic device may process image data according to the display position or the tilt angle of the display module. Fig. 11B As shown in , the image processing device of the electronic device can use the angular profile data corresponding to the angle range 1130 to generate processed image data for driving the pixel array 1111 according to the display position or tilt angle of the display module.

[0081] In one embodiment of the present disclosure, Fig. 11C In the embodiment, if the display module rotates counterclockwise by a certain angle, the motion sensor 1113 can detect that the display position or tilt angle of the display module has changed. Fig.11D As shown in , the image processing device of the electronic device can be changed to use the angular profile data corresponding to the angle range 1140 to generate processed image data for driving the pixel array 1111 to adjust the angle of the emission light provided by the pixel array 1111.

[0082] In one embodiment of the present disclosure, Fig.11E In FIG. 1 , if the observer 1120 rotates clockwise by a certain angle, the motion sensor 1113 can detect that the display position or tilt angle of the display module has changed. Fig.11F As shown in , the image processing device of the electronic device can be changed to use the angular profile data corresponding to the angle range 1150 to generate processed image data for driving the pixel array 1111 to adjust the angle of the emission light provided by the pixel array 1111.

[0083] Therefore, the image processing device of the electronic device can effectively process the angular profile data according to the display position or tilt angle of the display module to generate processed image data for driving the pixel array 1111. The pixel array 1111 can provide display content with realistic gloss variation that is within a wide range and changes based on the display position or tilt angle of the display module.

[0084] In summary, the image processing method and display system disclosed in the present invention can effectively implement image processing functions on image data with angular profile data to generate appropriate angular profile data required for use. The image processing method can effectively drive the display module according to the processed image data to provide display content with realistic gloss changes in a wide range.

[0085] Those skilled in the art will appreciate that various modifications and variations may be made to the disclosed embodiments without departing from the scope or spirit of the present disclosure. In view of the foregoing, the present disclosure is intended to cover modifications and variations as long as they are within the scope of the following claims and their equivalents.

Claims

1. An image processing method, characterized in that: include: receiving image data having angular profile data corresponding to a plurality of locations; processing the image data by performing at least one of noise filtering, image compensation, image modification, and image discontinuity mitigation to produce processed image data; as well as The processed image data is output.

2. The image processing method according to claim 1, characterized in that: The angular profile data includes a plurality of light intensity values ​​corresponding to a plurality of observation angles.

3. The image processing method according to claim 1, characterized in that: The image data includes a plurality of captured images, and the step of processing the image data includes: A plurality of diffuse light component images and a plurality of specular reflection light component images are separated from the plurality of captured images.

4. The image processing method according to claim 3, characterized in that: The step of separating the plurality of diffuse light component images and the plurality of specular reflection light component images from the plurality of captured images comprises: determining a minimum light intensity component corresponding to each position in the plurality of captured images; generating the plurality of diffuse light component images according to the minimum light intensity component in each of the plurality of captured images; and The minimum light intensity component is subtracted from the each of the plurality of captured images to generate the plurality of specular reflection light component images.

5. The image processing method according to claim 1, characterized in that: The image data includes a truncated light section image, and the step of processing the image data includes: At least one of lost peak data and missing angular profile data in the truncated light profile image is compensated.

6. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: Sampling step compensation is performed on the angular profile data.

7. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: At least one light intensity value in the angular profile data corresponding to at least one specific observation angle corresponding to each position is adjusted.

8. The image processing method according to claim 7, characterized in that: The step of adjusting the at least one light intensity value comprises: A corresponding minimum light intensity value is subtracted from each light intensity value, multiplied by a first adjustment factor, and then the corresponding minimum light intensity value is added back, wherein the first adjustment factor is greater than zero.

9. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: At least one light intensity value corresponding to at least one color corresponding to each position in the angular profile data is modified.

10. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: At least one light intensity value corresponding to each position in the angular profile data is adjusted to reduce the light profile of the angular discontinuity.

11. The image processing method according to claim 10, characterized in that: The step of adjusting the at least one light intensity value comprises: The corresponding minimum light intensity value is subtracted from each light intensity value, multiplied by a second adjustment factor, and then the corresponding minimum light intensity value is added back, wherein the second adjustment factor is between 0 and 1 and gradually becomes 0 from one end point angle of the light intensity distribution at each position to the other end point angle of the light intensity distribution.

12. The image processing method according to claim 10, characterized in that: After adjusting the at least one light intensity value corresponding to each position in the angular profile data, a first intensity value at one end point angle of the light intensity distribution corresponding to each position is equal to a second intensity value at another end point angle of the light intensity distribution corresponding to each position.

13. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: A plurality of light intensity values ​​of the angular profile data are replicated to extend the light profile at other observation angles in the angular profile data corresponding to each position.

14. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: Detecting eye position or head position; and The image data is processed according to the eye position or the head position to adjust the angle of the emitted light of the display module driven by the processed image data.

15. The image processing method according to claim 1, characterized in that: The step of processing the image data comprises: detecting a display position or a tilt angle of a display module; and The image data is processed according to the display position or the tilt angle to adjust the angle of the emitted light of the display module driven by the processed image data.

16. A display system, characterized in that: include: A camera, providing multiple captured images; an image processing device, coupled to the camera, receiving image data of the plurality of captured images, and generating processed image data according to the image data; as well as a display module, coupled to the image processing device, and receiving the processed image data, The image data includes angular profile data corresponding to a plurality of positions, and the image processing device processes the image data by performing at least one of noise filtering, image compensation, image modification, and image discontinuity mitigation to generate processed image data.

17. The display system according to claim 16, characterized in that: Also includes: an eye tracking sensor, coupled to the image processing device, The eye tracking sensor detects eye position or head position, and the image processing device processes the image data according to the eye position or the head position to adjust the angle of the emitted light of the display module driven by the processed image data.

18. The display system according to claim 16, characterized in that: Also includes: a motion sensor, coupled to the image processing device, The motion sensor detects a display position or a tilt angle of the display module, and the image processing device processes the image data according to the display position or the tilt angle to adjust an angle of light emitted by the display module driven by the processed image data.

19. The display system according to claim 16, characterized in that: The image processing device comprises: an input unit, coupled to the camera and receiving the image data of the plurality of captured images; a processing unit coupled to the input unit and processing the image data to generate the processed image data according to the image data; and The output unit is coupled to the processing unit and the display module, and outputs the processed image data to the display module.

20. The display system according to claim 16, characterized in that: The display module is a surface light field display device.

Images