Under-display camera

Abstract

An under-display camera is provided, comprising an image acquisition unit configured to generate an evaluation image based on a first light using an imaging unit including an image sensor. The first light is output by a light-emitting unit including a light source and passes through a display panel. The under-display camera is controlled based on a result of evaluating its performance, obtained by calculating feature values ​​based on the evaluation image, calculating reference values ​​based on a reference image, and comparing the feature values ​​with the reference values, wherein the reference image is obtained based on a second light output by the light-emitting unit, the second light not passing through the display panel.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2020-0189709, filed on December 31, 2020, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] An example embodiment of this disclosure relates to an under-display camera. Background Technology

[0004] With the development of the mobile industry, display technologies included in mobile phones have also been developed. Current display technologies have been developed to increase the screen share of the display panel. Furthermore, regarding the front-facing camera of a mobile phone, technologies that increase the screen share of the display panel may have excellent effects in product design, but there may be limitations that could lead to image quality degradation. Specifically, regarding the image quality degradation that occurs in under-display cameras (UDCs), which can increase the area where the display panel can be placed, there are difficulties in accurately evaluating the performance of UDCs. Summary of the Invention

[0005] One or more example embodiments provide an under-display camera (UDC) whose performance can be qualitatively evaluated by calculating digital feature values ​​based on an evaluation image obtained through an imaging unit via a display panel, and which may have improved performance.

[0006] According to one aspect of an example embodiment, an under-display camera is provided, configured to generate an image based on light transmitted through the display screen when disposed under the display screen, wherein an image acquisition unit including the under-display camera is configured to generate an evaluation image based on a first light using an imaging unit including an image sensor, the first light being output by a light-emitting unit including a light source and transmitted through the display panel, and wherein the result is controlled based on a result evaluating the performance of the under-display camera by calculating feature values ​​based on the evaluation image, calculating reference values ​​based on a reference image, and comparing the feature values ​​with the reference values, the reference image being obtained based on a second light output by the light-emitting unit, the second light not transmitted through the display panel.

[0007] According to one aspect of an example embodiment, an under-display camera is provided, configured to generate an image based on light transmitted through the display screen when disposed under the display screen, wherein an image acquisition unit including the under-display camera is configured to generate an evaluation image based on a first light using an imaging unit including an image sensor, the first light being output by a light-emitting unit including a light source and transmitted through the display panel, and wherein the under-display camera is controlled based on a result of evaluating the performance of the under-display camera, the result being obtained by the following steps: converting the evaluation image into a contour image; calculating a feature value based on the relative position of each of a plurality of positions on the contour line of the contour image relative to the center of the contour image at each of the plurality of positions; and comparing the feature value with a reference value obtained from a reference image, the reference image being obtained based on a second light not transmitted through the display panel.

[0008] According to one aspect of an example embodiment, an under-display camera is provided, configured to generate an image based on light transmitted through the display screen when disposed under the display screen. An image acquisition unit including the under-display camera is configured to generate a first evaluation image and a second evaluation image based on a first light using an imaging unit including an image sensor. The first light is output by a light-emitting unit including a light source and transmitted through the display panel. The under-display camera is controlled based on a result evaluating its performance, the result being obtained through the following steps: calculating a first feature value based on the first evaluation image and calculating a second feature value based on the second evaluation image; and obtaining a first performance evaluation result by comparing the first feature value with a reference value obtained from a reference image and obtaining a second performance evaluation result by comparing the second feature value with the reference value, the reference image being obtained based on a second light that does not transmit through the display panel. Attached Figure Description

[0009] The above and other aspects, features and advantages of this disclosure will become more apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings, in which:

[0010] Figure 1 A development diagram of the camera supplied with the display panel is shown;

[0011] Figure 2 This is a block diagram illustrating an apparatus for evaluating the performance of an under-display camera according to an example embodiment;

[0012] Figure 3 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment;

[0013] Figure 4A and Figure 4B The performance of an under-display camera is evaluated according to an example embodiment;

[0014] Figure 5This is a diagram of a display panel illustrating a method for evaluating the performance of an under-display camera according to an example embodiment;

[0015] Figure 6 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment;

[0016] Figure 7 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment;

[0017] Figure 8 and Figure 9 This is a diagram illustrating an image generated according to an example embodiment using a method for evaluating the performance of an under-display camera;

[0018] Figure 10 and Figure 11 This is a diagram illustrating the parameters used in image analysis relative to a method for evaluating the performance of an under-display camera, according to an example embodiment;

[0019] Figure 12 and Figure 13 This is a diagram illustrating a method for calculating reference values ​​and feature values ​​relative to a method for evaluating the performance of an under-display camera, according to an example embodiment;

[0020] Figure 14 This is a graph illustrating the performance evaluation results of a method for evaluating the performance of an under-display camera according to an example embodiment;

[0021] Figure 15 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment; and

[0022] Figure 16 This is a graph illustrating the performance evaluation results of a method for evaluating the performance of an under-display camera according to an example embodiment. Detailed Implementation

[0023] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0024] Figure 1 A development diagram of the camera supplied with the display panel is shown.

[0025] Reference Figure 1 The display panel and imaging unit (e.g., camera) included in mobile phones have been developed to improve the screen share (or screen ratio) of the display panel. For example, earlier mobile phones including rectangular display panels and imaging units were configured to expose the outer bezel, while in later mobile phones, the shape of the display screen has been changed to reduce the area of ​​the outer bezel, such as... Figure 1 As shown.

[0026] Recently, in order to further increase the screen share of display panels, methods have been developed that remove the outer bezel and install cameras by drilling holes in the display panel in a pop-up manner. In addition, products equipped with under-display cameras (UDC) located under the display screen have been developed as a method to maximize the screen share of display panels.

[0027] However, in under-display cameras, light must pass through the display panel to enter the camera lens, thus reducing light transmittance. Furthermore, opaque areas with a predetermined pattern on the display panel further reduce the transmittance of light entering them. Therefore, the performance of the under-display camera directly affects the quality of the acquired image, and its development requires overcoming issues such as color shift and resolution degradation.

[0028] In related technologies, methods that qualitatively evaluate camera performance using the naked eye are inaccurate. According to example embodiments, quantitative performance evaluation methods can be used to improve the performance of under-display cameras.

[0029] The improved under-display camera according to the example embodiment can be applied to mobile phones, and also to electronic devices such as laptops and televisions.

[0030] Figure 2 This is a block diagram illustrating an apparatus for evaluating the performance of an under-display camera according to an example embodiment.

[0031] Reference Figure 2 The performance evaluation device 1, used to evaluate the performance of the under-display camera in the example embodiment, can quantitatively evaluate the camera's performance. The performance evaluation device 1 may include a light-emitting unit 10, an evaluation image acquisition unit 20, a controller 30, a memory 40, and a reference image acquisition unit 50.

[0032] The light-emitting unit 10 allows light to be incident on the reference image acquisition unit 50 and / or the evaluation image acquisition unit 20 to generate a reference image and / or an evaluation image. The performance of the evaluation image acquisition unit 20 can be evaluated based on the reference image and the evaluation image, according to an example embodiment. Therefore, the light-emitting unit 10 allows light to be incident on the reference image acquisition unit 50 and the evaluation image acquisition unit 20 under the same conditions. However, the example embodiment is not limited thereto.

[0033] The evaluation image acquisition unit 20 can be configured to include an under-display camera and may include a display panel 22 and an imaging unit 25 including an image sensor. For example, the display panel 22 and the imaging unit 25 may be arranged sequentially in the direction of light incidence. Light incident from the light-emitting unit 10 can pass through the display panel 22 and can be input to the imaging unit 25. The display panel 22 may include opaque areas with regular patterns, and the imaging unit 25 may generate an evaluation image with degraded image quality due to the opaque areas of the display panel 22. The generated evaluation image can be converted or analyzed by the controller 30, and the data generated in the process can be stored in the memory 40.

[0034] The reference image acquisition unit 50 may be configured to exclude an under-display camera and may include an imaging unit 55. For example, light incident from the light-emitting unit 10 may be directly incident on the imaging unit 55. The imaging unit 55 may generate a reference image by imaging the incident light. Since the reference image is a comparison target for evaluating the performance of the evaluation image acquisition unit 20 by analyzing the evaluation image, the imaging unit 55 included in the reference image acquisition unit 50 may be configured to have the same performance as the imaging unit 25 included in the evaluation image acquisition unit 20.

[0035] The generated reference image can be converted or analyzed by the controller 30, and the data generated in the process can be stored in the memory 40. The frequency of generating the reference image is not limited to the example embodiment. As an example, the performance of the evaluation image acquisition unit 20 can be evaluated based on multiple evaluation images using a reference image generated at one time. As another example, the performance of the evaluation image acquisition unit 20 can be evaluated by generating a reference image corresponding to the evaluation image whenever an evaluation image is generated.

[0036] The controller 30 can quantify information about the evaluation image and reference image generated by the evaluation image acquisition unit 20 and the reference image acquisition unit 50, respectively, and can compare the quantified information to evaluate the performance of the evaluation image acquisition unit 20. For example, a method for calculating feature values ​​by quantifying information about the evaluation image can correspond to a method for calculating reference values ​​by quantifying information about the reference image. The process of calculating feature values ​​and reference values ​​will be described in more detail later.

[0037] As an example, controller 30 can quantify the degree of similarity between images of regions with the same light intensity from various images and circles, and can compare these degrees with each other. Controller 30 can output a performance evaluation result relative to evaluation image acquisition unit 20 by comparing feature values ​​of the evaluation image with reference values ​​of the reference image. For example, controller 30 can store the performance evaluation result of evaluation image acquisition unit 20 in memory 40.

[0038] However, the example embodiment is not limited thereto. As an example, the performance evaluation device 1 may also include additional components if needed. In addition, besides the functions described above, the controller 30 may also perform other functions, and the memory 40 may store at least a portion of the generated data after the evaluation image and reference image are generated until the result of evaluating the performance of the evaluation image acquisition unit 20 is obtained.

[0039] Figure 3 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment.

[0040] Reference Figure 3 The performance of an under-display camera can be evaluated using feature values ​​based on the evaluation image quantization generated according to the example embodiment.

[0041] Reference Figure 2 and Figure 3 Light can be incident on the imaging unit 25 through the display panel 22 (S110). The imaging unit 25 can generate an evaluation image by imaging the incident light (S120).

[0042] As an example, since the evaluation image is obtained by imaging light that passes through the display panel 22, the image quality of the evaluation image is degraded compared to a normal image obtained by imaging light that does not pass through the display panel 22. Information about the image quality degradation can be included in the evaluation image, and the controller 30 can calculate feature values ​​reflecting the information about the image quality degradation based on the evaluation image (S130). Based on the feature values, the controller 30 can output a result evaluating the performance of the evaluation image acquisition unit 20, which includes the under-display camera (S140).

[0043] However, Figure 3 The operations shown for evaluating performance are merely examples, and the example embodiments are not limited thereto. As examples, additional operations may be included before, during, or after operations S110 to S140, or they may be omitted or modified. Figure 3 Some of the operations described above.

[0044] Figure 4A and Figure 4B This is a diagram illustrating the performance evaluation of an under-display camera according to an example embodiment.

[0045] Figure 4A The process of light L output by the light-emitting unit 10 being incident on the evaluation image acquisition unit 20 is shown.

[0046] In an example embodiment, the light-emitting unit 10 may include a light source 12 and a shielding box 15. For example, the light-emitting unit 10 may be configured as a point light source, and the shielding box 15 may include slits, with the point light source disposed within the shielding box 15.

[0047] The slits in the shielding box 15 may have a predetermined width 'a'. As an example, the predetermined width 'a' may be a value between 1 mm and 3 mm. However, the example embodiment is not limited to this. For example, the predetermined width 'a' may have a value less than 1 mm or greater than 3 mm.

[0048] The light-emitting unit 10 can output light L that moves linearly in one direction through the slits in the shielding box 15. The output light L can be incident on the evaluation image acquisition unit 20. As an example, the evaluation image acquisition unit 20 can be configured to include an under-display camera. Therefore, the incident light L can be incident on the imaging unit 25 through the display panel 22. For example, the incident light L can be vertically incident on the display panel 22 and the imaging unit 25.

[0049] However, the example embodiments are not limited to Figure 4A The example shown, in which the light-emitting unit 10 is configured to output light L that moves linearly in one direction, can be implemented in various forms.

[0050] Figure 4B An example embodiment of the evaluation image acquisition unit 20 may be in the form of a mobile phone. However, the example embodiment is not limited to this, and the form of the evaluation image acquisition unit 20 may not be limited to the example shown.

[0051] As an example, the evaluation image acquisition unit 20 may include a display panel 22 and an imaging unit 25. The display panel 22 may correspond to the screen of a mobile phone, and the imaging unit 25 may correspond to the front-facing camera of the mobile phone. As an example, the display panel 22 may be configured to completely cover the imaging unit 25, and as referred to Figure 4A As described, light incident on the display panel 22 can pass through the display panel 22 and can be incident on the imaging unit 25. The imaging unit 25 can generate an evaluation image based on the incident light to evaluate the performance of the evaluation image acquisition unit 20.

[0052] Figure 5 This is a diagram of a display panel illustrating a method for evaluating the performance of an under-display camera according to an example embodiment.

[0053] Reference Figure 5 The evaluation image acquisition unit 20, used for evaluating the performance of an under-display camera in the example embodiment, may include a display panel 22 and an imaging unit 25.

[0054] Display panel 22 may include examples shown in Figure 5The predetermined pattern shown in (a) to (f) is a regular pattern. The predetermined pattern may include opaque areas, and light incident on the display panel 22 may undergo diffraction due to the predetermined pattern. Furthermore, the opaque areas included in the predetermined pattern can reduce the amount of light incident on the imaging unit 25. For example, the amount of light incident on the imaging unit 25 may be further reduced by approximately 50% to 90% compared to the light incident on the display panel 22. Therefore, the evaluation image generated by the imaging unit 25 may have degraded quality.

[0055] The shape of the evaluation image generated in the example embodiment can be determined based on a predetermined pattern included in the display panel 22. Compared to the method for evaluating the performance of the under-display camera in the example embodiment, since the performance evaluation result can be obtained from the evaluation image, the predetermined pattern included in the display panel 22 can directly affect the performance evaluation result.

[0056] Referring to pattern (a), the display panel 22 may include a pattern having regular circular holes. Referring to pattern (b), the display panel 22 may include a pattern having regular elliptical holes. Additionally, referring to patterns (c) and (d), the display panel 22 may include a pattern that includes holes having polygonal shapes such as hexagons or octagons.

[0057] However, the predetermined pattern included in the display panel 22 is not limited to this. The display panel 22 may include a pattern having circular holes and polygonal holes surrounding the circular holes, as shown in pattern (e). Additionally, as shown in pattern (f), the display panel 22 may include a pattern having holes of the same shape at various locations arranged in one direction and holes of different shapes at different locations in one direction. However, the pattern included in the display panel 22 is not limited to the examples shown in (a) to (f), and the display panel 22 may include patterns of various shapes.

[0058] Figure 6 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment.

[0059] Figure 6 A method for calculating reference values ​​to evaluate the performance of an under-display camera, according to an example embodiment, is shown.

[0060] Return to reference Figure 2 The reference image acquisition unit 50 can generate a reference image by imaging the light incident from the light-emitting unit 10 (S210). Since the light incident on the reference image acquisition unit 50 is directly incident on the imaging unit 55 and does not pass through the display panel, the reference image can be a normal image with less image quality degradation.

[0061] The controller 30 can convert the generated reference image into a grayscale image (e.g., a black and white image) (S220), and can convert the converted grayscale image into a contour image (S230). Based on the operations S220 and S230 of converting the reference image into a grayscale image to generate a contour image, the analysis accuracy of the contour image can be improved. However, the example embodiment is not limited to this; the reference image can be directly converted into a contour image without performing operation S220.

[0062] The controller 30 can calculate a reference value based on the generated contour image (S240). As an example, the reference value can be obtained by quantifying the degree to which regions of the reference image with the same light intensity resemble a circle. For instance, since the reference image is a normal image with minimal image quality degradation, regions with the same light intensity may appear similar to a circle. Therefore, the reference value can include image information when image quality degradation is minimal.

[0063] Figure 7 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment.

[0064] Figure 7 A method for calculating feature values ​​for evaluating the performance of an under-display camera, according to an example embodiment, is shown.

[0065] Return to reference Figure 2 The imaging unit 25 included in the evaluation image acquisition unit 20 can generate an evaluation image by imaging the incident light transmitted through the display panel 22 (S310). As described above, the evaluation image may appear to have degraded quality.

[0066] and Figure 6 The method for converting the reference image shown is similar. The controller 30 can convert the generated evaluation image into a grayscale image (e.g., a black and white image) (S320), and can convert the converted grayscale image back into a contour image (S330). However, the example embodiment is not limited to this; the evaluation image can be directly converted into a contour image without performing operation S320.

[0067] The controller 30 can calculate feature values ​​based on the generated contour image (S340). Similar to the reference value, the feature value can be obtained by quantifying the degree of similarity between the image and a circle in a region of equal light intensity in the evaluation image. Since the evaluation image has degraded image quality, the feature value can include image information that differs from the reference value.

[0068] In the example embodiment, the performance of the evaluation image acquisition unit 20 can be calculated by comparing the feature values ​​with reference values ​​(S350). Therefore, it is possible to predict the degree of similarity between the evaluation image and a circle, as well as how much noise was generated.

[0069] Figure 8 and Figure 9 This is a diagram illustrating an image generated according to an example embodiment using a method for evaluating the performance of an under-display camera.

[0070] Figure 8 and Figure 9 It shows Figure 6 and Figure 7 An example of an image described in the flowchart. As an example, Figure 8 The image shown can be compared with the corresponding Figure 6 It is related to the reference image acquisition unit. Figure 9 The image shown can be compared with the corresponding Figure 7 The evaluation is related to the image acquisition unit.

[0071] Reference Figure 8 The reference image 110 generated by the reference image acquisition unit can be converted into a grayscale image 120, and the grayscale image 120 can be converted into a contour image 130. For further detailed analysis, the contour image 130 can be converted into a three-dimensional (3D) contour image 140. However, the example embodiment is not limited to this, and the above operation can be changed, for example, by directly converting the grayscale image 120 into the 3D contour image 140 or by directly converting the reference image 110 into the contour image 130.

[0072] As described above, since the images 110, 120, 130 and 140 associated with the reference image acquisition unit are based on the reference image 110 with less image quality degradation, their shapes can be similar to circles.

[0073] Reference Figure 9 The evaluation image 210 generated by the evaluation image acquisition unit can be converted into a grayscale image 220, and the grayscale image 220 can be converted into a contour image 230. For further detailed analysis, the contour image 230 can be converted into a 3D contour image 240. However, the example embodiment is not limited to this, and the above operation can be changed to, for example, directly converting the grayscale image 220 into the 3D contour image 240 or directly converting the evaluation image 210 into the contour image 230.

[0074] As described above, since the images 210, 220, 230, and 240 associated with the evaluation image acquisition unit are based on the evaluation image 210 with degraded image quality, their shapes may not be circular, but may have different shapes. For example, the shapes of the images 210, 220, 230, and 240 may be determined based on a predetermined pattern of the display panel included in the evaluation image acquisition unit.

[0075] Figure 10 and Figure 11This is a diagram illustrating the parameters used in image analysis in a method for evaluating the performance of an under-display camera according to an example embodiment.

[0076] Figure 10 It shows the basis Figure 8 The magnified portion of the outline image 130 of the reference image 110 shown. Figure 11 It shows the basis Figure 9 The magnified portion of the contour image 230 of the evaluation image 210 shown.

[0077] According to the method for evaluating the performance of the under-display camera in the example embodiment, reference values ​​and feature values ​​can be calculated by quantifying the degree of similarity between areas of the same light intensity from the generated contour images 130 and 230 and circles.

[0078] Reference Figure 10 The coordinates on the contour image 130 (or the boundary of the contour image 130) are (x i y i In each of the multiple positions of ), it can be based on the coordinates (x) C1 y C1 The phase angle (θ) at the center of the contour image 130. i The reference value is calculated from the distance Di from the center of the contour image 130 to the plurality of locations.

[0079] Reference Figure 11 The coordinates on the contour image 230 (or the boundary of the contour image 230) are (x j y j In each of the multiple positions of ), it can be based on the coordinates (x) C2 y C2 The phase angle (θ) at the center of the contour image 230. j The feature values ​​are calculated from the distance Dj from the center of the contour image 230 to the plurality of locations.

[0080] The parameters used to calculate the reference and eigenvalues ​​can be derived from equations 1 and 2 below.

[0081] [Equation 1]

[0082]

[0083]

[0084] [Equation 2]

[0085]

[0086]

[0087] The phase angle and distance parameters derived from Equations 1 and 2 can be used to indicate the coordinates of multiple locations on the contour images 130 and 230. Additionally, the parameters can be used to calculate reference values ​​and eigenvalues.

[0088] Figure 12 and Figure 13 This is a diagram illustrating a method for calculating reference values ​​and feature values ​​relative to a method for evaluating the performance of an under-display camera, according to an example embodiment.

[0089] Figure 12 It can be shown that based on distance and phase angle Figure 10 The diagram shows various locations on the outline image 130. Figure 13 It can be shown that based on distance and phase angle Figure 11 The diagram shows various locations on the outline image 230. For example, it can be determined that... Figure 13 The smaller the fluctuations in the curve shown, the more the contour image 230 converted from the evaluation image 210 can resemble a circle.

[0090] Based on the above process, we can proceed from... Figure 12 and Figure 13 Calculate reference values ​​and eigenvalues. For example, reference values ​​and eigenvalues ​​may include the diffraction amplitude. diffraction ), standard deviation of diffraction (STDEV) diffraction ) and the average value of diffraction (Mean) diffraction At least one of them.

[0091] As an example, the amplitude of diffraction can be defined as... Figure 12 and Figure 13 The difference between the distance to the position with the largest distance and the distance to the position with the smallest distance. The amplitude of the diffraction can be calculated from Equation 3 below.

[0092] [Equation 3]

[0093] Amplitude diffraction =D max -D min

[0094] The standard deviation of diffraction can be defined as for Figure 12 and Figure 13 The standard deviation of the distances of all phase angles is given. The standard deviation of the diffraction can be calculated from Equation 4 below.

[0095] [Equation 4]

[0096]

[0097] Furthermore, the average value of diffraction can be defined as for... Figure 12 and Figure 13The average distance of all phase angles is given. The average value of the diffraction can be calculated from Equation 5 below.

[0098] [Equation 5]

[0099]

[0100] One or more of Equations 3 to 5 can be used to calculate the reference value and the feature value. Compared to the performance evaluation method in the example embodiment, the ratio of the reference value to the feature value can be output as the performance evaluation result. Therefore, it can be determined that the closer the performance evaluation result is to 100%, the smaller the difference between the evaluation image and the reference image, and the better the performance of the evaluation image acquisition unit.

[0101] When each of the reference value and feature value includes multiple values, multiple performance evaluation results may exist. In this case, a score for evaluating the performance of the image acquisition unit can be calculated based on the multiple performance evaluation results. For example, the performance score of the image acquisition unit may be the average of the multiple performance evaluation results. However, the example embodiment is not limited to this, and the performance score of the image acquisition unit can be calculated by assigning weights to each of the performance evaluation results. The scores calculated for multiple under-display cameras can be compared with each other to determine the under-display camera with better performance.

[0102] Figure 14 This is a graph illustrating the performance evaluation results of a method for evaluating the performance of an under-display camera according to an example embodiment.

[0103] It is possible Figure 14 The example shown is obtained according to the reference. Figures 1 to 13 The performance evaluation results and scores of the method for evaluating the performance of an under-display camera in the described example embodiments are shown.

[0104] Reference Figure 14 The reference values ​​calculated based on the reference image generated by the reference image acquisition unit can be X0, Y0, and Z0, and the feature values ​​calculated based on the evaluation image generated by the evaluation image acquisition unit can be X1, Y1, and Z1. For example, X, Y, and Z can correspond to the average value, standard deviation, and amplitude of the diffraction, respectively.

[0105] Based on the calculated reference and characteristic values, the performance of the under-display camera in the example embodiment can be evaluated as 75%, 10%, and 15%, respectively. When calculating the score from the average of the performance evaluation results, the score (or sub-score) can be determined to be 33.3%.

[0106] Figure 15 This is a flowchart illustrating a method for evaluating the performance of an under-display camera according to an example embodiment.

[0107] Reference Figure 15 , Figures 1 to 14 The method for evaluating the performance of the under-display camera in the example embodiment shown can also be improved by adding a feedback process.

[0108] Operations S410 to S460, which generate reference and evaluation images, calculate reference values ​​and feature values, and output performance evaluation results from them, can correspond to... Figure 3 , Figure 6 and Figure 7 The operation is illustrated. For example, in an example embodiment, light can be incident on the imaging unit through the display panel (S410), and an evaluation image can be generated by imaging the incident light (S420). The evaluation image can then be converted into a grayscale image (e.g., a black and white image) (S430), the grayscale image can be converted into a contour image (S440), and feature values ​​can be calculated (S450). In addition to imaging light that does not pass through the display panel and outputting a performance evaluation result for the evaluation image acquisition unit based on the feature values ​​and a reference value, a reference value can be calculated using the same method (S460).

[0109] The evaluation image generated in the above operation can be the first evaluation image, the calculated feature value can be the first feature value, and the performance evaluation result based on the first feature value can be the first performance evaluation result.

[0110] The system can output a first performance evaluation result and determine whether to change external conditions and whether to perform additional performance evaluation (S470). As an example, external conditions may include the presence and intensity of noise when an image is generated by the imaging unit. As an example, noise may refer to external light, but the example embodiment is not limited thereto. When additional imaging is necessary under changed conditions, the external conditions used to evaluate the performance of the image acquisition unit can be changed (S475).

[0111] Then, a second evaluation image can be generated through the same operations S410 to S460, and a second feature value can be calculated from the second evaluation image. A second performance evaluation result can be output by comparing the second feature value with a reference value. In this case, the reference value may have a different value than the existing reference value depending on the change of external conditions, but the example embodiment is not limited to this. By limiting the external conditions as evaluated above, the influence of noise can be reduced, and the performance of the under-display camera, which has been objectively quantified, can be evaluated.

[0112] Compared to the method for evaluating the performance of the under-display camera in the example embodiment, optimizations for improving the performance of the under-display camera can be performed separately from operation S470. For example, a first performance evaluation result can be output, and it can be determined whether the performance of the evaluation image acquisition unit can be improved by controlling the evaluation device (S480). When performance improvement is possible, the controller can control at least one of the light-emitting unit, display panel, and imaging unit based on the first performance evaluation result before generating the second evaluation image (S485).

[0113] Subsequently, a second evaluation image can be generated through the same operations S410 to S460, and a second feature value can be calculated from the second evaluation image. An improved second performance evaluation result can be output by comparing the second feature value with a reference value.

[0114] Although Figure 15 Although not shown, the performance evaluation result prediction based on the under-display camera adjustments can be performed separately from operations S470 and S480, compared to the method for evaluating the performance of the under-display camera in the example embodiment. For example, even without physically controlling the performance evaluation device as in S485, a second performance evaluation result based on changes in the light-emitting unit, display panel, and imaging unit can be predicted based on the first performance evaluation result. As an example, performance evaluation results can be predicted when the regular patterns included in the display panel have different shapes.

[0115] The development direction of under-display cameras can be easily determined from the predicted performance evaluation results based on changes in the performance evaluation device. Furthermore, the predicted performance evaluation results can be used for diffraction correction to improve the performance of under-display cameras, and also for calibration to maximize performance during mass production.

[0116] When the first performance evaluation result and the second performance evaluation result include multiple values, a first score calculated based on the first performance evaluation result and a second score calculated based on the second performance evaluation result can be calculated. For example, the score for evaluating the performance of the image acquisition unit can be the average of multiple performance evaluation results. However, the example embodiment is not limited to this, and the score for evaluating the performance of the image acquisition unit can be calculated by assigning weights to each of the performance evaluation results. Whether the performance has been improved can be confirmed by comparing the scores calculated based on the feedback operation of the under-display camera. As an example, based on Figure 15 The second score of the second performance evaluation result can be greater than the first score based on the first performance evaluation result.

[0117] Output performance evaluation results can help determine the processing standards for each component included in an electronic device, including an under-display camera. Furthermore, the output performance evaluation results can be used to determine the level of mass production for the product.

[0118] Figure 16 This is a graph illustrating the performance evaluation results of a method for evaluating the performance of an under-display camera according to an example embodiment.

[0119] Can Figure 16 The example shown is obtained according to the reference. Figures 1 to 15 The performance evaluation results and scores of the method for evaluating the performance of an under-display camera in the described example embodiments are shown.

[0120] For example, performance evaluation can be performed sequentially under a first condition of darkness and a second condition in which a specific amount of noise is present. Furthermore, the evaluation device can be optimized, and performance evaluation can again be performed sequentially under the first and second conditions.

[0121] Under the first condition, the reference values ​​calculated based on the reference image generated by the reference image acquisition unit can be X01, Y01, and Z01, and the first feature values ​​calculated based on the first evaluation image generated by the evaluation image acquisition unit can be X11, Y11, and Z11. Under the second condition, the reference values ​​can be X02, Y02, and Z02, and the first feature values ​​calculated based on the first evaluation image can be X12, Y12, and Z12. For example, X, Y, and Z can correspond to the average value, the standard deviation, and the amplitude of the diffraction, respectively.

[0122] Based on the reference value and the first feature value calculated under the first condition, the performance evaluation results of the under-display camera in the example embodiment are 75%, 10%, and 15%, respectively. When the score is calculated by averaging the performance evaluation results, the score is determined to be 33.3%.

[0123] Based on the reference value and the first feature value calculated under the second condition, the performance evaluation results of the under-display camera in the example embodiment can be obtained as 80%, 15% and 20%, respectively, and when the score is calculated by the average of the performance evaluation results, the score can be determined to be 38.3%.

[0124] As mentioned above, the performance evaluation of an under-display camera can achieve a high score when a certain amount of noise is present. However, the example embodiment is not limited to this. As an example, Figure 16 Each of the performance evaluation result values ​​shown may vary in the example embodiment. Additionally, under the second condition where noise is present, the result evaluating the performance of the under-display camera may be a lower score.

[0125] After the performance evaluation based on the first feature value is terminated, the operation of the control evaluation device can be performed to improve the performance of the image acquisition unit. Therefore, a second performance evaluation result different from the first performance evaluation result can be output.

[0126] The second feature value can be calculated under each of the first and second conditions, and can be compared with a reference value. For example, under the first condition, the second feature value calculated based on the second evaluation image generated by the evaluation image acquisition unit can be X21, Y21, and Z21. Under the second condition, the second feature value calculated based on the second evaluation image can be X22, Y22, and Z22.

[0127] Based on the reference value and the second characteristic value calculated under the first condition, the performance evaluation results of the under-display camera in the example embodiment are 85%, 30%, and 35%, respectively. When the score is calculated by averaging the performance evaluation results, the score is determined to be 50%.

[0128] Based on the reference value and the second characteristic value calculated under the second condition, the performance of the under-display camera in the example embodiment can be evaluated as 90%, 45%, and 50%, respectively, and the score can be 61.6% when the average of the performance evaluation results is used to calculate the score.

[0129] As described above, performance evaluation results can achieve higher scores by experiencing feedback operations through the under-display camera. However, the example embodiment is not limited to this. As an example, Figure 16 Each of the performance evaluation result values ​​shown may vary in the example embodiment. Additionally, under the second condition where noise is present, the result evaluating the performance of the under-display camera may be a lower score.

[0130] According to the foregoing example embodiments, the under-display camera can acquire evaluation images using an evaluation image acquisition unit including a UDC, and based on these images, characteristic values ​​and performance evaluation results for quantitatively evaluating the performance of the UDC can be obtained. Therefore, the UDC can be controlled based on the performance evaluation results. For example, the development direction of the UDC can be determined, and optimizations to improve the performance of the UDC can be performed.

[0131] According to exemplary embodiments, at least one of the components, elements, modules, and units described herein can be implemented as various numbers of hardware, software, and / or firmware structures performing the various functions described above. For example, at least one of these components, elements, modules, and units can use a direct circuit structure (such as a memory, processor, logic circuit, lookup table, etc.) that can perform its respective function under the control of one or more microprocessors or other control devices. Furthermore, at least one of these components, elements, modules, and units can be implemented by a module, program, or a portion of code containing one or more executable instructions for performing a specified logical function and executed by one or more microprocessors or other control devices. Additionally, at least one of these components, elements, modules, and units can also include or be implemented by a processor such as a central processing unit (CPU), microprocessor, etc., performing the corresponding function. Two or more of these components, elements, modules, and units can be combined into a single component, element, module, and unit that performs all the operations or functions of the combined two or more components, elements, modules, and units. Furthermore, at least a portion of the function of at least one of these components, elements, modules, and units can be performed by another of these components, elements, modules, and units. Furthermore, although the bus is not shown in the block diagram, communication between components, elements, modules, and units can be performed via the bus. The functional aspects of the above example embodiments can be implemented in algorithms executed on one or more processors. Moreover, the components, elements, modules, and units represented by blocks or processing operations can employ any number of related techniques for electronic configuration, signal processing and / or control, data processing, etc.

[0132] Although some exemplary embodiments have been described and illustrated above, it will be apparent to those skilled in the art that modifications and alterations may be made without departing from the scope of this disclosure as defined by the appended claims and their equivalents.

Claims

1. An under-display camera disposed beneath a display screen and configured to generate an image based on light transmitted through the display screen. in, The image acquisition unit, including the under-display camera, is configured to generate an evaluation image based on a first light using an imaging unit including an image sensor. This first light is output by a light-emitting unit including a light source and passes through a display panel. The under-display camera is controlled based on the performance evaluation results. The results are obtained by calculating feature values ​​based on the evaluation image, calculating reference values ​​based on a reference image, and comparing the feature values ​​with the reference values. The reference image is obtained based on second light output by the light-emitting unit, which does not pass through the display panel.

2. The under-display camera according to claim 1, wherein, The result for evaluating the performance is obtained based on the ratio of the reference value to the feature value.

3. The under-display camera according to claim 1, wherein, The evaluation image is converted into a grayscale image, the grayscale image is converted into a contour image, and the feature value is obtained based on the contour image.

4. The under-display camera according to claim 1, wherein, The evaluation image is converted into a contour image, and the feature value is obtained based on the contour image.

5. The under-display camera according to claim 1, wherein, The feature value is calculated based on the degree of similarity between the contour line of the evaluation image and a circle.

6. The under-display camera according to claim 5, wherein, The result evaluating the performance is obtained by calculating the reference value based on a second degree of similarity between the contour line of the reference image and the circle, and by comparing the first degree with the second degree.

7. An under-display camera configured to generate an image based on light transmitted through the display screen when positioned beneath the display screen. in, The image acquisition unit, including the under-display camera, is configured to generate an evaluation image based on a first light using an imaging unit including an image sensor. This first light is output by a light-emitting unit including a light source and passes through a display panel. The under-display camera is controlled based on the results of evaluating its performance, which are obtained through the following steps: converting the evaluation image into a contour image; calculating a feature value at each of a plurality of locations on the contour line of the contour image based on the relative position of each location with respect to the center of the contour image; and comparing the feature value with a reference value obtained from a reference image, which is obtained based on a second light that does not penetrate the display panel.

8. The under-display camera according to claim 7, wherein, The contour image is obtained by converting the evaluation image into a grayscale image and then converting the grayscale image into the contour image.

9. The under-display camera according to claim 7, wherein, Each of the reference value and the characteristic value includes at least one of the diffraction amplitude, the standard deviation of the diffraction, and the average value of the diffraction.

10. The under-display camera according to claim 9, wherein, Each of the reference value and the feature value includes the amplitude of the diffraction, which is defined by the difference between a maximum distance from the center of the contour image to a first position on the contour line of the contour image and a minimum distance from the center of the contour image to a second position on the contour line of the contour image.

11. The under-display camera according to claim 9, wherein, Each of the reference value and the feature value includes the standard deviation of diffraction, which is defined as the standard deviation of the distance from the center of the contour image to the plurality of locations.

12. The under-display camera according to claim 9, wherein, Each of the reference value and the feature value includes the average value of the diffraction, which is defined as the average distance from the center of the contour image to the plurality of locations.

13. The under-display camera according to claim 7, wherein, The relative position of each location is defined by the phase angle relative to the center of the contour image and the distance from the center of the contour image.

14. An under-display camera disposed beneath a display screen and configured to generate an image based on light transmitted through the display screen. in, The image acquisition unit, including the under-display camera, is configured to generate a first evaluation image and a second evaluation image based on a first light using an imaging unit including an image sensor. The first light is output by a light-emitting unit including a light source and passes through the display panel. The under-display camera is controlled based on the performance evaluation results, which are obtained through the following steps: calculating a first feature value based on the first evaluation image and calculating a second feature value based on the second evaluation image; and obtaining a first performance evaluation result by comparing the first feature value with a reference value obtained from a reference image and obtaining a second performance evaluation result by comparing the second feature value with the reference value, wherein the reference image is obtained based on a second light that does not pass through the display panel.

15. The under-display camera according to claim 14, wherein, The image acquisition unit is also configured to generate the first evaluation image and the second evaluation image under different external conditions.

16. The under-display camera according to claim 15, wherein, The different external conditions are related to the presence of noise and the intensity of the noise relative to the first evaluation image or the second evaluation image.

17. The under-display camera according to claim 14, wherein, The imaging unit of the image acquisition unit is controlled based on the first performance evaluation result before the second evaluation image is generated.

18. The under-display camera according to claim 14, wherein, Based on the first performance evaluation result and the second performance evaluation result, which each include multiple values, the performance of the under-display camera is evaluated by calculating a first score from the multiple values ​​of the first performance evaluation result and a second score from the multiple values ​​of the second performance evaluation result.

19. The under-display camera according to claim 14, wherein, Diffraction correction is performed on the under-display camera based on the results of the performance evaluation.

20. The under-display camera according to claim 14, wherein, The under-display camera is controlled based on the first performance evaluation result, such that after the control, the image acquisition unit is configured to generate a second evaluation image whose second feature value is closer to the reference value.

Images