Interference pattern detection methods, compensation methods, and circuitry suitable for under-screen cameras

Abstract

The present application provides an under-screen camera interference pattern detection method, compensation method and circuit system, the circuit system includes image processing circuit and under-screen camera module, the lens in the under-screen camera module shoots through the glass substrate of a display module, gets the image and transmits to the image processing circuit, detects the interference pattern in the image by the image processing circuit and compensates. In this method, a plurality of reference points in the image are set, the brightness average value and the brightness target value of each reference point are obtained, a region brightness ratio is calculated and a binary image is obtained after binarization, it is judged that there is an interference pattern according to the symmetry of the binary image, then the target matrix can be determined according to the weight value of all reference points in the image to compensate the interference pattern.

Description

Technical Field

[0001] This invention relates to a technique for processing interference patterns in images, and particularly to an image detection method, compensation method, and circuit system for interference patterns in images captured by an under-display camera. Background Technology

[0002] A camera module is typically installed on a mobile device. Generally, the lens in the camera module needs to be exposed. However, on mobile devices with a full-screen display design, it may be necessary to make openings or leave special spaces in the transparent surface structure (which may include thin-film transistor (TFT) panels, various optical films, surface glass, etc.) to expose the camera lens in order to take photos and videos smoothly.

[0003] Subsequently, to ensure a complete display area for full-screen displays, under-screen camera technology was developed, allowing the camera module and lens to be placed beneath the display screen. The design of an under-screen camera is as follows: Figure 1 The structural diagram shows that a camera module 10 is provided under the full-screen display panel 100. The camera module 10 includes a photosensitive element 103 and a lens 101. The diagram shows that the lens 101 is placed within the encapsulation structure of the full-screen display panel 100.

[0004] However, placing the under-display camera lens 101 inside the glass panel means that shooting will be done through the glass, and shooting through the glass may result in images such as... Figure 2 The interference patterns shown are optically known as Newton's rings, commonly seen as rainbow-like interference fringes. Furthermore, the phenomenon of Newton's rings appearing in under-display camera photography is due to the fact that when a single-wavelength light source passes through the full-screen display panel 100, the curved surfaces created by the gaps between different materials in its multi-layered structure cause an optical path difference between the incident and reflected light. Since the air film thickness is the same at the same radius of the ring, this optical path difference results in a ring pattern during imaging. When the light source has different wavelengths, rainbow-like rings are formed.

[0005] It is worth mentioning that the closer the light source is to a single wavelength, the more obvious the interference phenomenon becomes. Under a specific light source, the bright and dark fringes of Newton's rings will also change position due to the temporal coherence of the interference phenomenon, so it cannot be solved with a fixed correction value. Summary of the Invention

[0006] Given that under-display cameras can cause interference phenomena and generate Newton's rings (or rainbow rings) during shooting due to the structural design of the display panel (including structural materials, gaps, thickness, etc.), this invention proposes an interference pattern detection method, compensation method, and circuit system suitable for under-display cameras. The method utilizes image processing techniques to perform interference pattern detection and compensation, and can perform compensation based on the image characteristics of the interference pattern. In particular, it can adapt to various light source changes and perform dynamic compensation.

[0007] According to the embodiment, the main components of the proposed circuit system include an image processing circuit and an under-display camera module. The under-display camera module is located inside a display module. The lens in the under-display camera module takes a picture through the glass substrate of the display module to obtain an image and transmit it to the image processing circuit. Then, the image processing circuit detects and compensates for the interference pattern in the image.

[0008] The steps for detecting and compensating for interference patterns include: first, setting multiple reference points in the image and defining a first region for each reference point; then, calculating the average brightness value for the first region of each reference point; next, defining a second region for each reference point and calculating the target brightness value for the second region of each reference point; and finally, calculating a region brightness ratio based on the average brightness value and the target brightness value of each reference point. Next, a binarized image is generated based on the region brightness ratios of all reference points in the image. The symmetry of the patterns within this image can be used to determine whether an interference pattern exists. Weight values ​​are then determined, and a target matrix is ​​defined based on the weight values ​​of each reference point in the image. This target matrix describes the compensation coefficients for all reference points in the image, thus compensating for the identified interference pattern.

[0009] Preferably, if the target brightness value is greater than the average brightness value, there are dark fringes near the corresponding reference point; if the target brightness value is less than the average brightness value, there are bright fringes near the corresponding reference point. The ratio of bright areas in a region is calculated by dividing the target brightness value by the average brightness value.

[0010] Preferably, the interference pattern can be a Newton's rings, with wider rings near the center and thinner rings further away from the center. Thus, for each reference point, there is the same first region, and for the coverage of the second region, the closer to the center of the Newton's rings, the larger the second region needs to be.

[0011] Furthermore, the conditions for satisfying the symmetry of the binarized image include: the number of black and white dots at the black-white boundary of the binarized interference pattern is the same, and the positional deviation of the calculated center point at the boundary of each ring of the interference pattern is not greater than a deviation threshold.

[0012] Furthermore, the interference patterns generated in the image are located at different positions in the red, green, and blue channels. Specifically, the brightness ratio of each reference point in different color channels is calculated, and interference pattern compensation is performed for each color channel. Target matrices for different color channels in the image are also obtained to perform compensation for each channel.

[0013] Furthermore, the compensation coefficients of each reference point in the image are described by the target matrix. Each color channel of each pixel in the image can be corrected by bilinear interpolation to compensate for the interference pattern.

[0014] Furthermore, there are multiple sets of target matrices, and the interference pattern is dynamically compensated by switching between multiple sets of target matrices.

[0015] To further understand the features and technical content of the present invention, please refer to the following detailed description and illustrations of the present invention. However, the illustrations provided are for reference and illustration only and are not intended to limit the present invention. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of an under-display camera according to the present invention;

[0017] Figure 2 This is a schematic image of Newton's rings according to the present invention;

[0018] Figure 3 This is a flowchart illustrating an embodiment of the interference pattern detection and dynamic target matrix compensation method suitable for under-display cameras according to the present invention.

[0019] Figure 4 This is a flowchart illustrating an embodiment of calculating the brightness ratio of a region according to the present invention;

[0020] Figure 5 This is a schematic diagram of an embodiment of setting M×N reference points in an image according to the present invention;

[0021] Figure 6 This is a schematic diagram illustrating an embodiment of the present invention in which each reference point in an image has a different coverage area;

[0022] Figure 7 This is a flowchart illustrating an embodiment of the detection of interference patterns according to the present invention;

[0023] Figure 8A , 8B The binary image is obtained by comparing the brightness ratios of different color channels in the image shown according to the present invention with 8C.

[0024] Figure 9A , 9BThe binarized image obtained by dividing 9C by the brightness ratio of different color channels in the image shown according to the present invention;

[0025] Figure 10 This is a schematic diagram of an embodiment of the present invention, using a ring-shaped interference pattern as an example, for finding symmetrical points;

[0026] Figure 11 This is a flowchart illustrating an embodiment of determining image symmetry according to the present invention;

[0027] Figure 12 This is a graph illustrating the relationship between regional brightness ratio and weight value according to the present invention.

[0028] Figure 13 This is an example of an image according to the present invention;

[0029] Figure 14 This is a schematic diagram of compensation coefficient interpolation according to the present invention;

[0030] Figure 15 This is a flowchart illustrating an embodiment of the interference pattern detection and dynamic switching target matrix compensation method suitable for under-display cameras according to the present invention; and

[0031] Figure 16 This is a circuit system embodiment diagram illustrating the method for detecting and dynamically compensating interference patterns in an under-display camera according to the present invention. Detailed Implementation

[0032] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. Furthermore, the accompanying drawings of the present invention are for simple illustrative purposes only and are not depicted according to actual dimensions; this is stated in advance. The following embodiments will further describe the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention.

[0033] It should be understood that while terms such as “first,” “second,” and “third” may be used in this document to describe various components or signals, these components or signals should not be limited by these terms. These terms are primarily used to distinguish one component from another, or one signal from another. Furthermore, the term “or” as used herein should, as appropriate, include any combination of one or more of the associated listed items.

[0034] To address the issue of interference patterns, such as Newton's rings, appearing in images captured by under-display cameras, this invention proposes an interference pattern detection method, compensation method, and circuit system suitable for under-display cameras. The main reason for interference patterns in images captured by under-display cameras is interference caused by factors such as the structural material, layer gaps, and thickness of the display panel. Interference patterns, such as Newton's rings (or rainbow rings), are detected primarily based on the image characteristic of Newton's rings and similar interference fringes, which consist of multiple alternating bright and dark rings. The method involves analyzing the brightness distribution to detect interference patterns in the image, and specifically proposes a dynamic compensation method that adapts to changes in light source.

[0035] An interference pattern detection and compensation method suitable for under-display cameras is implemented in an image processing circuit. This circuit processes images captured by the under-display camera and can be applied to mobile devices. One of the technical objectives is to eliminate interference patterns in images captured by the under-display camera through post-processing. However, different light sources cause interference patterns to varying degrees. For example, the closer the light source is to a single wavelength, the more pronounced the interference phenomenon. Taking Newton's rings as an example, the ring-shaped interference pattern formed by bright and dark areas is more obvious. For instance, a 3000K U30 light source can clearly show Newton's rings, or rainbow rings. However, the same situation may not show obvious Newton's rings under specific light sources. Furthermore, even under the same light source, the bright and dark fringes formed by the interference phenomenon will still change position due to the temporal coherence of the interference phenomenon, so they cannot be solved with a fixed correction value.

[0036] According to the proposed method for interference pattern detection and compensation suitable for under-display cameras, please refer to the following embodiments. Figure 3 The flowchart of the embodiment shown is illustrated, in which the compensation method is dynamic compensation, which can adapt to changes in different light sources.

[0037] like Figure 3 As shown in step S301, an image is initially divided into multiple regions to calculate the brightness ratio of each region. Calculating region brightness is a method of averaging the brightness of a region, and its implementation is as follows: Figure 4 The flowchart for calculating the brightness ratio of the region is shown in step S401. Refer to... Figure 5 As shown, M×N reference points are set in an image, each representing a region within the image. In step S403, an appropriate square size is selected based on these reference points to define a first region. In the example shown, a first region 51 is selected at reference point 501, and a corresponding first region 52 is selected at another reference point 502. The other reference points can also have corresponding first regions defined. Next, as shown in step S405, the average reference point brightness (C) of the region covered by the first regions 51 and 52 corresponding to each reference point 501 and 502 is calculated.

[0038] Next, as shown in step S407, a second region corresponding to each reference point is set according to the actual situation of the generated interference pattern, such as... Figure 5 The second region 511 corresponding to reference point 501 and the second region 521 corresponding to reference point 502 shown in the image represent the area range for calculating the reference point brightness target value (T). That is, in step S409, the reference point brightness target value (T) is calculated from the pixel values ​​in the second regions 511 and 521 corresponding to each reference point 501 and 502.

[0039] Based on the average brightness (C) and target brightness (T) of each reference point, it can be concluded that if the target brightness (T) is greater than the average brightness (C), there are dark fringes in the vicinity of the corresponding reference point; conversely, if the target brightness (T) is less than the average brightness (C), there are bright fringes in the vicinity of the corresponding reference point. Then, as in step S411, the brightness ratio of a region is calculated using the target brightness (T) and average brightness (C) of each region (target brightness divided by average brightness, T / C). In this way, it can be determined that dark fringes require a ratio greater than 1 (T / C>1), while bright fringes require a ratio less than 1 (T / C<1).

[0040] It should be noted that when selecting the first and second regions, the size of the regions can be determined based on the actual observation of the bright and dark fringes produced by the interference pattern under different light sources. If the selected region is too small, it may highlight image noise. Therefore, the region size needs to be moderate and dynamically adjusted according to actual needs. For example, with Newton's rings, generally speaking, the part near the center of Newton's rings will have wider rings (composed of bright and dark fringes), while the part far from the center of Newton's rings will show thinner rings. Figure 5 The coverage area of ​​the second regions 511 and 521 shown is specifically determined by their distance from the center of the image (corresponding to the location of the under-display camera on the display panel). The closer the portion is to the center of the image, which could also be the center of Newton's rings, the larger the second regions 511 and 521 need to be. For example, see [reference needed]. Figure 6 The diagram shows that the first reference point 61 (coordinates 15, 12) near the center of the image and the second reference point 62 (coordinates 5, 6) far from the center of the image have the same first region size, such as 8×8 pixels; however, the second region of the first reference point 61 (52×52 pixels) near the center of the image is larger than the second region of the second reference point 62 (42×42 pixels) far from the center of the image. In this example, after calculating the region brightness ratio (T / C), the T / C obtained by the second reference point 62 at the bright fringes is 0.977, and the T / C obtained by the first reference point 61 at the dark fringes is 1.043.

[0041] Furthermore, because the interference patterns generated by the red, green, and blue channels are located at different positions, it is necessary to calculate the area luminance ratio (T / C) separately. That is, when actually calculating the average luminance (C) and the target luminance (T), the average luminance (C), the target luminance (T), and the area luminance ratio (T / C) for each region are calculated separately for the different color channel values ​​(such as red, green, and blue channel values) of the pixels in each region. Figure 5 Taking the reference point diagram in the image as an example, there are M×N reference points in the image. Each reference point has a regional brightness ratio (T / C) for the red, green and blue channels respectively. Therefore, subsequent compensation is also performed on the interference pattern for each channel.

[0042] After calculating the area brightness ratio (T / C) of the reference point in the image, proceed with... Figure 3 Step S303 involves detecting interference patterns in the image, such as ring-shaped interference patterns. This detection and compensation can be performed only on light sources that produce interference patterns, while neglecting to compensate for light sources that do not. Therefore, to adapt to changes in the light source environment, interference patterns need to be detected. For detailed procedures on interference pattern detection, please refer to [reference needed]. Figure 7 The flowchart of the embodiment shown is shown.

[0043] In step S701, the region brightness ratio (T / C) of each color channel at each reference point is first obtained, and then, as in step S703, a binarized image is obtained. In the above embodiment, the region brightness ratio (T / C) for each reference point is obtained, and based on this ratio, an M×N dimension binarized image can be obtained. In this way, the symmetry of the binarized image can be used to determine whether an interference pattern exists. However, binarizing solely based on the region brightness ratio may lead to misjudgments in determining the symmetry of the interference pattern. One major reason is that the rings of the interference pattern may not be clearly visible. Therefore, as in step S705, the region brightness ratios of different color channels can be divided to highlight the boundaries of the interference pattern, thereby improving the accuracy of interference pattern detection.

[0044] like Figure 8A , 8B The image shown, along with 8C, is a binarized image derived from the region brightness ratio (T / C) of different color channels (red, green, and blue); then referencing... Figure 9A , 9B The binarized image obtained by dividing the brightness ratio of different color channels in the image shown in 9C by the difference between the wavelengths of the different color channels is more likely to result in interference patterns that can be easily identified. For example... Figure 9AThe image shown is a binarized image obtained by dividing the brightness ratio of the green channel region by the brightness ratio of the red channel region (G / R). Figure 9B The image shown is a binarized image obtained by dividing the brightness ratio of the blue channel region by the brightness ratio of the red channel region (B / R). Figure 9C The image shown is a binarized image obtained by dividing the brightness ratio of the green channel region by the brightness ratio of the blue channel region (G / B), compared to... Figure 8A , 8B Compared to the binarized image shown in 8C, Figure 9A , 9B The binarized image obtained from 9C can improve the accuracy of detecting interference patterns.

[0045] After obtaining the binarized image, as in step S707, symmetrical points are found based on the characteristics of the interference pattern symmetry to determine the symmetry (step S709). This allows for the detection of the interference pattern in the image. (Refer to...) Figure 10 The diagram illustrates an embodiment for finding symmetrical points using a ring-shaped interference pattern as an example, and can also be referenced at the same time. Figure 11 The flowchart shown illustrates an embodiment for determining image symmetry.

[0046] Figure 10 This shows a binarized annular interference pattern. The boundary between white and black is found by image scanning along the X-axis. Figure 11 (Step S111) Take black dots and mark the intersection points B1, B2, B3, B4, B5 in sequence (from left to right in this example); Also, along the X-axis, find the intersection points where black turns white, take white dots, and mark the intersection points A1, A2, A3, A4, A5 in sequence (from left to right in this example). This allows you to determine the number of black and white dots. Figure 11 In step S113), to accurately determine whether an image contains a ring interference pattern, the characteristics of a ring interference pattern must be met. Therefore, the number of points A (white dots) and B (black dots) must be the same (condition one) before proceeding to the next step of calculating the center point of each ring. This example shows a ring interference pattern with 5 rings from the outside in. Next, the position of the center point of each ring is calculated. Figure 11Step S115 involves determining the boundary positions of each ring along the X-axis and then calculating the center point position of each ring. Specifically, the center point (closest to the center of the interference pattern) of the left boundary B1 and right boundary A5 of the first ring is (B1+A5) / 2; the center point of the left boundary A1 and right boundary B5 of the second ring is (A1+B5) / 2; and so on, calculating the center point position ring by ring until the last ring closest to the image center. This example shows the center point of the left boundary B3 and right boundary A3 of the fifth ring, which is (A3+B3) / 2. The center points of these five rings should be almost equal to the center point of the M×N binary image; this example only shows finding the center point along the X-axis.

[0047] Therefore, when determining symmetry, the positional deviation of the center point can be calculated based on the intersection positions of the above rings to determine whether the interference pattern has symmetry (Condition 2). Figure 11 (Step S117). According to an embodiment, a deviation threshold (k1) can be set, and the deviation of the calculated center point positions of each ring should be less than this deviation threshold (k1) to determine symmetry; further, in another embodiment, the deviations of the calculated center point positions of all rings from each other can be accumulated, and the number of deviations less than the deviation threshold (k1) can be accumulated. Figure 11 (Step S119) This cumulative number can serve as a basis for determining whether there is an interference pattern, especially for ring-shaped interference patterns such as Newton's rings.

[0048] according to Figure 7 In the illustrated embodiment of the process, in step S711 of determining whether there is an interference pattern, a cumulative quantity threshold (k2) is set. When the cumulative number of the deviation threshold is greater than or equal to this cumulative quantity threshold (k2), it can be said that the characteristics of the interference pattern are met. In this example, it is a ring interference pattern, that is, it is determined that the image has a ring interference pattern; otherwise, it is determined that the image does not have a specific interference pattern.

[0049] Back Figure 3 In the embodiment of the detection interference pattern shown, after... Figure 7 The illustrated process can determine whether an image contains an interference pattern. Figure 3 (Step S305) If the image does not contain an interference pattern (No) based on the characteristics of the interference pattern, the detection process ends (Step S307); if there is an interference pattern (Yes), the interference pattern is then compensated for.

[0050] continue Figure 3The illustrated process embodiment then proceeds to determine the calculation weight for each region's brightness ratio (T / C). This weight setting highlights areas with bright-dark interference patterns and eliminates potential erroneous judgments (step S309). Based on the region brightness ratio (T / C) calculated for each reference point in the image in step S301, it can be seen that in a grayscale image, the region brightness ratio is close to 1, while when there are other objects with boundaries in the image, the region brightness ratio deviates from 1. At this time, reference can be made... Figure 12 The diagram illustrates a formula for determining a calculation weight based on the region brightness ratio. The vertical axis represents the weight value, with a maximum weight value (max_wgt). The horizontal axis represents the absolute value of the region brightness ratio (T / C) minus 1 (|(T / C)-1|). A first threshold (thr1) and a second threshold (thr2) are defined for |(T / C)-1|. The diagram shows that a weight value is determined based on the value of |(T / C)-1| for each reference point. When |(T / C)-1| is less than or equal to the first threshold (thr1), the weight value is set to the maximum weight value. When it is greater than or equal to the second threshold (thr2), the weight value is 0. When |(T / C)-1| is between the first threshold (thr1) and the second threshold (thr2), the weight value is obtained through interpolation. This formula eliminates unreliable judgment results; for example, it ensures that values ​​that cumulatively conform to interference pattern characteristics are confirmed before being included in the calculation.

[0051] Here's an example: set the maximum weight value (max_wgt) to 0.5, the first threshold (thr1) to 0.05, and the second threshold (thr2) to 0.08, while also referring to... Figure 13 The image shown is an example of an interference pattern detection to be performed. In this example, the first reference point (coordinates (5, 2)) 131 is located in the gray image area, so the area brightness ratio (T / C) is close to 1. The absolute value of the difference between this and 1 is 0.008 (|(T / C)-1|=0.008). Since this is less than the first threshold (thr1), the obtained weight value is the maximum weight of 0.5. The second reference point (coordinates (20, 14)) 132's second area covers the color block in the image. The absolute value of the difference between the area brightness ratio and 1 (|(T / C)-1|) is 0.057. This value is between the first threshold (thr1) and the second threshold (thr2). The weight obtained by linear interpolation is 0.375, calculated as: (thr2-|(T / C)-1|) / (thr2-thrl)*Max_wgt=0.375.

[0052] The figure shows that the first region of the third reference point (coordinates (10, 23)) 133 falls within the color block in the image, while the second region covers most of the color block, resulting in a larger area brightness ratio. This means that when the area brightness ratio deviates too much, it is ignored, and the previously reliable area brightness ratio is used (stored in the circuit system's memory). In this example, the absolute value of the difference between the area brightness ratio and 1 (|(T / C)-1|) is 0.182, which is greater than the second threshold (thr2). Figure 12 The relation in the equation yields a weight of 0.

[0053] After determining the calculation weights (step S309), the next step is to determine a target matrix (step S311). The purpose of this step is to adjust the calculated area brightness ratio (T / C) in the current image and the target matrix for correction of the previous image based on the obtained weight values. An example of adjusting the target matrix is ​​shown below.

[0054] Cur R_Gain(j，i) Cur G_Gain(j，i) and Cur B_Gain(j，i) It is the ratio of the brightness of the red, green, and blue channels in the current image; Last R_Gain(j，i) Last G_Gain(j，i) And Last B_Gain(j，i) It is the target matrix used for color channel correction in the previous image; while R Gain(j，i) G Gain(j，i) And B Gain(j，i) This is the target matrix used for color channel correction in the current image. The target matrix describes the compensation coefficients (which can be represented by gain values) for each color channel at each reference point. After correction is completed, the results will be updated to Last. R_Gain(j，i) Last G_Gain(j，i) And Last B_Gain(j，i) The weight values ​​obtained in step S309 are then stored in the memory of the circuit system. The formula for adjusting the target matrix using the weight values ​​obtained in step S309 is as follows:

[0055] R Gain(j，i) =wgt*Cur R_Gain(j，i) +(1-wgt)*Last R_Gain(j，i)

[0056] G Gain(j，i) =wgt*Cur G_Gain(j，i) +(1-wgt)*Last G_Gain(j，i)

[0057] B Gain(j，i) =wgt*Cur B_Gain(j，i) +(1-wgt)*Last B_Gain(j，i)

[0058] Ultimately, as Figure 3 Step S313 in the process involves compensating the determined interference pattern using the target matrix, which can be referred to as... Figure 5 The image shown contains M×N reference points (the number is not limited), and the target matrix (R) Gain(j，i) G Gain(j，i) And B Gain(j，i) This describes the compensation coefficients for all reference points in the image on each color channel. It corrects the color channels of each pixel in the image using bilinear (two axes in a plane) interpolation to compensate for interference patterns. The method of performing the correction using bilinear interpolation can be found in [reference needed]. Figure 14 .

[0059] Figure 14 The image shows the current position of pixel 140, with several neighboring reference points labeled (i1, j1), (i2, j2), (i3, j3), and (i4, j4). The target matrix (R) of each reference point is used... Gain(j，i) G Gain(j，i) And B Gain(j，i) The pixel value of the current pixel 140 is obtained by interpolation. First, the distances between the current pixel 140 and its neighboring reference points are calculated. For example, distance D1 is the vertical distance between the current pixel 140 and the top left and top right; distance D2 is the vertical distance between the current pixel 140 and the bottom left and bottom right; distance D3 is the horizontal distance between the current pixel 140 and the top left and bottom left; and distance D4 is the horizontal distance between the current pixel 140 and the top right and bottom right. Then, interpolation is performed based on the distances (D1, D2, D3, D4) between the current pixel 140 and its neighboring reference points to obtain the pixel value R of the red channel (R) of the current pixel 140. Gain_Intp For example (other channels G) Gain_Intp B Gain_Intp (This can be extrapolated from the previous formula), the compensation of 140 for the current pixel is calculated using the following formula:

[0060] R Gain_Intp =(D2 / (D1+D2))×(D4 / (D3+D4))×R Gain(i1，j1) +(D2 / (D1+D2))×(D3 / (D3+D4))×R Gain(i2，j2) +(D1 / (D1+D2))×(D4 / (D3+D4))×R Gain(i3，j3) +(D1 / (D1+D2))×(D3 / (D3+D4))×R Gain(i4，j4) .

[0061] Thus, through Figure 3This document describes a flowchart and embodiment of an interference pattern detection and compensation method. It shows how to obtain the compensated channel values ​​for each pixel in an image. The target matrix for compensation can be dynamically determined by calculated weights, thus enabling dynamic switching of the target matrix for interferometric pattern compensation. Furthermore, this dynamic compensation technique can adapt to different interference patterns caused by changes in different light source environments. (See reference...) Figure 15 The flowchart shown is an embodiment of the interference pattern detection and dynamic compensation method suitable for under-display cameras. Both methods can adapt to situations where the position of the interference pattern in the image changes.

[0062] Step S151 is as described in the above embodiment (e.g.) Figure 4 ) Calculate the region brightness ratio (T / C) of M×N reference points in the image, step S153 as described in the above embodiment (e.g. Figure 7 The interference pattern is detected by the symmetry of the bright and dark fringes. Then, in step S155, it is determined whether an interference pattern exists. If no interference pattern is detected (No), the process ends as shown in step S157; otherwise, if an interference pattern is detected (Yes), the process continues. Figure 3 The difference from the process shown is that the target matrix can be obtained from a uniform gray card calibration environment and is not affected by scene changes. This example process proposes a dynamic switching method for the target matrix. There can be multiple sets of target matrices prepared in the circuit system. One of the target matrices that can eliminate the interference pattern is selected by detecting the interference pattern.

[0063] As in step S159, the k-th target matrix is ​​applied for compensation. This involves using the target matrix formed by the color channels of M×N reference points to interpolate and calculate the pixel values ​​of each channel for each pixel in the image, thus completing the compensation operation. Then, in step S161, the process continues as follows... Figure 4 The process shown calculates the regional brightness ratio of M×N reference points, and in step S163, through... Figure 7 The process shown detects interference patterns. Next, it determines whether there is an interference pattern (step S165). If there is no interference pattern (no), proceed as in step S167, and the process ends. If there is an interference pattern (yes), in step S169, the next target matrix (k = k+1) is determined, and the process returns to step S159. Compensation is performed using the (k+1)th target matrix. The interference pattern is dynamically compensated by switching between multiple target matrices until the compensation procedure ends.

[0064] Figure 16A circuit system for performing the above method is shown. The circuit system is used in a mobile device, and in particular includes an under-display camera module 163 located inside the display module 161 (which may include glass, a display panel, and a backlight module, etc.). The under-display camera module 163 is electrically connected to an image processing circuit 165. It takes pictures of external objects through the glass substrate of the display module 161 via its lens, and transmits the obtained images to the image processing circuit 165. The image processing circuit 165 detects interference patterns in the images and performs compensation. The image data can be stored in a memory 167. The memory 167 can be used to store multiple target matrices during dynamic compensation. After compensation is completed, the image is displayed on the display module 161 through the display control circuit 169.

[0065] In summary, this invention proposes an interference pattern detection method, compensation method, and circuit system suitable for under-display cameras. According to the description of the above embodiments, interference patterns, such as rainbow rings and other annular interference patterns, are first detected based on the brightness characteristics and symmetry in the image. Then, after determining the target matrix, compensation is performed, and a dynamic compensation method can be provided for situations where the position of the interference pattern changes.

[0066] The above-disclosed content is only a preferred embodiment of the present invention and is not intended to limit the scope of the patent application of the present invention. Therefore, any equivalent technical changes made based on the content of the present invention specification and illustrations shall fall within the scope of the patent application of the present invention.

[0067] Explanation of reference numerals in the attached figures:

[0068] 100: Full-screen display panel

[0069] 10: Camera Module

[0070] 103: Photosensitive element

[0071] 101: Lens

[0072] 501, 502: Reference points

[0073] 51, 52: First area

[0074] 511, 521 Second Area

[0075] 61: First Reference Point

[0076] 62: Second Reference Point

[0077] A1, A2, A3, A4, A5, B1, B2, B3, B4, B5: Intersection points

[0078] thr1: The First Threshold

[0079] thr2: The Second Threshold

[0080] max_wgt: Maximum weight value

[0081] 131: First Reference Point

[0082] 132: Second Reference Point

[0083] 133: Third Reference Point

[0084] 140: Current pixel

[0085] D1, D2, D3, D4: Distance

[0086] 161: Display Module

[0087] 163: Under-display camera module

[0088] 165: Image Processing Circuit

[0089] 167: Memory

[0090] 169: Display control circuit

[0091] Steps S301 to S313: Interference pattern detection and compensation process suitable for under-display cameras

[0092] Steps S401 to S411: Flowchart for calculating the regional brightness ratio

[0093] Steps S701 to S711: Procedure for detecting interference patterns

[0094] Steps S111 to S119: Procedure for determining image symmetry

[0095] Steps S151-S169: Dynamic compensation process

Claims

1. A method for detecting interference patterns suitable for under-display cameras, comprising: An image is acquired, in which multiple reference points are set. A first region is set for each reference point, and an average brightness value is calculated for the first region of each reference point. A second region is set for each reference point, and a target brightness value is calculated for the second region of each reference point. Calculate the brightness ratio of a region based on the average brightness value of each reference point and the target brightness value; Based on the brightness ratio of the regions at all reference points in the image, a binarized image is obtained; and Whether an interference pattern exists is determined based on the symmetry of the binarized image. The interference pattern is a Newton's ring, with a wider ring near the center and a thinner ring further away. For each reference point, there is the same first region. The second region is larger the closer it is to the center of the Newton's ring.

2. The interference pattern detection method suitable for under-display cameras according to claim 1, characterized in that, If the target brightness value is greater than the average brightness value, there are dark lines near the corresponding reference point; if the target brightness value is less than the average brightness value, there are bright lines near the corresponding reference point.

3. The interference pattern detection method suitable for under-display cameras according to claim 2, characterized in that, The regional brightness ratio is calculated by dividing the target brightness value by the average brightness value.

4. The interference pattern detection method suitable for under-display cameras according to claim 1, characterized in that, The conditions for satisfying the symmetry of the binarized image include: the number of black dots and white dots at the black-white boundary of the binarized interference pattern is the same, and the positional deviation of the calculated center point at the boundary of each ring of the interference pattern is not greater than a deviation threshold.

5. The interference pattern detection method suitable for under-display cameras according to any one of claims 1 to 4, characterized in that, The interference patterns generated in the image are located at different positions in the red, green, and blue channels. That is, the brightness ratio of each reference point in the region of the different color channels is calculated, and the interference pattern compensation is performed for different color channels.

6. The interference pattern detection method suitable for under-display cameras according to claim 5, characterized in that, Divide the brightness ratios of the regions of each reference point in different color channels by each other to highlight the boundaries of the interference pattern and improve the accuracy of detecting the interference pattern.

7. An interference pattern compensation method suitable for under-display cameras, comprising: An image is acquired, in which multiple reference points are set. A first region is set for each reference point, and an average brightness value is calculated for the first region of each reference point. A second region is set for each reference point, and a target brightness value is calculated for the second region of each reference point. Calculate the brightness ratio of a region based on the average brightness value of each reference point and the target brightness value; Based on the brightness ratio of the regions at all reference points in the image, a binarized image is obtained; Determine whether an interference pattern exists based on the symmetry of the binarized image; A weight value is determined for the brightness ratio of the regions at each reference point; and Based on the weight values ​​of each reference point in the image, a target matrix is ​​determined. This target matrix describes the compensation coefficients of all reference points in the image, and is used to compensate for the determined interference pattern. The interference pattern is a Newton's ring, with a wider ring near the center and a thinner ring further away. For each reference point, there is the same first region. The second region is larger the closer it is to the center of the Newton's ring.

8. The interference pattern compensation method suitable for under-display cameras according to claim 7, characterized in that, If the target brightness value is greater than the average brightness value, there are dark lines near the corresponding reference point; if the target brightness value is less than the average brightness value, there are bright lines near the corresponding reference point.

9. A circuit system, comprising: An image processing circuit; as well as An under-display camera module, electrically connected to the image processing circuit, is located inside a display module. An image is captured through a lens on the display module's glass substrate, and transmitted to the image processing circuit. The image processing circuit detects and compensates for an interference pattern in the image. The detection and compensation steps include: Multiple reference points are set in the image. A first region is set for each reference point. An average brightness value is calculated for the first region of each reference point. A second region is set for each reference point. A target brightness value is calculated for the second region of each reference point. Calculate the brightness ratio of a region based on the average brightness value of each reference point and the target brightness value; Based on the brightness ratio of the regions at all reference points in the image, a binarized image is obtained; The presence of the interference pattern is determined based on the symmetry of the binarized image. A weight value is determined for the brightness ratio of the regions at each reference point; and Based on the weight values ​​of each reference point in the image, a target matrix is ​​determined. This target matrix describes the compensation coefficients of all reference points in the image, and is used to compensate for the determined interference pattern. The interference pattern is a Newton's ring, with a wider ring near the center and a thinner ring further away. For each reference point, there is the same first region. The second region is larger the closer it is to the center of the Newton's ring.

Images