Image sensing device and imaging system

Abstract

The present application relates to image sensing devices and imaging systems. An image sensing device includes a first flare detection photodiode, a first mask disposed above the first flare detection photodiode, a second flare detection photodiode disposed adjacent to the first flare detection photodiode, and a second mask disposed above the second flare detection photodiode and configured to be larger than the first mask.

Description

Technical Field

[0001] The techniques and embodiments disclosed herein generally relate to image sensing devices and imaging systems. Background Technology

[0002] An imaging system is a device used to capture optical images by converting light into electrical signals using photosensitive semiconductor materials that react to light. With the development of the automotive, medical, computer, and communications industries, the demand for high-performance imaging systems is constantly increasing in various fields such as smartphones, digital cameras, game consoles, the Internet of Things (IoT), robotics, security cameras, and medical miniature cameras.

[0003] When images are captured by an imaging system, lens flares may occur due to strong light. In the event of lens flares, problems such as reduced image contrast may occur, thus requiring detection of such lens flares. Summary of the Invention

[0004] Various embodiments of this disclosure relate to an imaging apparatus including at least one flare detection pixel, which can use the flare detection pixel to determine whether a flare has occurred.

[0005] Various embodiments of this disclosure relate to an image sensing device including a mask that blocks a portion of the light incident on a photodiode.

[0006] According to embodiments of the present disclosure, an image sensing device may include: a first flare detection photodiode; a first mask disposed above the first flare detection photodiode; a second flare detection photodiode disposed adjacent to the first flare detection photodiode; and a second mask disposed above the second flare detection photodiode and configured to be larger than the first mask.

[0007] In some embodiments, the first mask may block a portion of the light incident from the exit pupil onto the first flare detection photodiode; and the second mask may block a portion of the light incident from the exit pupil onto the second flare detection photodiode.

[0008] In some embodiments, the image sensing device may further include: a first filter disposed above the first flare detection photodiode and adjacent to the first mask; and a second filter disposed above the second flare detection photodiode and adjacent to the second mask.

[0009] In some implementations, the first filter and the second filter may be transparent filters.

[0010] In some embodiments, the image sensing device may further include: a first microlens disposed above the first mask and offset by an amount from the central axis of the first flare detection photodiode; a second microlens disposed above the second mask and offset by the amount from the central axis of the second flare detection photodiode; an image photodiode configured to generate an image signal; a third filter disposed above the image photodiode; and a third microlens disposed above the third filter and offset by the amount from the central axis of the image photodiode.

[0011] In some embodiments, the third filter may be a color filter that includes at least one color selected from red, green, and blue.

[0012] In some embodiments, the offset may correspond to the incident angle of light from the exit pupil onto the first flare detection photodiode, the second flare detection photodiode, and the image photodiode.

[0013] In some embodiments, the first flare detection photodiode and the second flare detection photodiode may be disposed inside the image circle.

[0014] In some embodiments, the first flare detection photodiode and the second flare detection photodiode may be disposed inside an optical black area.

[0015] In some embodiments, the first flare detection photodiode and the second flare detection photodiode may be disposed outside the optical black area.

[0016] In some embodiments, the first mask may be positioned in the upper region of the first flare detection photodiode, at the position closest to the exit pupil.

[0017] In some embodiments, the second mask may be positioned in the upper region of the second flare detection photodiode, closest to the exit pupil.

[0018] In some implementations, the first mask and the second mask may be metal masks.

[0019] According to another embodiment of this disclosure, an imaging system may include an image sensing device and an image processing device. The image sensing device may include: a first flare detection pixel, the first flare detection pixel including a first mask blocking a portion of light from an exit pupil, detecting flare light, and generating a first flare detection signal; a second flare detection pixel, the second flare detection pixel including a second mask blocking a portion of light from the exit pupil, detecting the flare light, and generating a second flare detection signal; and a readout circuit configured to generate a first flare detection pixel value based on the first flare detection signal, and to generate a second flare detection pixel value based on the second flare detection signal. The image processing device may determine whether a flare has occurred based on the first flare detection pixel value and the second flare detection pixel value.

[0020] In some implementations, the first flare detection pixel and the second flare detection pixel may be disposed inside the image circle.

[0021] In some implementations, the first flare detection pixel and the second flare detection pixel may be located outside the optical black area.

[0022] In some implementations, the first flare detection pixel and the second flare detection pixel may be disposed inside the optical black area.

[0023] In some implementations, the image processing device can determine that a flare has occurred when either the first flare detection pixel value or the second flare detection pixel value is greater than a first threshold.

[0024] In some implementations, the image processing device may determine that a flare has occurred when the ratio of a first value obtained by subtracting the black level from the first flare detection pixel value to a second value obtained by subtracting the black level from the second flare detection pixel value is less than a second threshold, or when the ratio of the first value to the second value is greater than a third threshold.

[0025] According to another embodiment of this disclosure, an imaging system may include an image sensing device and an image processing device. The image sensing device includes: a first flare detection photodiode configured to generate a first flare detection signal; a second flare detection photodiode configured to generate a second flare detection signal; a first mask disposed above the first flare detection photodiode; a second mask disposed above the second flare detection photodiode; and a readout circuit configured to generate a first flare detection pixel value based on the first flare detection signal and a second flare detection pixel value based on the second flare detection signal. The image processing device can determine whether a flare has occurred based on the first flare detection pixel value, the second flare detection pixel value, and a black level.

[0026] It should be understood that the above general description of the embodiments of this disclosure and the following detailed description are illustrative and intended to provide further description of the claimed embodiments. Attached Figure Description

[0027] The above and other features and advantages of embodiments of the present disclosure will become apparent when considered in conjunction with the accompanying drawings and the following detailed description.

[0028] Figure 1 This is a block diagram illustrating an imaging system according to an embodiment of the present disclosure.

[0029] Figure 2 This is a diagram illustrating an image sensing apparatus according to an embodiment of the present disclosure.

[0030] Figure 3 This is a diagram illustrating pixel regions according to embodiments of the present disclosure.

[0031] Figure 4 This is a diagram illustrating pixels according to an embodiment of the present disclosure.

[0032] Figure 5 This is a diagram illustrating pixels according to an embodiment of the present disclosure.

[0033] Figure 6 This is a graph illustrating the sensitivity according to an embodiment of the present disclosure.

[0034] Figure 7 This is a diagram showing the offsets according to embodiments of the present disclosure.

[0035] Figure 8 This is a graph illustrating the sensitivity according to an embodiment of the present disclosure.

[0036] Figure 9This illustrates an embodiment according to the present disclosure. Figure 1 A block diagram of the computing device corresponding to the image processing device. Detailed Implementation

[0037] This disclosure includes embodiments of image sensing devices and imaging systems that can be used in configurations to substantially solve one or more technical or engineering problems and mitigate limitations or drawbacks encountered in some image sensing devices. Some embodiments of this disclosure relate to imaging devices including at least one flare detection pixel that can use the flare detection pixel to determine whether a flare has occurred. Some embodiments of this disclosure relate to image sensing devices including a mask that blocks a portion of the light incident on a photodiode. Recognizing the above problems, embodiments can advantageously provide imaging systems that can determine whether a flare has occurred by using the pixel value of the flare detection pixel. Because the image sensing device includes a mask that blocks a portion of the light incident on a photodiode, the image sensing device can accurately determine whether a flare has occurred by using the mask.

[0038] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. While the embodiments are readily adaptable to various modifications and alternatives, specific embodiments are shown in the drawings. However, the embodiments should not be construed as being limited to those described herein.

[0039] In the following description, various embodiments will be illustrated with reference to the accompanying drawings. However, it should be understood that the embodiments are not limited to the specific embodiments, but include various modifications, equivalents, and / or substitutions of the embodiments. The embodiments of this disclosure can provide various advantageous effects that can be directly or indirectly recognized by those skilled in the art.

[0040] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can readily realize the scope of the present disclosure. However, embodiments of the present disclosure can be implemented in various different forms and are not limited to the embodiments described herein.

[0041] In the following description of embodiments of this disclosure, detailed descriptions of known functions and configurations included herein will be omitted where such descriptions may make the subject matter of this disclosure quite unclear. In the accompanying drawings, components unrelated to the description of this disclosure have been omitted for clarity of description, and similar reference numerals will be used throughout this disclosure to refer to similar components.

[0042] In this disclosure, when a component is referred to as "connected," "joined," or "attached" to another component, it may include not only a direct connection but also an indirect connection in which the other component exists. Furthermore, when a component "comprises," "includes," or "has" another component, this means that, unless otherwise specified, the one component does not exclude other components and may include other components.

[0043] In this disclosure, terms such as "first" and "second" are used only to distinguish one element from other elements and are not used to limit the elements. Unless otherwise stated, these terms do not limit the order or importance of the elements. Therefore, within the scope of this disclosure, a first element in one embodiment may be referred to as a second element in another embodiment, and similarly, a second element in one embodiment may be referred to as a first element in another embodiment.

[0044] In the following description, components are distinguished from each other to clearly describe their characteristics, but this does not mean that they must be physically separated. That is, multiple components can be integrated into a single hardware or software module, and a single component can be divided into multiple hardware or software modules. Therefore, even without specific description, the integrated or divided implementations are within the scope of this disclosure.

[0045] In the following description, the components described with reference to various embodiments are not all necessary, and some components may be used selectively. Therefore, embodiments consisting of some components described in one embodiment are also within the scope of this disclosure. Furthermore, embodiments implemented by adding components to various embodiments are also within the scope of this disclosure.

[0046] In this disclosure, for ease of description, expressions of positional relationships such as "top," "upper," "bottom," "lower," "left," and "right" are used as in this specification, and the described positional relationships can be interpreted in reverse when the accompanying drawings shown in this disclosure are viewed in reverse.

[0047] In this disclosure, each of phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” can include any one or all possible combinations of the items listed together in the corresponding phrase within the phrase. In the description of this disclosure, the term “and / or” can include a combination of multiple items or any one of multiple listed items. For example, “A or B” can include “A only,” “B only,” or “both A and B.”

[0048] In the following text, reference will be made to Figures 1 to 9The embodiments of this disclosure are described in detail.

[0049] Figure 1 This is a block diagram illustrating an imaging system 10 according to an embodiment of the present disclosure.

[0050] Reference Figure 1 Imaging system 10 can refer to, for example, a digital still camera for capturing still images or a digital video camera for capturing moving images. For example, imaging system 10 can be implemented as a digital SLR (DSLR) camera, a mirrorless camera, or a smartphone, but is not limited thereto. Imaging system 10 may include a device with an image pickup element, enabling the device to capture (or photograph) a target object, thereby creating an image of the target object.

[0051] The imaging system 10 may include an image sensing device 100 and an image processing device 200. The image processing device 200 may correspond to an image signal processor (ISP).

[0052] Image sensing device 100 may be a complementary metal-oxide-semiconductor image sensor (CIS) configured to convert incident light into an electrical signal. Image sensing device 100 may include pixel array 110, row control circuitry 120, ramp generator 130, analog-to-digital converter (ADC) 140, output buffer 150, column driver 160, and timing controller 170. Figure 1 The components of the image sensing device 100 are shown, and the implementation includes many other variations, substitutions, modifications, alterations and alterations.

[0053] Pixel array 110 may include a plurality of pixels arranged in rows and columns. In one embodiment, the plurality of pixels may be arranged in a two-dimensional (2D) pixel array including rows and columns. In another embodiment, the plurality of image pixels may be arranged in a three-dimensional (3D) pixel array. The plurality of pixels may convert optical signals into electrical signals on a pixel-by-pixel or pixel-group basis, wherein pixels in a pixel group share at least some internal circuitry. Pixel array 110 may receive row control signals (RCS) from row control circuitry 120, including row selection signals, pixel reset signals, and transmission signals. Upon receiving an RCS, a corresponding pixel in pixel array 110 may be activated to perform an operation corresponding to the row selection signal, pixel reset signal, and transmission signal.

[0054] Each pixel in the pixel array 110 can have at least two different sensitivities. Here, sensitivity can refer to an increase in the amount of image data IDATA (or an increase in the amount of response) relative to an increase in the intensity of incident light. That is, as sensitivity increases, the increase in image data IDATA in response to an increase in the intensity of incident light increases. As sensitivity decreases, the increase in image data IDATA in response to an increase in the intensity of incident light decreases. In this disclosure, sensitivity can be determined by conversion gain.

[0055] The row control circuit 120 can activate the pixel array 110 based on command and control signals (TS1) provided by the timing controller 170 to perform specific operations on pixels included in corresponding rows. The row control circuit 120 may include a row driver. In some embodiments, the row control circuit 120 can select at least one pixel arranged in at least one row of the pixel array 110. The row control circuit 120 can generate a row selection signal to select at least one row among a plurality of rows. The row control circuit 120 can sequentially enable pixel reset signals and transmission signals for pixels corresponding to at least one selected row. Thus, a reference signal and an image signal, which are analog signals generated by each pixel in the selected row, can be sequentially transmitted to the ADC 140. The reference signal may be an electrical signal provided to the ADC 140 when the sensing node of the pixel (e.g., a floating diffusion region) is reset, and the image signal may be an electrical signal provided to the ADC 140 when photocharge generated by the pixel accumulates in the sensing node. The reference signal, indicating the unique reset noise of each pixel, and the image signal, indicating the intensity of the incident light, may be collectively referred to as the pixel signal (PS).

[0056] CMOS image sensors can use correlated double sampling (CDS) to remove unwanted offset values ​​(referred to as fixed-pattern noise) of pixels by sampling the pixel signal (PS) twice to remove the difference between the two samples. In one implementation, CDS can remove unwanted offset values ​​of pixels by comparing the pixel output voltages obtained before and after the photocharge generated by the incident light accumulates in the sensing node, allowing measurement of only the pixel output voltage based on the incident light. In some implementations, the ADC 140 can sequentially sample and hold reference signals and image signals provided from each of the multiple column lines of the pixel array 110.

[0057] The ramp generator 130 can generate the ramp signal (RS) required for the analog-to-digital conversion operation of the ADC 140 when it receives a timing signal (TS2) from the timing controller 170, and can provide the ramp signal to the ADC 140.

[0058] The ADC 140 can sample and hold the pixel signal (PS) from each column line of the pixel array 110 based on the timing signal (TS3) of the timing controller 170, convert the resulting signal into a digital signal, and output a digital signal. In some embodiments, the ADC 140 can be implemented as a ramp comparator ADC. A ramp comparator ADC may include comparator circuitry for comparing the analog pixel signal (PS) with a ramp signal that ramps up or down over time, and a counter for performing a count until the ramp signal matches the analog pixel signal (PS).

[0059] Output buffer 150 can temporarily hold column-based image data (i.e., data IDATA obtained through digital conversion of pixel signals (PS)) provided by ADC 140 to output image data (IDATA). The image data IDATA output from ADC 140 can be temporarily stored in output buffer 150 based on the timing signal (TS4) of timing controller 170. Output buffer 150 can provide an interface to compensate for differences in data rate or transmission rate between image sensing device 100 and other devices.

[0060] The column driver 160 can select a column of the output buffer 150 based on a control signal (TS5) from the timing controller 170, and can control the output buffer 150 to sequentially output image data IDATA temporarily stored in the selected column of the output buffer 150. In some embodiments, after receiving an address signal from the timing controller 170, the column driver 160 can generate a column selection signal (CSS) based on the address signal, and can select a column of the output buffer 150, thereby outputting the image data IDATA as an output signal from the selected column of the output buffer 150.

[0061] The timing controller 170 can control the operation of at least one of the row control circuit 120, the ramp generator 130, the ADC 140, the output buffer 150, or the column driver 160.

[0062] The timing controller 170 can provide at least one of the row control circuit 120, the ramp generator 130, the ADC 140, the output buffer 150, or the column driver 160 with clock signals required for the operation of corresponding components of the image sensing device 100, control signals for timing control, and address signals for selecting rows or columns. In some embodiments, the timing controller 170 may include logic control circuitry, phase-locked loop (PLL) circuitry, timing control circuitry, and communication interface circuitry.

[0063] The image processing apparatus 200 can perform image signal processing on image data IDATA received from the image sensing apparatus 100. The image processing apparatus 200 can reduce noise in the image data IDATA and can perform various image signal processing operations such as interpolation, compositing, gamma correction, color filter array interpolation, color matrix, color correction, color enhancement, and lens distortion correction to improve the image quality of the image data IDATA. Furthermore, the image processing apparatus 200 can compress the image data created by performing image signal processing for image quality improvement, allowing the image processing apparatus 200 to create an image file using the compressed image data. Alternatively, the image processing apparatus 200 can recover image data from an image file. In this case, the format used to compress such image data can be a reversible or irreversible format. As examples of compression formats, for still images, the Joint Photographic Experts Group (JPEG) format, JPEG 2000 format, etc., can be used. Furthermore, for moving images, multiple frames can be compressed according to the Moving Picture Experts Group (MPEG) standard to create a moving image file. For example, an image file can be created according to the Exif (Exchangeable Image File Format) standard.

[0064] Image processing apparatus 200 can generate an HDR image by synthesizing at least two images with different sensitivities. For example, image sensing apparatus 100 can output a low-sensitivity image generated from low-sensitivity pixels (e.g., low conversion gain (LCG) pixels) with relatively low sensitivity and a high-sensitivity image generated from high-sensitivity pixels (e.g., high conversion gain (HCG) pixels) with relatively high sensitivity. Image processing apparatus 200 can combine the low-sensitivity image and the high-sensitivity image to generate an HDR image. Here, low sensitivity and high sensitivity can correspond to relative concepts, and image sensing apparatus 100 can generate image data IDATA with at least N different sensitivities, where N is an integer greater than or equal to 2. Image processing apparatus 200 can generate an HDR image based on image data IDATA.

[0065] The image processing apparatus 200 can transmit image data that has undergone image processing to a host device (not shown). The host device (not shown) may be a processor (e.g., an application processor) for processing the image data received from the image processing apparatus 200, a memory (e.g., a non-volatile memory) for storing the image data, or a display device (e.g., a liquid crystal display, LCD) for visually displaying the image data.

[0066] In addition, the image processing device 200 can transmit control signals for controlling the operation of the image sensing device 100 (e.g., whether to operate, operation timing, operation mode, etc.) to the image sensing device 100.

[0067] Figure 2 This is a diagram illustrating an image sensing device 20 according to an embodiment of the present disclosure. The image sensing device 20 corresponds to... Figure 1 Image sensing device 100.

[0068] Reference Figure 2 The image sensing device 20 according to embodiments of the present disclosure may include a main lens 210 and a pixel region 250.

[0069] The main lens 210 may include multiple lenses that collect light and guide the collected light onto the pixel.

[0070] Based on the configuration of the main lens 210, the range of the exit pupil 220, the outer edges 231 and 232 of the exit pupil 220, and the range of the image ring 240 can be determined. In the imaging system, the path of light can be adjusted as it passes through multiple lenses, and the position of the imaginary exit port where light finally exits after passing through the lenses can be referred to as the exit pupil 220. Therefore, light typically enters the pixel region 250 from the exit pupil 220; however, due to the occurrence of lens flares, light may also enter the pixel region 250 from the outer edges 231 and 232 of the exit pupil 220. The image ring 240 can be set within the light incident area defined by the main lens 210.

[0071] Pixel region 250 may refer to an area in which pixels can be arranged. For example, pixel region 250 may include flare detection pixels 261, 262 and at least one image pixel 270.

[0072] Flare detection pixels 261 and 262 refer to pixels that detect flare light. By detecting flare light, flare detection pixels 261 and 262 can generate flare detection signals. More specific details regarding the configuration of flare detection pixels 261 and 262 will be described later.

[0073] Flare detection pixels 261 and 262 can be positioned inside the image ring 240. Light can enter from the outer side 231 and 232 of the exit pupil 220 toward the interior of the image ring 240. Therefore, since flare detection pixels 261 and 262 are positioned inside the image ring 240, they can detect flare light.

[0074] Image pixel 270 can refer to the pixel that generates the image signal. The image signal can be used to generate an image. Image pixel 270 can be set inside the image circle 240.

[0075] although Figure 2Although not shown, the image sensing device 20 may include a readout circuit. The readout circuit may generate pixel values ​​based on signals received from flare detection pixels 261, 262 and image pixels 270. For example, the readout circuit may generate flare detection pixel values ​​based on flare detection signals and image pixel values ​​based on image signals.

[0076] When a flare detection pixel value is equal to or greater than a first threshold, the image processing apparatus can determine that a flare has occurred. Here, the first threshold can be a value greater than the noise level. Furthermore, when the ratio of the number of pixel values ​​equal to or greater than the first threshold among the multiple flare detection pixel values ​​generated by multiple flare detection pixels is equal to or greater than a predetermined ratio, the image processing apparatus can determine that a flare has occurred. For example, when the ratio of pixel values ​​equal to or greater than the first threshold among the flare detection pixel values ​​is 10% or greater, the image processing apparatus can determine that a flare has occurred, but embodiments of this disclosure are not limited to this.

[0077] Figure 3 This is a diagram illustrating pixel region 250 according to an embodiment of the present disclosure.

[0078] In the following text, reference will be made to Figure 2 describe Figure 3 The implementation method.

[0079] Reference Figure 3 According to embodiments of the present disclosure, the pixel region 250 may include an image pixel region 320, an optical black region 330, and a flare detection pixel region 350. Figure 3 yes Figure 2 The top view of pixel region 250 is shown. Figure 3 In the middle, it is shown that... Figure 2 The exit pupil 220 corresponds to the exit pupil 310 and the exit pupil 310. Figure 2 Image circle 240 corresponds to image circle 340.

[0080] Image pixel region 320 can be the area in which image pixels are arranged.

[0081] The optical black area 330 can represent an area including pixels that remain in a light-blocking state. Since light is blocked, the optical black area 330 can be used to measure the black level of an image sensing device, or to remove noise from an image sensing device.

[0082] The flare detection pixel region 350 can be the region in which flare detection pixels are arranged. The flare detection pixel region 350 can be located inside the image ring 340. Alternatively, the flare detection pixel region 350 can be located inside the optical black region 330. Furthermore, the flare detection pixel region 350 can be located outside the image pixel region 320.

[0083] Figure 4 This is a diagram illustrating pixel regions according to embodiments of the present disclosure.

[0084] In the following text, reference will be made to Figure 2 describe Figure 4 The implementation method.

[0085] Reference Figure 4 According to embodiments of the present disclosure, the pixel region 250 may include an image pixel region 420, an optical black region 430, and a flare detection pixel region 450. Figure 4 yes Figure 2 The top view of pixel region 250 is shown. Figure 4 In the middle, it is shown that... Figure 2 The exit pupil 220 corresponds to the exit pupil 410 and the exit pupil 410. Figure 2 Image circle 240 corresponds to image circle 340.

[0086] Image pixel region 420 can be the region in which image pixels are arranged.

[0087] The optical black area 430 can represent an area including pixels that remain in a light-blocking state. Since light is blocked, the optical black area 430 can be used to measure the black level of the image sensing device, or to remove noise from the image sensing device.

[0088] The flare detection pixel region 450 can be the region in which flare detection pixels are arranged. The flare detection pixel region 450 can be located inside the image circle 340. Alternatively, the flare detection pixel region 450 can be located outside the optical black region 330.

[0089] Figure 5 This is a diagram illustrating pixels according to an embodiment of the present disclosure.

[0090] Figure 6 This is a graph illustrating the sensitivity according to an embodiment of the present disclosure.

[0091] Figure 7 This is a diagram showing the offsets according to embodiments of the present disclosure.

[0092] Figure 8 This is a graph illustrating the sensitivity according to an embodiment of the present disclosure.

[0093] In the following text, reference will be made to Figure 5 and Figure 8 describe Figure 5 The structure.

[0094] Reference Figure 5The image sensing device 500 may include a first flare detection pixel 510, a second flare detection pixel 520, a first image pixel 530 and / or a second image pixel 540.

[0095] The first flare detection pixel 510 may include a first flare detection photodiode 511, a first filter 512, a first mask 513, and a first microlens 514.

[0096] The first flare detection photodiode 511 can be a photodiode that generates a first flare detection signal by detecting flare light. The first flare detection photodiode 511 can be disposed inside the image circle. Furthermore, the first flare detection photodiode 511 can be disposed inside the optical black area and outside the image pixel area. Alternatively, the first flare detection photodiode 511 can be disposed outside the optical black area. Figure 5 In the diagram, the first flare detection photodiode 511 is shown disposed outside the first image pixel 530, but the arrangement of the first flare detection photodiode 511 according to embodiments of the present disclosure is not limited thereto.

[0097] The first filter 512 can be located at the upper end of the first flare detection photodiode 511. The first filter 512 can be a transparent filter.

[0098] The first mask 513 can block a portion of the light incident on the first flare detection photodiode 511. The first mask 513 can be a metal mask. The first mask 513 can be disposed at the upper end of the first flare detection photodiode 511. Specifically, the first mask 513 can be disposed in the upper region of the first flare detection photodiode 511, closest to the exit pupil, but is not limited thereto. Since the first mask 513 is located near the exit pupil, it can block a portion of the light incident on the first flare detection photodiode 511 from the exit pupil. Because the first mask 513 blocks a portion of the light incident on the first flare detection photodiode 511 from the exit pupil, it can determine the sensitivity of the first flare detection pixel 510. Specifically, refer to... Figure 6 The pixels included in the image sensing device may include photodiodes (PDs), color filters (CFs), and microlenses (MLs). Furthermore, the size of the mask can vary, and the sensitivity of the pixels can vary depending on the size of each mask. More specifically, since the amount of light incident on the photodiode (PD) can vary depending on the size of the mask, the sensitivity of the pixels can also vary depending on the size of the mask. For example, as the mask becomes larger, the amount of light incident on the photodiode from the exit pupil, etc., can decrease. Compared to a device without a mask... Figure 6 Compared to (a), the one with a mask Figure 6The pixel sensitivity of (b) can be lower in the predetermined direction. Furthermore, the mask has a higher sensitivity than... Figure 6 The larger size in (b) Figure 6 The sensitivity of pixel (c) in the predetermined direction can be lower than Figure 6 The pixels of (a).

[0099] The first microlens 514 can be disposed at the upper end of the first filter 512 or the first mask 513. Specifically, the first microlens 514 can be disposed at the upper end of the first filter 512 or the first mask 513 with a predetermined offset. For example, the first microlens 514 can be disposed at a position shifted by a predetermined offset from the central axis of the first flare detection photodiode 511. The offset can indicate the degree of relative displacement of the microlens from the center of the pixel. Specifically, refer to... Figure 6 The incident angles of light incident on the photodiodes (PDs) included in the image sensing device are different from each other, and the sensitivity of the pixels to the incident light varies depending on the position of the microlenses (MLs) positioned above the color filter (CF). Therefore, the offset of the microlenses (MLs) included in the pixels can vary. For example, Figure 7 The offset of the first microlens 514 shown in (a) can be less than Figure 7 The offset of the first microlens 514 described in (b), and Figure 7 The offset of the first microlens 514 shown in (b) can be less than Figure 7 The offset of the first microlens 514 shown in (c) is described. The first microlens 514 can be arranged such that the sensitivity of the first flare detection pixel 510 increases in the direction in which light enters from the exit pupil onto the first flare detection photodiode 511. For example, the first microlens 514 can be disposed above the first filter 512 or the first mask 513 to have a predetermined offset corresponding to the incident angle of light entering from the exit pupil onto the first flare detection photodiode 511. According to an embodiment, the offset of the first microlens 514 can be determined based on the incident angle of light, such that the tangent of the incident angle corresponds to the lateral displacement ratio of the first microlens 514 relative to the central axis of the first flare detection pixel 510. According to an embodiment, the offset can be predefined based on the position of the exit pupil 220 and the incident angle distribution of the main lens 210.

[0100] The second flare detection pixel 520 may include a second flare detection photodiode 521, a second filter 522, a second mask 523, and a second microlens 524.

[0101] The second flare detection photodiode 521 can be a photodiode that generates a second flare detection signal by detecting flare light. The second flare detection photodiode 521 can be disposed inside the image circle. Furthermore, the second flare detection photodiode 521 can be disposed inside the optical black area and outside the image pixel area. Alternatively, the second flare detection photodiode 521 can be disposed outside the optical black area. Figure 5 In the diagram, the second flare detection photodiode 521 is shown disposed outside the first image pixel 530, but the arrangement of the second flare detection photodiode 521 according to the embodiments of the present disclosure is not limited thereto.

[0102] The second filter 522 can be located at the upper end of the second flare detection photodiode 521. The second filter 522 can be a transparent filter.

[0103] The second mask 523 can block a portion of the light incident on the second flare detection photodiode 521. The second mask 523 can be a metal mask. The second mask 523 can be disposed at the upper end of the second flare detection photodiode 521. Specifically, the second mask 523 can be disposed in the upper region of the second flare detection photodiode 521, closest to the exit pupil, but is not limited thereto. Since the second mask 523 is located near the exit pupil, it can block a portion of the light incident on the second flare detection photodiode 521 from the exit pupil. Because the second mask 523 blocks a portion of the light incident on the second flare detection photodiode 521 from the exit pupil, it can determine the sensitivity of the second flare detection pixel 520. The size of the second mask 523 can be different from the size of the first mask 513. For example, the size of the second mask 523 can be larger than the size of the first mask 513. Therefore, the amount of light incident on the second flare detection photodiode 521 can be less than the amount of light incident on the first flare detection photodiode 511. For example, refer to Figure 8 The sensitivity 810 of the first flare detection pixel 510 is typically greater than the sensitivity 820 of the second flare detection pixel 520. Figure 8 In the accompanying drawings, reference numeral 801 can indicate a region located outside the exit pupil, and reference numeral 802 can indicate a region of the exit pupil. Since the first mask 513 and the second mask 523 are positioned close to the exit pupil, they can block light incident from the exit pupil. Therefore, the sensitivity 810 of the first flare detection pixel 510 and the sensitivity 820 of the second flare detection pixel 520 can be lower relative to the exit pupil region than relative to the outer region of the exit pupil.

[0104] The second microlens 524 can be disposed at the upper end of the second filter 522 or the second mask 523. Specifically, the second microlens 524 can be disposed at the upper end of the first filter 512 or the first mask 513 with a predetermined offset. The second microlens 524 can be arranged such that the sensitivity of the second flare detection pixel 520 increases in the direction in which light enters the second flare detection photodiode 521 from the exit pupil. For example, the second microlens 524 can be disposed above the second filter 522 or the second mask 523 with an offset corresponding to the incident angle of light entering the second flare detection photodiode 521 from the exit pupil. The offset of the first microlens 514 and the offset of the second microlens 524 can be the same as each other, but are not limited thereto.

[0105] The first image pixel 530 can refer to the pixel that generates the first image signal. The first image signal can be used to generate an image. The first image pixel 530 can be located inside the image circle.

[0106] The first image pixel 530 may include a first image photodiode 531, a third filter 532, and a third microlens 533.

[0107] The first image photodiode 531 can generate a first image signal in response to incident light.

[0108] The third filter 532 can be disposed at the upper end of the first image photodiode 531. The third filter 532 can be a color filter. Specifically, the third filter 532 can be a color filter that includes at least one color selected from red, green, or blue. For example, the third filter 532 can be a green filter.

[0109] The third microlens 533 can be disposed at the upper end of the third filter 532. Specifically, the third microlens 533 can be disposed at the upper end of the third filter 532 with a predetermined offset. For example, the third microlens 533 can be arranged such that the sensitivity of the first image pixel 530 increases in the direction in which light enters the first image photodiode 531 from the exit pupil. For example, the third microlens 533 can be disposed above the third filter 532 with an offset corresponding to the incident angle of light entering the first image photodiode 531 from the exit pupil. The offset of the second microlens 524 and the offset of the third microlens 533 can be the same as each other, but are not limited thereto.

[0110] The second image pixel 540 can refer to the pixel that generates the second image signal. The second image signal can be used to generate an image. The second image pixel 540 can be located inside the image circle.

[0111] The second image pixel 540 may include a second image photodiode 541, a fourth filter 542, and a fourth microlens 543.

[0112] The second image photodiode 541 can generate a second image signal in response to incident light.

[0113] The fourth filter 542 may be disposed at the upper end of the second image photodiode 541. The fourth filter 542 may be a color filter. Specifically, the fourth filter 542 may be a color filter that includes at least one color selected from red, green, or blue. For example, the fourth filter 542 may be a red filter.

[0114] The fourth microlens 543 can be disposed at the upper end of the fourth filter 542. Specifically, the fourth microlens 543 can be disposed at the upper end of the fourth filter 542 with a predetermined offset. For example, the fourth microlens 543 can be arranged such that the sensitivity of the second image pixel 540 increases in the direction in which light enters the second image photodiode 541 from the exit pupil. For example, the fourth microlens 543 can be disposed above the fourth filter 542 with an offset corresponding to the incident angle of light entering the second image photodiode 541 from the exit pupil. The offset of the fourth microlens 543 and the offset of the third microlens 533 can be the same as each other, but are not limited thereto.

[0115] although Figure 5 Although not shown, the image sensing device 500 may include a readout circuit. The readout circuit can generate a first flare detection pixel value based on a first flare detection signal, and can generate a second flare detection pixel value based on a second flare detection signal. Furthermore, the readout circuit can generate image pixel values ​​based on an image signal.

[0116] The image processing device can determine whether a flare has occurred based on a first flare detection pixel value and a second flare detection pixel value. For example, the image processing device can determine that a flare has occurred when a first ratio (obtained by subtracting the black level from the first flare detection pixel value) to a second value (obtained by subtracting the black level from the second flare detection pixel value) is less than a second threshold or greater than a third threshold. The above results can be expressed mathematically as shown in Equations 1 and 2.

[0117] [Equation 1]

[0118] [Equation 2]

[0119] In Equations 1 and 2, I1 can represent the pixel value of the first flare detection, and I2 can represent the pixel value of the second flare detection.d It can represent the black level, R lower The second threshold can be represented, and R upper It can represent the third threshold.

[0120] When no flare has occurred, the second and third thresholds can be determined based on the first and second flare detection pixel values. For example, when no flare has occurred, the second threshold can be the smallest ratio among multiple ratios of a first value (obtained by subtracting the black level from the first flare detection pixel value) and a second value (obtained by subtracting the black level from the second flare detection pixel value). Furthermore, when no flare has occurred, the third threshold can be the largest ratio among multiple ratios of the first value (obtained by subtracting the black level from the first flare detection pixel value) and a second value (obtained by subtracting the black level from the second flare detection pixel value). Threshold R lower and R upper The ratio can be determined experimentally when flare light superimposed. This deviates from the normal range of brightness variation. For example, when a flare occurs, the first pixel receives more off-axis light, causing the ratio to... Decrease.

[0121] Furthermore, the second and third thresholds can be expressed mathematically as shown in Equation 3.

[0122] [Equation 3]

[0123] In Equation 3, I1(n) (where n = 1, 2, ..., N) can represent the first flare detection pixel value obtained when the uniform brightness surface is captured N times, and I2(n) (where n = 1, 2, ..., N) can represent the second flare detection pixel value obtained when the uniform brightness surface is captured N times, and I d The black level can be represented. The second threshold can be the minimum value among the R(n) values, and the third threshold can be the maximum value among the R(n) values.

[0124] Furthermore, the image processing device can determine whether a flare has occurred based on Equations 4 and 5 below.

[0125] [Equation 4]

[0126] [Equation 5]

[0127] As mentioned above, I1 can represent the pixel value of the first flare detection, and I2 can represent the pixel value of the second flare detection. dIt can represent the black level, R lower The second threshold can be represented, and R upper It can represent the third threshold.

[0128] When the conditions of Equation 4 or Equation 5 are met, the image processing device can determine that a flare has occurred.

[0129] Figure 9 It is shown that... Figure 1 A block diagram of the computing device corresponding to the image processing device.

[0130] Reference Figure 9 The computing device 1000 is used to execute Figure 1 An implementation of the hardware configuration for operating the image processing device 200.

[0131] The computing device 1000 can be mounted on a chip independent of the chip on which the image sensing device is mounted. According to one embodiment, the chip on which the image sensing device is mounted and the chip on which the computing device 1000 is mounted can be implemented in a single package, such as a multi-chip package (MCP), but the scope of this disclosure is not limited thereto.

[0132] Furthermore, the internal configuration or arrangement of the computing device 1000 and the image sensing device can vary depending on the implementation. For example, at least a portion of the image sensing device may be included in the computing device 1000. Alternatively, at least a portion of the computing device 1000 may be included in the image sensing device. In this case, at least a portion of the computing device 1000 may be mounted together on a chip on which the image sensing device is mounted.

[0133] The computing device 1000 may include a processor 1010, a memory 1020, an input / output (I / O) interface 1030, and a communication interface 1040.

[0134] Processor 1010 can handle execution Figure 1 The data and / or instructions required for the operation of the components of the image processing apparatus 200 described herein.

[0135] The memory 1020 may store data and / or instructions required to perform the operation of the components of the image processing apparatus 200, and may be accessed by the processor 1010. For example, the memory 1020 may be volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.) or non-volatile memory (e.g., programmable read-only memory (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, etc.).

[0136] In other words, a computer program for performing the operation of the image processing apparatus 200 disclosed herein can be stored in the memory 1020 and executed and processed by the processor 1010, thereby realizing the operation of the image processing apparatus 200.

[0137] The input / output (I / O) interface 1030 may be an interface for connecting external input devices (e.g., keyboard, mouse, touch panel, etc.) and / or external output devices (e.g., display) to the processor 1010 to allow the transmission and reception of data.

[0138] The communication interface 1040 is a component capable of transmitting various data to and receiving various data from external devices (e.g., application processors, external memory, etc.), and can be a device that supports wired or wireless communication.

[0139] As can be clearly seen from the above description, the imaging system according to the embodiments of the present disclosure can determine whether a flare has occurred by using the pixel value of the flare detection pixel.

[0140] Since the image sensing device according to the embodiments of the present disclosure includes highly sensitive pixels arranged along the direction of incident flare light, the image sensing device can accurately determine whether a flare has occurred by using the highly sensitive pixels.

[0141] The embodiments disclosed herein can provide various beneficial effects that can be directly or indirectly recognized by those skilled in the art.

[0142] Those skilled in the art will understand that embodiments of this disclosure can be implemented in other specific ways besides those described herein. Furthermore, claims not expressly set forth in the appended claims may be proposed as embodiments in combination, or may be included as new claims through subsequent amendments after the filing of this application.

[0143] While several exemplary embodiments have been described, it should be understood that modifications and enhancements to the disclosed embodiments and other embodiments can be designed based on the description and / or illustration in this patent document. Furthermore, embodiments can be combined to form additional embodiments.

[0144] Cross-reference to related applications

[0145] This patent application claims priority and benefit to Korean Patent Application No. 10-2025-0046010, filed on April 9, 2025, the disclosure of which is incorporated herein by reference in its entirety.

Claims

1. An image sensing device, the image sensing device comprising: First flare detection photodiode; A first mask is disposed above the first flare detection photodiode; A second flare detection photodiode is configured to be adjacent to the first flare detection photodiode. as well as A second mask is disposed above the second flare detection photodiode, and the second mask has a larger area than the first mask.

2. The image sensing device according to claim 1, wherein, The first mask blocks a portion of the light incident from the exit pupil onto the first flare detection photodiode; and The second mask blocks a portion of the light incident from the exit pupil onto the second flare detection photodiode.

3. The image sensing device according to claim 1, further comprising: A first filter is disposed above the first flare detection photodiode and is positioned adjacent to the first mask; as well as A second filter is disposed above the second flare detection photodiode and is positioned adjacent to the second mask.

4. The image sensing device according to claim 3, wherein, The first filter and the second filter are transparent filters.

5. The image sensing device according to claim 1, further comprising: A first microlens is disposed above the first mask and is shifted by a predetermined offset from the central axis of the first flare detection photodiode; A second microlens is disposed above the second mask and is offset from the central axis of the second flare detection photodiode by the offset amount; An image photodiode, which generates an image signal; A third filter is disposed above the image photodiode; as well as A third microlens is disposed above the third filter and is offset from the central axis of the image photodiode by the offset amount.

6. The image sensing device according to claim 5, wherein, The third filter is a color filter that includes at least one of red, green, and blue.

7. The image sensing device according to claim 5, wherein, The offset corresponds to the incident angle of light from the exit pupil onto the first flare detection photodiode, the second flare detection photodiode, and the image photodiode.

8. The image sensing device according to claim 1, wherein, The first flare detection photodiode and the second flare detection photodiode are disposed inside the image ring within the light incident area defined by the main lens in the image sensing device.

9. The image sensing device according to claim 8, wherein, The first flare detection photodiode and the second flare detection photodiode are disposed inside the optical black area.

10. The image sensing device according to claim 8, wherein, The first flare detection photodiode and the second flare detection photodiode are disposed outside the optical black area.

11. The image sensing device according to claim 1, wherein, The first mask is positioned in the upper region of the first flare detection photodiode, closest to the exit pupil.

12. The image sensing device according to claim 1, wherein, The second mask is positioned in the upper region of the second flare detection photodiode, closest to the exit pupil.

13. The image sensing device according to claim 1, wherein, The first mask and the second mask are metal masks.

14. An imaging system, the imaging system comprising: Image sensing device, the image sensing device comprising: The first flare detection pixel includes a first mask that blocks a portion of the light from the exit pupil, detects flare light, and generates a first flare detection signal. A second flare detection pixel, comprising a second mask that blocks a portion of the light from the exit pupil, detects the flare light, and generates a second flare detection signal; and A readout circuit that generates a first flare detection pixel value based on the first flare detection signal and generates a second flare detection pixel value based on the second flare detection signal; and An image processing device that determines whether a flare has occurred based on the first flare detection pixel value and the second flare detection pixel value.

15. The imaging system according to claim 14, wherein, The first flare detection pixel and the second flare detection pixel are disposed inside the image ring within the light incident area defined by the main lens in the image sensing device.

16. The imaging system according to claim 15, wherein, The first flare detection pixel and the second flare detection pixel are located outside the optical black area.

17. The imaging system according to claim 15, wherein, The first flare detection pixel and the second flare detection pixel are located inside the optical black area.

18. The imaging system according to claim 14, wherein, When the first flare detection pixel value or the second flare detection pixel value is greater than the first threshold, the image processing device determines that a flare has occurred.

19. The imaging system according to claim 14, wherein, The image processing device: A flare is determined to have occurred when the ratio of a first value obtained by subtracting the black level of the image sensing device from the first flare detection pixel value to a second value obtained by subtracting the black level from the second flare detection pixel value is less than a second threshold, or when the ratio of the first value to the second value is greater than a third threshold.

20. An imaging system, the imaging system comprising: Image sensing device, the image sensing device comprising: A first flare detection photodiode generates a first flare detection signal; The second flare detection photodiode generates a second flare detection signal. A first mask is disposed above the first flare detection photodiode; A second mask, disposed above the second flare detection photodiode; and A readout circuit that generates a first flare detection pixel value based on the first flare detection signal and generates a second flare detection pixel value based on the second flare detection signal; and An image processing device that determines whether a flare has occurred based on a first flare detection pixel value, a second flare detection pixel value, and the black level of the image sensing device.

Images