Autofocus imaging apparatus and method for operating the same

Abstract

To provide an imaging apparatus for providing a phase-difference autofocus function affected less by noise, and a method for operating the imaging device.SOLUTION: An imaging apparatus 1 according to an embodiment of the present disclosure includes: an evaluation-value calculation unit 400 for calculating an evaluation-value set ESTN including evaluation values respectively corresponding to candidate disparities on the basis of a phase-image set ISAN; and a disparity-calculation unit 500 for calculating a target disparity TP on the basis of the evaluation-value set ESTN. Each evaluation values can indicate the likelihood that the candidate disparity corresponding to the evaluation value is the target disparity TP.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an imaging device that performs an autofocus function. [Background technology]

[0002] An imaging device is a device that includes an imaging unit that captures an optical image using the properties of a photosensitive semiconductor material that reacts to light. With the development of industries such as automobiles, medicine, computers, and communications, there is an increasing demand for high-performance imaging devices in various fields such as smartphones, digital cameras, game consoles, the Internet of Things, robots, security cameras, and medical microcameras.

[0003] Imaging devices can be broadly divided into those containing a charge coupled device (CCD) imager and those containing a complementary metal oxide semiconductor (CMOS) imager. CCD imagers tend to provide better image quality than CMOS imagers, but are implemented in larger sizes and consume more power.

[0004] In contrast, CMOS imagers can be realized in a smaller size and consume less power than CCD imagers, and because they are fabricated using CMOS fabrication techniques, the light-sensing elements and signal processing circuitry can be integrated onto a single chip, allowing for the production of low-cost, compact imaging devices.

[0005] The imaging device can perform an autofocus function based on the signals collected by the imaging unit. In various implementations, the imaging device can use phase detection autofocus or contrast based autofocus.

[0006] An imaging device that performs phase-based autofocus can determine the direction and amount of lens movement based on a single frame captured via an imaging unit, thereby increasing focusing speed compared to contrast-based autofocus.

[0007] However, if the captured signal contains noise, distortion of the phase image may occur, and accurate focusing may be difficult since autofocus is performed based on the distorted image. Summary of the Invention [Problem to be solved by the invention]

[0008] SUMMARY OF THE INVENTION It is an object of the present invention to provide an imaging device and method of operation that provides a phase-difference autofocus function with reduced noise effects.

[0009] The technical problems of the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0010] An imaging device according to one embodiment of the present disclosure includes an evaluation value calculation unit that calculates an evaluation value set including evaluation values ​​corresponding to respective candidate disparities based on a phase image set, and a disparity calculation unit that calculates a target disparity based on the evaluation value set, and the evaluation values ​​can indicate the likelihood that the candidate disparities corresponding to the respective evaluation values ​​are the target disparity.

[0011] According to one embodiment, the imaging device may further include an imaging unit including a lens and configured to collect a phase signal pair including a first phase signal for an object and a second phase signal for the object, wherein the phase image set may include a first phase image corresponding to the first phase signal and a second phase image corresponding to the second phase signal.

[0012] According to one embodiment, the target parallax may be a parallax for controlling the focus position of the lens so that the first phase image and the second phase image coincide with each other.

[0013] According to an embodiment, the evaluation value calculation unit may calculate the evaluation value set based on a sum of absolute differences between the first phase image and the second phase image.

[0014] According to an embodiment, the evaluation value calculation unit may calculate the evaluation value set based on a sum of square differences between the first phase image and the second phase image.

[0015] According to an embodiment, the evaluation value calculation unit may calculate the evaluation value set based on a phase correlation between the first phase image and the second phase image.

[0016] According to an embodiment, the evaluation value calculation unit may calculate the evaluation value set based on a cross correlation between the first phase image and the second phase image.

[0017] According to an embodiment, the imaging apparatus may further include an image generator configured to generate the phase image set based on the phase signal pair.

[0018] According to one embodiment, the image generator generates a plurality of time phase image sets based on a plurality of phase signal pairs collected at different times, the evaluation value calculator calculates a plurality of time evaluation value sets respectively corresponding to the plurality of time phase image sets, and the disparity calculator calculates the target disparity based on the plurality of time evaluation value sets.

[0019] According to one embodiment, the image generation unit generates a plurality of regional phase image sets based on one phase signal pair collected at any time, the evaluation value calculation unit calculates a plurality of regional evaluation value sets respectively corresponding to the plurality of regional phase image sets, and the disparity calculation unit can calculate the target disparity based on the plurality of regional evaluation value sets.

[0020] According to one embodiment, the imaging unit may transmit the phase signal pair to the image generating unit at a predetermined time interval, and the image generating unit may generate the phase image set at the predetermined time interval.

[0021] According to an embodiment, when a new phase image set is transmitted, the evaluation value calculation unit may discard an evaluation value set calculated based on an already transmitted phase image set.

[0022] According to one embodiment, the disparity calculation unit calculates the sum of evaluation values ​​corresponding to any candidate disparities, each of which is included in a different evaluation value set, and calculates the candidate disparity with the largest sum of the evaluation values ​​as the target disparity.

[0023] According to one embodiment, the evaluation value calculation unit converts a plurality of evaluation value sets corresponding to a plurality of phase image sets respectively into an evaluation value set corresponding to a current focus position of a lens included in the imaging device, and the parallax calculation unit can calculate the target parallax based on the evaluation value set corresponding to the current focus position of the lens.

[0024] According to one embodiment, the evaluation value calculation unit calculates a reliability for the evaluation value set based on noise in the phase image set, and the disparity calculation unit calculates the target disparity reflecting the reliability.

[0025] According to an embodiment, the disparity calculation unit may calculate the target disparity based on candidate disparities corresponding to maximum evaluation values ​​included in different sets of evaluation values.

[0026] An imaging device according to another embodiment of the present invention includes an imaging unit including a lens and collecting a plurality of phase signal pairs for an object; an image generation unit that generates a plurality of phase image sets based on the plurality of phase signal pairs; an evaluation value calculation unit that calculates a plurality of evaluation value sets including evaluation values ​​corresponding to candidate disparities based on the plurality of phase image sets; and a disparity calculation unit that calculates a target disparity based on the plurality of evaluation value sets, wherein the target disparity may be a disparity for controlling the focus position of the lens relative to the object. According to another embodiment, the evaluation values ​​may indicate the likelihood that the candidate disparities corresponding to the respective evaluation values ​​are the target disparities.

[0027] According to another embodiment, the plurality of phase signal pairs may be acquired at different times. According to another embodiment, the multiple phase signal pairs may be collected at the same time. [Effects of the Invention]

[0028] An imaging device according to an embodiment of the present invention may include an image processing device that reduces the influence of noise and more accurately calculates the target parallax.

[0029] An image processing device according to an embodiment of the present invention can calculate an evaluation value set including an evaluation value for each candidate disparity predicted as a target disparity, and calculate a target disparity based on the calculated evaluation value set. In addition, this document can provide various other benefits that can be perceived directly or indirectly. [Brief explanation of the drawings]

[0030] [Figure 1] 1 is a diagram illustrating an exemplary structure of an imaging device according to an embodiment of the present invention; [Figure 2] 1 is a block diagram specifically illustrating an image processing apparatus according to an embodiment of the present invention; [Figure 3a] 10A and 10B are diagrams for explaining a method for calculating a target parallax based on a plurality of phase signal pairs collected at different times. [Figure 3b] 10A and 10B are diagrams for explaining a method for calculating a target parallax based on a plurality of phase signal pairs collected at different times. [Figure 3c] 10A and 10B are diagrams for explaining a method for calculating a target parallax based on a plurality of phase signal pairs collected at different times. [Figure 4a] FIG. 10 is a diagram for explaining a method for calculating a target parallax based on one image collected at an arbitrary time. [Figure 4b] FIG. 10 is a diagram for explaining a method for calculating a target parallax based on one image collected at an arbitrary time. [Figure 4c] FIG. 10 is a diagram for explaining a method for calculating a target parallax based on one image collected at an arbitrary time. DETAILED DESCRIPTION OF THE INVENTION

[0031] Various embodiments of the present invention will now be described with reference to the accompanying drawings. The advantages and features of the present invention, and the manner in which they are achieved, will become apparent from the detailed description of the embodiments below in conjunction with the accompanying drawings. However, this does not limit the present invention to the specific embodiments.

[0032] It should be understood that the present invention is not limited to the embodiments, but can be realized in various different forms, and includes various modifications, equivalents, and / or alternatives to the embodiments of the present invention.

[0033] Furthermore, when assigning reference symbols to components in each drawing, it should be noted that the same symbols are assigned to the same components as far as possible even when they are displayed in other drawings.

[0034] In describing the embodiments of the present invention, if a detailed description of related known structures or functions is deemed to hinder understanding of the embodiments of the present invention, the detailed description will be omitted.

[0035] In the specification, the singular forms "a," "an," and "the" include the plural forms unless the context clearly dictates otherwise. As used in the specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, operations, and / or elements to a stated component, step, operation, and / or element.

[0036] 1 is a diagram illustrating an example of the structure of an imaging device 1 according to an embodiment of the present invention. A method for performing an AF (Auto-Focus) function in the imaging device 1 will be described with reference to FIG.

[0037] Referring to FIG. 1, an imaging device 1 may include an image capture unit 10 , an image processing unit 20 , and a motion detection unit 30 . The imaging unit 10 may include an image sensor 100 and a lens module 200. The image sensor 100 may include a pixel array 110, a driving circuit 120, a timing controller 130, and a readout circuit 140. In one embodiment, the pixel array 110 may include multiple unit pixels.

[0038] Incident light (optical signal) passing through the lens 210 included in the lens module 200 can be imaged and converted into an electrical signal by the unit pixels included in the pixel array 110. Each unit pixel can generate an electrical signal corresponding to an external object S.

[0039] Each unit pixel included in the pixel array 110 may include a photoelectric conversion element that absorbs light and generates an electric charge. Each unit pixel may provide a pixel signal corresponding to the electric charge generated by the photoelectric conversion element to the readout circuit 140.

[0040] According to an embodiment, at least some of the unit pixels included in the pixel array 110 may be phase difference pixels that generate different phase signals for the same object.

[0041] A pair of phase difference pixels may be arranged adjacent to each other in a vertical, horizontal, or diagonal direction on the pixel array so as to generate different phase signals for the same object. In this case, one of the pair of phase difference pixels generating different phase signals may be referred to as a first phase signal collector, and the other may be referred to as a second phase signal collector.

[0042] Furthermore, the signal generated by the first phase signal collecting unit can be referred to as a first phase signal, and the signal generated by the second phase signal collecting unit can be referred to as a second phase signal.

[0043] According to another embodiment, the phase difference pixel may include two photoelectric conversion elements adjacent to each other vertically or horizontally, each of which generates a phase signal different from the other.

[0044] A pair of photoelectric conversion elements, each generating a different phase signal and included in one phase difference pixel, can be referred to as a phase signal collecting unit. In this case, one of the pair of photoelectric conversion elements generating different phase signals can be referred to as a first phase signal collecting unit, and the other can be referred to as a second phase signal collecting unit. Furthermore, the signal generated by the first phase signal collecting unit can be referred to as a first phase signal, and the signal generated by the second phase signal collecting unit can be referred to as a second phase signal.

[0045] Each unit pixel included in the pixel array 110 may include a microlens, an optical filter, a photoelectric conversion element, and a wiring layer. According to an embodiment, one unit pixel may overlap one microlens.

[0046] The microlens can focus incident light that is incident on the pixel array 110 onto the optical filter and the photoelectric conversion element. The optical filter can selectively transmit the incident light that has passed through the microlens according to its wavelength.

[0047] Each unit pixel may include a photoelectric conversion element that receives incident light. The photoelectric conversion element may generate photocharges corresponding to the incident light that has passed through a microlens and an optical filter. The photoelectric conversion element may be a photodiode, a phototransistor, a photogate, a pinned photodiode (PPD), or a combination thereof. Hereinafter, the description will be made assuming that the photoelectric conversion element is a photodiode.

[0048] When the photoelectric conversion element is a photodiode, the photoelectric conversion element may include a structure in which an N-type impurity region and a P-type impurity region are vertically stacked. The photoelectric conversion element may be formed inside a semiconductor substrate. Exemplarily, the semiconductor substrate may be a P-type semiconductor substrate.

[0049] A wiring layer may be formed below the photoelectric conversion element, and a reset transistor, a transfer transistor, a floating diffusion region, a drive transistor, a selection transistor, and the like may be disposed in the wiring layer.

[0050] The reset transistor is activated in response to a reset control signal, thereby resetting the unit pixel potential to a predetermined level (pixel voltage level).

[0051] Also, when the reset transistor is activated, the transfer transistor can also be activated at the same time to reset the floating diffusion region.

[0052] The transfer transistor becomes active in response to a transfer control signal, thereby transferring photocharges accumulated in the photoelectric conversion element of each pixel to the floating diffusion region.

[0053] According to an embodiment, the unit pixel may include a transfer transistor corresponding to each photoelectric conversion element. The floating diffusion region can receive and store the charge generated by the photoelectric conversion element, and can be connected to the gate of the drive transistor.

[0054] The driving transistor may have a drain connected to a pixel voltage, a floating diffusion region connected to a gate of the driving transistor, and a source connected to a selection transistor.

[0055] The driving transistor can output a current corresponding to the voltage of the floating diffusion region connected to the gate electrode to the signal line through the selection transistor, in other words, the voltage of the floating diffusion region can be amplified through the driving transistor.

[0056] The selection transistor is activated in response to a selection control signal supplied to its gate electrode, thereby outputting the signal output from the drive transistor to the signal line. The pixel signal output to the signal line can be provided to the readout circuit 140. The pixel signal may be a signal that is output in response to the charge accumulated in the floating diffusion region included in the pixel array 110.

[0057] The readout circuit 140 may include a correlated double sampler (CDS), an analog-digital converter (ADC), a buffer, and the like.

[0058] The correlated double sampler can sample and hold pixel signals provided from the pixel array 110. The correlated double sampler can double sample the pixel signal level due to a specific noise level and incident light, and output a level corresponding to the difference. The correlated double sampler can remove noise from the pixel signal.

[0059] The analog-to-digital converter can convert the analog signal received from the correlated double sampler into a digital signal and transfer it to the buffer. The buffer can latch the received digital signals and sequentially output them to the image processing device 20. The buffer can include a memory for latching the digital signals and a sense amplifier for amplifying the digital signals.

[0060] The driving circuit 120 can drive a plurality of unit pixels included in the pixel array 110 in response to a signal from the timing controller 130 . For example, the driving circuit 120 can generate signals to control transistors included in the plurality of unit pixels included in the pixel array 110 (e.g., a transfer control signal to control a transfer transistor, a reset control signal to control a reset transistor, and a selection control signal to control a selection transistor, etc.) and provide them to the pixel array 110.

[0061] The driving circuit 120 can determine the activation and deactivation timings of the transfer control signal, the reset control signal, and the selection control signal provided to the unit pixel.

[0062] The timing controller 130 can control the driving circuit 120 to cause the pixel array 110 to absorb light and accumulate charges, or to temporarily store the accumulated charges, and to output an electrical signal based on the stored charges to the outside of the pixel array 110. The pixel signals may correspond to photocharges generated by the photoelectric conversion elements that the pixel array 110 includes.

[0063] The timing controller 130 can control the readout circuit 140 to sample and hold pixel signals provided from the pixel array 110. The timing controller 130 can control the analog-to-digital converter to convert the signal received from the correlated double sampler into a digital signal.

[0064] The timing controller 130 can generate / store a control signal for controlling the readout circuit 140 based on a signal received from the image processing device 20 .

[0065] According to one embodiment, pixel signals corresponding to the photocharges generated by a pair of phase signal collectors may be referred to as phase signals, and the phase signals corresponding to a pair of phase signal collectors may be referred to as a phase signal pair.

[0066] The imaging unit 10 can provide the phase signal pairs to the image processor 20. The image processor 20 can generate a set of phase images based on the received phase signal pairs.

[0067] The lens module 200 may be a component that receives light. Specifically, the lens module 200 may include a lens 210 and a lens driver 220. Lens 210 may refer to a single lens as well as a configuration including multiple lenses.

[0068] The lens driver 220 can control (focus) the position of the lens 210 in response to a control signal from the image processing device 20. By adjusting the position of the lens 210, the focus position of the lens 210 is adjusted.

[0069] The image processing device 20 generates a phase image and a phase image set based on the phase signal pair output from the imaging unit 10, and can calculate a target parallax for autofocusing based on the generated image set.

[0070] The target disparity may refer to the disparity calculated from the first phase image and the second phase image when the first phase image and the second phase image included in the phase image set are not affected by noise.

[0071] The image processing device 20 may generate a control signal for the lens driver 220 using the calculated target parallax and provide the generated control signal to the lens driver 220 . The lens driver 220 can control the focus position of the lens 210 in response to the control signal so that the object S is in focus.

[0072] The image processing device 20 can calculate an evaluation value corresponding to each candidate disparity based on the first phase image and the second phase image included in the phase image set. Parallax may refer to the difference between a pair of phase images of a single external object S.

[0073] Since the first phase image and the second phase image are generated based on the phase signals generated by a pair of phase signal collecting units, differences between the first phase image and the second phase image may occur depending on the positional difference between the phase signal collecting units and the path of the incident light passing through lens 210.

[0074] When the focus position of the lens 210 is controlled so that the pair of phase images move by the target parallax, the pair of phase images can be aligned with each other. The focus position of the lens 210 when the pair of phase images are aligned with each other can be referred to as the correct focus position.

[0075] When the first phase image and the second phase image are shifted by the target parallax, the difference between the shifted first phase image and the second phase image can be minimized. In other words, the target parallax may be a phase transition value that minimizes the difference between the first phase image and the second phase image. A candidate disparity may refer to any disparity that has a high probability of being a target disparity.

[0076] If the first and second phase images are distorted by noise, the disparity calculated when the difference between the first and second phase images is minimum may differ from the target disparity.

[0077] Therefore, the image processing device 20 can calculate an evaluation value corresponding to the likelihood that any candidate disparity is the target disparity. The evaluation value corresponding to the candidate disparity may be an error value between the first phase image and the second phase image after shifting the first phase image and the second phase image by the candidate disparity.

[0078] In other words, the candidate disparity for transitioning between the first phase image and the second phase image so as to minimize the error value between the first phase image and the second phase image can be most likely to be the target disparity.

[0079] According to one embodiment, the image processing device 20 may calculate the evaluation value set based on the sum of absolute differences (SAD) between the first and second phase images.

[0080] According to another embodiment, the image processor 20 may calculate the evaluation value set based on the sum of square differences (SSD) between the first and second phase images.

[0081] According to another embodiment, the image processing device 20 may calculate the evaluation value set based on the phase correlation between the first phase image and the second phase image.

[0082] According to yet another embodiment, the image processor 20 can calculate the evaluation value set based on the cross correlation between the first and second phase images.

[0083] However, the calculation method of the evaluation value set is merely an example, and any calculation method that can determine the error value between phase images can be included in the technical idea of ​​the present disclosure.

[0084] The motion detector 30 may include a sensor for detecting the motion of the imaging device 1 or the external object S. According to one embodiment, the motion detector 30 may include an external sensor such as a gyroscope sensor.

[0085] According to another embodiment, the motion detector 30 can receive images from the image processing device 20 and detect the motion of the imaging device 1 or the external device S based on the received images.

[0086] If motion above the threshold occurs, the motion detector 30 can provide information to the image processor 20 about the set of phase images corresponding to the motion above the threshold.

[0087] The image processing device 20 can exclude a phase image set corresponding to a motion exceeding a threshold based on information received from the motion detection unit 30, and calculate a target parallax amount based on the remaining phase image set.

[0088] FIG. 2 is a block diagram specifically showing an image processing device 20 according to an embodiment of the present invention. Referring to FIG. 2, the image processing device 20 may include an image generating unit 300, an evaluation value calculating unit 400, a disparity calculating unit 500, and a control signal generating unit 600.

[0089] The image generating unit 300 can generate general image data or phase-contrast image data based on the signal received from the imaging unit 10 . The image generating unit 300 can generate a first phase image based on a first phase signal output from a first phase signal collecting unit included in the pixel array (110 in FIG. 1) and can generate a second phase image based on a second phase signal output from a second phase signal collecting unit included in the pixel array (110 in FIG. 1).

[0090] The image generating unit 300 can generate a phase image set including a first phase image and a second phase image. The image generator 300 can also perform noise reduction, gain adjustment, waveform shaping, interpolation, white balance, gamma adjustment, edge enhancement, and the like on the image.

[0091] According to one embodiment, the phase image set may include phase image pairs generated based on phase signal pairs output from all of the phase signal collecting unit pairs included in the pixel array. In this case, each phase image may be an image corresponding to the entire area of ​​the pixel array.

[0092] According to another embodiment, the phase image set may include a phase image pair generated based on a phase signal pair output from a portion of the phase signal collecting unit pairs included in the pixel array. In this case, each phase image may be collected at an arbitrary time and may correspond to a partial region of the pixel array.

[0093] The imaging unit 10 may transmit the phase signals generated at preset time intervals to the image generating unit 300. The preset time interval may be a time period during which a sufficient amount of phase signals can be collected to calculate a target parallax.

[0094] The evaluation value calculation unit 400 can calculate an evaluation value set including evaluation values ​​corresponding to the candidate parallaxes based on the phase image set. As described above, the evaluation value may be a value corresponding to the likelihood that any candidate disparity is the target disparity.

[0095] The evaluation value calculation unit 400 can calculate an evaluation value using the sum of absolute differences (SAD), sum of square differences (SSD), phase correlation, or cross correlation between phase images. The set of evaluation values ​​may include evaluation values ​​for candidate disparities. The evaluation values ​​corresponding to the candidate disparities may be expressed in a two-dimensional plane.

[0096] The evaluation value calculation unit 400 may linearly interpolate the evaluation values ​​corresponding to the candidate parallaxes to calculate evaluation values ​​for candidate parallaxes for which evaluation values ​​have not been calculated. The evaluation value calculation section 400 can calculate, based on a plurality of phase image sets, a plurality of evaluation value sets corresponding to the respective phase image sets. According to one embodiment, the multiple phase image sets may correspond to phase signal pairs collected at different times.

[0097] More specifically, the evaluation value calculation unit 400 can calculate an evaluation value set based on a pair of a first phase image and a second phase image corresponding to phase signals collected at different times. In this case, the first phase image and the second phase image may be images corresponding to phase signals output from all pixels included in the pixel array.

[0098] In another embodiment, the multiple phase image sets may each correspond to a pair of phase signals collected at the same time. More specifically, the first phase image and the second phase image may be images corresponding to phase signals output from a portion of pixels in the pixel array.

[0099] The evaluation value conversion unit 410 can convert the evaluation value sets corresponding to the plurality of phase image sets into an evaluation value set corresponding to the current focus position of the imaging unit 10 . The phase image sets corresponding to the phase signals collected at different times may be collected with the imaging unit 10 at different focus positions.

[0100] The evaluation value conversion unit 410 converts the evaluation value sets corresponding to the phase image sets collected at different focus positions to correspond to the current focus position, so that the evaluation value sets collected at different focus positions can be used to calculate the target parallax.

[0101] The evaluation value conversion unit 410 receives focus position information corresponding to each phase image set from the imaging unit 10, and can convert the evaluation value set to correspond to the current focus position based on the received focus position information.

[0102] More specifically, the evaluation value conversion unit 410 can calibrate the candidate parallax to correspond to the current focus position based on the focus position information corresponding to the phase image set, and can convert the evaluation value by interpolating the evaluation value corresponding to the calibrated candidate parallax.

[0103] The reliability calculation unit 420 can calculate the reliability corresponding to each evaluation value set. According to an embodiment, the reliability calculation unit 420 displays the evaluation values ​​corresponding to the candidate disparities on a two-dimensional plane, and the reliability calculation unit 420 can calculate the evaluation values ​​based on the shape of the displayed candidate disparity graph.

[0104] More specifically, the reliability calculation unit 420 can display a continuous evaluation value graph corresponding to the candidate disparities by interpolation, and calculate the reliability by comparing it with a curvature function fitted to the evaluation value graph.

[0105] According to another embodiment, the reliability calculation unit 420 may calculate the reliability based on a comparison of the contrast between the first phase image and the second phase image. According to another embodiment, the reliability calculation unit 420 can calculate the reliability based on the difference between the maximum, minimum, and average values ​​of the evaluation values.

[0106] According to an embodiment, a confidence level can correspond to each set of evaluation values, and the confidence level can be used as a weight for each set of evaluation values ​​when calculating the target disparity.

[0107] The reliability calculation unit 420 can receive data for calculating the reliability via the imaging unit 10. The evaluation value storage unit 430 can store the calculated evaluation value set. The evaluation value set is stored in correspondence with the corresponding reliability, and the time when the phase image set was collected and the focus position of the lens 210 can also be stored in correspondence with the evaluation value set.

[0108] Furthermore, the evaluation value storage unit 430 can store the target parallax calculated by the parallax calculation unit 500 in association with the focal position of the lens 210 that collected the phase image set that is the basis for the calculation of the target parallax. The evaluation value calculation unit 400 can reflect the already stored target parallax and the corresponding focal position information in the calculation of the evaluation value.

[0109] The disparity calculation unit 500 can calculate the target disparity based on a plurality of evaluation value sets including candidate disparities and evaluation values. The target parallax may refer to the parallax calculated from the first phase image and the second phase image corresponding to the current focus position of the lens 210 in the absence of noise.

[0110] The disparity calculation unit 500 calculates the target disparity based on a plurality of sets of evaluation values, thereby reducing the influence of noise on an arbitrary set of phase images.

[0111] According to an embodiment, the disparity calculation unit 500 may calculate the sum of the evaluation values ​​corresponding to the candidate disparities, and determine the candidate disparity with the largest sum of the evaluation values ​​as the target disparity.

[0112] According to another embodiment, the disparity calculation unit 500 may calculate the target disparity by reflecting the reliability corresponding to the evaluation value set. For example, the disparity calculation unit 500 can multiply the evaluation values ​​included in the evaluation value set by the reliability, and calculate the candidate disparity with the largest sum of the evaluation values ​​multiplied by the reliability as the target disparity.

[0113] According to another embodiment, the disparity calculation unit 500 can calculate the target disparity based on the candidate disparity corresponding to the maximum evaluation value included in each evaluation value set. For example, the disparity calculation unit 500 may calculate the candidate disparity that most frequently has the maximum evaluation value as the target disparity.

[0114] The control signal generating section 600 can generate a control signal for controlling the imaging section 10 based on the calculated target parallax. The control signal generated by the control signal generating unit 600 may be a control signal for adjusting the focus position of the lens 210 included in the imaging unit 10. By generating the control signal based on the target parallax, the focus position of the lens 210 is adjusted so that the first phase image and the second phase image coincide with each other. The focus position of the lens 210 where the first phase image and the second phase image coincide with each other may be referred to as the correct focus position.

[0115] The control signal generator 600 may transmit the generated control signal to the lens module 200. The lens module 200 may control the lens driver 220 based on the control signal, and the lens driver 220 may adjust the focus position of the lens 210.

[0116] 3a to 3c are diagrams for explaining a method for calculating a target parallax based on a plurality of phase signal pairs collected at different times. Referring to FIG. 3a, the image generating unit (300 in FIG. 2) included in the image processing device (20 in FIG. 2) can receive time-dependent phase signals from the imaging unit 10 and generate image sets (IST1, IST2 to ISTN).

[0117] The first time phase image set IST1, the second time phase image set IST2 to the Nth time phase image set ISTN may correspond to phase signals collected at different times by the imaging unit (10 in FIG. 1).

[0118] The evaluation value calculation unit (400 in Figure 2) can calculate a first time evaluation value set EST1, a second time evaluation value set EST2, and an Nth time evaluation value set ESTN based on the first time phase image set IST1, the second time phase image set IST2, and the Nth time phase image set ISTN.

[0119] The first temporal evaluation value set EST1, the second temporal evaluation value set EST2, and the Nth temporal evaluation value set ESTN can be represented by evaluation value curves for candidate disparities as shown in Fig. 3b. The evaluation value sets for the candidate disparities can be calculated based on the phase images included in each of the phase image sets IST1, IST2, and ISTN.

[0120] For example, the evaluation value set can be calculated based on the sum of absolute differences between the first phase image and the second phase image included in each phase image set.

[0121] More specifically, the sum of absolute differences between the first phase image and the second phase image for each candidate parallax is calculated, and the inverse of the calculated sum of absolute differences can be used as the evaluation value. The candidate disparity that minimizes the sum of absolute differences between the first phase image and the second phase image may be the candidate disparity that is most likely to become the target disparity.

[0122] The evaluation value curve shown in Fig. 3b may be a continuous curve obtained by linear interpolation. The evaluation value corresponding to the candidate disparity may be the reciprocal of the sum of the absolute differences between the first and second phase images.

[0123] As shown in FIG. 3c, the disparity calculation unit (500 in FIG. 2) can calculate a target disparity TP based on a plurality of evaluation value sets (EST1, EST2 to ESTN).

[0124] According to the embodiment, the disparity calculation unit can calculate the target disparity TP based on the evaluation value set. More specifically, the disparity calculation unit can calculate the target disparity TP based on the sum of the evaluation values ​​corresponding to the respective candidate disparities. By using a plurality of evaluation value sets and calculating the sum of the evaluation values ​​corresponding to the respective candidate parallaxes, distortion due to noise can be prevented.

[0125] 3c, the sum of the evaluation values ​​for each candidate disparity can be represented by a continuous curve. The disparity calculation unit can calculate the candidate disparity with the largest sum of the evaluation values ​​in the curve of the sum of the evaluation values ​​for the candidate disparities as the target disparity TP. Furthermore, the evaluation value calculation unit can store the current focus position of the imaging unit that collected the phase signal and the target parallax TP calculated by the parallax calculation unit in association with each other.

[0126] According to one embodiment, the imaging unit can select a phase signal pair having the same focus position from among phase signal pairs acquired at different times, and transmit the selected phase signal pair to the evaluation value calculation unit.

[0127] When the evaluation value calculation unit calculates the evaluation value sets based on the phase image sets having the same focus position, it is not necessary to perform separate correction for each evaluation value set. Also, the disparity calculation unit can calculate the target disparity TP based on the evaluation value sets having the same focus position.

[0128] According to another embodiment, the imaging unit may transmit a pair of phase signals having different focus positions among the phase signals collected at different times to the evaluation value calculation unit, and the pair of phase signals transmitted by the imaging unit may be phase signals collected during a predetermined time interval.

[0129] The evaluation value calculation unit can calculate an evaluation value set based on a phase image set having different focus positions, and convert the calculated evaluation value set into an evaluation value set corresponding to the current focus position of the imaging unit.

[0130] According to an embodiment, the imaging unit can transmit a phase signal pair at a preset time interval. When a new phase image set is transferred, the evaluation value calculation unit can discard the evaluation value set calculated based on the already transferred phase image set.

[0131] According to another embodiment, the evaluation value calculation unit can calculate the reliability of each of the evaluation value sets EST1 and EST2. The disparity calculation unit can reflect the reliability of each evaluation value set EST1, EST2 when calculating the target disparity. For example, the reliability can be multiplied by each evaluation value set EST1, EST2 to obtain the sum of the evaluation values ​​for the candidate disparities. In other words, the disparity calculation unit can use the reliability as a weight for calculating the target disparity TP.

[0132] 4a to 4c are diagrams for explaining a method for calculating a target parallax based on one image collected at an arbitrary time. Referring to Fig. 4a (A), a first image I1 captured by the imaging unit (10 in Fig. 1) can be divided into a plurality of regions (A1, A2 to AN). The first image I1 may be an image collected by the imaging unit 10 at any time.

[0133] The first region A1 and the second region A2 to the Nth region AN may be different regions included in the first image I1. The image generating unit (300 in FIG. 2) receives the phase signals for each region from the imaging unit 10 and can generate an image set (ISA1, ISA2 to ISAN).

[0134] The first area phase image set ISA1, the second area phase image set ISA2, and the Nth area phase image set ISAN may correspond to different areas of the first image I1, respectively.

[0135] The image generation unit can transmit the generated phase image set (ISA1, ISA2 to ISAN) to the evaluation value calculation unit (400 in FIG. 2). The noise in each region of the first image I1 may vary depending on the physical structure of the image capture unit 10, the arrangement of the unit pixels included in the pixel array 110, and the like.

[0136] The first area phase image set ISA1 and the second area phase image set ISA2 may be image sets collected at the same time and at the same focal position.

[0137] The evaluation value calculation unit can calculate the first area evaluation value set ESA1, the second area evaluation value set ESA2 to the Nth area evaluation value set ESAN based on the received first area phase image set ISA1, the second area phase image set ISA2 to the Nth area phase image set ISAN.

[0138] The first region evaluation value set ESA1, the second region evaluation value set ESA2, and the Nth region evaluation value set ESAN can be represented by evaluation value curves for candidate disparities as shown in Figure 4b. The evaluation value sets for the candidate disparities can be calculated based on the phase images included in each image set (ISA1, ISA2 to ISAN). The calculation method for the evaluation value sets has been described with reference to Figures 3a to 3c above, so a duplicated description will be omitted.

[0139] The evaluation value curve shown in Fig. 4b may be a continuous curve obtained by linear interpolation. The evaluation value corresponding to the candidate disparity may be the reciprocal of the sum of the absolute differences between the first and second phase images included in each image set (ISA1, ISA2 to ISAN).

[0140] As shown in FIG. 4c, the disparity calculation unit (500 in FIG. 2) can calculate a target disparity TP based on a plurality of evaluation value sets (ESA1, ESA2 to ESAN).

[0141] According to the embodiment, the disparity calculation unit can calculate the target disparity TP based on the evaluation value set. The method for calculating the target disparity TP has been described with reference to Figures 3a to 3c above, so a duplicated description will be omitted.

[0142] Distortion due to noise can be prevented by calculating the sum of the evaluation values ​​corresponding to each candidate disparity using multiple evaluation value sets corresponding to different areas (A1, A2 to AN) included in one image I1.

[0143] The first area phase image set ISA1, the second area phase image set ISA2 to the Nth area phase image set ISAN are phase image sets with the same lens focal position, and therefore do not require separate correction for each evaluation value set.

[0144] According to the embodiment, the imaging unit 10 can collect phase signals at preset time intervals and transmit the collected phase signals to the image generating unit. When a new phase image set is transmitted from the image generating unit, the evaluation value calculating unit can discard the evaluation value set calculated based on the already transmitted phase image set.

[0145] According to another embodiment, the evaluation value calculation unit can calculate the reliability of each evaluation value set (ESA1, ESA2 to ESAN). The disparity calculation unit can reflect the reliability of each evaluation value set (ESA1, ESA2 to ESAN) when calculating the target disparity.

[0146] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art will understand that the present invention can be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and are not limiting. [Explanation of symbols]

[0147] 1. Imaging equipment 10. Imaging unit 20 Image Processing Device 30 Detector 100 image sensors 110 pixel array 120 Driving circuit 130 Timing controller 140 Readout circuit 200 Lens Module 210 Lens 220 Lens drive unit 300 Image Generation Unit 400 Evaluation value calculation unit 410 Evaluation value conversion unit 420 Reliability calculation unit 430 Evaluation value storage section 500 Parallax calculation section 600 control signal generation unit

Claims

1. an evaluation value calculation unit that calculates an evaluation value set including evaluation values ​​corresponding to the candidate parallaxes based on the phase image set; a parallax calculation unit that calculates a target parallax based on the evaluation value set; Including, The evaluation values ​​indicate the likelihood that the candidate parallaxes corresponding to the respective evaluation values ​​are the target parallaxes.

2. an imaging unit including a lens and configured to collect a phase signal pair including a first phase signal for the object and a second phase signal for the object; The phase image set includes:

2. The imaging device of claim 1, comprising a first phase image corresponding to the first phase signal and a second phase image corresponding to the second phase signal.

3. The imaging device according to claim 2 , wherein the target parallax is a parallax for controlling a focus position of the lens so that the first phase image and the second phase image coincide with each other.

4. The evaluation value calculation unit 3. The imaging apparatus of claim 2, wherein the evaluation value set is calculated based on a sum of absolute differences between the first phase image and the second phase image.

5. The evaluation value calculation unit 3. The imaging apparatus of claim 2, wherein the evaluation value set is calculated based on a sum of square differences between the first phase image and the second phase image.

6. The evaluation value calculation unit 3. The imaging apparatus of claim 2, wherein the evaluation value set is calculated based on a phase correlation between the first phase image and the second phase image.

7. The evaluation value calculation unit 3. The imaging apparatus of claim 2, wherein the set of evaluation values ​​is calculated based on a cross correlation between the first phase image and the second phase image.

8. The imaging device of claim 2 , further comprising an image generator that generates the phase image set based on the phase signal pair.

9. The image generation unit generating a plurality of sets of time phase images based on a plurality of pairs of phase signals collected at different times; The evaluation value calculation unit computing a plurality of sets of time estimates corresponding to the plurality of sets of time phase images; The parallax calculation unit The imaging device of claim 8 , wherein the target parallax is calculated based on the plurality of sets of time evaluation values.

10. The image generation unit generating a plurality of regional phase image sets based on one phase signal pair collected at any time; The evaluation value calculation unit calculating a plurality of sets of regional evaluation values ​​corresponding to the plurality of sets of regional phase images; The parallax calculation unit The imaging device according to claim 8 , wherein the target parallax is calculated based on the plurality of sets of region evaluation values.

11. the imaging unit transmits the phase signal pair to the image generating unit at a predetermined time interval; The imaging apparatus of claim 8 , wherein the image generating unit generates the phase image set at the preset time intervals.

12. The evaluation value calculation unit 12. The imaging apparatus of claim 11, wherein when a new set of phase images is transmitted, a set of estimates calculated based on a set of previously transmitted phase images is discarded.

13. The parallax calculation unit 2. The imaging device according to claim 1, wherein a sum of evaluation values ​​included in different evaluation value sets and corresponding to any candidate parallaxes is calculated, and the candidate parallax having the largest sum of the evaluation values ​​is calculated as the target parallax.

14. The evaluation value calculation unit converting a plurality of evaluation value sets corresponding to the plurality of phase image sets, respectively, into an evaluation value set corresponding to a current focus position of a lens included in the imaging device; The parallax calculation unit The imaging device of claim 1 , wherein the target parallax is calculated based on a set of evaluation values ​​corresponding to a current focus position of the lens.

15. The evaluation value calculation unit calculating a confidence level for the set of evaluation values ​​based on noise in the set of phase images; The imaging device according to claim 1 , wherein the parallax calculation unit calculates the target parallax by reflecting the reliability.

16. The parallax calculation unit The imaging device according to claim 1 , wherein the target disparity is calculated based on candidate disparities corresponding to maximum evaluation values ​​included in different sets of evaluation values.

17. an imaging unit including a lens for collecting a plurality of phase signal pairs for the object; an image generator for generating a plurality of phase image sets based on the plurality of phase signal pairs; an evaluation value calculation unit that calculates a plurality of evaluation value sets including evaluation values ​​corresponding to the candidate parallaxes based on the plurality of phase image sets; a parallax calculation unit that calculates a target parallax based on the plurality of evaluation value sets; Including, An imaging apparatus, wherein the target parallax is a parallax for controlling the focus position of the lens relative to the object.

18. The imaging device according to claim 17 , wherein the evaluation values ​​indicate a likelihood that the candidate parallax corresponding to each evaluation value is the target parallax.

19. 18. The imaging device of claim 17, wherein the plurality of phase signal pairs are acquired at different times.

20. 18. The imaging device of claim 17, wherein the multiple phase signal pairs are collected at the same time.

Images