Imaging device, image processing method, and program

Abstract

This invention provides an imaging device that achieves high-speed and accurate autofocus by improving the defocus amount detection performance. [Solution] An imaging device is provided, comprising: an image sensor in which a first pixel group having a first characteristic for detecting phase difference for autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged; an image stabilization unit for correcting camera shake; and an arithmetic processing unit that outputs a defocus amount based on pixel data of the first pixel group and pixel data of the second pixel group, where the pixel positions of the subject obtained from multiple exposures are different.

Description

Technical Field

[0001] The present disclosure relates to an imaging device, an image processing method, and a program.

Background Art

[0002] In cameras of smartphones and the like, an image sensor capable of performing phase difference detection is used. Recently, a configuration in which two photodiodes are provided for one pixel called Dual Pixel has been used, and the amount of defocus can be detected by detecting the phase difference between these two photodiodes. However, from the viewpoints of size and cost, an image sensor in which focus detection pixels are discretely arranged is still being utilized in the tele cameras of high-end smartphones, low-end smartphones, tablets, and the like. Patent Document 1 discloses an imaging device capable of acquiring the amount of defocus with high accuracy and at high speed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 was an imaging device that acquired the amount of defocus in one exposure. However, in the case of an image sensor in which focus detection pixels are discretely arranged, the number of pixels capable of focus detection may be small, and there are problems with the detection performance of the amount of defocus. Therefore, an object of the present disclosure is to provide an imaging device or the like that realizes fast and accurate autofocus by improving the defocus amount detection performance in an imaging device using discretely arranged pixels for phase difference detection.

Means for Solving the Problems

[0005] The imaging device of the present disclosure is The imaging device comprises an image sensor in which a first pixel group having a first characteristic for detecting phase difference for autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged; an image stabilization unit for correcting camera shake; and a calculation processing unit that outputs a defocus amount based on the pixel data of the first pixel group and the pixel data of the second pixel group, where the pixel positions of the subject obtained from multiple exposures are different.

[0006] With the above configuration, an imaging device using discretely arranged pixels for phase difference detection can be provided that improves the defocus amount detection performance, thereby enabling high-speed and accurate autofocus.

[0007] The imaging device disclosed herein is It has a shake detection unit that detects camera shake, The system is characterized by controlling the pixel position of the subject to differ in multiple exposures based on the detection results of the shake detection unit.

[0008] With the above configuration, even with camera shake, the pixel position relative to the subject can be varied across multiple exposures.

[0009] The imaging device disclosed herein is This method is characterized by correcting the positional shift of the subject's pixels due to camera shake between multiple exposures, and by causing a positional shift of the pixels of a predetermined subject.

[0010] With the above configuration, even with camera shake, it is possible to generate a shift in the position of pixels relative to a given subject.

[0011] The imaging device disclosed herein is Each time exposure, the system includes a matching degree calculation unit that calculates the degree of matching between the image shapes of the pixel data of the first pixel group and the pixel data of the second pixel group. The method is characterized by controlling the pixel positions of the subject to differ based on the degree of similarity.

[0012] With the above configuration, the pixel positions of the subject can be varied based on the degree of agreement between the image shapes of the pixel data of the first pixel group and the pixel data of the second pixel group.

[0013] The imaging device disclosed herein is The arithmetic processing unit performs output processing related to defocusing when a predetermined degree of agreement is obtained.

[0014] With the above configuration, it is possible to determine the timing of output processing related to defocusing based on the degree of match.

[0015] The imaging device disclosed herein is It has a reliability calculation unit that calculates the reliability of the output result regarding the amount of defocus each time it is exposed, The method is characterized by determining the effectiveness of the output result regarding the amount of defocus based on the aforementioned reliability.

[0016] With the above configuration, the validity of the output results regarding the amount of defocus can be determined.

[0017] The imaging device disclosed herein is The method is characterized in that, when outputting the defocus amount multiple times, the pixel data of the first pixel group and the pixel data of the second pixel group used for each output are shared multiple times.

[0018] With the above configuration, data can be shared and the amount of defocus can be quickly output.

[0019] The image processing method disclosed herein is: This image processing method uses an image sensor in which a first pixel group having a first characteristic for detecting phase difference for autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged, and an image stabilization unit for correcting camera shake, to output an output regarding the amount of defocus based on pixel data of the first pixel group and the second pixel group, where the pixel positions of the subject obtained from multiple exposures are different.

[0020] With the above configuration, in an imaging device using pixels for phase difference detection that are discretely arranged, by improving the defocus amount detection performance, an image processing method capable of realizing high-speed and accurate autofocus can be provided.

[0021] The program of the present disclosure is an image sensor in which a first pixel group having a first characteristic for phase difference detection of autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged, and a shake correction unit for correcting shake, and causing an information processing device to perform an output related to a defocus amount based on pixel data of the first pixel group and pixel data of the second pixel group in which pixel positions of a subject obtained by a plurality of exposures are different.

[0022] With the above configuration, in an imaging device using pixels for phase difference detection that are discretely arranged, by improving the defocus amount detection performance, a program capable of realizing high-speed and accurate autofocus can be provided.

Effects of the Invention

[0023] According to the present disclosure, in an imaging device using pixels for phase difference detection that are discretely arranged, by improving the defocus amount detection performance, an imaging device or the like capable of realizing high-speed and accurate autofocus can be provided.

Brief Description of the Drawings

[0024] [Figure 1] It is a schematic diagram of a pixel group for detecting a phase difference according to an embodiment. [Figure 2] It is a diagram showing an example of an image captured by a pixel group for detecting a phase difference according to an embodiment. [Figure 3] It is a block diagram showing the configuration of an imaging device according to an embodiment. [Figure 4] It is a flowchart of an image processing method according to an embodiment. [Figure 5A] It is a flowchart for determining the degree of coincidence of the shape of an image of pixel data according to an embodiment. [Figure 5B] This figure shows an example of determining the degree of matching of the image shape of pixel data according to the embodiment. [Figure 6A] This is a flowchart for determining the reliability of the phase difference calculation process according to the embodiment. [Figure 6B] This figure shows an example of reliability determination for the phase difference calculation process according to the embodiment. [Figure 7] This figure shows an example of an image processing method for sharing data according to the embodiment. [Figure 8] This is a first schematic diagram showing an example of the arrangement of a group of pixels for detecting a phase difference according to the embodiment. [Figure 9] This is a second schematic diagram showing an example of the arrangement of a pixel group for detecting phase difference according to the embodiment. [Figure 10] This diagram illustrates the phase difference due to the focus according to the embodiment. [Modes for carrying out the invention]

[0025] Embodiment Embodiments of the present invention will be described below with reference to the drawings. However, the invention claimed is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential for solving the problem. For clarity of explanation, the following descriptions and drawings have been omitted and simplified as appropriate. In each drawing, the same elements are denoted by the same reference numerals, and redundant explanations have been omitted where necessary.

[0026] (Description of the pixel group for detecting phase difference according to the embodiment) Figure 1 is a schematic diagram of the pixel group for detecting phase difference according to the embodiment. Figure 2 is a diagram showing an example image captured by the pixel group for detecting phase difference according to the embodiment. Figure 10 is a diagram illustrating the phase difference due to focus according to the embodiment. The pixel group for detecting phase difference according to the embodiment will be described with reference to Figures 1, 2 and 10.

[0027] As shown in Figure 1, the pixel group 100 for detecting phase difference comprises a first pixel group 101 whose right half is shielded from light and a second pixel group 102 whose left half is shielded from light. The term "pixel group" is used because there are multiple sets of shielded pixels. As shown in the lower part of Figure 1, when light passes through the lens, the right-side light enters the pixel sensor in the first pixel group 101, where the right half is shielded from light. Similarly, the left-side light enters the second pixel group 102, where the left half is shielded from light. The first pixel group 101, with its right half shielded from light, is designated as the first pixel group having a first characteristic. The second pixel group, with its left half shielded from light, is designated as the second pixel group having a second characteristic different from the first characteristic.

[0028] As shown in Figure 1, when the first pixel group 101 and the second pixel group 102 are arranged discretely, the image captured by the first pixel group 101 will look like Figure 2. In other words, an image with gaps in places is acquired. The arrows in Figure 2 indicate the state as the number of exposures increases. As shown by the arrows in Figure 2, by exposing the image multiple times and increasing the number of images, an image of the entire subject is acquired.

[0029] As shown in Figure 10, when the lens is in focus, there is no phase shift. However, when the lens is in a front-focus state, close to the object, the focal point is formed in front of the image sensor. As a result, a phase difference occurs between the first pixel group 101 and the second pixel group 102, causing the signals to be detected out of sync.

[0030] In a back-focus state where the lens is far from the subject, the focal point is connected behind the image sensor. As a result, the signals detected in the first pixel group 101 and the second pixel group 102 are misaligned.

[0031] To detect such a phase difference, it is necessary to acquire images of the same position in the first pixel group 101 and the second pixel group 102. Therefore, if the first pixel group 101 and the second pixel group 102 are discretely arranged, the accuracy of phase difference detection can be improved by taking multiple exposures and acquiring the same image while varying the pixel positions of the subject.

[0032] Multiple images are acquired by finely adjusting the lens position using an image stabilization mechanism to compensate for camera shake. By superimposing the discrete phase difference information obtained from these images, the amount of defocus can be detected by interpolating data with insufficient resolution and removing noise. In this way, phase difference detection is used to calculate the amount of defocus for autofocus.

[0033] (Description of the imaging device according to the embodiment) Figure 3 is a block diagram showing the configuration of the imaging device according to the embodiment. The imaging device according to the embodiment will be described with reference to Figure 3. The imaging device is equipped with an autofocus function.

[0034] As shown in Figure 3, the imaging device 300 according to this embodiment includes a lens 301, an image sensor 302, an image sensor control unit 303, a focus detection drive influence removal unit 304, an image signal processing unit 305, an image storage unit 306, an image display unit 307, an autofocus main control unit 310, a focus adjustment lens control unit 311, a shake correction main control unit 313, a shake detection unit 314, and a shake correction lens control unit 312.

[0035] Lens 301 is an autofocus lens located between the subject and the image sensor 302. It automatically moves back and forth to adjust the focal position and direct the image of the subject onto the image sensor. Lens 301 may be composed of multiple lenses combined to form an image, with one of the lenses moving back and forth to autofocus.

[0036] The image sensor 302 may be, for example, a camera using a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image sensor 302 converts the light incident from the lens 301 into an electrical signal.

[0037] The image sensor control unit 303 controls, for example, the timing of image acquisition by the image sensor 302. The image sensor control unit 303 also works in cooperation with the discrete autofocus pixels and the autofocus main control unit 310 to acquire pixels captured by multiple exposures.

[0038] The autofocus main control unit 310 comprises a phase difference calculation unit 308 and a defocus amount calculation unit 309. The autofocus main control unit 310 can also be called the calculation processing unit. First, the autofocus main control unit 310 acquires pixel data from the first pixel group 101 and the second pixel group 102. The autofocus main control unit 310 then determines whether sufficient pixel data from the first pixel group 101 and the second pixel group 102 has been acquired. If sufficient pixel data has not been acquired, the image is repeatedly captured by changing the pixel position of the subject to acquire pixel data. If sufficient pixel data has been acquired, the phase difference calculation unit 308 detects the phase difference from the images obtained from multiple exposures. There are several methods for determining whether sufficient pixel data has been acquired. The defocus amount calculation unit 309 calculates the defocus amount based on the phase difference calculated by the phase difference calculation unit 308. The defocus amount is used for autofocus in cooperation with the focus adjustment lens control unit.

[0039] The focus adjustment lens control unit 311 performs the lens control necessary for autofocus based on the amount of defocus. The focus adjustment lens control unit 311 transmits the current lens position to the autofocus main control unit 310. The focus adjustment lens control unit 311 also obtains the lens drive target position from the autofocus main control unit 310.

[0040] The image stabilization main control unit 313 is the part that controls image stabilization and other functions. The image stabilization main control unit 313 can also be called the image stabilization unit. The image stabilization main control unit 313 obtains the lens drive amount from the autofocus main control unit 310. The image stabilization main control unit 313 also obtains image stabilization data from the image stabilization detection unit 314. When the image stabilization main control unit 313 wants to move the image +2 to the right, and obtains data indicating that the image stabilization is +1 to the right, it controls the lens 301 so that the image moves +1 to the right. In this way, the image stabilization main control unit 313 corrects the positional shift of the subject pixels due to image stabilization between multiple exposures and generates a predetermined positional shift of pixels.

[0041] The shake detection unit 314 detects camera shake. The shake detection unit 314 is an acceleration sensor or the like. Based on the detection result of the shake detection unit 314, the shake correction main control unit 313 controls the pixel position of the subject to be different in multiple exposures.

[0042] The image stabilization lens control unit 312 controls the position of the lens 301 based on camera shake. The image stabilization lens control unit 312 transmits the current lens position to the image stabilization main control unit 313. The image stabilization lens control unit 312 obtains the lens drive target position from the image stabilization main control unit 313. The image stabilization lens control unit 312 controls the position of the lens 301 based on the obtained lens drive target position.

[0043] The focus detection drive effect removal unit 304 eliminates the effect of moving the lens to acquire an image with the first pixel group 101 and the second pixel group 102 by changing the pixel positions of the subject. In other words, although the acquired image is moving, it appears to the photographer as if the image is stationary. The focus detection drive effect removal unit 304 corrects the image shake.

[0044] The image signal processing unit 305 processes the image signal in various ways. The image signal processing unit 305 automatically or manually adjusts the brightness of the image, such as the color and exposure conditions, to make it look good.

[0045] The image storage unit 306 stores images of the subject. The image storage unit 306 consists of memory and other components.

[0046] The image display unit 307 displays the captured subject. The image display unit 307 is a display device such as an EL (Electro Luminescence) display device or a liquid crystal display device.

[0047] The above configuration provides an imaging device that achieves high-speed and accurate autofocus by improving the defocus amount detection performance.

[0048] (Description of the image processing method according to the embodiment) Figure 4 is a flowchart of the image processing method according to the embodiment. The image processing method according to the embodiment will be explained with reference to Figure 4.

[0049] First, exposure takes place (step S401). An image of the subject is captured through the lens 301. Discrete autofocus pixel data is then output (step S402). The subject is captured by the first pixel group 101 and the second pixel group 102, and pixel data is output. Next, the pixel data is combined (step S403). The pixel data captured by the first pixel group 101 and the second pixel group 102 are combined.

[0050] Next, it is determined whether there is enough pixel data to perform the phase difference calculation and whether the position shift shooting is complete (step S405). If there is not enough pixel data (if NO in step S405), the autofocus main control unit 310 performs a position shift drive (step S406) and exposure is performed again (step S401). If sufficient pixel data is obtained to perform the phase difference calculation (if YES in step S405), the phase difference calculation unit 308 performs the phase difference calculation (step S407). If all pixel data is obtained, it may be considered that there is enough data to perform the phase difference calculation. Alternatively, it may be considered that there is enough data to perform the phase difference calculation after a predetermined number of pixel data composites. For example, if all data can be obtained in 64 composites, it may be considered that there is enough data to perform the phase difference calculation after 62 or 63 composites. Furthermore, there may be cases where the phase difference calculation cannot be performed even if all data is obtained. For example, when imaging a low-contrast subject. In such cases, the lens movement is stopped and the autofocus processing is stopped, but exposure continues and pixel data is taken. The autofocus process is stopped until sufficient pixel data is obtained for the phase difference calculation. The determination of whether there is enough data for other phase difference calculations will be described later.

[0051] Based on the phase difference calculation process, the defocus amount calculation unit 309 performs the defocus amount calculation process (step S408). The defocus amount calculation unit 309 outputs the defocus amount (step S409). Then, the process ends. The defocus amount is used in the autofocus process.

[0052] The above configuration provides an image processing method that achieves high-speed and accurate autofocus by improving the defocus amount detection performance.

[0053] The above image processing method can be executed by an information processing device. The information processing device comprises a processor that executes and processes a program, and a memory that stores the program. The information processing device is incorporated into the imaging device 300. The information processing device is preferably composed of one device, but may be composed of multiple devices.

[0054] Some or all of the processing in the information processing device and imaging device 300 can be implemented as a computer program. Such a program can be stored using various types of non-temporary computer-readable media and supplied to a computer. Non-temporary computer-readable media include various types of tangible recording media. Examples of non-temporary computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Alternatively, the program may be supplied to the computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can be supplied to the computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0055] (Explanation of obtaining sufficient pixel data using the image shape matching determination method for pixel data according to the embodiment) Figure 5A is a flowchart for determining the degree of matching of the image shape of the pixel data according to the embodiment. Figure 5B is a diagram showing an example of determining the degree of matching of the image shape of the pixel data according to the embodiment. The acquisition of sufficient pixel data using the image shape matching determination of the pixel data according to the embodiment will be explained with reference to Figures 5A and 5B.

[0056] As shown in the upper part of Figure 5B, the first pixel group 101 and the second pixel group 102 are shifted in phase within a predetermined phase range, and the sum of the absolute values ​​of the differences between the first pixel group and the second pixel group at each phase position is calculated. As shown in the lower part of Figure 5B, if the minimum value of the calculated data is less than or equal to a predetermined judgment value, it is determined that there is agreement, that is, that sufficient data has been obtained to calculate the phase difference, and the process proceeds to the phase difference calculation.

[0057] To explain the process using a flowchart, as shown in Figure 5A, once the pixel data is combined, the degree of agreement calculation unit of the autofocus main control unit 310 calculates the degree of agreement between the image shapes of the pixel data of the first pixel group and the pixel data of the second pixel group. The autofocus main control unit 310 then performs a degree of agreement determination (step S501). It is determined whether the minimum value of the degree of agreement is less than or equal to a predetermined determination value (step S502). If the minimum value of the degree of agreement is not less than or equal to the predetermined determination value (if NO in step S502), the autofocus main control unit 310 performs a position shift drive (step S406). If the minimum value of the degree of agreement is less than or equal to the predetermined determination value (if YES in step S502), the phase difference calculation unit 308 executes a phase difference calculation process (step S407).

[0058] With the above configuration, pixel positions can be varied based on the degree of matching. Furthermore, the timing of the output processing related to defocusing can be determined.

[0059] (Explanation of obtaining sufficient pixel data using reliability determination of phase difference calculation processing according to the embodiment) Figure 6A is a flowchart of the reliability determination of the phase difference calculation process according to the embodiment. Figure 6B is a diagram showing an example of the reliability determination of the phase difference calculation process according to the embodiment. The acquisition of sufficient pixel data using the reliability determination of the phase difference calculation process according to the embodiment will be explained with reference to Figures 6A and 6B.

[0060] As shown in Figure 6B, the first and second pixel groups within the detection area are phase-shifted within a predetermined phase range, and the sum of the absolute values ​​of the differences between the first and second pixel groups at each phase is calculated. The reliability value is calculated by dividing the difference value (YM) at the phase position that yields the minimum value with an accuracy of less than the pixel pitch, obtained from the phase position of the minimum value of the calculated data and the data before and after that phase position, by the contrast value of the pixel group (YM / C). If the reliability value is less than or equal to a predetermined judgment value, it is determined that the reliability is high, and the process proceeds to the defocus calculation. The contrast value of the pixel group is the sum of the absolute values ​​of the differences between the pixel group and adjacent pixels.

[0061] To explain the process using a flowchart, as shown in Figure 6A, when the pixel data is combined, the phase difference calculation unit 308 performs a phase difference calculation (step S407). After the phase difference calculation, the reliability calculation unit of the autofocus main control unit 310 calculates the reliability of the output result regarding the defocus amount each time there is exposure. The autofocus main control unit 310 then determines the reliability (step S602). It is determined whether the reliability is below a predetermined determination value (step S602). If the reliability is not below a predetermined determination value (if step S602 is NO), the autofocus main control unit 310 performs a position shift drive (step S406). If the reliability is below a predetermined determination value (if step S602 is YES), the defocus amount calculation unit 309 performs a defocus amount calculation (step S408).

[0062] With the above configuration, the validity of the output results regarding the amount of defocus can be determined.

[0063] (Description of an image processing method for sharing data according to the embodiment) Figure 7 shows an example of an image processing method for sharing data according to the embodiment. The image processing method for sharing data according to the embodiment will be explained with reference to Figure 7.

[0064] As shown in Figure 7, pixel data synthesis 709 is obtained using shift positions 701 and 704. Shift position 701 is the data for shift position 1, shift position 702 is the data for shift position 2, shift position 703 is the data for shift position 3, and shift position 704 is the data for shift position 4.

[0065] Next, pixel data synthesis 710 is obtained using shift position 702 and shift position 705. The data at shift position 705 is the same as the data at shift position 1. Therefore, 705 of shift position 1 is input instead of 701 of shift position 1.

[0066] Next, pixel data synthesis 711 is obtained using shift positions 703 and 706. The data at shift position 706 is the data at shift position 2. In this way, pixel data synthesis is obtained by sharing the repeated data from shift position 1 to shift position 4. That is, when outputting the defocus amount multiple times, the pixel data of the first pixel group and the pixel data of the second pixel group used for each output are shared multiple times. By doing this, the data can be shared and the defocus amount can be output quickly.

[0067] (Description of an example of the arrangement of pixel groups for detecting phase difference according to the embodiment) Figure 8 is a first schematic diagram showing an example of the arrangement of the pixel group for detecting phase difference according to the embodiment. Figure 9 is a second schematic diagram showing an example of the arrangement of the pixel group for detecting phase difference according to the embodiment. The pixel group for detecting phase difference according to the embodiment will be described with reference to Figures 8 and 9.

[0068] As shown in Figure 8, various shading patterns are possible. The spacing between the first pixel group and the second pixel group, and the spacing between the first pixel groups themselves, may be any of these. Furthermore, any density is acceptable as long as the density of the first pixel group and the density of the second pixel group are the same. In addition, shading of the first pixel group and the second pixel group may be done not only from the left and right but also from the top and bottom.

[0069] As shown in Figure 9, two pixels may be designated as the first pixel and the second pixel. Similarly, any density is acceptable as long as the density of the first and second pixel groups is the same. The arrangement of the two pixels may be horizontal, vertical, or a combination thereof. The color may also be any of green, red, blue, or white, and may span both pixels. The color filter array may be a Bayer array or a quad-Bayer array.

[0070] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. [Explanation of Symbols]

[0071] 100 Pixel group for detecting phase difference, 101 First pixel group, 102 Second pixel group, 300 Imaging device, 301 Lens, 302 Image sensor, 303 Image sensor control unit, 304 Focus detection drive influence removal unit, 305 Image signal processing unit, 306 Image storage unit, 307 Image display unit, 308 Phase difference calculation unit, 309 Defocus amount calculation unit, 310 Autofocus main control unit, 311 Focus adjustment lens control unit, 312 Shake correction lens control unit, 313 Shake correction main control unit, 314 Shake detection unit, 701 Shift position, 702 Shift position, 703 Shift position, 704 Shift position, 705 Shift position, 706 Shift position, 709 Pixel data synthesis, 710 Pixel data synthesis, 711 Pixel data synthesis

Claims

1. An imaging device comprising: an image sensor in which a first pixel group having a first characteristic for detecting phase difference for autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged; an image stabilization unit for correcting camera shake; and a calculation processing unit that outputs a defocus amount based on pixel data of the first pixel group and pixel data of the second pixel group, where the pixel positions of the subject obtained in multiple exposures are different.

2. It has a shake detection unit that detects camera shake, The imaging apparatus according to claim 1, which controls the pixel position of the subject to differ in multiple exposures based on the detection result of the shake detection unit.

3. The imaging apparatus according to claim 2, which corrects the positional shift of the subject pixels due to camera shake between multiple exposures and generates a predetermined pixel positional shift.

4. Each time exposure occurs, the system includes a matching degree calculation unit that calculates the degree of matching between the image shapes of the pixel data of the first pixel group and the pixel data of the second pixel group. The imaging apparatus according to claim 1, which controls the pixel positions of the subject to differ based on the degree of agreement.

5. The imaging apparatus according to claim 4, wherein the calculation processing unit performs output processing related to defocus when a predetermined degree of agreement is obtained.

6. It has a reliability calculation unit that calculates the reliability of the output result regarding the amount of defocus each time it is exposed, The imaging apparatus according to claim 1, which determines the effectiveness of the output result regarding the amount of defocus based on the reliability.

7. The imaging apparatus according to claim 1, wherein when outputting the defocus amount multiple times, the pixel data of the first pixel group and the pixel data of the second pixel group used for each output are shared multiple times.

8. An image processing method that uses an image sensor in which a first pixel group having a first characteristic for detecting phase difference for autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged, and an image stabilization unit for correcting camera shake, to output an amount of defocus based on pixel data of the first pixel group and pixel data of the second pixel group, where the pixel positions of the subject obtained from multiple exposures are different.

9. A program that causes an information processing device to perform an output regarding the amount of defocus based on pixel data of the first pixel group and pixel data of the second pixel group, where the pixel positions of the subject obtained from multiple exposures are different, using an image sensor in which a first pixel group having a first characteristic for detecting phase difference for autofocus and a second pixel group having a second characteristic different from the first characteristic are discretely arranged, and an image stabilization unit for correcting camera shake.

Images