Control device, lens device, imaging device, control method, and program

Abstract

To provide, for example, a control unit advantageous in terms of precise focusing.SOLUTION: A control unit (112) has: a signal processing unit (502) that performs signal processing on a pair of image signals from a pair of image sensors (501) by using correction values different among pixels of the pair of image signals; and a correlation operation unit (503) that performs correlation operation on the pair of image signals output from the signal processing unit. The signal processing unit performs, by using the correction values, weighting addition on the pixels adjacent in a direction in which the correlation operation is performed.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a control device, a lens device, an imaging device, a control method, and a program.

Background Art

[0002] Conventionally, an imaging device is known that forms an image on a pair of sensors with a light beam split at the pupil plane through a pair of lenses, performs a correlation operation based on the signals of the two obtained images, obtains a phase difference amount corresponding to the image displacement amount of the two images, and performs phase difference type autofocus (AF). In this method, if there are manufacturing errors of the lens or mounting errors of the sensor, a difference occurs in the phase difference amount in each area on the sensor due to the influence of distortion in the subject image on the sensor.

[0003] Patent Document 1 discloses a method of performing distortion correction using one correction value representing a correlation operation area.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When the distortion in the subject image is large, the variation in the phase difference amount within the correlation operation area is also large. Therefore, sufficient correction cannot be performed only by using one correction value representing the correlation operation area as disclosed in Patent Document 1. As a result, it is difficult to perform accurate focusing.

[0006] An object of the present invention is to provide, for example, a control device that is advantageous in terms of accurate focusing.

Means for Solving the Problems

[0007] As one aspect of the present invention, the control device is based on an image sensor containing multiple pixels. of The system has a processing unit that processes the signal using different correction values ​​for each pixel, and a calculation unit that performs correlation calculations on the signal output from the processing unit, wherein the calculation unit obtains a correction value for a predetermined pixel based on the phase difference amount at the time of focus obtained by correlation calculation in a correlation calculation area including a predetermined pixel among the plurality of pixels, obtains a correction value for pixels other than the predetermined pixel based on the phase difference amount for the predetermined pixel, and the processing unit uses the correction values ​​to perform the The target pixel in the correlation calculation domain and the target pixel and Weighted summation is performed on signals corresponding to adjacent pixels. This causes the center of gravity of the signal of the target pixel to shift in order to correct the amount of image misalignment. .

[0008] Other objects and features of the present invention are described in the following examples. [Effects of the Invention]

[0009] According to the present invention, for example, a control device that is advantageous in terms of accurate focusing can be provided. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram of the imaging system in Example 1. [Figure 2] This is a block diagram of the focus detection unit in Example 1. [Figure 3] This is an explanatory diagram of the pixel sequences of images A and B in Example 1. [Figure 4] This is an explanatory diagram of the image displacement at each position on the AF sensor in Example 1. [Figure 5] This is an explanatory diagram of the image misalignment within the correlation calculation area on the AF sensor in Example 1. [Figure 6] This is a flowchart showing the process for obtaining the correction value in Example 1. [Figure 7] This is an explanatory diagram of the adjustment points for acquiring the phase difference amount on the AF sensor in Example 1. [Figure 8] This is a flowchart showing the AF process in Example 1. [Figure 9] It is an explanatory diagram of the phase difference correction method in Example 1. [Figure 10] It is a diagram showing the relationship between the pixel centroid movement amount and the correction error amount in Example 2. [Figure 11] It is a flowchart showing the AF processing in Example 2. [Figure 12] It is an explanatory diagram of the pixel centroid movement amount after offset in Example 2. [Figure 13] It is a flowchart showing the acquisition process of the second correction value in Example 3. [Figure 14] It is an explanatory diagram of the two-dimensional correlation operation in Example 4.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. 〔Example 1〕 First, referring to FIG. 1, the imaging system in Example 1 of the present invention will be described. FIG. 1 is a block diagram of the imaging system 10. The imaging system 10 includes a camera body (imaging device) 200 and a lens device 100 that is detachable from the camera body 200. The lens device 100 and the camera body 200 are mechanically and electrically connected via a mount 300 that is a coupling mechanism. Note that this embodiment is also applicable to an imaging device in which the camera body and the lens device are integrally configured.

[0012] The lens device 100 has an imaging optical system. The imaging optical system includes a focus lens 101 that performs first focus adjustment, a zoom lens 102 that performs zooming, an aperture unit (aperture diaphragm unit) 103 that adjusts the light amount, and a beam splitter prism (splitting means) 104 that splits light.

[0013] The focus lens 101 moves in the direction along the optical axis OA (optical axis direction) by the focus lens driving unit 106. The focus lens detector 107 detects the position of the focus lens 101. The zoom lens 102 moves in the optical axis direction by the zoom lens driving unit 108. The zoom lens detector 109 detects the position of the zoom lens 102. The aperture unit 103 is configured to include aperture blades. The aperture driving unit 110 drives the aperture unit 103 to perform a light amount adjustment operation. The aperture detector 111 detects the F value (aperture value) of the aperture unit 103.

[0014] The focus lens driving unit 106, the zoom lens driving unit 108, and the aperture driving unit 110 are each configured to include, for example, an ultrasonic motor (vibration wave motor). However, this embodiment is not limited thereto, and other motors such as a voice coil motor, a DC motor, or a stepping motor may be applied. The focus lens detector 107, the zoom lens detector 109, and the aperture detector 111 are each configured to include, for example, a potentiometer or an encoder.

[0015] The beam splitting prism 104 separates (divides) the light that has passed through the aperture unit 103 into transmitted light and reflected light. The light (transmitted light) that has passed through the beam splitting prism 104 is incident on the imaging element 201 of the camera body 200. Also, the light (reflected light) reflected by the beam splitting prism 104 is incident on the focus detection unit 112. The focus detection unit 112 calculates a phase difference amount by performing a correlation operation on a pair of image signals and converts it into a defocus amount. The lens control unit 120 drives the focus lens 101 and controls the zoom lens 102 and the aperture unit 103 based on the defocus amount obtained by the focus detection unit 112.

[0016] The image sensor 201 is equipped with a CMOS sensor or CCD sensor, and converts the optical image (subject image) formed by the imaging optical system of the lens device 100 into an optical signal. The signal processing circuit 202 generates a video signal by performing signal processing on the electrical signal output from the image sensor 201 and outputs the video signal to the video display device 400. As a result, the video display device 400 can display the video.

[0017] Next, the focus detection unit (control device) 112 will be described with reference to Figure 2. Figure 2 is a block diagram of the focus detection unit 112. Light reflected by the branching prism 104 is split into two light beams by a pair of phase difference detection lenses (not shown). The AF sensor 501 is a pair of image sensors that perform photoelectric conversion on a pair of images (image A and image B) formed by the two split light beams to generate image signals for the two images. The sensor signal processing unit (signal processing unit) 502 performs signal processing on the image signals for the two images. The correlation calculation processing unit (correlation calculation unit) 503 performs correlation calculations using the image signals for the two images from the sensor signal processing unit 502. The correction value storage unit 504 stores correction values ​​for correcting the amount of misalignment between the two images, which will be described later. The sensor signal processing unit 502 corrects the image signals for the two images using the correction values ​​read from the correction value storage unit 504.

[0018] Next, the calculation performed by the correlation calculation unit 503 will be explained with reference to Figure 3. Figure 3 is an explanatory diagram of the pixel sequences of image A and image B. As shown in Figure 3, the correlation calculation unit 503 performs correlation calculations using the pixel sequences of the paired image A and image B. In performing correlation calculations, the correlation calculation unit 503 calculates the correlation amount by adding the absolute values ​​of the differences between the pixel signals of image A and image B across the correlation calculation area. The correlation amount is calculated by fixing one of image A or image B and shifting the other by one pixel at a time, and performing calculations for every k shifts. When the number of shifts is k, the correlation amount COR(k) is expressed as shown in equation (1) below.

[0019]

number

[0020] In equation (1), A i and B i These are the i-th pixel value of image A and the i-th pixel value of image B, respectively. When the number of shifts k is varied, the correlation amount COR(k) is maximized, which means that the signals of image A and image B are most in sync (focused). The resolution of the number of shifts k that can be calculated by the above calculation is one pixel. Therefore, in order to calculate the number of shifts k with a resolution of less than one pixel, the correlation amount difference ΔCOR(k) between the two images when shifted by k pixels is calculated using equation (2), using the correlation amount COR(k) when shifted by k pixels and the correlation amount COR(k+1) when shifted by (k+1) pixels.

[0021]

number

[0022] The shift number k at which the correlation amount COR(k) is maximized indicates the point of focus and can be calculated at the zero-crossing point where the correlation amount difference ΔCOR(k) changes from negative to positive. This shift number k is called the phase difference amount. The correlation calculation processing unit 503 converts the obtained phase difference amount into a defocus amount and outputs it to the lens control unit 120. The lens control unit 120 calculates the focus lens drive amount based on the defocus amount and drives the focus lens 101.

[0023] Next, the shift in the two images formed at each position on the AF sensor 501 will be explained with reference to Figures 4(a), (b) and 5(a), (b). Figures 4(a) and (b) are explanatory diagrams of the image shift at each position on the AF sensor 501. Figure 4(a) shows the subject on the image of the image display device 400. Figure 4(b) shows the signal levels of the two images when the subject in Figure 4(a) is imaged on the AF sensor 501, and shows the signal levels corresponding to the dotted lines in Figure 4(a). In Figure 4(b), the solid line shows image A and the dotted line shows image B. In this embodiment, the left-right direction in Figure 4(b) is the correlation direction. In Figure 4(b), the amount of shift in the two images formed at the nine locations on the AF sensor 501 is different at each position, and the amount of phase difference obtained as a result of correlation calculation at each position is also different. This discrepancy is due to errors in the optical distance from the branched optical system (branched prism 104) to the AF sensor 501 and the optical distance from the branched optical system to the image sensor, as well as the precision of the installation position of the AF sensor 501.

[0024] Conventionally, the amount of misalignment between the two images within the correlation calculation area shown in Figure 4(b) was treated as a negligible amount. Therefore, the correction value (phase difference amount in the focused state) for a certain correlation calculation area could be used as, for example, the correction value for the position of the pixel in the center of that correlation calculation area, and one correction value for that correlation calculation area was sufficient. In this case, as a correction process to correct the amount of misalignment between the two images at each position of the AF sensor 501, a process of subtracting the correction value from the phase difference amount of the correlation calculation result can be applied.

[0025] Figures 5(a) and (b) show an example where the amount of misalignment between the two images is large due to the optical and mechanical design, and this misalignment cannot be ignored in the correlation calculation area. Figure 5(a) shows a subject on the image of the image display device 400. Figure 5(b) shows the signal levels of the two images when the subject in Figure 5(a) is imaged by the AF sensor 501, and shows the signal levels corresponding to the dotted lines in Figure 5(a). It can be seen that the amount of misalignment between the two images differs at the left edge, center, and right edge of the correlation calculation area. In such cases, the correction value of the pixel in the center of the correlation calculation area cannot be used as the correction value for the entire correlation calculation area, as in the conventional method. If the correction value of the pixel in the center is used as the correction value for the entire correlation calculation area, for example, good focusing accuracy can be obtained when the subject is in the center of the correlation calculation area, but focusing accuracy decreases when the subject is at the left edge or right edge of the correlation calculation area.

[0026] Thus, when the amount of displacement between the two images differs at each position in the correlation calculation area, it is necessary to correct the amount of displacement at each position in the correlation calculation area. Next, we will explain the procedure for obtaining correction values ​​and the correction process for correcting the amount of displacement at each position in the correlation calculation area for each pixel.

[0027] First, the procedure for acquiring the correction value will be explained with reference to Figure 6. Each step in Figure 6 is mainly performed by the focus detection unit 112 or the lens control unit 120.

[0028] First, in step S101, the lens control unit 120 adjusts the position (zoom position) of the aperture unit 103 and the zoom lens 102. Specifically, the lens control unit 120 sets the aperture unit 103 to the wide-open position and sets the zoom position to the wide-angle end. Note that the zoom position may be other positions.

[0029] Next, in step S102, the user manually drives the focus lens 101 to adjust the focus while checking the image on the image display device 400, so that it is in focus on the subject. Alternatively, the imaging system 10 may calculate the contrast based on the image information and adjust the focus to the position of the contrast peak.

[0030] Next, in step S103, the lens control unit 120 sets up a correlation calculation area to calculate the phase difference amount for each adjustment point. Figures 7(a) and 7(b) are explanatory diagrams of the adjustment points on the AF sensor 501 from which the phase difference amount is obtained. Figure 7(a) shows a subject (vertical bar) displayed on the image of the image display device 400. Note that the subject is not limited to vertical bars and may be other subjects. Figure 7(b) shows nine adjustment points p1 to p9 on the AF sensor 501 as black pixels. The subject shown in Figure 7(a) is arranged to correspond to the adjustment points p1 to p9 in Figure 7(b). In Figure 7(b), the correlation calculation area for calculating the phase difference amount of adjustment point p1 is shown by diagonal lines (including the black adjustment point in the center), and the correlation calculation area is set so that adjustment point p1 is located in the center of the correlation calculation area. The width of the correlation calculation area is determined according to the size of the subject.

[0031] Next, in step S104 of Figure 6, the sensor signal processing unit 502 outputs the pixel signal of the correlation calculation area corresponding to the adjustment point p1 set in step S103 to the correlation calculation processing unit 503. The correlation calculation processing unit 503 calculates the phase difference amount by performing a correlation calculation in the correlation calculation area and obtains a correction value.

[0032] Next, in step S105, the lens control unit 120 determines whether it has completed acquiring the correction values ​​for all adjustment points p1 to p9. If there are any points that have not yet been acquired, it returns to step S103 and resets the correlation calculation area corresponding to the adjustment points. Then, in step S104, the correlation calculation processing unit 503 acquires the correction values ​​in the same way. On the other hand, if it has completed acquiring the correction values ​​for all adjustment points p1 to p9, it proceeds to step S106.

[0033] In step S106, the correlation calculation processing unit 503 performs interpolation calculations for correction values ​​of pixels other than the adjustment points p1 to p9. In the interpolation calculation, the phase difference amount of each pixel in the correlation direction and in the direction orthogonal to the correlation direction is calculated using the phase difference amount of the adjustment points, for example by linear interpolation. Note that the interpolation method is not limited to linear interpolation, and other methods such as polynomial approximation may be used. In this embodiment, there are nine adjustment points, but the number of adjustment points may be increased or decreased depending on the magnitude of the displacement of the two images formed at each position of the AF sensor 501. After calculating the correction values ​​of all pixels by interpolation calculation, the process proceeds to step S107. In step S107, the correction value storage unit 504 stores the calculated correction values ​​and terminates this flow.

[0034] Next, with reference to Figure 8, the AF processing using the acquired correction values ​​will be explained. Figure 8 is a flowchart of the AF processing. Each step in Figure 8 is mainly performed by the focus detection unit 112 or the lens control unit 120.

[0035] First, in step S201, the lens control unit 120 sets the correlation calculation area by having the user operate an operation unit (not shown). The correlation calculation area may be one-dimensional or two-dimensional. Next, in step S202, the sensor signal processing unit 502 reads the correction value corresponding to the correlation calculation area set in step S201 from the correction value storage unit 504.

[0036] Next, in step S203, the sensor signal processing unit 502 calculates the pixel centroid shift of each pixel in the correlation calculation area. Here, the pixel centroid shift will be explained with reference to Figure 9. Figure 9 is an explanatory diagram of the phase difference correction method, corresponding to the pixels in the correlation calculation area shown by the shaded area in Figure 7(b), and showing pixels (A image pixels) A1 to A9 and pixels (B image pixels) B1 to B9, as well as correction values ​​c1 to c9 for each pixel position. Here, the correction of pixels A5 and B5 will be described. The correction value for both pixels A5 and B5 is c5. For other pixels as well, the pixel number and the correction value number correspond. The correction value c5 is, for example, 0.4, and this value means that the two images are shifted by 0.4 pixels. In this case, the corrected pixels (signal levels) of pixels A5 and B5 are calculated by shifting the centroid of 0.2 pixels, which is half of the correction value c5. The centroids of pixels A5 and B5 move in opposite directions to eliminate the displacement between the two images.

[0037] The corrected pixel A5' of pixel A5 and the corrected pixel B5' of pixel B5 are calculated using the following equations (3) and (4).

[0038]

number

[0039]

number

[0040] For pixel A5, the calculation is performed by using the adjacent pixel A6 to its right and weighting it by a correction value c5. By weighting and adding the adjacent pixel to its right, the center of gravity shifts to the left. On the other hand, for pixel B5, the calculation is performed by using the adjacent pixel B4 to its left and weighting it by a correction value c5. By weighting and adding the adjacent pixel to its left, the center of gravity shifts to the right. In this way, the sensor signal processing unit 502 performs weighting and adding of the first output signal from the first pixel of the AF sensor 501 (for example, pixel A5 or pixel B5) and the second output signal from the second pixel adjacent to the first pixel (for example, pixel A6 or pixel B4). Note that the amount of pixel centroid shift may also take a negative value. In that case, the direction of the pixel centroid shift of images A and B should be reversed compared to the direction in the case of a positive value as described above.

[0041] Here, images A and B were moved by half the correction value, but it is also possible to move the centroid of only one of the pixels in either image A or image B. In that case, the amount of centroid movement will be c, not c / 2. Furthermore, weighted addition may be performed using multiple adjacent pixels instead of just one adjacent pixel. In addition, to secure adjacent pixels to be used for pixel centroid movement, one pixel may be added to the left and right ends of the correlation calculation area, increasing the number of pixels in the correlation calculation area by two pixels.

[0042] As described above, the sensor signal processing unit 502 performs the same pixel centroid shift on the other pixels in the correlation calculation area as it did on pixels A5 and B5. After the sensor signal processing unit 502 has performed the pixel centroid shift, the process proceeds to step S204. In step S204, the correlation calculation processing unit 503 performs a correlation calculation and calculates the phase difference amount. Next, in step S205, the correlation calculation processing unit 503 calculates the defocus amount based on the phase difference amount and transmits the calculated defocus amount to the lens control unit 120. Next, in step S206, the lens control unit 120 calculates the focus lens drive amount based on the defocus amount and drives the focus lens 101 (performs AF drive).

[0043] In this embodiment, the amount of image displacement at each pixel position due to lens manufacturing errors or sensor mounting errors is obtained as a correction value, and before correlation calculation, the pixel centroid is shifted based on the correction value of each pixel to correct the amount of image displacement for each pixel. By correcting the amount of image displacement for each pixel, even if there is a difference in the amount of image displacement of each pixel within the correlation calculation area, this difference can be suppressed. As a result, even when the distortion of the subject image due to lens manufacturing errors or sensor mounting errors is large, it becomes possible to detect focus with high accuracy. [Example 2] Next, Embodiment 2 of the present invention will be described. Components similar to those described in Embodiment 1 are indicated by the same reference numerals, and their descriptions will be omitted. Embodiment 1 described a method for correcting the amount of misalignment between two images within the correlation calculation area by shifting the pixel centroid. In correction by shifting the pixel centroid, since there is no physical pixel at that position, there is a possibility of correction errors due to the shifting of the pixel centroid. The correction error tends to increase as the amount of shifting the pixel centroid increases.

[0044] Figure 10 shows the relationship between the pixel centroid shift and the correction error. In Figure 10, the horizontal axis represents the pixel centroid shift, and the vertical axis represents the correction error. As the pixel centroid shift increases from 0 pixels to 0.5 pixels, the correction error increases. Subsequently, as it increases from 0.5 pixels to 1 pixel, the correction error decreases. In this example, the correction error is maximum when the pixel centroid shift is 0.5 pixels. To reduce the correction error, it is necessary to make the pixel centroid shift as small as possible (or move it away from 0.5 pixels). In this example, a correction processing method for suppressing the pixel centroid shift will be described. In this example, the procedure for obtaining the correction value is the same as in Example 1.

[0045] Next, with reference to Figure 11, the AF processing using the acquired correction values ​​will be explained. Figure 11 is a flowchart of the AF processing. Each step in Figure 11 is mainly performed by the focus detection unit 112 or the lens control unit 120.

[0046] The processing in steps S301 and S302 is the same as steps S201 and S202 in Figure 8, respectively. Subsequently, in step S303, the sensor signal processing unit 502 performs an offset process by subtracting the same value (offset amount) from the correction value (or pixel centroid shift amount) of each pixel in the correlation calculation area. That is, the sensor signal processing unit 502 searches for the maximum and minimum values ​​of the correction value of each pixel in the correlation calculation area and uses the intermediate value as the offset amount.

[0047] Figures 12(a) to (c) are explanatory diagrams of the pixel centroid shift amount after offset, showing the relationship between the correction value c, the pixel centroid shift amount c / 2 of image A and image B, and the pixel centroid shift amount after offset. As explained in Example 1, half the amount of the correction value c is used as the pixel centroid shift amount of image A and image B. The offset amount is 0.4, which is the midpoint between the maximum value of 0.6 and the minimum value of 0.2 for the pixel centroid shift amount in the correlation calculation area. The pixel centroid shift amount after offset is generally smaller than the pixel centroid shift amount before offset. If the correction values ​​of each pixel in the correlation calculation area are monotonically decreasing or monotonically increasing from the left edge to the right edge of the area, the correction value of the pixel in the center of the correlation calculation area may be used as the offset amount. In addition, the method of calculating the offset amount is not limited to these.

[0048] The sensor signal processing unit 502 subtracts the offset amount from the correction value of each pixel, and then performs steps S304 and S305 in the same manner as the processing described in Example 1 (steps S203 and S204 in Figure 8) to calculate the phase difference amount. Subsequently, in step S306, the sensor signal processing unit 502 adds the offset amount subtracted from each pixel in step S303 to the calculated phase difference amount in order to remove the offset amount. The following steps S307 and S308 are the same as steps S205 and S206 in Figure 8, respectively.

[0049] According to this embodiment, the amount of pixel centroid movement can be reduced by offset processing, and correction errors due to pixel centroid movement can be suppressed. [Example 3] Next, Embodiment 3 of the present invention will be described. Components similar to those described in Embodiments 1 and 2 are indicated by the same reference numerals, and their descriptions will be omitted. Embodiment 2 described an offset process for suppressing correction errors due to pixel centroid shift. Although the offset process can suppress correction errors, there is a possibility that correction errors may remain. Therefore, in this embodiment, the procedure for obtaining a correction value (second correction value) to further suppress correction errors will be described.

[0050] Figure 13 is a flowchart showing the process for acquiring the second correction value. Each step in Figure 13 is mainly performed by the focus detection unit 112 or the lens control unit 120.

[0051] First, in step S401, the focus detection unit 112 (sensor signal processing unit 502, correction value storage unit 504) acquires and stores the correction value (first correction value). Step S401 is the same as the processing in steps S101 to S107 in Figure 6.

[0052] Next, in step S402, the sensor signal processing unit 502 applies correction and offset processing based on the pixel centroid movement amount based on the first correction value, and reacquires the correction values ​​for adjustment points p1 to p9 in the same way as steps S101 to S105 in Figure 6. Ideally, the reacquired correction values ​​for adjustment points p1 to p9 should be 0. However, if there is a correction error due to the pixel centroid movement, the correction value will not be 0. Therefore, in step S403, the sensor signal processing unit 502 corrects the first correction value using the reacquired correction value and calculates the second correction value. When the first correction values ​​for adjustment points p1 to p9 are c(p1) to c(p9), and the reacquired correction values ​​for adjustment points p1 to p9 are cc(p1) to cc(p9), the second correction values ​​c'(p1) to c'(p9) are expressed as shown in equation (5) below.

[0053] c'(p--)=c(p--)+cc(p--) …(5) ("--" represents a number between 1 and 9.) In other words, the second correction value c' can be calculated by adding the first correction value c and the newly acquired correction value cc.

[0054] Next, in step S404, the sensor signal processing unit 502 performs interpolation calculations for the correction values ​​of pixels other than the adjustment points, based on the second correction values ​​at adjustment points p1 to p9, in the same manner as in step S106 of Figure 6 described in Example 1. Next, in step S405, the correction value storage unit 504 replaces the stored first correction value with the second correction value and stores the second correction value.

[0055] Furthermore, as shown in Figure 10, if the amount of pixel centroid movement is known in advance, the second correction value may be determined without reacquiring the correction value at each adjustment point. For example, in Figure 10, the correction error increases approximately linearly as the amount of pixel centroid movement increases from 0 pixels to 0.5 pixels. Therefore, if the amount of pixel centroid movement across the entire range of the AF sensor 501 falls within the range of 0 pixels to 0.5 pixels, the first correction value may be multiplied by a coefficient based on the slope of a linearly approximated straight line to obtain the second correction value.

[0056] According to this embodiment, the correction error can be further reduced by obtaining a second correction value. [Example 4] Next, Embodiment 4 of the present invention will be described. In this embodiment, components similar to those described in Embodiments 1 to 3 are indicated by the same reference numerals, and their descriptions are omitted. In this embodiment, the process for performing correlation calculations in two dimensions will be described.

[0057] Figure 14 is an explanatory diagram of the two-dimensional correlation calculation. In Figure 14, the two-dimensional correlation calculation area for each pixel of the AF sensor 501 is indicated by diagonal lines. In Figure 14, the left-right direction is the correlation direction, and the directions orthogonal to the correlation direction are shown as line0 to line4. The correlation amount COR(k) is calculated for each shift amount k of the two images in each line, and the correlation amounts of line0 to line4 are designated as COR1(k) to COR4(k). The correlation amounts COR1(k) to COR4(k) of each line are added together for each shift amount, and the resulting correlation amount is designated as COR_all(k). Based on COR_all(k), the phase difference amount is calculated in the same manner as described in Example 1.

[0058] The process of calculating a single phase difference amount in a two-dimensional correlation calculation area by adding correlation amounts is well known and is effective in suppressing variations in phase difference amounts, such as when the illumination of the subject is low. When adding the correlation amounts of each line, if there is variation in the amount of image shift between the two images in each line, it is not possible to obtain the correct correlation calculation result. Therefore, as a preprocessing step before calculating the correlation amount of each line, a process is performed to correct the amount of image shift due to pixel centroid movement in each line. This process improves the accuracy of the correlation calculation result obtained by adding correlation amounts, even if there is a large distortion of the subject image due to lens manufacturing errors and there is a difference in the amount of image shift in each line and each pixel when adding correlation amounts.

[0059] As described above, in each embodiment, the control device (focus detection unit 112) has a signal processing unit (sensor signal processing unit 502) and a correlation calculation unit (correlation calculation processing unit 503). The signal processing unit performs signal processing on each of the pair of image signals from the pair of image sensors (AF sensor 501) using different correction values ​​for each pixel of the pair of image signals. The correlation calculation unit performs a correlation calculation on the pair of image signals output from the signal processing unit. The signal processing unit also performs weighted addition of adjacent pixels in the direction of the correlation calculation using the correction values.

[0060] In each embodiment, the signal processing unit uses the value obtained by subtracting the offset amount from the correction value to perform weighted addition of adjacent pixels in the direction of correlation calculation. The correlation calculation unit adds the offset amount to the phase difference amount obtained by the correlation calculation. Preferably, the offset amount is the correction value of the central pixel in the correlation calculation area among the correction values, or the midpoint between the minimum and maximum values ​​among the correction values.

[0061] Preferably, the control device has a storage unit (correction value storage unit 504) for storing correction values. More preferably, the correction values ​​are measured values ​​for some pixels in a pair of image sensors, and interpolated values ​​based on the measured values ​​for the other pixels in a pair of image sensors.

[0062] Preferably, the correction value is used to correct the amount of image misalignment in each pixel of a pair of image sensors. Preferably, the correction value is determined based on the phase difference amount obtained by correlation calculation. Preferably, each of the pair of image sensors has a two-dimensional pixel array. The signal processing unit performs signal processing using the correction value for each pixel row in the direction in which correlation calculation is performed. The correlation calculation unit obtains the correlation amount for the image signal of each pixel row output from the signal processing unit and sums the obtained correlation amounts for each pixel row.

[0063] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0064] According to each embodiment, even when the distortion of the subject image caused by manufacturing errors in the lens or mounting errors in the sensor is large, focus detection can be performed with high accuracy. Therefore, according to each embodiment, it is possible to provide a control device, lens device, imaging device, control method, and program that are advantageous in terms of accurate focusing, for example.

[0065] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its essence.

[0066] For example, in each embodiment, the focus detection unit 112 is provided on the lens device 100, but it is not limited to this, and at least a part of the function of the focus detection unit 112 (such as the AF sensor 501) may be provided on the camera body 200. Also, in each embodiment, focus detection is performed using the light beam branched by the branching prism 104, but it is not limited to this, and focus detection using a phase difference detection method (image plane phase difference AF) may be performed using the image signal output from the image sensor 201. [Explanation of symbols]

[0067] 112 Focus detection unit (control device) 501 AF sensor (a pair of image sensors) 502 Sensor signal processing unit (signal processing unit) 503 Correlation Calculation Processing Unit (Correlation Calculation Section)

Claims

1. A processing unit that processes signals from an image sensor containing multiple pixels using different correction values ​​for each pixel, It has a calculation unit that performs correlation calculations on the signals output from the processing unit, The calculation unit obtains a correction value for a predetermined pixel based on the phase difference amount at the time of focus obtained by correlation calculation in a correlation calculation area including a predetermined pixel among the plurality of pixels, and obtains a correction value for pixels other than the predetermined pixel based on the phase difference amount for the predetermined pixel. The control device is characterized in that the processing unit performs weighted summation of signals corresponding to the target pixel and adjacent pixels in the correlation calculation domain using the correction value, thereby shifting the centroid of the signal of the target pixel to correct the amount of image misalignment.

2. The control device according to Claim 1, wherein the calculation unit obtains a correction value for pixels other than the predetermined pixel by interpolation calculation using the phase difference amount for the predetermined pixel.

3. A processing unit that processes signals from an image sensor containing multiple pixels using different correction values ​​for each pixel, It has a calculation unit that performs correlation calculations on the signals output from the processing unit, The processing unit uses the value obtained by subtracting the offset amount from the correction value to perform weighted summation of signals corresponding to adjacent pixels among the plurality of pixels. The control device is characterized in that the calculation unit adds the offset amount to the phase difference amount obtained by the correlation calculation.

4. The control device according to claim 3, characterized in that the offset amount is the correction value of the central pixel of the correlation calculation area among the correction values, or an intermediate value between the minimum and maximum values ​​among the correction values.

5. The control device according to claim 3 or 4, characterized in that the calculation unit obtains a correction value for pixels other than the predetermined pixel by performing an interpolation calculation using a phase difference amount for a predetermined pixel among the plurality of pixels.

6. The control device according to any one of claims 1 to 5, characterized in that it has a storage unit for storing the correction value.

7. The aforementioned plurality of pixels are arranged in a two-dimensional array, The processing unit performs the processing for each pixel sequence in the direction in which the correlation calculation is performed. The control device according to any one of claims 1 to 6, characterized in that the calculation unit sums the correlation amounts for each pixel sequence signal output from the processing unit.

8. An optical apparatus characterized by having a control device according to any one of claims 1 to 7 and an optical system.

9. An imaging apparatus comprising a control device according to any one of claims 1 to 7 and the image sensor.

10. A processing step that processes signals from an image sensor containing multiple pixels using different correction values ​​for each pixel, The process includes a calculation step in which a correlation calculation is performed on the signal output in the processing step, In the calculation step, a correction value is obtained for a predetermined pixel based on the phase difference amount at the time of focus obtained by correlation calculation in a correlation calculation area including a predetermined pixel among the plurality of pixels, and a correction value is obtained for pixels other than the predetermined pixel based on the phase difference amount for the predetermined pixel. A control method characterized in that, in the processing step, the centroid of the signal of the target pixel is shifted in such a way that the amount of image misalignment is corrected by performing weighted addition of the signals corresponding to the target pixel and adjacent pixels in the correlation calculation domain using the correction value.

11. A processing step that processes signals from an image sensor containing multiple pixels using different correction values ​​for each pixel, The process includes a calculation step in which a correlation calculation is performed on the signal output in the processing step, In the processing step described above, the value obtained by subtracting the offset amount from the correction value is used to perform weighted summation of signals corresponding to adjacent pixels among the plurality of pixels. A control method characterized in that, in the calculation step, the offset amount is added to the phase difference amount obtained by the correlation calculation.

12. A program characterized by causing a computer to execute the control method described in claim 10 or 11.

Images