Processing apparatus, imaging apparatus, processing method, program and storage medium

Abstract

To provide a processing apparatus that can perform high-speed processing.SOLUTION: A processing apparatus (200) comprises: a storage unit (203) that stores first data in which information for each pixel position of an imaging device (202) is stored in a first arrangement (106) having a sequence from an optical axis of an imaging optical system (201) toward the periphery or a sequence from the periphery toward the optical axis, and second data in which the first data is stored in a second arrangement (104, 105) associated with an optical condition; and a processing unit (207) that calculates third data using a continuous plurality of pieces of adjacent data of a part of the first data in the second data, and stores the third data in the storage unit in a third arrangement being different from the first arrangement.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a processing device, an imaging device, a processing method, a program, and a storage medium. [Background technology]

[0002] Conventionally, focus detection methods using an image sensor with pupil separation based on an image sensor have been known. Furthermore, in interchangeable lens cameras, certain characteristics are determined by both the optical characteristics of the interchangeable lens and the optical characteristics of the image sensor. In particular, the baseline length between focus detection pixels is necessary for calculating the focal plane, and this characteristic must be calculated.

[0003] Patent Document 1 discloses a method for calculating a conversion coefficient for converting the image shift amount of a pupil-split image into a defocus amount using information on the vignetting shape of the lens frame for each lens. Parameters for calculating a conversion coefficient for the vignetting shape are stored as a table in a memory unit, and parameters corresponding to the determined vignetting shape are retrieved from the memory unit and used for calculation. Patent Document 2 discloses a method for correcting an image by performing deconvolution processing using a restoration filter stored in a memory unit. In Patent Documents 1 and 2, the necessary parameters are stored in a memory unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-219576 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-65187 [Patent Document 3] Japanese Patent Application Publication No. 2023-42434 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the methods disclosed in Patent Documents 1 and 2 require different parameters for each pixel position, resulting in an enormous table even when interpolating discrete optical conditions such as pixel position, F-value, and PO value. Furthermore, when discrete optical conditions are used, pixel position, F-value, and PO value must be interpolated, which requires interpolation of a total of 16 parameters (two interpolation F-values ​​and two interpolation PO values) for four interpolation pixel positions corresponding to one pixel position.

[0006] In recent years, the number of distance measurement points that can be measured simultaneously has increased, and frame rates have also become faster, requiring a large number of coefficient calculations for distance maps and distance images, resulting in a huge number of table lookups.In addition, when distance calculations are performed, a large amount of traffic occurs in the DRAM where the tables are stored, due to image development processing, etc., and table lookups occur during these long burst transfers.This reduces the throughput of both the development processing and the distance measurement processing.

[0007] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a processing device capable of high speed processing. [Means for solving the problem]

[0008] An optical coefficient calculation means according to one aspect of the present invention includes a memory unit that stores first data, in which information for each pixel position of an imaging element is stored in a first array in an order from the optical axis of the imaging optical system toward the periphery or from the periphery toward the optical axis, and second data, in which the first data is stored in a second array related to optical conditions, and a processing unit that calculates third data using a plurality of consecutive adjacent data of a portion of the first data in the second data, and stores the third data in the memory unit in a third array different from the first array.

[0009] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a processing device capable of high-speed processing. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 10 is a diagram showing an arrangement in raster order. [Figure 2] FIG. 1 is a block diagram of an imaging device according to a first embodiment. [Figure 3] 2 is a cross-sectional view of each pixel of the image sensor according to the first embodiment. FIG. [Figure 4] FIG. 3 is an explanatory diagram of a base line length in the first embodiment. [Figure 5] FIG. 3 is an explanatory diagram of a vignetting shape in the first embodiment. [Figure 6] FIG. 2 is an explanatory diagram of the position of the imaging surface and lens information in the first embodiment. [Figure 7] FIG. 3 is an explanatory diagram of interpolation in the first embodiment. [Figure 8] 5 is a flowchart showing a procedure for creating a defocus map in the first embodiment. [Figure 9] FIG. 2 is an explanatory diagram of an arrangement order corresponding to pixel positions in the first embodiment. [Figure 10] FIG. 10 is an explanatory diagram of the difference between a raster arrangement and an arrangement in order of distance from the optical axis. [Figure 11] 4A to 4C are explanatory diagrams of a lens information map and a reference table map in the first embodiment. [Figure 12] FIG. 10 is an explanatory diagram of an arrangement order corresponding to pixel positions in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] (First embodiment) First, an imaging device (processing device) 200 according to a first embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram of the imaging device 200. The imaging device 200 includes an imaging optical system 201, an image sensor 202, a recording and display unit 204, a signal processing unit 205, a control unit 206, a focus detection unit 207, and a lens control unit 208. The focus detection unit 207 is a processing unit that has a function of creating a defocus map. The imaging device 200 also includes a dynamic random access memory (DRAM) 203 and a read-only memory (ROM) 209. The DRAM 203 and the ROM 209 are storage units that store various data such as control programs and parameter tables used by the control unit 206 or the focus detection unit 207.

[0014] The image sensor 202 is a photoelectric conversion element such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. The image sensor 202 photoelectrically converts an optical image formed by the imaging optical system 201 and outputs an imaging signal to a DRAM 203. The control unit 206 controls the entire imaging device 200 (the entire system). The signal processing unit 205 processes the imaging signal stored in the DRAM 203 and converts it into image information (image data). The recording and display unit 204 displays or records the image data stored in the DRAM 203. The focus detection unit 207 processes the imaging signal stored in the DRAM 203 to perform focus detection. The lens control unit 208 controls each lens (lens group) constituting the imaging optical system 201 based on a signal from the focus detection unit 207. The ROM 209 stores control programs operating in the control unit 206 and focus detection unit 207, as well as a parameter table having the data structure of this embodiment.

[0015] The control unit 206 controls the entire system of the imaging device 200 based on user operations via an operation unit (not shown). When the system starts up, the control unit 206 moves the control program and parameter table from the ROM 209 to the DRAM 203, and also starts up the imaging optical system 201 and the imaging element 202 to transfer the output of the imaging element 202 to the DRAM 203.

[0016] The signal processing unit 205 reads the output of the image sensor 202 from the DRAM 203 and writes the signal-processed image data back to the DRAM 203. The recording and display unit 204 reads the image data written by the signal processing unit 205 and displays it on a display unit (not shown). At the same time, the focus detection unit 207 reads the output of the image sensor 202, performs focus detection, and controls the lens control unit 208 to adjust the focus.

[0017] The image sensor 202 has a first focus detection pixel that receives a light beam that passes through a first pupil partial region of the image capturing optical system 201, and a second focus detection pixel that receives a light beam that passes through a second pupil partial region of the image capturing optical system 201 that is different from the first pupil partial region. The focus detection unit 207 calculates the amount of image shift using a first focus detection signal (image A signal) generated from the light reception signal of the first focus detection pixel and a second focus detection signal (image B signal) generated from the light reception signal of the second focus detection pixel. The focus detection unit 207 then calculates the amount of defocus using the image shift amount and a conversion coefficient.

[0018] Next, the pixel structure of the image sensor 202 will be described with reference to FIG. 3. FIG. 3 is a cross-sectional view of each pixel (pixel unit) of the image sensor 202. In FIG. 3, 301 is a microlens, 304 is a color filter, and 302 and 303 are photoelectric conversion units. As described above, two photoelectric conversion units 302 and 303 are arranged as two divided pixels (a first focus detection pixel and a second focus detection pixel) corresponding to one microlens 301. This results in two pupil-separated image signals (a first focus detection signal (image A signal) and a second focus detection signal (image B signal)). The image A signal and the image B signal are stored in the DRAM 203 and then added together in the signal processing unit 205 for use as image data. The image A signal and the image B signal are also used to detect the amount of image shift when a correlation calculation is performed by the focus detection unit 207.

[0019] Once the amount of image shift is known, it can be converted into the direction and amount of defocus, and the lens can be driven according to the amount of defocus to achieve focus. Here, to calculate the amount of defocus from the amount of image shift, a coefficient (conversion coefficient) is required to convert the amount of image shift into the amount of defocus.

[0020] Next, the base line length will be described with reference to FIG. 4. FIG. 4 is an explanatory diagram of the base line length. 403 is the pixel unit shown in FIG. 3, 401 is the imaging optical system, 402 is the chief ray of the imaging optical system, and 404 and 405 are the chief ray for each divided pixel (divided pixel). 407 is the defocus amount, which corresponds to the distance between the position where the divided chief rays 404 and 405 intersect and the pixel, i.e., the imaging surface. The chief rays 404 and 405 are determined by the center of gravity of the light rays incident on the divided pixel. 406 is the base line length, which corresponds to the distance between the two centers of gravity of the light rays incident on two divided pixels. The defocus amount 407 can be calculated using the base line length 406 and the image shift amount.

[0021] Next, the relationship between pixel position, base line length, and vignetting shape will be described with reference to Fig. 5. Fig. 5 is an explanatory diagram of the vignetting shape. 501 shows the image sensor 202 when viewed from the front side. 507 is the shape of the pupil (exit pupil) when the imaging optical system 201 is viewed from the center 502 of the image sensor 202. 510 is the base line length at pixel 506. 509 is the shape of the exit pupil at the diagonal part 504 of the image sensor 202.

[0022] The imaging optical system 201 is configured by combining multiple lenses in a cylindrical shape, so the centers of the lens frames are aligned in a straight line when viewed from the central portion 502. However, as the distance from the optical axis (optical axis center) increases, a deviation occurs according to the distance from the imaging surface to the lens, and the resulting vignetting causes the shape of the exit pupil at the diagonal portion 504 to become a vignetting shape 509. This shortens the base length 511 of the pixel 508.

[0023] The vignetting shape changes concentrically around the optical axis, but the divided pixels are divided left and right. Therefore, the angle of the pixel and the vignetting shape differ depending on the pixel position. The pixel unit of the image sensor 202 has structures other than the photoelectric conversion units 302 and 303, so it is not symmetrical vertically or horizontally. In addition, manufacturing errors of the microlenses 301 must be taken into consideration. Therefore, the information required to calculate the base line length is specific to the pixel position. Therefore, in this embodiment, a parameter table corresponding to the F-number and PO-number (exit pupil distance) for each pixel position is used to calculate a coefficient equivalent to the base line length.

[0024] Next, a procedure for the focus detection unit 207 to create a defocus map will be described with reference to Fig. 8(a). Fig. 8(a) is a flowchart showing the procedure for creating a defocus map. Each step in Fig. 8(a) is mainly executed by the focus detection unit 207.

[0025] When the defocus map creation process starts, first in step S801, the focus detection unit 207 calculates a defocus coefficient and creates a coefficient map. The coefficient map can be created once the state of the imaging optical system 201 is determined, so it may be created in parallel with exposure. Next, in step S802, the focus detection unit 207 reads the exposed A and B image signals from the DRAM 203 and performs a correlation calculation to create an image shift map. Next, in step S803, the focus detection unit 207 creates a defocus map by multiplying the image shift map created in step S802 by the coefficient map created in step S801.

[0026] Next, the procedure for creating the coefficient map in step S801 will be described in detail with reference to Fig. 8(b). Fig. 8(b) is a flowchart showing the procedure for creating the coefficient map. First, in step S811, the focus detection unit 207 acquires lens information (information related to the imaging optical system 201). Next, in step S812, a lens information map (interpolated lens information map) is created.

[0027] Here, the procedure for acquiring lens information in step S811 and creating a lens information map in step S812 will be described in detail with reference to Fig. 6. Fig. 6 is an explanatory diagram of the position of the imaging surface and lens information.

[0028] Reference numeral 601 denotes a front view of the image sensor 202, 602 denotes the optical axis, and 603 to 606 denote discrete distances from the optical axis 602, which correspond to distances having discrete parameters of the lens (image sensor 201). If the parameters of distances 603 to 606 are available, the parameters of points between them can be created by interpolating from the left and right. Reference numeral 610 denotes the distance between the first lens frame and the image sensor 202, 612 denotes the radius of the first lens frame, 609 denotes the distance between the second lens frame and the image sensor 202, and 611 denotes the radius of the second lens frame. If these four pieces of information (parameters) are obtained from the lens (image sensor 201), the vignetting shape at the pixel position can be calculated.

[0029] However, since the lens frame itself is located in a space distorted by the refraction of the lens, the distance to the lens frame and the radius vary depending on the pixel position. Therefore, it is necessary to acquire four pieces of information (parameters) corresponding to the current lens state regarding the optical axis 602 and distances 603 to 606.

[0030] In this embodiment, because the interchangeable lens has the imaging optical system 201, the interchangeable lens has these four pieces of information, and this information is transferred from the interchangeable lens to the camera body via communication. The grid 607 indicates the position where the coefficient parameter table is located. The coefficient parameter table is a table corresponding to the position where the grid 607 intersects. This table has discrete information for the entire screen, but the four frame information is obtained by interpolation using the distance from the optical axis 602. Therefore, if only one-fourth of the screen is interpolated, the interpolation results for the other three pieces of information that have a common distance from the optical axis 602 can be shared. The lens information map calculated in step S812 is a map of only the area surrounded by the dashed line 608.

[0031] The focus detection unit 207 stores index data in order of distance from the optical axis 602 that indicates the correspondence between data in order of distance from the optical axis 602 of the entire screen and a lens information map of one-fourth of the screen, and accesses the lens information map by referring to the index data.

[0032] In step S812, the focus detection unit 207 creates a lens information map (interpolated lens information map). Specifically, the focus detection unit 207 first obtains the distance to the two lens frames and the radius by interpolation, and then calculates the size and distance of the exit pupil. The lens information map is a map of the F-number, which relates to the size of the exit pupil, and the PO-value, which relates to the distance to the exit pupil (exit pupil distance), for each pixel position.

[0033] Next, in step S813, the focus detection unit 207 creates a reference table map using the lens information map created in step S812. Next, in step S814, the focus detection unit 207 divides the reference table map created in step S813 into identical reference groups.

[0034] Next, with reference to Figures 1(a) and (b), the data arrangement order stored in a storage unit such as the DRAM 203 or the ROM 209, i.e., the order from the optical axis to the periphery, will be described. Note that this embodiment will describe the order from the optical axis to the periphery, but this is not limited to this, and the present invention is also applicable to an order from the periphery to the optical axis. Figure 1(a) is a diagram showing an arrangement in a general raster order (as a comparative example). Figure 1(b) is a diagram showing an arrangement in this embodiment (the order from the optical axis to the periphery, i.e., an arrangement in a spiral order based on the optical axis).

[0035] Reference numeral 101 denotes parameters P00 to P05 corresponding to pixel position (y1, x1). Reference numeral 102 denotes an array of parameters 101 in which pixels are arranged adjacently horizontally (horizontal pixel positions x0 to x20). Reference numeral 103 denotes an array of parameters 101 in which pixels are arranged adjacently vertically (vertical pixel positions y0 to y10). The arrays 102 and 103 form one plane (the imaging plane of the image sensor 202). This is a common array for image data, etc. Reference numeral 104 denotes an array (PO0 to PO310) in which the planes (arrays 102 and 103) are arranged in PO order. Reference numeral 105 denotes an array (F1.0 to F32) in which the planes (arrays 102 and 103) are arranged in F-number order. Arrays 104 and 105 are second arrays (arrays in order of values ​​related to the optical conditions) related to optical conditions (F-number, PO value).

[0036] Fig. 1(a) shows parameters 101 for a pixel position (y1, x1) under the condition that the F value is F1.2 and the PO value is PO1. In Fig. 1(a), the first pixel position is the pixel position (y0, x0) on the optical axis.

[0037] On the other hand, FIG. 1B shows an array (first array) 106 of this embodiment. The first pixel position is not (y0, x0), but (y5, x10), i.e., the pixel position on the optical axis, as the first data, followed by pixel positions (y5, x9) and (y5, x11) on the left and right of the optical axis, in this array, arranged in order of proximity to the optical axis. That is, the storage unit of this embodiment stores first data in which information for each pixel position of the image sensor 202 is stored in a first array in order from the optical axis of the imaging optical system 201 to the periphery, and second data in which the first data is stored in a second array related to optical conditions. As described below, the focus detection unit 207 calculates third data (coefficient data) using a portion of the first data in the second data, which are consecutive adjacent data, and stores the third data in the storage unit in a third array (raster array) different from the first array.

[0038] 9 is an explanatory diagram of the arrangement order (order of the arrangement 106) corresponding to pixel positions in this embodiment, and shows the arrangement positions relative to positions on the screen (horizontal pixel positions x0 to x20, vertical pixel positions y0 to y10). The central pixel position is "0", the pixel positions to the left and right of that are "1" and "2", then the pixel positions above and below are "3" and "4", and so on. The final pixel positions "227", "228", "229", and "230" are the diagonal parts (four corners) of the screen.

[0039] Since the F-stop and PO-values ​​are both discrete, four parameters must be obtained for interpolation. For example, if the F-stop is F1.3 and the PO-value is 20, the four parameters to be interpolated are "F1.2, PO1", "F1.2, PO35", "F1.4, PO1", and "F1.4, PO35".

[0040] 7(a) and (b) are explanatory diagrams of interpolation in this embodiment. In Fig. 7(a), 706 is a parameter set (corresponding to parameter 101), and 702, 703, 704, and 705 are parameters obtained from a table, representing four points each having six parameters 706. If the conditions corresponding to point 701 to be obtained are F1.3 and PO20, the six parameters 706 corresponding to point 701 are obtained by interpolation using the parameters 706 corresponding to the four surrounding points 702, 703, 704, and 705.

[0041] The reference table map created in step S813 of Figure 8(b) is data in which four discrete data references for interpolating F-stops and PO-stops are arranged in order of distance from the optical axis. When the discrete values ​​cross a threshold, the four combinations change, but the F-stops and PO-stops change gradually as they move away from the optical axis. For this reason, if the data are arranged in order of distance from the optical axis, there is a high probability that adjacent data will have the same reference.

[0042] Next, the lens information map created in step S812 and the reference table map created in step S813 will be described with reference to Fig. 11. Fig. 11 is an explanatory diagram of a pixel position table, F-number PO value, and reference F-number PO value.

[0043] Reference numeral 1101 denotes the lens information map (F-number and PO-value) created in step S812, and includes information about each pixel in the pixel position table 1105. The F-number and PO-value each change gradually as the distance from the optical axis (the center position of the image sensor 202) increases. Also, since the same value is assigned to pixels at the same distance from the center position, two consecutive pixels will have the same value if the pixel is located on the optical axis (center) in either the vertical or horizontal direction. On the other hand, if the pixel is located at a position different from the optical axis (center) in either the vertical or horizontal direction, four consecutive pixels will have the same value.

[0044] Reference numeral 1102 denotes a reference table map (reference F-number and reference PO-value) created in step S813, which indicates the values ​​of discrete data used for interpolation. The reference table map 1102 has a reference data set in region 1103 that includes reference data common to each corresponding pixel. The reference table map 1102 also has a reference data set in region 1104 that includes reference data different from the reference data of region 1103. In step S814, the focus detection unit 207 divides the reference table map into groups such as regions 1103 and 1104, and creates management data for the start and consecutive number of each reference group (divided group) (dividing into identical reference groups).

[0045] Next, in step S815, the focus detection unit 207 DMA transfers a parameter table corresponding to one of the divided groups from the DRAM 203 to an internal calculation memory of the focus detection unit 207. Next, in step S816, the focus detection unit 207 interpolates parameters based on the F-number and PO-number, as described with reference to FIG. 7A. Next, in step S817, the focus detection unit 207 calculates a defocus coefficient (conversion coefficient) using the parameters for which interpolation has been completed. Note that the calculation formula for the defocus coefficient is not essential to this embodiment, and therefore will not be described here, but for example, a formula such as that shown in Patent Document 1 can be used.

[0046] The focus detection unit 207 then stores the obtained defocus coefficients (coefficient data) in the corresponding locations in the raster array (horizontal or vertical order of pixels, third array) in the storage unit. The storage unit stores a pixel position table 1105 that associates positions in the array in order of distance from the optical axis (positions in the first array) with positions in the raster array (positions in the third array). The focus detection unit 207 references the pixel position table 1105 and stores the coefficient data obtained by calculation in the order of the first array in the raster array in the storage unit.

[0047] Next, in step S818, the focus detection unit 207 determines whether defocus coefficients corresponding to all groups (all pixel positions) have been stored in a raster array (whether processing for all groups has been completed). If there are defocus coefficients that have not yet been stored, the process returns to step S815, and the focus detection unit 207 performs processing for the next parameter table. By repeating steps S815 to S814, defocus coefficients corresponding to all pixel positions are stored in a raster array. If all defocus coefficients have been stored, the process proceeds to step S819.

[0048] Next, with reference to FIGS. 10(a) and 10(b), the difference in the number of transfers between a raster array and an array in order of distance from the optical axis (spiral order) will be explained. FIGS. 10(a) and 10(b) are explanatory diagrams of the difference between a raster array and an array in order of distance from the optical axis. 10(a) is a diagram showing blocks (same reference group) divided in step S814. In the example shown in FIG. 10(a), 16 transfers are performed, 4 times ((1) to (4)) x 4 types. On the other hand, FIG. 10(b) is a diagram showing transfers in the case of a raster array. Just transferring line 1001 requires 20 transfers, 5 times ((1) to (5)) x 4 types, exceeding the number of transfers in FIG. 10(a). Line 1002 requires 28 transfers, 7 times ((1) to (7)) x 4 types. Thus, in the case of the raster array of FIG. 10(b), the positions and numbers of each line are different.

[0049] Next, in step S819, the focus detection unit 207 performs interpolation at the ranging mesh position. Up until step S818, a mesh is formed at a density necessary to ensure accuracy. However, ranging must be performed at an even finer division. For this reason, the density of the image deviation map created in step S802 is higher than the density of the defocus coefficients in the raster array created in step S817.

[0050] Points 708, 709, 711, and 710 in FIG. 7B correspond to pixel positions in the coefficient map created in step S817. Meanwhile, point 707 in FIG. 7B corresponds to a pixel position in the image disparity map created in step S802. In step S819, the focus detection unit 207 interpolates values ​​from the distances between each of points 708, 709, 711, and 710 and point 707 to create a defocus coefficient map corresponding to the pixel positions in the image disparity map. In order to interpolate and create coefficients to be multiplied by the raster-arranged image disparity map, it is necessary to store the defocus coefficients in a raster array in advance in step S817.

[0051] In this embodiment, the pixel position table 1105 is expressed by vertical and horizontal indexes of pixels, but the same effect can be obtained by expressing it by offset information of the memory address of the storage destination.

[0052] Furthermore, in this embodiment, parameters are managed using the F value and PO value, but similar effects can be obtained with other values ​​as long as they relate to information that represents the shape of the exit pupil, such as the upper and lower lines.

[0053] Although this embodiment calculates a defocus coefficient, the present invention is not limited to this. For example, the same effect as this embodiment can be obtained by calculating a point spread function or a line spread function linked to the exit pupil and performing deconvolution processing using the restoration filter disclosed in Patent Document 2.

[0054] Furthermore, in this embodiment, the lens information map 1101 is configured so that it is sorted in order of distance from the optical axis for the entire screen, but this is not limited to this. For example, an intermediate effect can be obtained by configuring the lens information map 1101 so that it is sorted in order of distance from the optical axis within a divided area divided into blocks or within one raster line.

[0055] Furthermore, in this embodiment, the focus detection unit 207 is controlled using a microcomputer, but the same effect can be obtained even if it is configured using dedicated hardware.

[0056] (Second embodiment) Next, a second embodiment of the present invention will be described with reference to Fig. 12. Fig. 12 is an explanatory diagram of the arrangement order corresponding to pixel positions, and shows the arrangement positions relative to positions on the screen (pixel positions).

[0057] In recent years, as disclosed in Patent Document 3, interchangeable lenses that form stereo images on a single imaging element have been put into practical use. The imaging optical system for obtaining stereo images in Patent Document 3 is considered to have two optical axes (multiple optical axes). FIG. 12 corresponds to FIG. 9 described in the first embodiment, and shows a state in which an order is assigned to the distance from each optical axis when the imaging optical system has two optical axes as in this embodiment. As shown in FIG. 12, the first arrangement in this embodiment is an arrangement in an order from each of the two optical axes toward the periphery or an order from the periphery toward each of the multiple optical axes (an arrangement in a spiral order based on each optical axis).

[0058] Just as the lens information map created in step S812 in the first embodiment is one-fourth of the entire screen, in this embodiment, by configuring it so that there are eight locations at the same distance from the optical axis, the lens information map can be reduced to one-eighth of its original size.

[0059] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0060] According to each embodiment, it is possible to provide a data structure and a calculation procedure that enable a reduction in DRAM transactions for table lookup based on optical conditions, and therefore, according to each embodiment, it is possible to provide a processing device, an imaging device, a processing method, a program, and a storage medium that are capable of high-speed processing.

[0061] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) a storage unit that stores first data in which information for each pixel position of the image sensor is stored in a first array in an order from the optical axis of the imaging optical system toward the periphery or from the periphery toward the optical axis, and second data in which the first data is stored in a second array related to optical conditions; a processing unit that calculates third data using a plurality of consecutive adjacent data of a portion of the first data in the second data, and stores the third data in the memory unit in a third array different from the first array. (Configuration 2) the storage unit stores a pixel position table that associates positions in the first array with positions in the third array; The processing device according to configuration 1, wherein the processing unit stores the third data obtained by calculation in the order of the first array in the third array in the storage unit by referring to the pixel position table. (Configuration 3) the imaging optical system has a plurality of optical axes, 3. The processing device according to configuration 1 or 2, wherein the first arrangement is an arrangement in an order from each of the plurality of optical axes toward the periphery or an order from the periphery toward each of the plurality of optical axes. (Configuration 4) 4. The processing device according to any one of configurations 1 to 3, wherein the first arrangement is an arrangement in a spiral order with the optical axis as a reference. (Configuration 5) 5. The processing device according to any one of configurations 1 to 4, wherein the second array is an array in order of values ​​related to the optical conditions. (Configuration 6) 6. The processing device according to configuration 5, wherein the values ​​relating to the optical conditions include at least one of an F value or a PO value. (Configuration 7) 7. The processing device according to any one of configurations 1 to 6, wherein the third array is in the order of the horizontal or vertical direction of pixels. (Configuration 8) 8. The processing device according to any one of configurations 1 to 7, wherein the third data is coefficient data used to calculate a defocus amount. (Configuration 9) The processing unit calculating an amount of image shift using a first focus detection signal generated from a light reception signal of a first focus detection pixel that receives a light beam that passes through a first pupil partial region of the imaging optical system, and a second focus detection signal generated from a light reception signal of a second focus detection pixel that receives a light beam that passes through a second pupil partial region of the imaging optical system that is different from the first pupil partial region; 9. The processing device according to any one of configurations 1 to 8, wherein the processing device calculates a defocus amount using the image shift amount and the third data. (Configuration 10) 10. An imaging device comprising an imaging element and the processing device according to any one of configurations 1 to 9. (Method 1) referencing first data stored in a storage unit in a first array in which information for each pixel position of the image sensor is in an order from the optical axis of the image pickup optical system toward the periphery or from the periphery toward the optical axis, and second data stored in the storage unit in a second array in which the first data is related to optical conditions; calculating third data using a plurality of consecutive adjacent data of a part of the first data in the second data; and storing the third data in the storage unit in a third array different from the first array. (Configuration 11) A program that causes a computer to execute the processing method described in Method 1. (Configuration 12) 12. A computer-readable storage medium storing the program according to claim 11.

[0062] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0063] 104, 105 Second array 106 1st Array 200 Imaging device (processing device) 201 Imaging Optical System 202 Image sensor 207 Focus detection unit (processing unit) 203 DRAM (storage unit)

Claims

1. a storage unit that stores first data in which information for each pixel position of the image sensor is stored in a first array in an order from the optical axis of the imaging optical system toward the periphery or from the periphery toward the optical axis, and second data in which the first data is stored in a second array related to optical conditions; a processing unit that calculates third data using a plurality of consecutive adjacent data that are part of the first data in the second data, and stores the third data in the memory unit in a third array that is different from the first array.

2. the storage unit stores a pixel position table that associates positions in the first array with positions in the third array; 2. The processing device according to claim 1, wherein the processing unit stores the third data obtained by performing calculations in the order of the first array in the third array in the storage unit with reference to the pixel position table.

3. the imaging optical system has a plurality of optical axes, 2. The processing device according to claim 1, wherein the first arrangement is an arrangement in order from each of the plurality of optical axes toward the periphery or an arrangement in order from the periphery toward each of the plurality of optical axes.

4. 2. The processing device according to claim 1, wherein the first arrangement is a spiral arrangement with the optical axis as a reference.

5. The processing device according to claim 1 , wherein the second array is an array in order of values ​​related to the optical conditions.

6. The processing device according to claim 5 , wherein the value relating to the optical condition includes at least one of an F-number and a PO value.

7. 2. The processing device according to claim 1, wherein the third array is in horizontal or vertical order of pixels.

8. 8. The processing device according to claim 1, wherein the third data is coefficient data used to calculate a defocus amount.

9. The processing unit calculating an amount of image shift using a first focus detection signal generated from a light reception signal of a first focus detection pixel that receives a light beam that has passed through a first pupil partial region of the imaging optical system, and a second focus detection signal generated from a light reception signal of a second focus detection pixel that receives a light beam that has passed through a second pupil partial region of the imaging optical system that is different from the first pupil partial region; 8. The processing device according to claim 1, wherein a defocus amount is calculated using the image shift amount and the third data.

10. An imaging device comprising an imaging element and the processing device according to any one of claims 1 to 7.

11. referencing first data stored in a storage unit in a first array in which information for each pixel position of the image sensor is in an order from the optical axis of the image pickup optical system toward the periphery or from the periphery toward the optical axis, and second data stored in the storage unit in a second array in which the first data is related to optical conditions; calculating third data using a plurality of consecutive adjacent data of a part of the first data in the second data; and storing the third data in the storage unit in a third array different from the first array.

12. A program causing a computer to execute the processing method according to claim 11.

13. A computer-readable storage medium storing the program according to claim 12.

Images