Image processing apparatus, imaging apparatus, image processing method, and program

Abstract

To provide an image processing apparatus that can perform appropriate optical correction processing on a composite image obtained by combining a plurality of images after positional deviation correction.SOLUTION: An image processing apparatus (10) processes a plurality of images obtained by consecutively picking up images of a subject, and has: detection means (108) that detects the amount of positional deviation between the plurality of images; correction means (109) that corrects the amount of positional deviation; composition means (107) that combines the plurality of images for which the amount of positional deviation is corrected, and thereby creates a composite image; and acquisition means (110) that acquires an optical center position of the composite image based on the amount of positional deviation. The optical center position is a reference position in performing optical correction processing on the composite image, and the amount of correction in the optical correction processing changes according to the distance from the reference position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an imaging device, an image processing method, and a program. [Background technology]

[0002] Conventionally, a method is known in which optical correction and development processes are performed on digital signals (RAW data) output from an imaging element by divided exposure, and then the positional misalignment between the multiple images is corrected and the multiple images are combined. When multiple images are combined after optical correction and development processes, the combined image to which the above-mentioned processes have been performed is recorded, and therefore the parameters of the optical correction and development processes in the recorded combined image cannot be changed. On the other hand, by correcting the amount of positional misalignment using RAW data before optical correction and development processes and recording the combined image, it is possible to change the parameters of the optical correction and development processes in the recorded combined image.

[0003] Patent Document 1 discloses an imaging device that generates a composite image after correcting positional deviations between multiple captured images, and performs vignetting correction on the composite image taking into account the vignetting characteristics of the optical system. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-92548 Summary of the Invention [Problem to be solved by the invention]

[0005] When a composite image is generated by correcting the positions using RAW data, the captured images are combined in a state where they are shifted by the amount of the correction of the optical center positions of the individual captured images, making it impossible to apply desired optical correction processing to the composite image.The method disclosed in Patent Document 1 cannot be widely applied to optical correction processing for distortion aberration, chromatic aberration, etc.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image processing device capable of performing appropriate optical correction processing on a composite image obtained by combining a plurality of images after positional deviation correction. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present invention is an image processing device that processes a plurality of images obtained by successively capturing an image of a subject, and includes a detection means for detecting the amount of positional shift between the plurality of images, a correction means for correcting the amount of positional shift, a synthesis means for synthesizing the plurality of images whose amounts of positional shift have been corrected to generate a synthetic image, and an acquisition means for acquiring the optical center position of the synthetic image based on the amount of positional shift, wherein the optical center position is a reference position when performing optical correction processing on the synthetic image, and the amount of correction in the optical correction processing changes depending on the distance from the reference position.

[0008] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image processing device capable of performing appropriate optical correction processing on a composite image obtained by combining a plurality of images after positional deviation correction. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an imaging apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a pixel arrangement of an image sensor according to the present embodiment. [Figure 3] FIG. 10 is an explanatory diagram of a position alignment process according to the present embodiment. [Figure 4] 4 is a flowchart of divided exposure photography in this embodiment. [Figure 5] FIG. 4 is an explanatory diagram of a method for calculating optical center coordinates in the present embodiment. [Figure 6] FIG. 10 is a diagram showing the relationship between image height and optical correction value in the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] First, an imaging device (image processing device) 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram of the imaging device 10. The imaging device 10 is capable of processing a plurality of images obtained by successively capturing subject images. When a shutter switch SW2 (not shown) is pressed, light rays incident on a lens (imaging optical system) 100 pass through a diaphragm (aperture stop) 101 and form a subject image on an imaging element 102. Note that in this embodiment, the imaging device 10 is configured such that a lens device including the lens 100 and a camera body including the imaging element 102 are detachable, but they may also be configured as an integrated unit.

[0013] The image sensor 102 converts an optical image of a subject into an electrical signal and outputs an image signal (analog signal). The image sensor 102 has a Bayer array configuration in which R (red), G1 (green), G2 (green), and B (blue) pixels are regularly arranged. The analog signal output from the image sensor 102 is converted into a digital signal (RAW data) by the A / D converter 103 and temporarily stored in memory 115. Note that RAW data is data that has not undergone a predetermined development process.

[0014] The pixel arrangement of the image sensor 102 will now be described with reference to FIG. 2. FIG. 2 is a diagram of the pixel arrangement of the image sensor 102. As shown in FIG. 2, the image sensor 102 includes an effective pixel area 200, which is made up of photodiodes (photoelectric conversion elements) and is illuminated with light, and an optical black area (OB area) 201, in which a predetermined area of ​​the photodiodes is shielded from light by an aluminum thin film or the like. In other words, the effective pixel area 200 is not shielded from light, while the OB area 201 is shielded from light. The OB integration unit 104 integrates the pixel values ​​of the OB area 201 for each of R, G1, G2, and B in the Bayer array and outputs an average value of the OB values. The output value of the OB integration unit 104 is set as a dark level (black level), and the OB clamping unit 105 performs OB clamping processing. The OB clamping processing can prevent problems such as black floating and color shift from occurring.

[0015] The image stabilization unit (image stabilization mechanism) 106 has a gyro sensor and calculates the amount of image stabilization from information on the camera's rotation angle (pitch direction, yaw direction, roll direction) acquired during exposure, and passes the image stabilization amount to the control unit (control means) 114. The control unit 114 optically suppresses image stabilization during exposure by driving the physical position of the image sensor 102 in a direction perpendicular to the optical axis by the amount of image stabilization. Furthermore, the lens 100 serving as the imaging optical system has an image stabilization lens that corrects image stabilization, and optically suppresses image stabilization during exposure by acquiring the amount of image stabilization obtained by the image stabilization unit 106 and driving the image stabilization lens. The memory 115 stores RAW data, image data processed by the signal processing unit 112, and the like.

[0016] The composition unit (composition means) 107 generates a composite image by combining multiple images for which the amount of misalignment has been corrected. The composition unit 107 also performs various calculations depending on the composition method of the divided exposure shooting mode. In this embodiment, the composition unit 107 performs various calculations depending on three composition methods: addition mode, average addition mode, and relatively bright mode. The luminance value of each image before composition is I_i(x, y) (i = 1 to N, x, y represent coordinates within the screen), and the luminance value of the image after composition of these N images is I(x, y). In this case, the addition mode is expressed by the following equation (1), and the composite image data is obtained by adding the luminance values ​​of the N images for each pixel.

[0017] I(x,y)=I_1(x,y)+I_2(x,y)+···+I_N(x,y) …(1) The averaging mode is expressed by the following equation (2), and the composite image data is obtained by averaging the brightness values ​​of N images for each pixel.

[0018] I(x,y)=(I_1(x,y)+I_2(x,y)+···+I_N(x,y)) / N …(2) The comparatively bright mode is expressed by the following formula (3), and the maximum luminance value of N images is selected for each pixel to become the composite image data.

[0019] I(x,y)=max(I_1(x,y),I_2(x,y),···,I_N(x,y)) …(3) The signal processing unit 112 performs development processing such as white balance processing, color matrix processing, or gamma processing on the RAW data or the RAW data combined by the combining unit 107. The recording unit (recording means) 113 records the RAW data, the RAW data combined by the combining unit 107, or image data developed by the signal processing unit 112. The control unit 114 performs overall control of the imaging device 10.

[0020] The misalignment detection unit (detection means) 108 detects the amount of misalignment between multiple images captured continuously on a pixel-by-pixel basis. In this embodiment, template matching is performed on a reference image (first image) and a registered image to detect the amount of misalignment (misalignment amount) on a pixel-by-pixel basis. Template matching is a technique in which a specific region of an image is cut out, the absolute value of the difference between the specific region is measured while shifting the template position, and the amount of shift with the smallest difference is determined as the amount of misalignment. Note that in this embodiment, the amount of misalignment is detected by template matching of images, but this is not limited to this. For example, a gyro sensor may be used to obtain the rotation angle of the imaging device between consecutive images, and the amount of misalignment between images may be calculated from the focal length and rotation angle.

[0021] The misalignment correction unit (correction means) 109 corrects the amount of misalignment detected by the misalignment detection unit 108. More specifically, the misalignment correction unit 109 corrects the position of the RAW data based on the amount of misalignment for each pixel detected by the misalignment detection unit 108, and outputs the corrected data to the memory 115. The misalignment correction unit 109 also reads out a portion of the RAW data and changes the amount of misalignment correction for each region.

[0022] The optical center correction unit (acquisition means) 110 acquires (calculates) the optical center position (optical center coordinates) of the composite image based on the amount of positional deviation detected by the positional deviation detection unit 108, and outputs information about the acquired optical center position to memory 115. Here, the optical center position is a reference position when optical correction processing is performed on the composite image, and the correction amount in the optical correction processing changes depending on the distance from the reference position. The method for calculating the optical center coordinates will be described later.

[0023] The lens 100 is detachable from the camera body, and when the lens 100 is attached to the camera body, the control unit 114 acquires optical correction data from the lens 100 and outputs the data to the memory 115. The optical correction unit 111 calculates an optical correction value based on the optical correction data acquired from the lens 100. Based on the optical correction value calculated by the optical correction unit 111, the control unit 114 performs control (optical correction processing) related to optical correction such as peripheral illumination correction, distortion correction, chromatic aberration of magnification correction, or image restoration.

[0024] The optical correction unit 111 calculates a distortion aberration correction value from the distortion aberration correction information. The control unit 114 then sets the calculated distortion aberration correction value and performs distortion aberration correction. The optical correction unit 111 also calculates a magnification chromatic aberration correction value from the magnification chromatic aberration correction information. The control unit 114 then sets the calculated correction value and performs magnification chromatic aberration correction processing. The optical correction unit 111 also calculates a peripheral illumination correction value from the peripheral illumination correction information. The control unit 114 then sets the calculated peripheral illumination correction value and performs peripheral illumination correction processing on the image data. The optical correction unit 111 also calculates a restoration coefficient to be used for image restoration from the image restoration processing information. The control unit 114 then sets the calculated image restoration coefficient and performs image restoration processing. Note that the optical correction processing described above is just an example, and a configuration other than the above may be performed, or a configuration may be configured that does not perform one or more of the above optical correction processing.

[0025] Next, the optical correction unit 111 will be described in detail with reference to FIG. 6. FIGS. 6(a) to 6(d) are diagrams showing the relationship between image height and each correction value (optical correction value). FIG. 6(a) shows the correction values ​​for performing magnification chromatic aberration correction. The correction values ​​are obtained by dividing the image height into multiple discrete data, and are calculated by interpolation. In FIG. 6(a), the horizontal axis represents the image height to be corrected, and the vertical axis represents the amount of correction. Magnification chromatic aberration correction is performed using a correction value according to the image height.

[0026] In FIG. 6( a), 600 denotes the red lateral chromatic aberration correction value, and 601 denotes the blue lateral chromatic aberration correction value. Lateral chromatic aberration is a phenomenon in which an image is distorted for each red and blue component. Lateral chromatic aberration is corrected by shifting the inner or outer pixels by the correction value to match the green component. Lateral chromatic aberration correction has correction values ​​corresponding to the shooting distance, aperture value, and focal length, and correction is performed using the correction values ​​corresponding to the shooting conditions. In this embodiment, correction is performed using correction values ​​corresponding to the shooting distance, aperture value (maximum aperture value), and focal length (maximum focal length). Details of the optical aberration correction method using correction values ​​corresponding to the shooting distance, aperture value, and focal length are well known and therefore will be omitted. The correction values ​​in this embodiment are set to perform correction using correction data that is point-symmetric with respect to the optical axis. It is possible to store correction values ​​corresponding to the shooting distance, aperture value, and focal length in a small memory area. In this way, the correction amount changes depending on the distance from the reference position (image height). That is, the closer to the optical center position, the smaller the correction amount, and the farther from the optical center position, the larger the correction amount.

[0027] Peripheral illumination correction, distortion correction, and image restoration processing each have two-dimensional correction data with the horizontal axis indicating the image height to be corrected and the vertical axis indicating the amount of correction. Similar to chromatic aberration correction, correction is performed using a correction value corresponding to the image height. Figure 6(b) shows the distortion correction value for distortion correction processing. Distortion correction is performed by shifting pixels inward or outward by the amount of the distortion correction value. 602 is the distortion correction value corresponding to the image height. Figure 6(c) shows the peripheral illumination correction value for peripheral illumination correction processing. Peripheral illumination correction is performed by multiplying the pixel values ​​of pixels at the same image height according to the peripheral illumination correction value. 603 is the peripheral illumination correction value corresponding to the image height. Figure 6(d) shows the gain amount of the correction coefficient for image restoration. Image restoration processing involves performing multiple multi-tap filter processes on pixels at the same image height according to the image restoration correction coefficient, and applying a gain amount according to the shooting conditions to each filter to perform image restoration processing according to the image height. Reference numeral 604 denotes the gain amount of the image restoration coefficient according to the image height.

[0028] Next, divided exposure photography in this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart of divided exposure photography. Each step in Fig. 4 is mainly executed by the control unit 114 or by each unit based on an instruction from the control unit 114.

[0029] First, in step S101, when the divided exposure shooting mode is started by a user operation, the control unit 114 sets the number of images to be shot. Next, in step S102, the user issues a shooting instruction by pressing the shutter switch SW2. Next, in step S103, the control unit 114 takes continuous images until the number of images set in step S101 has been shot, and repeats combining while correcting positional deviations (steps S103 to S106). Steps S103 to S106 will be described in detail below.

[0030] First, in step S103, in response to the shooting instruction in step S102, the control unit 114 controls the image sensor 102 to capture RAW data. Next, in step S104, the misalignment detection unit 108 uses the first image as a reference image and detects the amount of misalignment between the currently captured image and the first image by template matching. At this time, the misalignment detection unit 108 arranges templates for multiple regions of the image and calculates the amount of misalignment for each region. The misalignment detection unit 108 then creates a histogram of the calculated amounts of misalignment and adopts the most frequently occurring amount of misalignment as the final amount of misalignment. Next, in step S105, the misalignment correction unit 109 uses the amount of misalignment detected in step S104 to align the image (RAW data) captured most recently with the reference image (alignment process).

[0031] The alignment process will now be described with reference to Fig. 3. Fig. 3 is an explanatory diagram of the alignment process, and schematically shows a method of performing synthesis processing by aligning a second frame image (RAW data) 302 with a first frame image (RAW data) 301, which is a reference image.

[0032] Image 301 has an effective pixel region 301a and an OB region 301b. Similarly, image 302 has an effective pixel region 302a and an OB region 302b. Here, it is assumed that the misalignment correction unit 109 is set to a mode that aligns only the image of the effective pixel region. Therefore, in the aligned image 302', only the effective pixel region 302a' has moved, and the OB region 302b has not moved. As a result, an area 302c where no pixels exist has appeared in the aligned image 302'. Note that when the misalignment correction unit 109 is set to a mode that aligns the entire frame, the coordinate information is changed internally, but the appearance of the image 302 before and after alignment remains unchanged.

[0033] Next, in step S106 of FIG. 4, the composition unit 107 combines the image (image 302' of FIG. 3) that has been position-corrected (aligned) in step S105 with the reference image or the combined image (composite RAW data) up to the previous frame. The composition unit 107 performs the composition process as described above depending on the set mode. FIG. 3 shows an example of composition process in the averaging mode or the relatively bright mode, in which an area 302c without pixels that occurs due to alignment is excluded from the composition target. As a result, the effective pixel area 310a of the composite image 310 includes an area 310c where the image of the second frame is not combined. For the OB area 310b of the composite image 310, the OB area 301b of the first frame and the OB area 302b of the second frame are combined using a method depending on the mode.

[0034] 4, the control unit 114 determines whether or not the number of shots set in step S101 has been taken. If it is determined that the number of shots has not been taken, the process returns to step S103. On the other hand, if it is determined that the number of shots has been taken, the process proceeds to step S108.

[0035] Subsequently, in step S108, the optical center correction unit 110 calculates (acquires) the optical center coordinates (optical center position) of the combined RAW image (composite image). Here, a method for calculating the optical center coordinates will be described with reference to Fig. 5. Fig. 5 is an explanatory diagram of the method for calculating the optical center coordinates.

[0036] When combining two images, 400 is the first image (development area) that serves as the reference image for misalignment correction, 401 is the optical center coordinate of the first image, 402 is the second image that is the target of misalignment correction, and 403 is the optical center coordinate of the second image. The optical center coordinates of each image are the center coordinates of the development area for the first image, and the center coordinates of the development area for the second and subsequent images, plus the amount of misalignment correction in the horizontal and vertical directions.

[0037] The optical center coordinates 401 of the first image and the optical center coordinates 403 of the second image are averaged in the horizontal and vertical directions to obtain the optical center coordinates 404 of the composite image. In this embodiment, the optical center position of the composite image is thus the average value of the optical center positions of multiple images that are composited based on the amount of misalignment. Note that while this example shows the composition of two images, even if the number of images increases, the optical center coordinates of the images to be subjected to alignment correction can be calculated in the same way, and the average values ​​in the horizontal and vertical directions can be obtained using the optical center coordinates of the images to be composited.

[0038] When optical image stabilization is performed using the image stabilization unit 106 during exposure, the optical center coordinates of each image (each captured image) may not be at the center of the development area. In this case, the average value of the optical center coordinates during exposure for each image may be calculated, and the amount of misalignment correction due to positioning may be added to the optical center coordinates thus obtained to determine the optical center coordinates of each image. In other words, the optical center correction unit 110 may obtain the optical center coordinates of the composite image using the average value of the optical center coordinates of multiple images when the optical image stabilization mechanism is operating during exposure.

[0039] 4, the control unit 114 determines whether or not to record RAW data (whether or not the user has enabled recording of RAW data). If it is determined that RAW data is to be recorded, the process proceeds to step S112. On the other hand, if it is determined that RAW data is not to be recorded, the process proceeds to step S110.

[0040] In step S110, the optical correction unit 111 and control unit 114 perform optical correction processing on the combined RAW data (composite image) based on the optical center coordinates. The optical correction processing is processing that changes the correction amount depending on the image height. Therefore, the optical correction unit 111 calculates optical correction values ​​for magnification chromatic aberration correction, distortion aberration correction, peripheral illumination correction, and image restoration, using the optical center coordinates of the combined image calculated in step S108 as the image height 0 (the reference position for the optical correction processing). Then, the control unit 114 performs optical correction processing on the combined image based on each calculated optical correction value. Subsequently, in step S111, the signal processing unit 112 performs development processing on the combined RAW data that has been optically processed in step S110.

[0041] In step S112, the control unit 114 records the composite image in an external file such as an SD card. If it is determined in step S109 that the RAW data is to be recorded, the composited RAW data generated in step S106 is recorded. On the other hand, if it is determined in step S109 that the RAW data is not to be recorded, the developed image (developed image) is recorded in step S111. The control unit 114 also records the optical center coordinates calculated in step S108 as meta information in the file. The information recorded at this time may include a shift amount from the center coordinates of the development area. The shift amount from the center coordinates can be calculated from the difference between the center coordinates of the development area and the optically corrected center coordinates of the composite image. With respect to the recorded composite RAW data, an image processing device other than the imaging device 10 may perform at least a part of the processing in steps S110 to S111.

[0042] As described above, in this embodiment, the control unit 114 determines whether or not to record RAW data based on the user's settings. If the control unit 114 determines not to record RAW data, it performs optical correction processing on the composite image and records a developed image of the composite image after the optical correction processing in the recording unit 113. On the other hand, if the control unit 114 determines to record RAW data, it records the composite image and the optical center position of the composite image in the recording unit 113.

[0043] Furthermore, with regard to the peripheral illumination correction performed in the optical correction in step S110, the peripheral illumination correction data itself may be stored as image map information of the horizontal image height, vertical image height, and correction value, as disclosed in Patent Document 1. In this case, the positions are corrected and combined based on the correction amount for positional deviation, and peripheral illumination correction can be performed on the combined RAW data.

[0044] In the present embodiment, the imaging device performs steps S101 to S112. However, the step of acquiring multiple images and the step of correcting misalignment and combining the multiple images may be performed by different devices. For example, the imaging device may capture images multiple times to acquire images for image composition, and an image processing device that acquires the captured images may detect and correct misalignment between the multiple images to perform image composition. In such a configuration, the image processing device that detects and corrects misalignment between the multiple images and combines the images may not have an imaging function. Alternatively, the image processing device that performed the image composition may perform steps S108 to S112. Alternatively, the image processing device that performed the image composition may not have the functions of optical correction processing or development processing, and may omit step S109 and perform step S112 after step S108.

[0045] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0046] According to each embodiment, it is possible to provide an image processing device, an imaging device, an image processing method, and a program that are capable of performing appropriate optical correction processing on a composite image obtained by combining multiple images after positional deviation correction.

[0047] The disclosure of each embodiment includes the following configurations and methods.

[0048] (Configuration 1) An image processing device that processes a plurality of images obtained by successively capturing an image of a subject, a detection means for detecting a positional deviation amount between the plurality of images; a correction means for correcting the amount of positional deviation; a synthesis unit that synthesizes the plurality of images whose positional deviations have been corrected to generate a synthesized image; an acquisition unit for acquiring an optical center position of the composite image based on the amount of positional deviation; the optical center position is a reference position when optical correction processing is performed on the composite image, 10. An image processing device according to claim 9, wherein the amount of correction in the optical correction process varies depending on the distance from the reference position. (Configuration 2) 2. The image processing device according to configuration 1, further comprising a control means for performing the optical correction process on the composite image. (Configuration 3) 2. The image processing device according to configuration 1, further comprising control means for recording the composite image and the optical center position of the composite image in recording means. (Configuration 4) 5. The image processing device according to any one of configurations 1 to 4, wherein the plurality of images are RAW data. (Configuration 5) The method further includes a control means for determining whether or not to record the RAW data based on a setting made by a user, The control means If it is determined that the RAW data is not to be recorded, the optical correction processing is performed on the composite image, and a developed image of the composite image after the optical correction processing is recorded in a recording means; 5. The image processing device according to configuration 4, wherein, when it is determined that the RAW data is to be recorded, the composite image and the optical center position of the composite image are recorded in the recording means. (Configuration 6) 6. The image processing device according to any one of configurations 1 to 5, wherein the optical correction processing is peripheral illumination correction. (Configuration 7) 6. The image processing device according to any one of configurations 1 to 5, wherein the optical correction processing is correction of chromatic aberration of magnification. (Configuration 8) 6. The image processing device according to any one of configurations 1 to 5, wherein the optical correction processing is distortion correction. (Configuration 9) 6. The image processing device according to any one of configurations 1 to 5, wherein the optical correction processing is an image restoration processing. (Configuration 10) 10. The image processing device according to any one of configurations 1 to 9, wherein the optical center position of the composite image is an average value of the optical center positions of the plurality of images that are composited based on the amount of positional deviation. (Configuration 11) The image processing device according to any one of configurations 1 to 10, wherein the acquisition means acquires the optical center position of the composite image by using an average value of the optical center positions of the plurality of images when an optical image stabilization mechanism is operating during exposure. (Configuration 12) 12. The image processing device according to any one of configurations 1 to 11, wherein the correction means corrects the amount of positional deviation using a first image of the plurality of images as a reference image. (Configuration 13) 13. An imaging device comprising: the image processing device according to any one of configurations 1 to 12; and an imaging element. (Method 1) An image processing method for processing a plurality of images obtained by successively capturing an image of a subject, comprising: detecting a positional deviation amount between the plurality of images; correcting the amount of positional deviation; generating a composite image by combining the plurality of images whose positional deviation amounts have been corrected; acquiring an optical center position of the composite image based on the amount of positional deviation; the optical center position is a reference position when optical correction processing is performed on the composite image, An image processing method, wherein the amount of correction in the optical correction processing changes depending on the distance from the reference position. (Configuration 14) A program that causes a computer to execute the image processing method described in Method 1.

[0049] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0050] 10 Imaging device (image processing device) 107 Synthesis unit (synthesis means) 108 Position deviation detection unit (detection means) 109 Position deviation correction unit (correction means) 110 Optical center correction section (obtaining means)

Claims

1. An image processing device that processes a plurality of images obtained by successively capturing an image of a subject, a detection means for detecting a positional deviation amount between the plurality of images; a combining unit that combines the plurality of images that have been aligned based on the amount of misalignment to generate a combined image; an acquisition unit for acquiring an optical center position of the composite image based on the amount of positional deviation; the optical center position is a reference position when optical correction processing is performed on the composite image, 10. An image processing device according to claim 9, wherein the amount of correction in the optical correction process varies depending on the distance from the reference position.

2. 2. The image processing apparatus according to claim 1, further comprising a control unit for performing the optical correction process on the composite image.

3. 2. The image processing apparatus according to claim 1, further comprising control means for recording the composite image and the optical center position of the composite image in recording means.

4. 2. The image processing apparatus according to claim 1, wherein the plurality of images are RAW data.

5. The recording device further includes a control unit that determines whether or not to record the RAW data based on a setting made by a user, The control means If it is determined that the RAW data is not to be recorded, the optical correction processing is performed on the composite image, and a developed image of the composite image after the optical correction processing is recorded in a recording means; 5. The image processing apparatus according to claim 4, wherein, when it is determined that the RAW data is to be recorded, the composite image and the optical center position of the composite image are recorded in the recording means.

6. 2. The image processing apparatus according to claim 1, wherein the optical correction processing is peripheral illumination correction.

7. 2. The image processing device according to claim 1, wherein the optical correction processing is correction of chromatic aberration of magnification.

8. 2. The image processing device according to claim 1, wherein the optical correction processing is distortion correction.

9. 2. The image processing apparatus according to claim 1, wherein the optical correction processing is an image restoration processing.

10. 2. The image processing apparatus according to claim 1, wherein the optical center position of the composite image is an average value of the optical center positions of the plurality of images that are composited based on the amount of positional deviation.

11. 2. The image processing device according to claim 1, wherein the acquisition means acquires the optical center position of the composite image using an average value of the optical center positions of the plurality of images when an optical image stabilization mechanism is operating during exposure.

12. 2. The image processing apparatus according to claim 1, wherein the combining means combines the plurality of images, which are aligned based on the amount of positional deviation, using a first image of the plurality of images as a reference image, to generate a combined image.

13. An imaging device comprising: the image processing device according to claim 1; and an imaging element.

14. An image processing method for processing a plurality of images obtained by successively capturing an image of a subject, comprising: detecting a positional deviation amount between the plurality of images; generating a composite image by combining the plurality of images that have been aligned based on the amount of misalignment; acquiring an optical center position of the composite image based on the amount of positional deviation; the optical center position is a reference position when optical correction processing is performed on the composite image, An image processing method, wherein the amount of correction in the optical correction process varies depending on the distance from the reference position.

15. A program causing a computer to execute the image processing method according to claim 14.

Images