Camera device, control method thereof, and non-transitory computer-readable storage medium

Abstract

The present invention provides an imaging device and a control method thereof and a non-transitory computer-readable storage medium. The imaging device capable of exchanging lens units includes: an imaging unit for capturing images using an image sensor; a synthesis unit for synthesizing a plurality of images; and a control unit for controlling a predetermined synthesis process, wherein the predetermined synthesis process is used to create a synthetic image by causing the synthesis unit to synthesize a plurality of images obtained by the imaging unit performing multiple imaging operations, wherein, in the case where it is determined that the lens unit replaced after acquiring a first image for synthesis processing is a lens unit having an image circle different from that of the lens unit installed before the replacement, the control unit controls the imaging unit to not capture a new image for creating the synthetic image.

Description

[0001] (This application is a divisional application of an application filed on June 25, 2019, with application number 201910554373.6, and invention name “IMAGING DEVICE, CONTROL METHOD THEREOF, AND NON-TEMPORARY COMPUTER-READABLE STORAGE MEDIUM”.) Technical Field

[0002] The present invention relates to an imaging device, a control method thereof, and a non-transitory computer-readable storage medium. Background Art

[0003] There are digital cameras that can perform so-called multiple exposure imaging that obtains one composite image by synthesizing a plurality of captured images and records the composite image. There are digital cameras that can select the number of images to be synthesized or a synthesis processing method (such as weighted averaging, synthesis of relatively bright parts, or synthesis of relatively dark parts) before shooting, and there are also digital cameras that can select an image that has already been shot as a basic image and generate a multiplexed image by synthesizing an image to be shot on the basic image.

[0004] Japanese Patent Laid-Open No. 2012-19343 and Japanese Patent Laid-Open No. 2007-221237 disclose a technology for changing settings during multiple exposure imaging. Japanese Patent Laid-Open No. 2012-19343 discloses a technology that, in a digital camera capable of multiple exposure imaging, can change settings for the aspect ratio of an image to be recorded during shooting. In addition, Japanese Patent Laid-Open No. 2007-221237 discloses a technology that, in a digital camera capable of multiple exposure imaging, can change settings such as recording quality and recording size during shooting.

[0005] In an interchangeable lens type camera, the lens unit can be replaced while multiple exposure imaging is being performed. There are various types of interchangeable lens units, and it is conceivable that an unnatural composite image is generated depending on the lens unit that is replaced. Summary of the invention

[0006] The present invention, in its first aspect, provides an imaging device capable of exchanging lens units, the imaging device comprising: an imaging unit for capturing images using an image sensor; a synthesis unit for synthesizing multiple images; and a control unit for controlling a predetermined synthesis process, the predetermined synthesis process being used to create a composite image by causing the synthesis unit to synthesize multiple images obtained by multiple imaging operations performed by the imaging unit, wherein, in a case where it is determined that the lens unit replaced after a first image for the synthesis process is acquired is a lens unit having an image circle different from that of the lens unit installed before the replacement, the control unit controls the imaging unit so as not to capture a new image for creating the composite image.

[0007] The present invention, in its second aspect, provides a camera device capable of exchanging lens units, the camera device comprising: a camera unit for capturing images using an image sensor; a synthesis unit for synthesizing multiple images; and a control unit for controlling a predetermined synthesis process, the predetermined synthesis process being used to create a composite image by causing the synthesis unit to synthesize multiple images obtained by multiple imaging operations performed by the camera unit, wherein, in a case where it is determined that the lens unit replaced after acquiring a first image for the synthesis process is a lens unit requiring predetermined image correction, the control unit controls the camera unit so as not to capture a new image for creating the composite image.

[0008] In its third aspect, the present invention provides a control method for a camera device, wherein the camera device is capable of exchanging lens units and includes a camera unit for capturing images using an image sensor, the control method comprising: a synthesis step for synthesizing multiple images; and a control step for controlling a predetermined synthesis process, wherein the predetermined synthesis process is used to create a composite image by causing the synthesis step to synthesize multiple images obtained by multiple imaging operations performed by the camera unit, wherein, in a case where it is determined that the lens unit replaced after acquiring a first image for the synthesis process is a lens unit having an image circle different from that of the lens unit installed before the replacement, the camera unit is controlled not to capture a new image for creating the composite image.

[0009] The present invention, in its fourth aspect, provides a control method for a camera device, wherein the camera device is capable of exchanging lens units and includes a camera unit for capturing images using an image sensor, the control method comprising: a synthesis step for synthesizing a plurality of images; and a control step for controlling a predetermined synthesis process, wherein the predetermined synthesis process is used to create a composite image by causing the synthesis step to synthesize a plurality of images obtained by multiple imaging operations performed by the camera unit, wherein, in a case where it is determined that the lens unit replaced after acquiring a first image for the synthesis process is a lens unit requiring predetermined image correction, the camera unit is controlled not to capture a new image for creating the composite image.

[0010] In its fifth aspect, the present invention provides a non-temporary computer-readable storage medium storing a program which, when read and executed by a computer, causes the computer to execute steps of a control method for a camera device, the camera device being capable of exchanging lens units and including a camera unit for capturing images using an image sensor, the control method comprising: a synthesis step for synthesizing a plurality of images; and a control step for controlling a predetermined synthesis process, the predetermined synthesis process being used to create a composite image by causing the synthesis step to synthesize a plurality of images obtained by multiple imaging operations performed by the camera unit, wherein, in a case where it is determined that the lens unit replaced after acquiring a first image for the synthesis process is a lens unit having an image circle different from an image circle of the lens unit installed before the replacement, the camera unit is controlled not to capture a new image for creating the composite image.

[0011] In its sixth aspect, the present invention provides a non-temporary computer-readable storage medium storing a program which, when read and executed by a computer, causes the computer to execute the steps of a control method for a camera device, the camera device being capable of exchanging lens units and including a camera unit for capturing images using an image sensor, the control method comprising: a synthesis step for synthesizing a plurality of images; and a control step for controlling a predetermined synthesis process, the predetermined synthesis process being used to create a composite image by causing the synthesis step to synthesize a plurality of images obtained by multiple imaging operations performed by the camera unit, wherein, in a case where it is determined that the lens unit replaced after acquiring a first image for the synthesis process is a lens unit requiring predetermined image correction, the camera unit is controlled not to capture a new image for creating the composite image.

[0012] The present invention, in its seventh aspect, provides an imaging device capable of exchanging lens units, the imaging device comprising an imaging unit, and the imaging device being used to create a composite image based on multiple images obtained by performing multiple imaging operations by the imaging unit, the imaging device comprising: a detection unit for detecting reinstallation of the lens unit; a determination unit for determining an area based on an image circle of the reinstalled lens unit when the detection unit detects reinstallation of the lens unit, wherein image data of an image sensor included in the imaging unit is extracted relative to the area; an acquisition unit for acquiring image data related to the area determined by the determination unit from the image sensor; a judgment unit for judging whether the image size of the image data of interest acquired by the acquisition unit is the same as the image size of the image data to be synthesized when synthesizing the image data of interest acquired by the acquisition unit and the image data to be synthesized; and a synthesis unit for New composite image data to be synthesized is created by synthesizing the focus image data and the image data to be synthesized based on the judgment result of the judgment unit, wherein the synthesis unit performs the following operations: when the judgment unit judges that the image size of the focus image data is the same as the image size of the image data to be synthesized, the new image data to be synthesized having the same image size as the image data to be synthesized is created by synthesizing the focus image data and the image data to be synthesized, and when the judgment unit judges that the image size of the focus image data is different from the image size of the image data to be synthesized, the synthesis position of the image represented by the focus image data and the image represented by the image data to be synthesized and the size of the image represented by the synthesized image data are set, and the new image data to be synthesized is created based on the settings.

[0013] The present invention provides, in its eighth aspect, a control method for an imaging device, wherein the imaging device is capable of exchanging lens units, the imaging device includes an imaging unit, and the imaging device is used to create a composite image based on multiple images obtained by performing multiple imaging operations with the imaging unit, the control method comprising: a detection step for detecting reinstallation of the lens unit; a determination step for determining an area based on an image circle of the reinstalled lens unit when reinstallation of the lens unit is detected in the detection step, wherein image data of an image sensor included in the imaging unit is extracted relative to the area; an acquisition step for acquiring image data related to the area determined in the determination step from the image sensor; a judgment step for judging whether the image size of the image data of interest acquired in the acquisition step is the same as the image size of the image data to be synthesized when the image data of interest acquired in the acquisition step and the image data to be synthesized are synthesized; and and a creation step for creating new image data to be synthesized by synthesizing the focus image data and the image data to be synthesized based on the judgment result of the judgment step, wherein in the creation step, when it is judged in the judgment step that the image size of the focus image data is the same as the image size of the image data to be synthesized, the new image data to be synthesized having the same image size as the image data to be synthesized is created by synthesizing the focus image data and the image data to be synthesized, and when it is judged in the judgment step that the image size of the focus image data is different from the image size of the image data to be synthesized, the synthesis position of the image represented by the focus image data and the image represented by the image data to be synthesized is set, the size of the image represented by the synthesized image data is set, and the new image data to be synthesized is created based on the settings.

[0014] The present invention provides, in its ninth aspect, a non-transitory computer-readable storage medium storing a program, which, when read and executed by a computer, causes the computer to execute the steps of a control method for an imaging device, the imaging device being capable of exchanging lens units, the imaging device including an imaging unit, and the imaging device being used to create a composite image based on a plurality of images obtained by performing multiple imaging operations by the imaging unit, the control method comprising: a detection step for detecting reinstallation of the lens unit; a determination step for determining, when reinstallation of the lens unit is detected in the detection step, an area, wherein image data of an image sensor included in the imaging unit is extracted relative to the area; an acquisition step for acquiring, from the image sensor, image data related to the area determined in the determination step; and a judgment step for judging, when the image data of interest acquired in the acquisition step and the image data to be synthesized are synthesized, the image size of the image data of interest. whether the image size is the same as the image size of the image data to be synthesized; and a creation step for creating new image data to be synthesized by synthesizing the image data of interest and the image data to be synthesized based on the judgment result of the judgment step, wherein in the creation step, when it is judged in the judgment step that the image size of the image data of interest is the same as the image size of the image data to be synthesized, the new image data to be synthesized having the same image size as the image data to be synthesized is created by synthesizing the image data of interest and the image data to be synthesized, and when it is judged in the judgment step that the image size of the image data of interest is different from the image size of the image data to be synthesized, the synthesis position of the image represented by the image data of interest and the image represented by the image data to be synthesized is set, the size of the image represented by the synthesized image data is set, and the new image data to be synthesized is created based on the settings.

[0015] Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings). BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram of the structure of an image pickup apparatus according to an embodiment.

[0017] Figure 2 : is a diagram showing an example of a setting menu screen related to multiple exposure imaging.

[0018] Figures 3A to 3C : is a diagram showing an example of an image to be captured using multiple exposure imaging.

[0019] Figure 4 is a flowchart showing the operation of the image pickup apparatus according to the first embodiment.

[0020] Figure 5A and 5B : is a diagram showing the difference in image circles according to lens units.

[0021] Figure 6 is a diagram showing one example of a warning message according to the first embodiment.

[0022] Figure 7 is a flowchart showing the operation of the image pickup apparatus in the second embodiment.

[0023] Figure 8 is a diagram showing one example of a warning message in the second embodiment.

[0024] Fig. 9 is a flowchart showing the operation of the image pickup apparatus in the third embodiment.

[0025] Fig.10 is a flowchart showing the operation of the image pickup apparatus in the fourth embodiment.

[0026] Fig.11A and 11B is a diagram showing the relationship between the image circle and the size of the obtained image.

[0027] Fig.12 : is a flowchart showing the processing when the lens unit is replaced in the fifth embodiment.

[0028] Fig.13 Detailed description of the process of performing multiple exposure shooting in the fifth embodiment.

[0029] Fig.14 : is a diagram showing one example of a synthesis sequence of image synthesis processing in the fifth embodiment.

[0030] Fig.15 : is a diagram showing one example of a synthesis sequence of image synthesis processing in the fifth embodiment.

[0031] Fig.16 : is a diagram showing one example of a synthesis sequence of image synthesis processing in the fifth embodiment.

[0032] Fig.17 : is a diagram showing one example of a synthesis sequence of image synthesis processing in the fifth embodiment.

[0033] Fig.18 is a flowchart of the multiple synthesis processing in the fifth embodiment.

[0034] Fig.19is a flowchart showing the process for specifying a combining position in the fifth embodiment.

[0035] Fig. 20A and 20B : is a diagram showing an example of a GUI when a combining position is specified in the fifth embodiment. DETAILED DESCRIPTION

[0036] Hereinafter, embodiments of the present invention will be described in detail based on the accompanying drawings. Note that the structures shown in the following embodiments are merely examples, and the present invention is not limited to the structures shown.

[0037] Figure 1 1 is a block diagram of the structure of an image pickup apparatus represented by an interchangeable lens type digital camera in this embodiment. The image pickup apparatus in this embodiment includes an image pickup apparatus body 100 and a lens unit 180 that can be attached to / detached from the image pickup apparatus body 100.

[0038] In the main body 100 of the imaging apparatus, the shutter 120 controls the exposure amount of the imaging device 122 described later. The imaging device 122 includes an image sensor including a CCD or CMOS element for converting an optical image into an electrical signal. R, G, and B color filters are arranged on the surface of the image sensor in a Bayer arrangement. Therefore, image data immediately after the image is obtained by the imaging device 122 includes pixel data obtained from the pixels of the Bayer arrangement. In addition, the imaging device 122 has an A / D conversion processing function. The AF evaluation value detection unit 123 calculates the AF evaluation value based on the contrast information obtained from the digital image signal or the phase difference obtained from the parallax image, and the imaging device 122 outputs the obtained AF evaluation value to the system control unit 150. Note that the image sensor in the present embodiment is a so-called full-size sensor.

[0039] The image processing unit 124 performs predetermined pixel interpolation processing, resizing processing such as reduction, and color conversion processing on the image data output from the imaging device 122 or the image data from the memory control unit 115. In addition, the image processing unit 124 can obtain information related to the distance between the subject and the imaging device. Specifically, the image processing unit 124 obtains the distance to the subject by detecting the phase difference between two given parallax images, and obtains information related to the distance from the imaging device to the subject for each pixel. In addition, the image processing unit 124 performs predetermined calculation processing using the image data obtained by capturing the image, and the system control unit 150 performs exposure control and distance measurement control based on the calculation result. Therefore, TTL (through the lens) type AE (automatic exposure) processing and EF (automatic flash adjustment) processing are performed. In addition, the image processing unit 124 performs AF (auto focus) processing, and the output of the AF evaluation value detection unit 123 included in the imaging device 122 can be used when performing the AF processing. In addition, the image processing unit 124 performs predetermined calculation processing using image data obtained by capturing an image, and performs TTL type AWB (auto white balance) processing using the obtained calculation result. In addition, the image processing unit 124 generates a multiplexed image by performing processing for synthesizing a plurality of images. In addition, the image processing unit 124 converts the RAW image data of the Bayer arrangement into image data including three components for each pixel by performing debayer processing, and also performs encoding and decoding processing.

[0040] The output data of the imaging device 122 is written to the memory 132 via the image processing unit 124 and the memory control unit 115, or is directly written to the memory 132 via the memory control unit 115. The memory 132 stores the image data acquired by the imaging device 122 and subjected to A / D conversion, and the image data for display in the display unit 128. The memory 132 has a sufficient storage capacity for storing a predetermined number of still images and a predetermined length of moving images and audio. In addition, the memory 132 is used as a memory for image display (video memory). The D / A converter 113 converts the data for image display stored in the memory 132 into an analog signal, and supplies the analog signal to the display unit 128. In this way, the image data for display written in the memory 132 is displayed in the display unit 128 via the D / A converter 113. The display unit 128 displays on a display such as an LCD according to the analog signal from the D / A converter 113. The digital signal obtained by A / D conversion of the imaging device 122 and stored in the memory 132 is converted into an analog signal by the D / A converter 113, and the analog signal is continuously sent to the display unit 128 for display, so that an electronic viewfinder that displays a through-the-lens image can be realized.

[0041] The nonvolatile memory 156 is an electrically erasable and recordable memory, and is a flash memory or the like. For example, constants and programs for operating the system control unit 150 are stored in the nonvolatile memory 156. In this context, "program" may refer to a program for executing various flowcharts to be described later. In addition, the nonvolatile memory 156 stores a lens table 156a in which the model names of lens units that can be connected to the main body 100 of the image pickup apparatus are associated with feature information of the lens units. The feature information of the lens unit includes the size of the image circle.

[0042] The system control unit 150 controls the processing unit of the main body 100 of the imaging apparatus as a whole by executing the program stored in the above-mentioned nonvolatile memory 156. Reference numeral 152 denotes a system memory, and is a RAM. Constants and variables for operating the system control unit 150 and programs read out from the nonvolatile memory 156 and the like are arranged in the system memory 152.

[0043] The system timer 153 is a timer unit for measuring the time period of various types of control and the time of the built-in clock. The mode switching switch 160, the first shutter switch 162 and the second shutter switch 163 included in the shutter button 161, and the operation unit 170 are used as operation members for inputting various instructions of the system control unit 150. The mode switching switch 160 switches the operation mode of the system control unit 150 to any one of the still image recording mode, the moving image recording mode, and the reproduction mode. The still image recording mode includes an automatic shooting mode, an automatic scene determination mode, a manual mode, various scene modes that configure different settings for each shooting scene, a program AE mode, a custom mode, and a multiple exposure camera mode. By using the mode switching switch 160, the mode can be directly switched to any mode of these modes. Alternatively, it is also possible to switch to the still image recording mode using the mode switching switch 160, and then switch to any mode of these modes included in the still image recording mode using another operation member. Similarly, the moving image recording mode can also include a plurality of modes.

[0044] When the shutter button 161 provided in the main body 100 of the image pickup apparatus is operated, i.e., half-pressed (shooting preparation instruction), the first shutter switch 162 is turned on and generates a first shutter switch signal SW1. Upon receiving the first shutter switch signal SW1, the system control unit 150 starts AF (auto focus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, EF (auto flash adjustment) processing, and the like.

[0045] When the operation of the shutter button 161 is completed, that is, the shutter button 161 is fully pressed (shooting instruction), the second shutter switch 163 is turned on and generates a second shutter switch signal SW2. Upon receiving the second shutter switch signal SW2, the system control unit 150 starts a series of shooting processes from reading out the signal from the imaging device 122 to writing the image data to the recording medium 190.

[0046] By selecting various function icons displayed on the display unit 128, appropriate functions for each situation are assigned to the operation members of the operation unit 170, and the operation members thus serve as various function buttons. Examples of these function buttons include an end button, a return button, an image scroll button, a jump button, a zoom-out button, and an attribute change button. For example, by pressing the menu button, a menu screen that enables various settings to be set is displayed on the display unit 128. The user can intuitively set various settings by using the menu screen displayed in the display unit 128, the four-way (up, down, left, right) button, and the "Set" button. The menu screen that enables various settings to be set includes a screen for selecting an image that has been captured in the past and recorded as a basic image in the recording medium 190 when the multiple exposure imaging mode is selected, and it can also be multiple exposure imaging that synthesizes a captured image on the selected basic image. In addition, a screen for enabling selection of whether to crop the size of the image to be read out as a captured image to the APS-C sensor size is also included, and if the setting for cropping is selected, it is possible to shoot in the APS-C sensor size even in the case where a lens unit of a full-size sensor is mounted. Will be used later Figure 5A and 5B The cropping process will be described below.

[0047] The power control unit 141 includes, for example, a battery detection circuit, a DC / DC converter, and a switch circuit for switching blocks to be powered, and detects whether a battery has been inserted, the type of the battery, and its remaining capacity. In addition, the power control unit 141 controls the DC / DC converter according to the detection result and the instruction of the system control unit 150, and supplies a required voltage for a required length of time to each unit including the recording medium 190.

[0048] The power supply unit 140 includes a main battery such as an alkaline battery or a lithium battery, a sub-battery such as a NiCd battery, a NiMH battery, or a lithium ion battery, or an AC adapter, etc. The storage medium I / F 118 is an interface with a recording medium 190 such as a memory card or a hard disk. The recording medium 190 is a storage medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, etc.

[0049] The connector 171 electrically connects the main body 100 of the image pickup apparatus to the lens unit 180. The connector 171 exchanges control signals, status signals, data signals, etc. between the main body 100 of the image pickup apparatus and the lens unit 180, and also has a function of supplying current at various voltages. In addition, whether the lens unit 180 is mounted to the connector 171 is detected by the lens unloading / mounting detection unit 154.

[0050] Next, the structure of the lens unit 180 will be described. The image pickup lens 181 is composed of a plurality of lenses including a focus lens for performing focus adjustment by moving in the optical axis direction and a zoom lens for performing zooming. In addition, the aperture 182 adjusts the amount of light to the image pickup device 122 of the main body 100 of the image pickup apparatus. The lens unit control unit 183 controls the entire lens unit 180 according to a control signal from the system control unit 150 of the main body 100 of the image pickup apparatus, and controls the driving of the focus lens and the zoom lens of the image pickup lens 181 and the aperture 182. In addition, the lens unit control unit 183 includes a memory for storing constants and variables used for operation, programs, and the like. In addition, the lens unit control unit 183 also includes a non-volatile memory for storing identification information such as a model name specific to the lens unit 180 and its manufacturing number, management information, functional information such as a maximum aperture, a minimum aperture, a focal length, an aberration amount (such as distortion aberration), and current and past setting values, and the like. The connector 184 electrically connects the lens unit 180 to the imaging apparatus body 100. The connector 184 exchanges control signals, status signals, data signals, and the like between the imaging apparatus body 100 and the lens unit 180, and also has a function of receiving currents having various voltages.

[0051] Next, use Figure 2 and 3 to describe the basic operation of multiple-exposure photography. Figure 2 3 is a setting menu screen for multiple-exposure imaging, and FIG. 4 shows images to be captured in the multiple-exposure imaging.

[0052] When the user presses the menu button of the operation unit 170, a menu screen for various settings is displayed. When the user selects a setting item for multiple exposure photography by operating the four-way button of the operation unit 170, the menu screen for various settings is displayed on the display unit 128. Figure 2 A submenu screen 200 for multiple-exposure imaging is shown, and the user can change the settings of multiple-exposure imaging.

[0053] The multiple exposure image capturing menu screen 200 includes setting items 201 to 204 and a button 205 for selecting a reference image. Among these items, the setting item 201 is a setting item for selecting whether to perform multiple exposure image capturing, and one of "Yes" and "No" is selected. If the user exits the menu after selecting "Yes" and returns to shooting, the multiple exposure image capturing mode is entered.

[0054] Setting item 202 is a setting item for selecting a method of overlapping exposure when performing multiple exposure photography, and the user can select one of "Addition", "Arithmetic Mean", "Comparison (Bright)", and "Comparison (Dark)". When "Addition" is selected, the brightness of each of the multiple images to be synthesized is simply added. When "Arithmetic Mean" is selected, the brightness of each of the multiple images to be synthesized is added to obtain a brightness that is the average brightness of the image to be overlapped described later. When "Comparison (Bright)" is selected, overlapping is performed so that the brightness is compared between the images to be synthesized, and the brighter part is retained. When "Comparison (Dark)" is selected, overlapping is performed so that the brightness is compared between the images to be synthesized, and the darker part is retained.

[0055] Setting item 203 is a setting item for selecting the number of images to be synthesized when performing multiple exposure photography, and the user can specify the number of images to be overlapped. In the multiple exposure photography mode, once the set number of images are taken and the synthesized image is recorded, the mode returns from the multiple exposure photography mode to the normal photography mode.

[0056] The setting item 204 is a setting item for selecting images to be recorded to the recording medium 190 in the multiple exposure imaging, and the user can select one of “all images” and “multiplexed images only”. When “all images” is selected, both the multiple images on which overlap is performed and the multiple exposure composite image are recorded in the recording medium 190. When “multiplexed images only” is selected, only the multiple exposure composite image is recorded in the recording medium 190, and the individual images on which overlap is performed are not recorded.

[0057] When the user presses button 205, one image file already stored in recording medium 190 will be selected, and a multiple-exposure composite image will be created and recorded. Note that if the user sets "5" in setting item 203, and selects one recorded image file (RAW image file) by pressing button 205, a multiple-exposure composite image is created using the image indicated by the selected image file as the first image and the remaining four images to be captured.

[0058] Note that the structure may enable selection of two or more image files. In this case, for example, if two image files are selected, a multiple exposure composite image can be created based on the two images indicated by the selected two files and the three images to be captured later.

[0059] Figure 3A Reference numeral 301 in FIG. 1 denotes a first captured image in the multiple exposure imaging mode, and Figure 3B Reference numeral 302 in FIG. 1 represents a second captured image. Figure 3C Reference numeral 303 in FIG. 1 denotes a multiple-exposure composite image at the end of the second shooting.

[0060] First embodiment

[0061] Figure 4 1 is a flowchart showing the processing procedure of the system control unit 150 in the multiple exposure imaging mode according to the first embodiment. When the user selects the multiple exposure imaging mode using the mode switching switch 160 or by selecting multiple exposure imaging in the menu setting, the main body 100 of the imaging apparatus enters the multiple exposure imaging mode and performs Figure 4 in the processing. Figure 4 The processing in each step in is realized by the system control unit 150 developing a predetermined program read out from the nonvolatile memory 156 in the system memory 152 and controlling the operation and processing of each unit constituting the main body 100 of the image pickup apparatus. Note that it is assumed that the user has made various settings (such as the number of images to be synthesized and the designation of image files to be synthesized) and the set information is stored in the system memory 152. Note that the structure may be such that the number of images to be synthesized when the previous multiple-exposure image pickup is performed is the number of images to be synthesized this time unless otherwise changed. In this case, the number of images to be synthesized may be stored in the nonvolatile memory 156.

[0062] In step S401, the system control unit 150 acquires the number N of images to be multiple-synthesized, which is one of setting information in the multiple-exposure imaging mode, from the system memory 152 or the nonvolatile memory 156. Next, in step S402, the system control unit 150 determines whether basic image selection has been performed, in which a basic image is selected as the first image from the captured images, in order to start multiple-exposure imaging, based on information stored in the system memory 152 or the nonvolatile memory 156. If it is determined that basic image selection has not been performed (No in step S402), the system control unit 150 advances the processing to step S403, and substitutes an initial value "0" into a variable n for counting the number of shots. On the other hand, if it is determined that basic image selection has been performed (Yes in step S402), the system control unit 150 advances the processing to step S404, and substitutes an initial value "1" into the variable n.

[0063] In step S405, the system control unit 150 determines whether remounting of the lens unit has been performed based on a signal from the lens unmounting / mounting detection unit 154. Remounting of the lens unit includes, for example, a case where a mounted lens unit is removed once and then mounted again, and a case where a mounted lens unit is replaced with a lens unit different from the mounted lens unit.

[0064] If it is determined that the lens unit has been remounted (YES in step S405), the system control unit 150 advances the processing to step S406, and if it is determined that the lens unit has not been remounted (NO in step S405), the system control unit 150 advances the processing to step S409.

[0065] In step S406, the system control unit 150 determines whether the variable n is "0" (i.e., no base image is selected and no image is captured after entering the multiple-exposure imaging mode). In other words, the system control unit 150 determines whether there is an image to be subjected to multiple synthesis. If it is determined that the variable n is "0" ("Yes" in step S406), the system control unit 150 advances the processing to step S409. On the other hand, if the variable n is not "0" ("No" in step S406), the system control unit 150 advances the processing to step S407.

[0066] In step S407 , the system control unit 150 determines whether the image circle of the newly mounted lens unit is different from the image circle of the previously mounted lens unit.

[0067] Specifically, the system control unit 150 can communicate with the mounted lens unit 180 via the connector 171 and acquire identification information such as a model name, etc. In addition, the system control unit 150 can acquire the image circle of the mounted lens unit by searching in the lens table 156a in the nonvolatile memory 156 using the acquired identification information. The system control unit 150 performs the determination processing in step S407 by comparing the image circle of the lens unit before remounting the lens unit (assuming that it is stored in a predetermined address of the nonvolatile memory 156) and the image circle of the lens unit after remounting.

[0068] Note that the determination processing in step S407 may be performed based on whether or not it is necessary to switch the size of the image to be read out from the imaging device 122 and to be recorded according to the mounted lens unit. Furthermore, the determination processing may be performed based on whether or not the amount of vignetting in the peripheral portion of the image read out from the imaging device 122 detected by scanning using the image processing unit 124 or the system control unit 150 is different from the previous amount of vignetting.

[0069] Here, we will use Figure 5A and 5B to describe the differences in the image circle. Figure 5A and 5B Reference numerals 501 and 502 in the figure show the manner in which light that has passed through the lens unit forms an image on the image sensor of the imaging device 122. Reference numeral 501 shows the case where a lens unit with a full-size sensor is attached, and reference numeral 502 shows the case where a lens unit with an APS-C size sensor is mounted. As described above, the image sensor in the present embodiment is a full-size sensor. However, when a lens unit of an APS-C size sensor is mounted, light is not incident on the peripheral portion of the image sensor, and "vignetting" occurs around the image. Therefore, when a lens unit of an APS-C size sensor is mounted, the system control unit 150 performs a cropping process by controlling the imaging device 122, in which the image of the area indicated by the illustrated reference numeral 503 inside the image sensor is cropped out as a captured image. As a result of the cropping process performed in this manner, although the size of the image is reduced, an image with the vignetting portion removed can be obtained.

[0070] On the other hand, in the case of multiple exposure shooting, since the images are synthesized as shown in FIG. 3, when a lens unit having a different image circle is mounted during multiple exposure shooting, if synthesis is performed as usual, the vignetted portion is also synthesized, which results in an unnatural synthesized image. Furthermore, when the size of the readout image is reduced to the APS-C sensor size (i.e., the portion where light is incident is cropped and readout), the size of the image is different and synthesis cannot be performed.

[0071] Therefore, in the present embodiment, if it is determined that the image circle of the remounted lens unit is the same as the image circle of the lens unit mounted last time (No in step S407), the system control unit 150 advances the processing to step S409. Also, if it is determined that the image circle of the remounted lens unit is different from the image circle of the lens unit mounted last time (Yes in step S407), the system control unit 150 advances the processing to step S408 to interrupt the subsequent image capturing processing.

[0072] In step S408, the system control unit 150 displays a message indicating that the multiple exposure imaging has ended on the display unit 28 ( Figure 6 ). Furthermore, the system control unit 150 advances the process to step S415 to record the composite image obtained by the capturing that has been performed to the recording medium 190 as an image file.

[0073] In step S409, the system control unit 150 determines whether the second shutter switch 163 is turned on (whether the second shutter switch signal SW2 is generated). If it is determined that the second shutter switch 163 is not turned on (No in step S409), the system control unit 150 returns the process to step S405 and waits for the second shutter switch 163 to be turned on. Then, if it is determined that the second shutter switch 163 is turned on, the system control unit 150 advances the process to step S410.

[0074] In step S410, the system control unit 150 uses the imaging device 122 to shoot in a still image frame, stores the obtained image data (RAW image data of the Bayer arrangement) in the memory 132, and advances the process to step S411. In step S411, the system control unit 150 increments the value of the variable n by "1", and advances the process to step S412.

[0075] In step S412, the system control unit 150 determines whether the value of the variable n is greater than "1". That is, the system control unit 150 determines whether an image to be processed in the synthesis process already exists. If it is determined that the variable n is "1" or less ("No" in step S412), the system control unit 150 returns the process to step S405. In addition, if it is determined that the variable n is greater than "1" ("Yes" in step S412), the system control unit 150 advances the process to step S413.

[0076] In step S413, the system control unit 150 synthesizes the plurality of pieces of image data obtained by capturing the image in step S410 and stored in the memory 132 by controlling the image processing unit 24, generates synthesized image data, and stores the synthesized image data in the memory 132. If the variable n is greater than "2", that is, if the image obtained by capturing in step S410 is the third or later image, the system control unit 150 synthesizes the image data obtained by capturing the image with the synthesized image data obtained in the previous synthesizing process, creates new synthesized image data, and stores the newly created synthesized image data in the memory 132. When the process in step S413 ends, the system control unit 150 advances the process to step S414. Note that in the synthesizing process in step S413, the image data using the Bayer arrangement is synthesized as usual.

[0077] In step S414, the system control unit 150 determines whether the variable n is equal to "N", that is, whether to end the multiple exposure shooting. If it is determined that the variable n is not equal to "N" ("No" in step S414), the system control unit 150 returns the processing to step S405. Otherwise, if it is determined that the variable n is equal to "N", the system control unit 150 advances the processing to step S415.

[0078] In step S415, the system control unit 150 performs development processing (including debayer processing) (i.e., processing such as white balance adjustment and pixel interpolation) on the synthesized image data created in step S413 by controlling the image processing unit 124, converts the obtained image data into YUV data, and writes the YUV data into the memory 132. After completing the processing in step S415, the system control unit 150 advances the processing to step S416.

[0079] In step S416, the system control unit 150 that controls the image processing unit 24 compression-encodes the development-processed composite image data created in step S415 into JPEG-formatted still image data, writes the JPEG-formatted still image data into the memory 132, and advances the process to step S417.

[0080] In step S417, the system control unit 150 records the still image data in the JPEG format compression-encoded in step S416 to the recording medium 190 via the storage medium I / F 118, and ends a series of multiple-exposure imaging. Figure 2 If “save captured image” is selected in the setting item 204 in , the captured image (the image obtained in step S410 ) that has been stored in the memory 132 and used for synthesis is also recorded in the recording medium 190 .

[0081] As described above, in a case where re-mounting of a new lens unit is detected during multiple-exposure imaging and the image circle of the newly mounted lens unit is different from the image circle of the lens unit mounted last time, a message is displayed and the multiple-exposure imaging is terminated. Therefore, in a case where a lens unit having an image circle different from the image circle of the lens unit mounted last time is mounted during multiple-exposure imaging, incorrect multiplexed images can be prevented from being synthesized. For example, in a case where multiple-exposure imaging is started with a lens unit of a full-size sensor mounted and the lens unit is switched to a lens unit of an APS-C size sensor during multiple-exposure imaging, incorrect multiplexed images can be prevented from being synthesized. In addition, synthesized images using images obtained by shooting before switching the lens unit can be stored in the recording medium 190, although the number thereof is less than the N that was initially set.

[0082] Second embodiment

[0083] Next, the processing of the image pickup apparatus according to the second embodiment will be described. The configuration of the image pickup apparatus according to the second embodiment is similar to that of the first embodiment ( Figure 1 ) and its detailed description will be omitted.

[0084] In the first embodiment, in the case where a new lens unit is mounted during multiple-exposure photography, the lens characteristic acquired by the device is the image circle of the lens, and if the image circle of the newly mounted lens unit is different from the image circle of the lens unit mounted last time, a message is displayed and the multiple-exposure photography is ended. In contrast, in the second embodiment, in the case where a new lens unit is mounted during multiple-exposure photography and the image circle of the newly mounted lens unit is different from the image circle of the lens unit mounted last time, a warning message is displayed, and if a lens unit having the same image circle is mounted again, photography can be continued. In this way, the device controls creation of a composite image based on the acquired lens characteristic (in this case, the image circle). Thus, even if a lens unit having an image circle different from that of the lens unit mounted last time is mounted, incorrect multiplexed images can be prevented from being synthesized. If a lens unit having an image circle different from that of the lens unit mounted last time is mistakenly mounted, photography will not be ended, and a series of multiple-exposure photography can be continued after the correct lens unit is remounted.

[0085] Figure 7 1 is a flowchart showing the processing procedure of the system control unit 150 in the multiple-exposure imaging mode according to the second embodiment. Note that it is assumed that the user has already made various settings related to the multiple-exposure imaging (such as the number of images to be synthesized and the designation of image files to be synthesized), and the set information is stored in the system memory 152 or the nonvolatile memory 156.

[0086] Figure 7 The processing in steps S701 to S707 and steps S709 to S717 in Figure 4 The processing in steps S401 to S407 and steps S409 to S417 is the same, so their description will be omitted. Figure 7 Steps S721 to S726 in step S707, i.e., the steps after the judgment of "No" in step S707, (corresponding to Figure 4 The processing in the step after the judgment of "No" in step S407 is the same as Figure 4 The processing is different in .

[0087] In step S721, the system control unit 150 displays a message ( Figure 8 ) for prompting the user to switch the lens unit again because the mounted lens unit has an image circle different from that of the lens unit mounted last time. In addition, the system control unit 150 displays a menu in the display unit 128, with which the user can select whether to record a composite image obtained by shooting performed before remounting and end multiple-exposure imaging, and whether to discard the composite image and end multiple-exposure imaging.

[0088] In step S722, the system control unit 150 determines whether the user has selected an option to end multiple-exposure imaging using the menu displayed in step S721. If it is determined that the user has not selected the option to end multiple-exposure imaging (No in step S722), the system control unit 150 advances the processing to step S724, and if it is determined that the user has selected the option to end multiple-exposure imaging (Yes in step S722), the system control unit 150 advances the processing to step S723.

[0089] In step S723, the system control unit 150 determines whether the user has selected an option to record a composite image obtained by shooting performed before remounting using the menu displayed in step S721. If it is determined that the user has not selected the option to record a composite image (No in step S723), the system control unit 150 ends a series of multiple-exposure imaging. Also, if it is determined that the user has selected the option to record a composite image (Yes in step S723), the system control unit 150 advances the processing to step S715.

[0090] In step S724, the system control unit 150 determines whether the lens unit has been remounted using the lens unmount / mount detection unit 154. If it is determined that the lens unit has been remounted ("Yes" in step S724), the system control unit 150 advances the process to step S725, and if it is determined that the lens unit has not been remounted ("No" in step S724), the system control unit 150 returns the process to step S721 to wait for remounting.

[0091] In step S725, the system control unit 150 determines whether the image circle of the lens unit that has been remounted is different from the image circle of the lens unit used in the immediately preceding shooting. If it is determined that the image circle of the remounted lens unit is still the same as the image circle of the lens unit used in the immediately preceding shooting ("Yes" in step S725), the system control unit 150 causes the processing to proceed to step S726. If it is determined that the image circle of the lens unit that has been remounted is different from the image circle of the lens unit used in the immediately preceding shooting ("No" in step S725), the system control unit 150 causes the processing to return to step S721.

[0092] In step S726 , the system control unit 150 hides the warning message displayed in step S721 , returns the process to step S705 , and continues a series of multiple-exposure imaging.

[0093] The processing of a series of multiple exposure imaging from step S709 to step S717 is the same as the processing in steps S409 to S417 according to the first embodiment, and thus a description thereof will be omitted.

[0094] According to the second embodiment as described above, in the case where a lens unit having an image circle different from that of the lens unit installed last time is installed during multiple exposure photography, a warning message is displayed, and a series of multiple exposure photography can be ended according to the user's selection. In addition, in the case where the user does not select an option to end a series of multiple exposure photography and reinstalls a lens unit having the same image circle, a series of multiple exposure photography can be continued. Thus, in the case where a lens unit having an image circle different from that of the lens unit installed last time is installed, incorrect multiple images can be prevented from being synthesized, and if such a lens unit is mistakenly installed, shooting will not be ended, and a series of multiple exposure photography can be continued after another lens unit is installed. For example, in the case where multiple exposure photography is started with a lens unit of a full-size sensor installed and the lens unit is switched to a lens unit of an APS-C size sensor during multiple exposure photography, incorrect multiple images can be prevented from being synthesized. In addition, if a lens unit of a full-size sensor is reinstalled, a series of multiple exposure photography can be continued.

[0095] Third embodiment

[0096] Next, the processing of the image pickup apparatus according to the third embodiment will be described. The configuration of the image pickup apparatus according to the third embodiment is similar to that of the first embodiment ( Figure 1 ) and its detailed description will be omitted.

[0097] In the first embodiment, when a new lens unit is mounted during multiple-exposure imaging, if the image circle of the mounted lens unit is different from the image circle of the previously mounted lens unit, a message is displayed and the multiple-exposure imaging is ended.

[0098] Likewise, in the third embodiment, in the case where a new lens unit is mounted during multiple-exposure imaging, the device acquires lens characteristics indicating whether the lens unit requires predetermined image correction, and if the newly mounted lens unit enables settings for image correction such as distortion correction, that is, if a lens unit for which image correction is necessary is mounted, a message is displayed, and multiple-exposure imaging is terminated. In this way, the device controls creation of a composite image based on the acquired lens characteristics (in this case, indicating that the lens unit requires predetermined image correction). Thus, in the case where a lens unit for which image correction is necessary is mounted during multiple-exposure imaging, creation of an unnatural multiplexed image due to synthesis of an image for which image correction is not necessary and an image for which image correction is necessary can be prevented.

[0099] Fig. 9 1 is a flowchart showing the processing procedure of the system control unit 150 in the multiple exposure image capturing mode in the third embodiment. Note that it is assumed that the user has made various settings related to multiple exposure image capturing (such as the number of images to be synthesized and the designation of image files to be synthesized), and the set information is stored in the system memory 152 or the nonvolatile memory 156. In addition, when the multiple exposure image capturing mode is started, the settings of image correction such as distortion correction are fixed to be invalid, and the multiple exposure image capturing mode is started after the settings are thus configured.

[0100] Fig. 9 The processing in steps S901 to S905 and steps S909 to S917 in Figure 4 The processing in steps S401 to S405 and steps S409 to S417 is the same, so their description will be omitted. Fig. 9 Steps S921 to S923 (corresponding to steps S921 to S923) in the above procedure as steps after the judgment of "No" in step S905 Figure 4 The processing in the step after the judgment of "No" in step S405 is the same as Figure 4 The processing is different in .

[0101] In step S921, the system control unit 150 determines whether the newly mounted lens unit needs to make the setting of the image correction effective, that is, whether a lens unit that must perform image correction is installed. Whether the setting of the image correction needs to be fixed to be effective can be determined based on the lens table 156a that stores information on whether correction is required in association with the lens unit identification information (model name). The system control unit 150 can make the determination by receiving the identification information from the mounted lens unit via the connector 171 and acquiring information on whether the lens unit needs to be corrected from the lens table 156a. In addition, depending on the situation, the system control unit 150 can make the determination using information on the correction function of the distortion aberration, etc. of the lens unit 180 acquired from the lens unit 180 via the connector 171. In addition, the system control unit 150 can make the determination by scanning the image read out from the imaging device 122 by itself or using the image processing unit 124 and detecting the aberration amount of the image.

[0102] If it is determined that the newly mounted lens unit does not need to have the image correction setting fixed to be enabled (No in step S921), the system control unit 150 advances the process to step S909. If it is determined that the newly mounted lens unit needs to have the image correction setting fixed to be enabled (Yes in step S921), the system control unit 150 advances the process to step S922.

[0103] In step S922, the system control unit 150 displays a message indicating that the multiple-exposure imaging will be ended in the display unit 128, and advances the processing to step S923. In step S923, the system control unit 150 determines whether the value of the variable n is greater than "1", that is, whether at least two images have been captured for the multiple-exposure imaging. If the variable n is greater than "1" ("Yes" in step S923), the system control unit 150 advances the processing to step S915. If it is determined that the variable n is "1" or less ("No" in step S923), the system control unit 150 ends a series of multiple-exposure imaging.

[0104] The processing from still image capturing in step S909 to the end of a series of multiple exposure image capturing in step S917 is the same as the processing in steps S409 to S417 according to the first embodiment, and thus a description thereof will be omitted.

[0105] According to the third embodiment as described above, in the case where a new lens unit is mounted during multiple-exposure imaging, if the mounted lens unit has the setting of image correction such as distortion correction fixed to be effective, that is, if a lens unit for which image correction must be performed is mounted, a message is displayed and multiple-exposure imaging is terminated. Therefore, in the case where a lens unit for which image correction must be performed is mounted during multiple-exposure imaging in which the setting of image correction is fixed to be ineffective, it is possible to prevent incorrect multiplexed images from being synthesized. For example, in the case where the lens unit is switched to a lens unit for which distortion aberration is large despite its low cost and it is necessary to have the setting of image correction fixed to be effective during multiple-exposure imaging, it is possible to prevent incorrect multiplexed images from being created.

[0106] Fourth embodiment

[0107] Next, the processing of the image pickup apparatus according to the fourth embodiment will be described. The configuration of the image pickup apparatus according to the fourth embodiment is similar to that of the first embodiment ( Figure 1 ) and its detailed description will be omitted.

[0108] In the third embodiment, in the case where a newly mounted lens unit during multiple exposure photography requires that settings for image correction such as distortion correction be validated (i.e., in the case where a lens unit for which image correction must be performed is mounted), a message is displayed and multiple exposure photography is terminated. In contrast, in the fourth embodiment, the device acquires lens characteristics of whether a lens unit requires predetermined image correction, and in the case where a lens unit for which image correction must be performed is mounted during multiple exposure photography, a warning message is displayed, and in the case where a lens unit for which image correction must be performed is remounted, photography can be continued. In this manner, the device controls creation of a composite image based on the acquired lens characteristics (in this case, indicating that the lens unit requires predetermined image correction). Thus, in the case where a lens unit for which image correction must be performed is mounted during multiple exposure photography for which settings for image correction are invalidated, it is possible to prevent unnatural multiplexed images from being synthesized. Furthermore, if such a lens unit is mistakenly mounted, photography is not terminated, and a series of multiple exposure photography can be continued after another lens unit is mounted.

[0109] Fig.10 1 is a flowchart showing the processing procedure of the system control unit 150 in the multiple exposure image capturing mode in the fourth embodiment. Note that it is assumed that the user has already made various settings related to multiple exposure image capturing (such as the number of images to be synthesized and the designation of image files to be synthesized), and the set information is stored in the system memory 152 or the nonvolatile memory 156. In addition, when the multiple exposure image capturing mode is started, the settings of image correction such as distortion correction are fixed to be invalid, and the multiple exposure image capturing mode is started after the settings are thus configured.

[0110] The processes in steps S1001 to S1005 and steps S1009 to S1017 are the same as the processes in steps S401 to S405 and steps S409 to S417 in the first embodiment, and thus description thereof will be omitted.

[0111] In step S1021, the system control unit 150 determines whether a lens unit for which image correction must be performed has been mounted. If it is determined that a lens unit for which image correction is not required is mounted (No in step S1021), the system control unit 150 advances the processing to step S1009. If it is determined that a lens unit for which image correction is required is mounted (Yes in step S1021), the system control unit 150 advances the processing to step S1022. In step S1022, since a lens unit for which image correction must be performed has been mounted, the system control unit 150 displays a message prompting the user to switch the lens unit again in the display unit 128. In addition, the system control unit 150 displays a menu in the display unit 128 with which the user can select to record a composite image obtained by shooting performed before re-mounting and end the processing of multiple-exposure imaging, or to discard the composite image and end the processing of multiple-exposure imaging, and the processing advances to step S1023.

[0112] In step S1023, the system control unit 150 determines whether the user has selected an option to end multiple-exposure imaging on the menu displayed in step S1022. If it is determined that the user has not selected an option to end multiple-exposure imaging (No in step S1023), the system control unit 150 advances the processing to step S1025. Otherwise, if it is determined that the user has selected an option to end multiple-exposure imaging (Yes in step S1023), the system control unit 150 advances the processing to step S1024.

[0113] In step S1024, the system control unit 150 determines whether the user has selected an option to record a composite image obtained by shooting performed before remounting using the menu displayed in step S1022. If it is determined that the user has not selected the option to record a composite image (No in step S1024), the system control unit 150 ends a series of multiple-exposure imaging. Also, if it is determined that the user has selected the option to record a composite image (Yes in step S1024), the system control unit 150 advances the processing to step S1015.

[0114] In step S1025, the system control unit 150 determines whether the lens unit has been remounted using the lens unmount / mount detection unit 154. If it is determined that the lens unit has been remounted ("Yes" in step S1025), the system control unit 150 advances the process to step S1026, and if it is determined that the lens unit has not been remounted ("No" in step S1025), the system control unit 150 returns the process to step S1022.

[0115] In step S1026, the system control unit 150 determines whether the remounted lens unit needs to validate the image correction settings, that is, whether a lens unit that must perform image correction has already been mounted. If it is determined that the remounted lens unit does not need to validate the image correction settings (No in step S1026), the system control unit 150 advances the process to step S1027. If it is determined that the remounted lens unit needs to validate the image correction settings (Yes in step S1026), the system control unit 150 returns the process to step S1022.

[0116] In step S1027 , the system control unit 150 hides the warning message displayed in step S1022 , returns the process to step S1005 , and continues a series of multiple-exposure imaging.

[0117] A series of processing of multiple exposure image capturing from step S1009 to step S1017 is the same as the processing in steps S409 to S417 according to the first embodiment, and thus a description thereof will be omitted.

[0118] According to the fourth embodiment as described above, in the case where the lens unit installed during multiple exposure shooting requires the setting of image correction to be fixed to be effective (that is, a lens unit that must perform image correction is installed), a warning message is displayed, and a series of multiple exposure shooting can be ended according to the user's selection. In addition, in the case where the user does not select the option to end the series of multiple exposure shooting and reinstalls the lens unit that does not require image correction, the series of multiple exposure shooting can be continued. Thus, in the case where the lens unit that must perform image correction is installed during multiple exposure shooting in which the setting of image correction is fixed to be invalid, it is possible to prevent incorrect multiplexed images from being synthesized. In addition, if such a lens unit is installed by mistake, shooting is not ended, and a series of multiple exposure shooting can be continued after another lens unit is installed.

[0119] Fifth embodiment

[0120] Next, the multiple exposure image capturing mode in the fifth embodiment will be described.

[0121] First, refer to Fig.11Aand 11B The relationship between the image sensor mounted in the image pickup device 122 and the image circle of the mounted lens unit will be described.

[0122] like Fig.11A As shown, if the image circle is larger than the image sensor surface (in the case of a full-size lens unit), a captured image using the entire image sensor (RAW image data of the Bayer arrangement) can be obtained.

[0123] On the other hand, Fig. 11B As shown in FIG. 1 , in the case of an APS-C lens unit, the size of the image circle is smaller than the size of the image sensor. Therefore, the rectangular range 1100 inscribed in the image circle must be set as the imaging range. That is, when the size of the image circle is smaller than the size of the image sensor, as shown in FIG. Fig. 11B As shown, the size of the captured image needs to be reduced to less than Fig.11A The size of the captured image is shown.

[0124] The system control unit 150 receives identification information (such as a model name, etc.) from the mounted lens unit 180 when the mounting of the lens unit 180 is detected based on a signal from the lens unmounting / mounting detection unit 154 or when performing initialization processing after power-on. In addition, the system control unit 150 searches the lens table 156a using the received identification information as a keyword and acquires the image circle of the mounted lens unit. Then, the system control unit 150 determines whether the lens unit 180 has been realized based on the acquired image circle. Fig.11A and 11B Which state is in . Fig.11A In the case of the state in, the system control unit 150 sets the image processing unit 124 so that the captured image is acquired using the entire area of ​​the image sensor. Fig. 11B In the case of the state in , the system control unit 150 sets the image processing unit 124 so as to acquire an image within a region 1100 inside the image sensor as a captured image.

[0125] Fig.12 1 is a flowchart showing processing of the system control unit 150 in the case where a detection signal indicating that a lens unit has been mounted is received from the lens unmounting / mounting detection unit 154.

[0126] In step S1201, the system control unit 150 communicates with the lens unit control unit 183 via the connector 171. In step S1202, the system control unit 150 acquires identification information of the lens unit such as a model name, etc. In addition, in step S1203, the system control unit 150 acquires the image circle of the mounted lens unit by referring to the lens table 156a, and determines which is larger, the image circle or the image sensor included in the imaging device 122.

[0127] In step S1203, if it is determined that the image circle is larger than the image sensor, the system control unit 150 advances the processing to step S1204. In step S1204, the system control unit 150 determines that the captured RAW image size is "large". On the other hand, if it is determined that the image circle is smaller than the size of the image sensor, the system control unit 150 advances the processing to step S1205. In step S1205, the system control unit 150 determines that the captured RAW image size is "small". Furthermore, in step S1206, the system control unit 150 sets, to the image processing unit 124, the size of the RAW image of the Bayer arrangement when acquiring the RAW image from the imaging device 122 based on the determined size (the entire area of ​​the image sensor or the area 1100 in the present embodiment).

[0128] Fig.13 1 is a flowchart showing the flow of multiple synthesis processing in the image processing unit 124 which is a feature of the fifth embodiment. When the user selects the multiple exposure imaging mode using the mode switching switch 160 or selects multiple exposure imaging in the menu setting, the system control unit 150 enters the multiple exposure imaging mode and performs the multiple exposure imaging according to the image processing unit 124. Fig.13 The flowchart shown in the figure is used to perform the multiple exposure shooting process. Fig.13 The processing in each step is realized by the system control unit 150 deploying a predetermined program read out from the nonvolatile memory 156 into the system memory 152, executing the program, and controlling the operation and processing of the units constituting the main body 100 of the image pickup apparatus. Note that it is assumed that the user has made various settings (such as the number of images to be synthesized and the designation of image files to be synthesized) and the set information is stored in the system memory 152. Note that the structure may be such that, unless otherwise changed, the number of images to be synthesized in the previous multiple-exposure shooting is the number of images to be synthesized this time. In this case, the number of images to be synthesized may be stored in the nonvolatile memory 156.

[0129] In step S1301, the system control unit 150 acquires the number N of images to be multiple-synthesized in the multiple-exposure imaging mode from the system memory 152 or the nonvolatile memory 156. Next, in step S1302, the system control unit 150 determines whether the user has made a setting to use the image file recorded in the recording medium 190 as the first base image to be used in the multiple-exposure imaging. If it is determined that a base image has not been selected (NO in step S1302), the system control unit 150 advances the processing to step S1303, and if it is determined that a base image has been selected (YES in step S1302), the system control unit 150 advances the processing to step S1304.

[0130] In step S1303, the system control unit 150 initializes a variable n indicating the number of captured images to “0”, and advances the process to step S1305. Furthermore, in step S1304, the system control unit 150 initializes a variable n to “1”, and advances the process to step S1305.

[0131] In step S1305, the system control unit 150 uses the lens unmounting / mounting detection unit 154 to determine whether a lens unit has been newly mounted (whether the lens unit has been remounted). Remounting of a lens unit includes, for example, a case where the mounted lens unit is removed once and then mounted again and a case where the mounted lens unit is replaced with a lens unit different from the mounted lens unit. If it is determined that a lens unit has been newly mounted ("Yes" in step S1305), the system control unit 150 advances the processing to step S1306, and if it is determined that the lens unit is the same as the lens unit used when the previous shooting was performed, that is, unmounting / mounting has not been performed ("No" in step S1305), the system control unit 150 advances the processing to step S1308.

[0132] In step S1306 , the system control unit 150 determines whether the image circle of the currently mounted lens unit is different from the image circle of the lens unit used when obtaining image data to be synthesized.

[0133] Specifically, the system control unit 150 can communicate with the mounted lens unit 180 via the connector 171 and acquire identification information such as a model name, etc. In addition, the system control unit 150 can acquire lens characteristics such as an image circle of the mounted lens unit, etc., by searching the lens table 156a in the nonvolatile memory 156 using the acquired identification information. The system control unit 150 performs the determination processing in step S1306 by comparing the image circle of the lens unit used when shooting was performed before the lens unit was remounted (assuming that it is stored in a predetermined address of the nonvolatile memory 156) with the image circle of the lens unit after the lens unit is remounted.

[0134] Note that if the structure is such that the size of the image to be read out from the imaging device 122 and to be recorded is switched according to the mounted lens unit, the determination process may be performed based on the size of the image read out from the imaging device 122. Furthermore, the determination process may be performed by performing image processing on the image read out from the imaging device 122 using the image processing unit 124 and by detecting the amount of vignetting in the peripheral portion of the image.

[0135] If it is determined that the image circle of the currently mounted lens unit is different from the image circle of the lens unit used when obtaining the image data to be synthesized (YES in step S1306), the system control unit 150 causes the processing to proceed to step S1307; otherwise (NO in step S1306), the system control unit 150 causes the processing to proceed to step S1308.

[0136] In step S1307 , the system control unit 150 performs processing for specifying a combining position (details will be described later), and advances the processing to step S1308 .

[0137] In step S1308, the system control unit 150 determines whether the second shutter switch 163 is turned on (whether the second shutter switch signal SW2 is generated). If it is determined that the second shutter switch 163 is not turned on (No in step S1308), the system control unit 150 returns the processing to step S1305, and if it is determined that the second shutter switch 163 is turned on (Yes in step S1308), the system control unit 150 advances the processing to step S1309.

[0138] In step S1309, the system control unit 150 acquires image data by causing the imaging device 122 to capture a still image frame, and stores the image data in the memory 132. Furthermore, in step S1310, the system control unit 150 increments the value of the variable n by "1". In step S1311, the system control unit 150 determines whether the value of the variable n is greater than 1, that is, whether the captured image is the second or later image and whether image data to be subjected to synthesis processing already exists. If it is determined that the variable n is "1" or less ("No" in step S1311), the system control unit 150 returns the processing to step S1305, and if it is determined that the variable n is greater than "1" ("Yes" in step S1311), the system control unit 150 advances the processing to step S1312.

[0139] In step S1312, the system control unit 150 creates new composite image data by controlling the image processing unit 124, by synthesizing the focus image data that has been obtained by using the image captured in step S1309 and stored in the memory 132 with the image data to be synthesized, and stores the new composite image data in the memory 132.

[0140] Here, if the value of the variable n is "2", the image of interest obtained in the most recent step S1309 is the second image. Therefore, the system control unit 150 stores the composite image obtained by synthesizing the second image of interest and the first image in the memory 132. In addition, if the value of the variable n is greater than "2", the image of interest is the third or later image. Therefore, in this case, the system control unit 150 creates a new composite image by synthesizing the image of interest obtained in the most recent step S1309 and the previous composite image, and stores the new composite image in the memory 132.

[0141] In step S1313, the system control unit 150 compares the value of the variable n with "N". That is, the system control unit 150 determines whether the number of images in the multiple-exposure imaging has reached the set number. If it is determined that the value of the variable n is less than "N" ("No" in step S1313), the system control unit 150 returns the processing to step S1305. Otherwise, if it is determined that the value of the variable n is "N" ("Yes" in step S1313), the system control unit 150 advances the processing to step S1314.

[0142] In step S1314, the system control unit 150 performs development processing (i.e., processing such as white balance adjustment and pixel interpolation) on the composite image data created in step S1312 by controlling the image processing unit 124, converts the obtained image data into YUV data, and writes the YUV data into the memory 132. Furthermore, in step S1315, the system control unit 150 compression-encodes the composite image data generated in step S1314 and subjected to development processing into still image data in JPEG format by controlling the image processing unit 124, and writes the still image data in JPEG format into the memory 132. Furthermore, in step S1316, the system control unit 150 records the still image data in JPEG format compression-encoded in step S1315 to the recording medium 190 via the storage medium I / F 118, and ends a series of multiple-exposure imaging.

[0143] Next, refer to Fig.19 The process for specifying the combining position in step S1307 will be described.

[0144] In step S1901, the system control unit 150 determines whether the image circles when the two images to be synthesized were taken are the same. If it is determined that the image circles are the same, or if the image data to be synthesized and the image data to be synthesized are not ready (the number of captured images is less than 2), the system control unit 150 does not perform processing for specifying a synthesis position and ends the processing. In addition, if it is determined that the image circles when the two images to be synthesized were taken are different (No in step S1901), the system control unit 150 advances the processing to step S1902.

[0145] In step S1902 , the system control unit 150 displays a screen for designating a combining position by controlling the display unit 128 , and accepts a user operation via the operation unit 170 .

[0146] To simplify the description, a case will be described where the image circle of image data to be synthesized (first image data or synthesized image data obtained in the synthesis process in the most recent step S1312) is smaller than the image circle of the currently mounted lens unit.

[0147] Fig. 20AThe initial screen of the screen for specifying the synthesis position in this state is shown. The system control unit 150 displays the live video obtained by the currently mounted lens unit in the entire area of ​​the area 2001 of the display unit 128. Then, the system control unit 150 displays the image to be synthesized in the area 2002 which can be moved according to the user operation. When the user views this screen, the synthesized image obtained by the most recent synthesis process is movably displayed in the image of the live video. In addition, the system control unit 150 moves the area 2002 within the area 2001 according to the user operation performed on the operation unit 170. Upon receiving an instruction for determining the position from the user via the operation unit 170, the system control unit 150 determines the position of the area 2002. That is, the system control unit 150 determines the position of the image to be synthesized relative to the image to be synthesized.

[0148] Note that if the image circle of the image to be synthesized is larger than the image circle of the currently mounted lens unit, the system control unit 150 displays the image to be synthesized in the entire area of ​​the area 2001 and displays the live video in the area 2002. Then, similarly to the above operation, the user specifies and determines the position of the area 2002. The difference for the user is that the position of the live video is determined instead of the position of the image to be synthesized.

[0149] In step S1903, the system control unit 150 prompts the user to select one of the size of the image having the image circle indicated by the area 2001 (full size) and the size of the image having a smaller image circle indicated by the area 2002 as the final combined size. The system control unit 150 then waits for reception of a selection instruction from the user via the operation unit 170.

[0150] In step S1904, the system control unit 150 determines whether the final combined size selected by the user is the full size. If it is determined that the final combined size selected by the user is the full size ("Yes" in step S1904), the system control unit 150 advances the processing to step S1906. On the other hand, if it is determined that the final combined size selected by the user is not the full size ("No" in step S1904), the system control unit 150 advances the processing to step S1905.

[0151] In step S1905, the system control unit 150 keeps the display state of the display unit 128 as Fig. 20A Status shown.

[0152] In step S1906, the system control unit 150 displays the live composite image of the area with a smaller image circle in the entire area of ​​the area 2001 of the display unit 128 during the period from now until the second shutter switch 163 is turned on. Fig. 20B As shown, even if the currently mounted lens unit is a lens unit of a full-size image sensor, the system control unit 150 displays a composite image obtained by cropping in the area 2001, similarly to the case of a smaller image circle. Fig. 20A It is easier to understand that the image in area 2002 is enlarged to the size of the entire area of ​​area 2001 and displayed.

[0153] In step S1907, the system control unit 150 configures the settings of the image processing unit 124 so that synthesis is performed at the final synthesis size using the information indicating the synthesis position determined as described above. As a result, the system control unit 150 causes the image processing unit 124 to perform synthesis at the final synthesis size. Fig.13 The synthesis process is performed in step S1312 of the flowchart shown in the figure with the above-mentioned setting. Of course, the synthesized image obtained in step S1312 is one of the full-size image and the APS-C size image.

[0154] Next, refer to Figures 14 to 17 An example of a process of combining four images by the process for specifying a combining position in step S1307 performed by the image processing unit 124 will be described.

[0155] Fig.14 An example is shown in which a first image and a second image are captured using a lens unit with a small image circle (APS lens unit), a third image and a fourth image are captured using a lens unit with a large image circle (full-size lens unit), and the size of the final composite image specified by the user is the size of the image with the small image circle.

[0156] When the first image and the second image are captured, the image processing unit 124 obtains images 1401 and 1402 by cropping an area corresponding to the APS image circle from the full-size image. Since the images 1401 and 1402 are images captured using a lens unit with a small image circle (cropped images), the user does not designate a synthesis position. The image processing unit 124 creates a synthesized image 1411 (cropped image) by simply synthesizing the two images 1401 and 1402. The third image is captured using a lens unit with a large image circle, and the user selects a smaller image size as the size of the final synthesized image. Therefore, the image processing unit 124 acquires an image obtained by cropping the full-size image as the third image 1403. Then, the image processing unit 124 creates a new synthesized image 1412 (cropped image) by synthesizing the synthesized image 1411 and the image 1403. Note that here, the user specifies the position where the image 1403 is to be cropped from the full-size image obtained in the third capture. In addition, the fourth image is captured using a lens unit having a large image circle, and the user selects a smaller image size as the size of the final composite image. Therefore, the image processing unit 124 acquires an image obtained by cropping the full-size image as the fourth image 1404. Then, the image processing unit 124 creates a final composite image 1413 (cropped image) by synthesizing the composite image 1412 and the image 1404. Note that here, the user also specifies the position of the cropped image 1404 from the full-size image obtained in the fourth capture.

[0157] Fig.15 An example is shown in which the first image and the second image are captured using a lens unit with a large image circle (full-size lens unit), the third image and the fourth image are captured using a lens unit with a small image circle (APS lens unit), and the size of the final composite image specified by the user is the size of the image with the small image circle.

[0158] Since images 1501 and 1502 are full-size image data obtained by shooting using a lens unit with a large image circle, the user does not specify the synthesis position when synthesizing these images. The image processing unit 124 creates a synthesized image 1511 (full-size image) by simply synthesizing these two images 1501 and 1502. Since the third image is shot by using a lens unit with a small image circle, the image processing unit 124 creates an image 1503 by cropping the full-size image obtained by the imaging device 122. The user has selected a smaller image size as the size of the final synthesized image. Therefore, the image processing unit 124 creates a synthesized image 1512 (cropped image) by synthesizing the synthesized image 1511 (full-size image) and the image 1503 (cropped image). Note that the synthesis position of the image 1503 on the synthesized image 1512 is specified by the user. In addition, since the fourth image is shot by using a lens unit with a small image circle, the image processing unit 124 creates an image 1504 by cropping the full-size image obtained by the imaging device 122. The user has selected a smaller image size as the size of the final composite image. Therefore, the image processing unit 124 creates a final composite image 1513 (cropped image) by composite image 1512 (cropped image) and fourth image 1504 (cropped image). Note that since composite image 1512 and image 1504 are cropped images, the user does not perform position designation.

[0159] Fig.16 An example is shown in which the first image and the second image are captured using a lens unit with a small image circle (APS lens unit), the third image and the fourth image are captured using a lens unit with a large image circle (full-size lens unit), and the size of the final composite image specified by the user is the size of the image with the large image circle.

[0160] When the first image and the second image are captured, the image processing unit 124 obtains images 1601 and 1602 by cropping an area corresponding to the APS image circle from the full-size image. Since the images 1601 and 1602 are images captured using a lens unit with a small image circle (cropped images), the user does not designate a synthesis position. The image processing unit 124 creates a synthesized image 1611 (cropped image) by simply synthesizing the two images 1601 and 1602. The third image is captured using a lens unit with a large image circle, and the user selects a larger image size as the size of the final synthesized image. Therefore, the image processing unit 124 obtains a full-size synthesized image 1612 by synthesizing the synthesized image 1611 (cropped image) and the full-size image 1603. Note that here, the user specifies the synthesis position of the synthesized image 1611 on the full-size image 1603 obtained in the third shooting. The fourth image is captured using a lens unit with a large image circle, and the user selects a larger image size as the size of the final synthesized image. Therefore, the image processing unit 124 creates a final synthesized image 1613 (full-size image) by synthesizing the full-size synthesized image 1612 and the fourth full-size image 1604. Note that since the synthesized image 1612 and the image 1604 are full-size images, the user does not perform position designation.

[0161] Fig.17 An example is shown in which the first image and the second image are captured using a lens unit with a large image circle (full-size lens unit), the third image and the fourth image are captured using a lens unit with a small image circle (APS lens unit), and the size of the final composite image specified by the user is the size of the image with the large image circle.

[0162] Since images 1701 and 1702 are full-size image data obtained by shooting using a lens unit with a large image circle, the user does not specify the synthesis position when synthesizing these images. The image processing unit 124 creates a synthesized image 1711 (full-size image) by simply synthesizing these two images 1701 and 1702. Since the third image is shot by using a lens unit with a small image circle, the image processing unit 124 creates an image 1703 by cropping the full-size image obtained by the imaging device 122. The user has selected a larger image size as the size of the final synthesized image. Therefore, the image processing unit 124 creates a synthesized image 1712 (full-size image) by synthesizing the synthesized image 1711 (full-size image) and the image 1703 (cropped image). Note that the synthesis position of the image 1703 on the synthesized image 1711 is specified by the user. In addition, since the fourth image is shot by using a lens unit with a small image circle, the image processing unit 124 creates an image 1704 by cropping the full-size image obtained by the imaging device 122. The user has selected a larger image size as the size of the final composite image. Therefore, the image processing unit 124 creates a final composite image 1713 (full-size image) by composite image 1712 (full-size image) and fourth image 1704 (cropped image). Note that the composite position of image 1704 on composite image 1712 is specified by the user.

[0163] Next, refer to Fig.18 The multiple synthesis processing of the image processing unit 124 in this embodiment is described below. Fig.13 The processing in step S1312 in .

[0164] In step S1801, the image processing unit 124 determines whether the size of the RAW captured image obtained in step S1309 is consistent with the size of the image to be synthesized. If it is determined that the sizes are the same ("Yes" in step S1801), the image processing unit 124 advances the process to step S1807, otherwise ("No" in step S1801), the image processing unit 124 advances the process to step S1802.

[0165] In step S1802, the image processing unit 124 determines whether comparison synthesis has been selected as the synthesis method of multiple exposure imaging in the menu setting. If it is determined that comparison synthesis has been selected as the synthesis method of multiple exposure imaging ("Yes" in step S1802), the image processing unit 124 advances the processing to step S1803, and if it is determined that comparison synthesis has not been selected as the synthesis method of multiple exposure imaging ("No" in step S1802), the image processing unit 124 advances the processing to step S1804.

[0166] In step S1803 , the image processing unit 124 configures settings so that the pixel value of the full-size image outside the area of ​​the cropped RAW image is maintained at 100%, and advances the process to step S1807 .

[0167] In step S1804, the image processing unit 124 determines which one of light and dark is selected in the comparison synthesis set in the menu. If it is determined that the synthesis of the relatively bright portion has been selected ("Yes" in step S1804), the image processing unit 124 advances the processing to step S1806, and if it is determined that the synthesis of the relatively dark portion has been selected ("No" in step S1804), the image processing unit 124 advances the processing to step S1805.

[0168] In step S1806, the image processing unit 124 configures settings so that the area outside the area of ​​the cropped RAW image is a black area (all components are at minimum brightness), and advances the process to step S1807. In step S1805, the image processing unit 124 configures settings so that the area outside the area of ​​the cropped RAW image is a white area (all components are at maximum brightness), and advances the process to step S1807.

[0169] In step S1807 , if the combining position has been designated by the user, the image processing unit 124 performs combining processing at the designated combining position according to the conditions set in steps S1803 , S1805 , and S1806 , and stores the combined image data to the memory 132 .

[0170] According to the present embodiment as described above, even when the lens unit is switched to a lens unit having a different image circle size in a camera in which the lens unit can be removed and attached, multiple exposure imaging can be continued, and further, synthesis can be performed at a user's desired position in the image.

[0171] Note that in the above-described embodiment, when multiple exposure imaging is performed, at least the live image is displayed in the display unit 128, but the structure may be such that a composite image of two images can be displayed only when positioning is performed in the synthesis process. In addition, in the above-described embodiment, the user selects the size of the final composite image each time images of different image circles are synthesized, but the structure may be such that the final composite size is provided as a setting for Fig.12 The setting items for multiple exposure shooting in and the structure that determines the final composite size when configuring the settings.

[0172] Other embodiments

[0173] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0174] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures and functions.

Claims

1. An image pickup device capable of interchangeably mounting a lens unit, the image pickup device comprising: A camera component for capturing images using an image sensor; A synthesis component, used for synthesizing multiple images; as well as a control section for controlling a predetermined synthesis process to perform a plurality of imaging operations using the image sensor, create a synthesized image from a plurality of images obtained by the plurality of imaging operations, and record the created synthesized image in a recording medium, In which, when the lens unit reinstalled after the first image for the predetermined synthesis processing is acquired has an image circle different from the image circle of the lens unit installed before the reinstallation, the control component causes the predetermined synthesis processing to record the synthetic image created by synthesizing the image for the predetermined synthesis processing taken before the reinstallation.

2. The imaging device according to claim 1, wherein: The control section ends the predetermined synthesis process in a case where a lens unit remounted after the first image for the predetermined synthesis process is acquired has an image circle different from that of the lens unit mounted before the remounting.

3. The imaging device according to claim 1 or 2, further comprising a communication component for communicating with the mounted lens unit, in, In a case where the lens unit reinstalled after acquiring the first image for the predetermined synthesis processing has an image circle identical to that of the lens unit installed before the reinstallation, the control component continues the predetermined synthesis processing that has been performed since before the reinstallation and enables the camera component to capture a new image for creating the synthesized image.

4. The imaging device according to claim 1 or 2, further comprising a communication component for communicating with the mounted lens unit, in, The control section determines whether the lens unit remounted after acquiring the first image for the predetermined synthesis process has the same image circle as the lens unit mounted before remounting based on the lens unit information received from the remounted lens unit via the communication section. 5 . The imaging device according to claim 1 , further comprising a cropping means for cropping an image acquired from the imaging means in a case where a lens unit has an image circle smaller than a size of the image sensor.

6. The imaging device according to claim 1 or 2, wherein: The predetermined synthesis processing is processing for obtaining a multiplexed image by synthesizing a plurality of images obtained in multiple-exposure imaging.

7. The imaging device according to claim 1 or 2, wherein: Each image used for the predetermined synthesis process is obtained when the user performs an operation to instruct image capture.

8. The imaging device according to claim 1, wherein: When a lens unit reinstalled after acquiring the first image for the predetermined synthesis processing has an image circle different from that of the lens unit installed before the reinstallation, the control component waits for a lens unit having the same image circle as that of the lens unit installed before the reinstallation.

9. The imaging device according to claim 1, wherein: The control section displays a predetermined guide in a case where a lens unit remounted after the first image for the predetermined synthesis process is acquired has an image circle different from an image circle of the lens unit mounted before the remounting.

10. An image pickup device capable of interchangeably mounting a lens unit, the image pickup device comprising: A camera component for capturing images using an image sensor; A synthesis component, used for synthesizing multiple images; as well as a control section for controlling a predetermined synthesis process to perform a plurality of imaging operations using the image sensor, create a synthesized image from a plurality of images obtained by the plurality of imaging operations, and record the created synthesized image in a recording medium, wherein, in a case where the lens unit reinstalled after the first image for the predetermined synthesis processing is acquired is a lens unit of a second type different from the lens unit of the first type installed before the reinstallation, the control unit causes the predetermined synthesis processing to record the created synthesized image created by synthesizing the image for the predetermined synthesis processing captured before the reinstallation, The first type of lens unit is a lens unit for a first type of image sensor, and the second type of lens unit is a lens unit for a second type of image sensor, wherein a size of the first type of image sensor is different from a size of the second type of image sensor.

11. The imaging device according to claim 10, further comprising a communication component for communicating with the mounted lens unit, in, The control section determines the type of the lens unit remounted after the first image for the predetermined synthesis process is acquired, based on the lens unit information received from the lens unit via the communication section.

12. The imaging device according to claim 10 or 11, wherein: The image circle of the first type of lens unit is larger than the image circle of the second type of lens unit.

13. The imaging device according to claim 10 or 11, wherein: The first type of lens unit is for a full-size image sensor, and the second type of lens unit is for an APS-C size image sensor.

14. The image pickup apparatus according to claim 12, further comprising a cropping section for cropping an image acquired from the image pickup section when the second type lens unit is mounted.

15. The imaging device according to claim 10 or 11, wherein: The predetermined synthesis processing is processing for obtaining a multiplexed image by synthesizing a plurality of images obtained in multiple-exposure imaging.

16. A control method of an image pickup device, the image pickup device being capable of interchangeably mounting a lens unit and having an image pickup component for capturing an image using an image sensor, the control method comprising: Combining multiple images; as well as controlling a predetermined synthesis process to perform a plurality of imaging operations using the image sensor, create a synthesized image from a plurality of images obtained by the plurality of imaging operations, and record the created synthesized image in a recording medium, In which, when the lens unit reinstalled after acquiring the first image for the predetermined synthesis processing has an image circle different from the image circle of the lens unit installed before the reinstallation, the predetermined synthesis processing is controlled to record the created synthetic image created by synthesizing the image for the predetermined synthesis processing taken before the reinstallation.

17. A control method of an image pickup device, the image pickup device being capable of interchangeably mounting a lens unit and having an image pickup component for capturing an image using an image sensor, the control method comprising: Combining multiple images; as well as controlling a predetermined synthesis process to perform a plurality of imaging operations using the image sensor, create a synthesized image from a plurality of images obtained by the plurality of imaging operations, and record the created synthesized image in a recording medium, wherein, in a case where the lens unit reinstalled after the first image for the predetermined synthesis process is acquired is a lens unit of a second type different from the lens unit of the first type installed before the reinstallation, the predetermined synthesis process is controlled to record the created synthesized image created by synthesizing the image for the predetermined synthesis process captured before the reinstallation, The first type of lens unit is a lens unit for a first type of image sensor, and the second type of lens unit is a lens unit for a second type of image sensor, wherein a size of the first type of image sensor is different from a size of the second type of image sensor.

18. A non-transitory computer-readable storage medium storing a program which, when read and executed by a computer, causes the computer to execute steps of a control method of an image pickup device, the image pickup device being capable of interchangeably mounting a lens unit and having an image pickup component for capturing an image using an image sensor, the control method comprising: Combining multiple images; as well as controlling a predetermined synthesis process to perform a plurality of imaging operations using the image sensor, create a synthesized image from a plurality of images obtained by the plurality of imaging operations, and record the created synthesized image in a recording medium, In which, when the lens unit reinstalled after acquiring the first image for the predetermined synthesis processing has an image circle different from the image circle of the lens unit installed before the reinstallation, the predetermined synthesis processing is controlled to record the created synthetic image created by synthesizing the image for the predetermined synthesis processing taken before the reinstallation.

19. A non-transitory computer-readable storage medium storing a program which, when read and executed by a computer, causes the computer to execute steps of a control method of an image pickup device, the image pickup device being capable of interchangeably mounting a lens unit and having an image pickup component for capturing an image using an image sensor, the control method comprising: Combining multiple images; as well as controlling a predetermined synthesis process to perform a plurality of imaging operations using the image sensor, create a synthesized image from a plurality of images obtained by the plurality of imaging operations, and record the created synthesized image in a recording medium, wherein, in a case where the lens unit reinstalled after the first image for the predetermined synthesis process is acquired is a lens unit of a second type different from the lens unit of the first type installed before the reinstallation, the predetermined synthesis process is controlled to record the created synthesized image created by synthesizing the image for the predetermined synthesis process captured before the reinstallation, The first type of lens unit is a lens unit for a first type of image sensor, and the second type of lens unit is a lens unit for a second type of image sensor, wherein a size of the first type of image sensor is different from a size of the second type of image sensor.

Images