Image Production Method

Abstract

The present invention provides an image production method that allows an image obtained by extracting a part of a captured image to have good image quality even when the position of the extracted area is changed. The image production method of the present invention comprises: a setting step, in which a plurality of areas with a second field of view angle smaller than the first field of view angle are set in a shooting area of ​​a first image with a first field of view angle; a designation step, in which a second area with an appropriate exposure amount smaller than the first area among the first area and the second area included in the plurality of areas is designated as a reference area; and a condition determination step, in which at least one condition of the exposure time when shooting the first image and the aperture amount relative to the incident light to the shooting lens is determined based on the appropriate exposure amount of the reference area.

Description

Technical Field

[0001] An embodiment of the present invention relates to an image production method for producing an image document by shooting an image using a shooting device. Background Art

[0002] When acquiring images of a plurality of subjects in a certain scene, a single imaging device may capture an image of the scene and extract the image of each subject from the captured image. As an example of such a technique, the techniques described in Patent Documents 1 to 3 can be cited.

[0003] Patent Document 1 discloses a technique of cutting out a part of an original image captured by one camera to generate a cutout image, controlling the cutout position, and outputting the cutout image so as to change the position of a subject.

[0004] Patent Document 2 discloses a technology for detecting a plurality of objects based on a high-resolution wide-angle image captured by one camera, cutting out the image of each detected object from the wide-angle image, and displaying the images of the plurality of objects side by side on a display.

[0005] Patent Document 3 discloses a technique in which a plurality of arbitrary regions are designated from an image captured by one imaging device, and images corresponding to the designated plurality of arbitrary regions are cut out and output.

[0006] Previous technical literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Laid-Open No. 11-220653

[0009] Patent Document 2: Japanese Patent Application Publication No. 2004-194309

[0010] Patent Document 3: Japanese Patent Application Publication No. 2014-42357 Summary of the invention

[0011] Technical issues to be solved by the invention

[0012] When a part of an image is extracted, the quality of the extracted image is determined by conditions such as exposure, white balance, and lens focus (focus) during shooting. If these conditions are set to appropriate values, the extracted image will have good quality.

[0013] On the other hand, the appropriate conditions for the extracted image are determined according to the position of the extraction region in the original image. In view of this, it is required to ensure good image quality for the extracted image even when the position of the extraction region changes.

[0014] However, the techniques described in Patent Documents 1 to 3 do not optimize conditions such as exposure, white balance, and lens focus according to the position of the extracted image. Therefore, even if the techniques described in these documents are used, the quality of the extracted image may not be properly ensured, especially when the position of the extracted area changes.

[0015] One embodiment of the present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image production method that can obtain an image by extracting a part of a captured image and achieve good image quality even when the position of the extracted area is changed.

[0016] Means for solving technical problems

[0017] In order to achieve the above-mentioned purpose, an image production method of one embodiment of the present invention is an image production method for producing an image document based on an image captured by a shooting device having a shooting lens and an imaging element, and is characterized in that it comprises: a setting step of setting multiple areas of a second field of view angle smaller than the first field of view angle in a shooting area of ​​a first image of a first field of view angle; a designation step of designating a first area included in the multiple areas and a second area whose appropriate exposure amount is smaller than that of the first area as a reference area; and a condition determination step of determining at least one condition of the exposure time when shooting the first image and the aperture amount relative to the incident light to the shooting lens according to the appropriate exposure amount of the reference area.

[0018] Furthermore, an image production method according to an embodiment of the present invention may include: a selection process of selecting a selection area as a shooting area for recording an image from a plurality of areas; a switching process of reselecting the selection area from a plurality of areas and switching the selection area after the selection process; and a production process of separately recording an image of the selection area before the switching process and an image of the selection area after the switching process to produce an image document.

[0019] Furthermore, in the image production method of one embodiment of the present invention, a dynamic image document as an image document can be produced by combining the image of the selected area before the switching process and the image of the selected area after the switching process in the production process. In this case, a dynamic image document of an image with good image quality can be obtained before and after the switching process.

[0020] Furthermore, it is preferable that the designated step is performed when the switching step is performed, and the condition determination step is performed every time the designated step is performed.

[0021] Furthermore, the image production method of one embodiment of the present invention may also include: a sensitivity determination step, which, when the first area is selected as the selected area, determines the sensitivity of the pixels corresponding to the first area among the multiple pixels of the imaging element based on the appropriate exposure amount of the first area and the conditions determined by the condition determination step.

[0022] Furthermore, in the sensitivity determination step, the sensitivity of the pixels corresponding to the first region is determined to be higher than the sensitivity of the pixels corresponding to the second region serving as the reference region.

[0023] Furthermore, the image production method according to one embodiment of the present invention may further include: an adjustment step of adjusting the exposure amount when shooting the first image over time by changing the condition over time. In this case, the condition determination step may include: a first determination step of determining the condition based on the appropriate exposure amount of the reference area; and a second determination step of determining the adjustment amount of the exposure amount in the adjustment step based on the appropriate exposure amount of the selected area. Furthermore, the adjustment step may be implemented based on the condition determined in the first determination step and the adjustment amount determined in the second determination step.

[0024] In addition, it is preferable that the first determination step is performed when the switching step is performed, and the second determination step is performed in the adjustment step after the first determination step is performed.

[0025] Furthermore, the image production method according to one embodiment of the present invention may include a display step of displaying the first image on a display screen. In this case, in the setting step, the plurality of areas are set according to a setting operation performed by a user through the first image displayed on the display screen.

[0026] Furthermore, according to one embodiment of the present invention, it is an image production method for producing an image document based on an image captured by a shooting device having a shooting lens and an imaging element, and it comprises: a setting process, in which a plurality of areas of a second field of view angle smaller than the first field of view angle are set in a shooting area of ​​a first image of a first field of view angle; a selection process, in which a selection area is selected from a plurality of areas as a shooting area of ​​a recorded image; a switching process, in which, after the selection process, the selection area is reselected from a plurality of areas and the selection area is switched; a first recording process, which is implemented within a period from the implementation of the switching process to the first period, and the image of the selection area in the first period is recorded; a second recording process, which is implemented within a second period other than the first period, and the image of the selection area in the second period is recorded; and an adjustment process, in each of the first period and the second period, at least one condition among the exposure amount of the selection area, the white balance of the image of the selection area, and the focus of the shooting lens is adjusted, and the image production method can realize an adjustment process implemented in the first period, in which the adjustment speed of the conditions is faster than that of the adjustment process implemented in the second period.

[0027] Furthermore, in the above-mentioned image production method, it is more preferred that in the adjustment process, the aperture drive unit that changes the aperture amount relative to the incident light to the shooting lens is driven to adjust the exposure amount, and in the adjustment process implemented in the first period, the driving speed of the aperture drive unit is faster than that in the adjustment process implemented in the second period.

[0028] Furthermore, in the above-mentioned image production method, it is more preferred that in the adjustment process, the focusing optical component of the shooting lens is moved to adjust the focus, and in the adjustment process implemented in the first period, the moving speed of the focusing optical component is faster than that in the adjustment process implemented in the second period.

[0029] Furthermore, according to one embodiment of the present invention, it is an image production method for producing an image document based on an image captured by a shooting device having a shooting lens and an imaging element, and it comprises: a setting process, in which a plurality of areas of a second field of view angle smaller than the first field of view angle are set in a shooting area of ​​a first image of a first field of view angle; a selection process, in which a selection area is selected from a plurality of areas as a shooting area of ​​a recorded image; a switching process, in which, after the selection process, the selection area is reselected and switched from a plurality of areas; a first recording process, which is carried out within a period from the implementation of the switching process to the first period, and the image of the selection area in the first period is recorded; a second recording process, which is carried out within a second period excluding the first period, and the image of the second period is recorded. an image of a selected area in the image; a first adjustment process, which is implemented in a first period and uses the first adjustment data to adjust the exposure of the selected area to a target exposure; and a second adjustment process, which is implemented in a second period and uses the second adjustment data to adjust the exposure of the selected area to the target exposure, and can achieve, in the first adjustment process and the second adjustment process, determining the values ​​of multiple parameters including exposure time, sensitivity of pixels possessed by an imaging element, and aperture amount relative to incident light of a shooting lens according to the first adjustment data or the second adjustment data, and adjusting the exposure to the target exposure, and an image production method in which the values ​​of multiple parameters specified for the target exposure are different from each other between the first adjustment data and the second adjustment data.

[0030] Furthermore, in the above-mentioned image production method, the first adjustment data and the second adjustment data may define a change pattern when changing the values ​​of the plurality of parameters in order to adjust the exposure to the target exposure. In this case, it is preferred that the first adjustment data define a change pattern in which the degree of change of the aperture amount relative to the target exposure is smaller than the change pattern in the second adjustment data.

[0031] Furthermore, in the above-mentioned image production method, the photographing device may be provided with an electronic dimming filter capable of electronically changing the dimming degree. Furthermore, the dimming degree of the electronic dimming filter may be included in the plurality of parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a perspective view showing an example of the appearance of an imaging device according to a first embodiment of the present invention.

[0033] Figure 2 This is a rear view showing the rear side of the imaging device according to the first embodiment of the present invention.

[0034] Figure 3 This is a block diagram showing the configuration of an imaging device according to a first embodiment of one embodiment of the present invention.

[0035] Figure 4 This is a diagram showing the correspondence between pixels of the imaging element and the imaging area of ​​the first image.

[0036] Figure 5 This is a diagram showing the steps of setting the image area to be extracted from the first image.

[0037] Figure 6 This is a diagram showing a state in which a plurality of image regions extracted from a first image are set.

[0038] Figure 7 A diagram showing an image of a selection area displayed on a display screen.

[0039] Figure 8 This is an explanatory diagram of a moving image file of recorded images.

[0040] Fig. 9 This is a diagram showing the flow of a video production process according to a first embodiment of the present invention.

[0041] Fig.10 It is a diagram showing the flow of a video production process according to a second embodiment of one embodiment of the present invention.

[0042] Fig.11 It is a diagram showing the flow of a video production process according to a third embodiment of one embodiment of the present invention.

[0043] Fig.12 It is a diagram showing the change over time of the appropriate exposure amount of each of the reference area and the selected area.

[0044] Fig.13 This is a diagram showing the exposure amount after adjustment of the selected area when the adjustment is continued based on the appropriate exposure amount of the reference area.

[0045] Fig.14 It is a diagram showing the target exposure amount in the third embodiment.

[0046] Fig.15 It is a diagram showing transition of the exposure amount after adjustment of each of the reference area and the selected area in the third embodiment.

[0047] Fig.16 It is an explanatory diagram of the video production flow involved in the fourth embodiment of one embodiment of the present invention.

[0048] Fig.17 It is an explanatory diagram of the video production flow involved in the fifth embodiment of one embodiment of the present invention.

[0049] Fig.18 This is a diagram showing an example of a change pattern specified by the adjustment data. DETAILED DESCRIPTION

[0050] Hereinafter, preferred embodiments (first embodiment to sixth embodiment) of the present invention will be described in detail with reference to the drawings.

[0051] However, the embodiment described below is only an example for facilitating understanding of one embodiment of the present invention, and does not limit the present invention. That is, one embodiment of the present invention can be changed or improved from the embodiment described below as long as it does not deviate from its purpose. In addition, equivalents are included in one embodiment of the present invention.

[0052] <<First embodiment>>

[0053] A first embodiment as one embodiment of the present invention relates to a method using Figure 1 to Figure 3 The image production method of the shooting device 10 is shown. Figure 1 and Figure 2 1 shows the appearance of the camera 10. Figure 1 is a three-dimensional diagram of the photographing device 10, Figure 2 This is a diagram showing the imaging device 10 as viewed from the rear side. Figure 3 It is a block diagram showing the structure of the imaging device 10 .

[0054] [Basic structure of the camera]

[0055] The photographing device 10 is, for example, a digital camera, and is used for photographing images. In the following description, unless otherwise specified, "image" refers to a real-time image (real-time view image), that is, an image photographed in real time.

[0056] Figure 1 and Figure 2The imaging device 10 shown is a lens-interchangeable digital camera, and includes an imaging device body 12 and an imaging lens 14. The imaging lens 14 is replaceably mounted on a mount 13 of the imaging device body 12. However, the present invention is not limited thereto, and the imaging device 10 may also be a lens-integrated digital camera.

[0057] (Photographing lens)

[0058] like Figure 3 As shown, the photographing lens 14 includes an optical component unit 18 , an aperture 20 , an electronic neutral density filter 21 , a focus driving section 22 , an aperture driving section 23 , and a voltage applying section 24 .

[0059] The optical component unit 18 includes a plurality of lenses, and includes a focusing optical component 19 (focus lens). When the focusing optical component 19 moves in the direction of the optical axis L1, the focus (focal point) of the photographing lens 14 changes.

[0060] There are two focus adjustment modes: a manual focus mode and an automatic focus mode. In the manual focus mode, the user manually rotates a focus ring 16 provided on the lens barrel of the photographing lens 14 to move the focus optical component 19 in the direction of the optical axis L1.

[0061] In the auto focus mode, the control unit 46 drives the focus drive unit 22 to move the focus optical component 19 in the direction of the optical axis L1, thereby automatically adjusting the focus to focus on the subject in the image. In addition, for example, when the user half-presses the release button 26 of the camera body 12, the focus adjustment in the auto focus mode is implemented as a trigger.

[0062] The driving motor of the focus driving section 22 is constituted by an ultrasonic motor or the like, and is provided in the imaging device body 12 or in the imaging lens 14 .

[0063] The optical component unit 18 includes: a wide-angle lens, an ultra-wide-angle lens, a 360-degree lens or an anamorphic lens, etc. Thus, the camera 10 can capture images with a wide field of view in the horizontal direction. Here, the maximum field of view angle (hereinafter referred to as the first field of view angle) when the camera 10 captures an image is determined based on the use of the optical component unit 18 and the imaging element 40 described later, and the image captured at the first field of view angle is equivalent to the "first image" of one embodiment of the present invention.

[0064] Furthermore, the imaging device 10 may include a plurality of optical component units 18 having different viewing angles.

[0065] The aperture 20 is disposed in the optical path of the photographing lens 14 and has a variable opening shape. It is an optical component that adjusts the aperture amount (specifically, the aperture value or F value) relative to the incident light to the photographing lens 14. The aperture 20 changes the opening shape by, for example, mechanically driving a component for changing the opening size. However, the invention is not limited to this, and may also be an aperture (physical property aperture) that drives a liquid crystal or an electrochromic element to change the opening shape.

[0066] The opening shape of the diaphragm 20 , that is, the aperture amount, is adjusted by the diaphragm driving unit 23 .

[0067] The electronic neutral density filter 21 is disposed in the optical path of the photographing lens 14 and is an ND (Neutral Density) filter that can electronically change the dimming degree. The electronic neutral density filter 21 is formed by mixing a light-absorbing dimming material into liquid crystal, for example, and controls the dimming degree (specifically, the transmittance of light) by changing the tilt angle of the liquid crystal molecules.

[0068] In addition, the electronic dimming filter 21 is not limited to a liquid crystal type electronic ND filter, and may be a precipitation type electronic ND filter utilizing a redox reaction of a reaction substance (e.g., silver chloride). Furthermore, as the electronic dimming filter 21, for example, an electronic ND filter described in Japanese Patent Application Publication No. 2013-88596 or Japanese Patent Application Publication No. 2019-68402 may be utilized.

[0069] The dimming degree of the electronic dimming filter 21 is adjusted by changing the magnitude of the voltage (applied voltage) applied to the electronic dimming filter 21 by the voltage applying section 24 .

[0070] (Camera main body)

[0071] like Figure 1 and Figure 2 As shown, the camera body 12 includes an operation unit operated by the user. For example, a release button 26 is disposed on the upper surface of the camera body 12. For example, when the user fully presses the release button 26, recording of an image captured by the camera 10 or an image based on the captured image begins. As an image based on the captured image, for example, an image of a selected area described later can be cited. In addition, the user's release instruction (i.e., recording instruction) can be input to the control unit 46 via a detection function based on a touch operation of the display 28.

[0072] A display 28 constituting a display screen is arranged on the back side of the imaging device body 12 , and a plurality of buttons 30 , 32 , and 34 are arranged around the display 28 .

[0073] The display 28 is composed of, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electroluminescence) display, an LED (Light Emitting Diode) display, or electronic paper, etc. The display screen of the display 28 displays an image captured by the camera 10 or an image based on the captured image.

[0074] Furthermore, on the display screen of the display 28 , a selection menu related to shooting conditions and the like, notification information to the user including warnings and the like, and a playback image of an image acquired in the past and the like are displayed.

[0075] The display 28 has a function of detecting a touch operation of a user's finger. In addition, a transmissive touch panel 36 overlaps with the display 28 or is installed inside the display 28. The touch panel 36 detects the contact position and displacement of the user's finger or stylus, etc., and outputs a signal based on the detection result to a specified output destination. For example, it is assumed that the user performs the following operation (so-called zoom-in operation): after touching the touch panel 36 with two fingers close to each other, the interval between the fingers is widened. In this case, the touch panel 36 detects the positions of the operation start time and the operation end time of each of the two fingers, and outputs a signal corresponding to the detection result.

[0076] like Figure 3 As shown, a shutter 38 , an imaging element 40 , an analog signal processing circuit 44 , a control unit 46 , an internal memory 50 , a card slot 52 , and a buffer 56 are provided in a housing of the imaging device body 12 .

[0077] The imaging element 40 is an image sensor, and is composed of a solid imaging element such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor Image Sensor). Figure 4 As shown, the imaging element 40 has a plurality of pixels 42 arranged in a grid pattern. Figure 4 The correspondence between the pixels 42 of the imaging element 40 and the imaging area of ​​the first image is shown.

[0078] Each pixel 42 has an on-chip microlens, a color filter, and a photodiode (photoelectric conversion element).

[0079] And, if Figure 4As shown, each pixel 42 corresponds to one of the unit areas A0 constituting the shooting area A1 of the image. The shooting area A1 is the shooting area when the first image is shot by the shooting device 10, and its field of view angle is equivalent to the first field of view angle. The unit area A0 is the area that becomes the smallest unit in the shooting area A1, and is the area corresponding to the pixels 42 arranged in the longitudinal and transverse directions of the imaging element 40.

[0080] In other words, the first image projected on the imaging area A1 is composed of the same number of unit images as the number of unit areas A0 (ie, the number of pixels 42). In this specification, the number of unit images in the first image is simply referred to as the "number of pixels".

[0081] The imaging element 40 receives light from the subject through the photographing lens 14, converts the received light image into an electric signal (image signal), and outputs the electric signal. The imaging element 40 is configured to be able to perform an exposure operation using a so-called electronic shutter.

[0082] In the following description, unless otherwise specified, "exposure" means exposure in the imaging element 40 using the electronic shutter while the shutter 38 is maintained in an open state. Furthermore, "exposure time" means the time corresponding to the shutter speed by the electronic shutter, which strictly means the charge accumulation time.

[0083] exist Figure 3 In the structure shown, the analog signal processing circuit 44 reads out the electrical signal (image signal) of one frame output from the imaging element 40 to each pixel. The analog signal processing circuit 44 amplifies the read image signal through AGC (Auto Gain Controller) and performs signal processing such as correlated double sampling processing on the amplified signal. The processed signal is sent to the image processing unit 48 of the control unit 46.

[0084] The control unit 46 controls each unit of the imaging device 10 to execute various processes related to the creation of image files. Figure 3 As shown, the control unit 46 includes a controller 47 and an image processing unit 48 .

[0085] The control unit 46 is composed of, for example, one or more processors, such as a CPU (Central Processing Unit) control program. However, the present invention is not limited thereto, and the above-mentioned processor may be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), an MPU (Micro-Processing Unit), or other ICs (Integrated Circuits), or a combination of these.

[0086] Furthermore, as represented by SoC (System on Chip) or the like, the above-mentioned processor may be constituted by a single IC (Integrated Circuit) chip, which includes the functions of the entire control unit 46 including the controller 47 and the image processing unit 48 .

[0087] In addition, the hardware structure of each processor listed above can be implemented by a circuit (Circuitry) formed by combining circuit elements such as semiconductor elements.

[0088] The controller 47 controls the entire imaging device 10 according to the user's operation or a predetermined control pattern. For example, the controller 47 controls the imaging element 40 and the analog signal processing circuit 44 to capture an image (moving image) at a predetermined frame rate.

[0089] The controller 47 determines the shooting conditions according to the shooting environment, and controls the driving units, the imaging element 40, the analog signal processing circuit 44, and the image processing unit 48 so that the shooting conditions become the determined conditions. The shooting conditions include the exposure, white balance, and focus (focal point) of the shooting lens 14 when shooting an image.

[0090] Furthermore, the controller 47 controls the image processing unit 48 so that the captured image or an image based on the captured image is recorded in a recording medium.

[0091] The image processing unit 48 converts the signal sent from the analog signal processing circuit 44 into digital image data, and then performs various processes on the digital image data, such as gamma correction, white balance correction, and scratch correction. Furthermore, the image processing unit 48 compresses the processed digital image data in a compression format that complies with a predetermined standard.

[0092] Then, the image processing unit 48 generates compressed digital image data at a specific frame rate during the image shooting, and obtains an image (strictly speaking, a frame image) from the data. The image (frame image) obtained at this time is equivalent to the image shot at the first field of view angle, that is, the first image.

[0093] Furthermore, the image processing unit 48 executes various processes (for example, extraction processing described later) on the acquired image under the control of the controller 47 , and outputs the processed image to the display 28 frame by frame at a specific frame rate.

[0094] The image processing unit 48 records the processed image in a recording medium and creates an image file. In this way, the image processing unit 48 has a function of creating an image file, and the imaging device 10 having the image processing unit 48 is used as an image creating device.

[0095] In addition, unless otherwise specified, the operations and processes of the controller 47 and the image processing unit 48 are described below as the operations and processes of the control unit 46. The processes performed by the control unit 46 will be described in detail later.

[0096] A memory card 54 that can be loaded and unloaded from the camera body 12 via the internal memory 50 built into the camera body 12 and the card slot 52 is a recording medium, and the image is recorded on the recording medium by the control unit 46. The internal memory 50 and the memory card 54 are composed of a flash memory or a ferroelectric memory. In addition, the internal memory 50 and the memory card 54 may exist outside the camera body 12. In this case, the control unit 46 can record the image on the external recording medium by wire or wirelessly.

[0097] The buffer 56 functions as a working memory of the control unit 46 and is composed of, for example, a DRAM (Dynamic Random Access Memory) or a ferroelectric memory.

[0098] [About the processing by the control unit]

[0099] The processing by the control unit 46 includes display processing, extraction processing, selection processing, switching processing, recording processing, exposure adjustment processing, focus adjustment processing, and white balance adjustment processing.

[0100] Hereinafter, each of the above-mentioned processes will be described separately.

[0101] (Display Processing)

[0102] The display process is a process of displaying various images on the display screen of the display 28. For example, if the control unit 46 starts the display process after the camera 10 is turned on, then at this point in time, the first image captured by the camera 10 at the first viewing angle, that is, Figure 4 The image shown in the imaging area A1 is displayed.

[0103] Furthermore, when the extraction process is executed, the control unit 46 displays the image of the selected area described later on the display screen during the display process. Furthermore, when the selected area is switched (i.e., when the switching process described later is implemented), the control unit 46 switches the image displayed on the display screen to the image of the selected area after the switching process.

[0104] The above-described execution of the display processing by the control unit 46 corresponds to the display step. In the present embodiment, the image displayed on the display 28 by the display processing is a real-time image (live view image) captured in real time at that point in time.

[0105] (Extraction Processing)

[0106] The extraction process is a process of extracting a part of the first image. The extracted image is also called a cropped image (cut image), and is called a recording target in the recording process.

[0107] In the extraction process, the user performs an operation for setting the area to be extracted from the captured area of ​​the first image. Figure 5 , and the setting operation is described in detail. Figure 5 This shows the steps of setting the image area to be extracted from the first image.

[0108] like Figure 5 As shown, the user displays the first image on the display screen of the display 28 (in Figure 5 Specifically, if the user performs a predetermined button operation while the first image P1 is displayed, Figure 5 As shown, the rectangular region setting frame FR is displayed overlapping the first image P1. The region A2 surrounded by the region setting frame FR becomes the image region extracted from the first image P1. The field of view angle of the extracted image is smaller than the field of view angle of the first image P1, that is, the first field of view angle, and the following is the second field of view angle.

[0109] The region setting frame FR is displayed in a state where its position, size, and aspect ratio are changeable. For example, the user touches one side of the region setting frame FR on the touch panel 36 with a finger and drags the finger to change the display position of the region setting frame FR. In addition, the user touches the region setting frame FR with two fingers on the touch panel 36 and expands or reduces the distance between the fingers to change the size and aspect ratio of the region setting frame FR. Thus, by changing the position, size, and aspect ratio of the region setting frame FR, the position, size, and aspect ratio (AspectRatio) of the image region extracted from the first image P1 are changed.

[0110] The above operation is equivalent to the setting operation performed by the user. Figure 6 As shown, by repeating the above series of operations a plurality of times, the regions A2 of the plurality of extracted images can be set in the imaging region A1 of one first image P1. Figure 6 This shows a state where a plurality of regions A2 extracted from the first image P1 are set.

[0111] In addition, when a plurality of areas A2 are set, the size (second viewing angle) of each area A2 may be the same between the areas A2 or may be different between the areas A2.

[0112] Furthermore, the shape of each area A2 is not limited to a rectangle, and may be a square, a parallelogram, a trapezoid, a rhombus, a circle or an ellipse, a triangle or a polygon with more than one pentagon, or an irregular shape.

[0113] Furthermore, there is no particular limitation on the number of areas A2 to be set. In this embodiment, Figure 6 As shown, set it to 2 or more.

[0114] When the setting operation by the user is repeated multiple times, the control unit 46 receives each setting operation and sets the number of areas A2 corresponding to the number of setting operations in the imaging area A1 of the first image P1. This operation of the control unit 46 corresponds to the setting step.

[0115] Furthermore, when the first image P1 is a high-quality image (for example, an image having 10 million or more pixels), the image of the second viewing angle extracted from the first image P1 is an image of sufficiently high quality.

[0116] There is no particular limitation on the number of pixels of the first image P1, and the lower limit thereof may be set to 10 million or more, and more preferably 60 million or more. Furthermore, the upper limit of the number of pixels may be set to 1 billion or less, and more preferably 500 million or less. If the number of pixels exceeds the above lower limit, the visual recognition of the image of the second viewing angle extracted from the first image P1 is ensured. If the number of pixels is lower than the above upper limit, the data volume of the first image P1 is reduced, and the processing speed based on the control unit 46 is increased.

[0117] (Selection and switching processing)

[0118] The selection process is a process of selecting a selected area to be recorded from a plurality of areas A2 set in the imaging area A1 of the first image P1.

[0119] The switching process is a process of reselecting a selected area from the plurality of areas A2 and switching the selected area after the selection process.

[0120] In the present embodiment, the number of regions set as the selected region in the selection process and the switching process is one, but the present invention is not limited thereto, and two or more regions may be selected as the selected region.

[0121] The steps of the selection process and the switching process are described. After the user performs the aforementioned setting operation to set a plurality of areas A2, he selects an arbitrary area A2 as the selected area. After that, the user inputs the selected area A2 by operating the buttons 30, 32, 34 or the touch panel 36, and the control unit 46 receives the input operation. The control unit 46 determines the selected area from the plurality of areas A2 according to the user's input operation. This operation based on the control unit 46 is equivalent to the selection process.

[0122] Furthermore, after the selection process, if the user reselects the selected area to the other area A2 and performs input operation again, the control unit 46 receives the re-input operation and switches the previous selected area to the other area A2 according to the re-input operation. This operation by the control unit 46 corresponds to the switching process.

[0123] In addition, the input operation (re-input operation) performed by the user when reselecting the selection area is not particularly limited. For example, the re-input operation may be an operation of tapping the area A2 to be selected on the display 28, or an operation of pressing any one of the up / down buttons or the left / right buttons on the first operation button 30 as a cross key.

[0124] As described above, in this embodiment, in the selection process, the selected area is selected from the plurality of areas A2, and the image of the selected area is recorded as an image file. Figure 7 The display screen shown is displayed on the display 28. Figure 7 Indicates the image displayed in the selected area on the display screen.

[0125] Furthermore, by performing a switching process after the selection process, the selection area can be switched, and the image of the recording object can be changed from the image of a certain subject to the image of another subject. Thus, for each of a plurality of subjects in the same scene (same position), a separate image (specifically, a close-up image of each subject) can be simultaneously captured by a single camera 10 without using a plurality of devices.

[0126] Furthermore, the area A2 where the recorded image is projected can be switched in time sequence according to the user's operation, and an image file in which the images before and after the switching are projected can be easily acquired.

[0127] Furthermore, in conjunction with the switching process, the image displayed on the display screen of the display 28 is switched from the image of the selected area before the switching process to the image of the selected area after the switching process. Thus, the user can confirm the switching of the selected area on the display screen.

[0128] (Record Processing)

[0129] The recording process is a process in which the control unit 46 records the image of the selected area on the recording medium and creates an image file (more specifically, a dynamic image file) related to the image. In the recording process, the action of the control unit 46 recording the image of the selected area is equivalent to the recording step. In addition, in the recording process, the action of the control unit 46 creating the image file is equivalent to the creation step.

[0130] In the present embodiment, recording of images and creation of image files have the same meaning, and the creation process is equivalent to the recording process.

[0131] Furthermore, when the switching process is implemented, the control unit 46 records the image of the selected area before the switching process and the image of the selected area after the switching process to create an image document. To explain in detail, in the creation process, the control unit 46 combines the image (dynamic image) of the selected area before the switching process and the image (dynamic image) of the selected area after the switching process to create a dynamic image document as an image document. Figure 8 As shown, a dynamic image file of an image in which the subject changes due to switching of the selection area is acquired. Figure 8 This is an explanatory diagram of a moving image file of recorded images.

[0132] (Exposure adjustment processing)

[0133] The exposure amount adjustment process is a process of adjusting the exposure amount of the selected area, and is executed using the AE (Automatic Exposure) function of the control unit 46 .

[0134] The exposure amount is determined based on a plurality of parameters including the exposure time (i.e., the shutter speed of the electronic shutter), the sensitivity (ISO sensitivity) of the pixel 42 of the imaging element 40, and the aperture amount (F value) relative to the incident light of the shooting lens 14. The value of each parameter is determined, and the aperture drive unit 23, the imaging element 40, and the image processing unit 48 are controlled in such a way that each parameter becomes the determined value, thereby adjusting the exposure amount. Here, the adjustment of the aperture 20 based on the aperture drive unit 23 is performed by another well-known automatic light adjustment (Automatic Light Control: ALC) function. In addition, the shutter speed of the electronic shutter in the imaging element 40 is adjusted using the well-known electronic light control (Electronic Light Control: ELC) function.

[0135] Furthermore, when the electronic neutral density filter 21 is provided in the photographing lens 14, the plurality of parameters include the dimming degree of the electronic neutral density filter 21. In this case, the value of the dimming degree of the electronic neutral density filter 21 is determined, and the voltage applying unit 24 is controlled so that the dimming degree becomes the determined value, thereby adjusting the exposure amount.

[0136] The appropriate value of the exposure amount of the selected area (appropriate exposure amount) changes depending on which area among the plurality of areas A2 set in the imaging area A1 of the first image P1 is the selected area.

[0137] To explain in more detail, when the first image P1 is photographed, the exposure amount at each part of the photographing area A1 including a plurality of areas A2 changes according to the subject and environment present at each part. That is, the appropriate exposure amount of each area A2 in the photographing area A1 can change according to the position of the area A2, and is therefore determined for each area A2.

[0138] The appropriate exposure amount of each area A2 is determined by a known procedure. For example, based on the image signal output from the pixel 42 corresponding to the unit area A0 constituting a certain area A2 in the imaging element 40, the brightness of R (red), G (green), and B (blue) (i.e., RGB pixel value) is accumulated for the area A2. Here, the pixel 42 corresponding to the unit area A0 is Figure 4 In the positional relationship shown, the pixel 42 is located at the same configuration position as the unit area A0. For example, Figure 4The unit area A0 in the area A2 surrounded by a thick frame in the figure corresponds to the pixel 42 in the range surrounded by a thick frame in the same figure. Then, based on the cumulative value of the RGB pixel values ​​calculated for the above-mentioned area A2 (specifically, the average value in the area A2), the appropriate exposure amount for the image of the area A2 is determined.

[0139] The exposure of the selected area is adjusted according to the appropriate exposure determined in the above-mentioned method. Specifically, the control unit 46 determines the values ​​of the exposure time (shutter speed) and the aperture value (F value) according to the appropriate exposure of the specified area. This operation is equivalent to the condition determination process based on the control unit 46. The values ​​of each parameter determined in the condition determination process are values ​​representing the exposure conditions when the first image including the image of the selected area is photographed.

[0140] In addition, in the present embodiment, both the exposure time and the aperture amount are determined, but the present invention is not limited to this, and either the exposure time or the aperture amount may be determined.

[0141] After determining the exposure time and the aperture value, the control unit 46 determines the sensitivity (ISO sensitivity) of the pixels 42 corresponding to the selected area based on the combination of these values. Strictly speaking, the gain (amplification ratio) of the output signal from the pixels 42 corresponding to the selected area is determined. This operation corresponds to the sensitivity determination process by the control unit 46.

[0142] The gain may be, for example, an analog gain for an analog signal in the analog signal processing circuit 44 or the like, or a digital gain for a signal (data) after digital conversion in the video processing unit 48 or the like.

[0143] After determining the exposure time, the aperture amount, and the sensitivity, the control unit 46 controls the aperture drive unit 23, the imaging element 40, the image processing unit 48, and the like so that these parameters have the determined values.

[0144] During the image recording, the exposure adjustment process is repeatedly performed periodically (for example, every frame) according to the above steps. Then, the values ​​of exposure time, aperture value and sensitivity are determined every time the exposure adjustment process is performed.

[0145] Specifically, in the shooting of the image, the cumulative value of the RGB pixel values ​​in the specified area of ​​the Nth (N is a natural number) frame image is calculated. Then, the appropriate exposure amount based on the calculated cumulative value is calculated, and the exposure time and aperture amount when shooting the next (i.e., the N+1th) frame image are determined based on the calculated appropriate exposure amount.

[0146] Then, the integrated value of the RGB pixel values ​​in the selected area in the Nth next frame image is obtained, and the appropriate exposure of the selected area is calculated based on the integrated value obtained. Then, the sensitivity of the pixel 42 corresponding to the selected area in the Nth next frame image is determined based on the calculated appropriate exposure and the exposure time and aperture value at that time.

[0147] In this embodiment, the exposure time and the aperture value change over time, and accordingly, the exposure value when shooting the first image is adjusted over time. In addition, the sensitivity of the pixels 42 corresponding to the selected area changes over time, and as a result, the exposure value of the selected area is adjusted over time.

[0148] In the present embodiment, “adjustment over time” means that an adjustment amount is determined for each frame for a parameter to be adjusted, and each parameter is increased or decreased by the adjustment amount for each frame.

[0149] (Focus adjustment processing)

[0150] The focus adjustment process is a process in which the control unit 46 controls the focus drive unit 22 in the autofocus mode to automatically adjust the focus of the photographing lens 14. In the focus adjustment in the autofocus mode, a known autofocus technique can be used.

[0151] (White balance adjustment processing)

[0152] The white balance adjustment process is a process for automatically adjusting the white balance of the image of the selected area by the control unit. The white balance adjustment process is performed using a known AWB (Auto White Balance) function, for example, the technology described in Japanese Patent Application Laid-Open No. 2009-33410.

[0153] [Regarding the Process of Video Production According to the First Embodiment]

[0154] refer to Fig. 9 , the process of image production involved in the first embodiment is explained. Fig. 9 This is an explanatory diagram of the video production flow involved in the first embodiment.

[0155] In addition, for convenience of explanation below, a period of capturing one first video (frame image) is referred to as “one frame”.

[0156] In the image production process, the user first starts the imaging device 10. After the device is started, the control unit 46 performs initial settings for various conditions including imaging conditions and exposure conditions (S001). In this step S001, each of the multiple parameters that determine the exposure amount, such as exposure time (shutter speed), aperture value (F value) and sensitivity (ISO sensitivity), is preset to an initial value.

[0157] Then, the image capturing device 10 starts capturing the first image at the first viewing angle (ie, capturing the first image). In conjunction with this, the control unit 46 performs a display process, and displays the first image as a real-time image on the display screen of the display 28 (S002).

[0158] The user performs the above setting operation multiple times, and the control unit 46 receives each setting operation to implement the setting process (S003). In this step S003, the control unit 46 sets multiple areas A2 in the shooting area A1 of the first image according to the setting operation performed by the user through the first image displayed on the display screen.

[0159] When the user selects a selected area from the plurality of areas A2 and inputs the selection result through the touch panel 36, etc., the control unit 46 implements a selection process (S004). In this step S004, the control unit 46 selects a selected area from the plurality of areas A2 according to the user's input operation.

[0160] Thereafter, when the user performs an operation such as pressing the release button 26 to start recording an image, the steps after step S005 are performed.

[0161] First, in step S005, the control unit 46 acquires one frame of the first image (i.e., frame image). In the next step S006, the control unit 46 calculates the cumulative value of the RGB pixel values ​​of the image of each area A2 for each of the multiple areas A2 set in step S003 in the acquired first image. In the subsequent step S007, the control unit 46 determines the appropriate exposure amount for each area A2 based on the cumulative value of the RGB pixel values ​​calculated for each area A2.

[0162] After executing step S007, the control unit 46 executes a sensitivity determination process to determine the sensitivity (strictly speaking, gain) of the pixels 42 corresponding to the selected area in the imaging element 40 (S008). In the first (first) step S008 in the image production process, the sensitivity of the pixels 42 corresponding to the selected area is determined based on the appropriate exposure amount of the selected area, the exposure time and the aperture amount set as initial values ​​in advance.

[0163] Then, in step S008 , the control unit 46 changes the gain to adjust the exposure amount of the selected area so that the sensitivity of the pixels 42 corresponding to the selected area becomes the determined sensitivity.

[0164] After that, the control unit 46 performs a recording process (production process) to record the image of the selected area on the recording medium, and starts to produce a dynamic image file as the image file (S009). In this step S009, the control unit 46 records the image of the selected area at the sensitivity (gain) determined in step S008. In addition, the control unit 46 displays the image of the selected area on the display screen of the display 28.

[0165] After that, the control unit 46 performs a designation process to designate a predetermined area among the plurality of areas A2 set in step S003 as a reference area (S010). In step S010, the second area of ​​the first area and the second area included in the plurality of areas A2, in which the appropriate exposure is smaller than that of the first area, is designated as the reference area. Here, the first area and the second area are relative concepts, and when comparing the plurality of areas A2, the area with a larger appropriate exposure is equivalent to the first area, and the area with a smaller appropriate exposure is equivalent to the second area.

[0166] In the present embodiment, the region A2 having the smallest appropriate exposure among the plurality of regions A2 is designated as the reference region, but the present invention is not limited thereto, and the region A2 having a smaller appropriate exposure than any other region A2 may be designated as the reference region.

[0167] After designating the reference area, the control unit 46 implements a condition determination process to determine the exposure time and aperture value when capturing the first image in the next frame (hereinafter referred to as the next exposure time and aperture value) (S011). In this step S011, the control unit 46 determines the next exposure time and aperture value based on the appropriate exposure value of the reference area.

[0168] Then, in step S011, the control unit 46 adjusts the exposure time and the aperture value so that the values ​​become the values ​​determined in step S011. Thus, in the next frame, the first image (frame image) is captured with the exposure time and the aperture value of the appropriate exposure value based on the reference area specified immediately before.

[0169] In the steps described above, a series of operations after step S005 of acquiring the first image are repeatedly performed during the image recording period. Therefore, if the exposure time and aperture value are determined in step S011, the first image is acquired with the above-mentioned exposure time and aperture value in the frame immediately thereafter (i.e., step S005 after the second time).

[0170] When the user selects a selection area once and then reselects and performs input operation again (S012), the control unit 46 performs a switching process (S013). In this step S013, the control unit 46 reselects and switches the selection area from the plurality of areas A2 according to the user's input operation.

[0171] When step S013 is implemented and the selected area is switched, the control unit 46 records the image of the selected area before the switching process and the image of the selected area after the switching process in step S009 in the frame immediately following the switching process to create an image document. To explain in more detail, the control unit 46 combines the image of the selected area before the switching process and the image of the selected area after the switching process to create a dynamic image document as an image document.

[0172] Then, until the user performs a predetermined operation and issues an end instruction, the control unit 46 repeats the above-mentioned steps S005 to S013 ( S014 ).

[0173] As described above, the present embodiment is characterized in that an area with a smaller appropriate exposure value among the plurality of areas A2 is set as a reference area, and the next exposure time and aperture value are determined based on the appropriate exposure value of the reference area. Thus, overexposure can be suppressed when switching the selected image, so that the image after switching has good image quality.

[0174] The above-mentioned effect will be described in detail. Usually, when the selected area is switched, the exposure time and the aperture value are determined so that the exposure amount of the selected area after switching becomes an appropriate exposure amount.

[0175] On the other hand, among the multiple areas A2, sometimes the brightness of the area A2 other than the selected area (hereinafter referred to as the unselected area) is greater than the brightness of the selected area. In this case, if the exposure of the selected area is determined based on its appropriate exposure, the RGB pixel values ​​of the image of the unselected area may be saturated. As a result, it may be impossible to accurately calculate the appropriate exposure of each area A2. In addition, when the unselected area (strictly speaking, the unselected area whose appropriate exposure is smaller than that of the selected area) is switched to the selected area, overexposure may occur.

[0176] In contrast, in this embodiment, the area A2 with a smaller appropriate exposure (strictly speaking, the area A2 with the smallest appropriate exposure) is set as the reference area, and the exposure time and aperture value are determined based on the appropriate exposure of the reference area. This can suppress the above-mentioned problems such as overexposure, and can more appropriately determine conditions such as exposure time in image capture.

[0177] Furthermore, in this embodiment, as described above, during the recording of the image, the above steps S005 to S013 are performed for each frame. Then, when the first area with a larger appropriate exposure amount among the plurality of areas A2 is selected as the selected area, in the second and subsequent frames, the exposure amount of the selected area is determined based on the conditions determined in the immediately preceding frame.

[0178] Specifically, in the second and subsequent frames, the sensitivity (gain) of the pixel 42 corresponding to the selected area (i.e., the first area) is determined in the sensitivity determination process implemented in step S008. At this time, the sensitivity is determined based on the appropriate exposure of the first area as the selected area at this time and the exposure time and aperture determined in step S011 (i.e., the condition determination process) in the immediately preceding frame. And, in this embodiment, the above-mentioned sensitivity is determined to be higher than the sensitivity of the pixel 42 corresponding to the second area as the reference area. Thus, the sensitivity of the pixel 42 corresponding to the selected area is appropriately determined, and the exposure of the first area as the selected area is adjusted to be the appropriate exposure.

[0179] In addition, as a method of increasing the exposure of the selected area for optimization, other methods other than the method of increasing the sensitivity of the pixel 42 corresponding to the selected area as described above can also be considered. For example, the exposure of the selected area can be increased by performing frame synthesis on the image of the selected area. Frame synthesis refers to signal processing that adds image signals of a plurality of frames continuous on the time axis and performs signal addition.

[0180] However, if a moving subject is reflected in the image being frame-combined, an afterimage of the subject may occur due to the frame-combination. Therefore, the adjustment of the exposure amount based on frame-combination is only applicable to non-moving subjects.

[0181] <<Second embodiment>>

[0182] refer to Fig.10 , a second embodiment of redesignating the reference area only when the selected area is switched will be described. Fig.10 It is an explanatory diagram about the video production process involved in the second embodiment.

[0183] In addition, the following mainly describes the differences between the second embodiment and the first embodiment.

[0184] As compared Fig. 9 and 10 It can be seen that each step in the video production flow according to the second embodiment is substantially the same as each step in the video production flow according to the first embodiment.

[0185] Specifically, steps S021 to S029 in the video production process of the second embodiment are the same as steps S001 to S009 in the video production process of the first embodiment. In addition, during the video recording period, the steps after step S025 are repeated for each frame. In this respect, the video production process of the second embodiment is the same as the video production process of the first embodiment.

[0186] On the other hand, in the second embodiment, once the reference area is designated after the designated process is performed, the designated process is performed only when the switching process is performed. In addition, in the second embodiment, the condition determination process is performed every time the designated process is performed. In the above points, the second embodiment is different from the first embodiment.

[0187] The differences between the image production process involved in the second embodiment and the first embodiment are described as follows. Fig.10 As shown, in the second embodiment, it is determined whether a reference area has been designated in each frame ( S030 ).

[0188] When the reference area is not specified, the control unit 46 performs a specifying process and specifies the second area (specifically, the area with the smallest appropriate exposure amount) among the plurality of areas A2 as the reference area (S031). Thereafter, the control unit 46 performs a condition determination process and determines the next exposure time and aperture value based on the appropriate exposure amount of the reference area specified in step S031 (S032).

[0189] Furthermore, under normal circumstances, steps S031 and S032 are performed only immediately after recording of the captured image starts (ie, the first frame).

[0190] When the reference area has been designated, it is determined whether the user has reselected the selected area after selecting it once and performed the input operation again (S033). When the selected area has not been reselected, the reference area at that time (i.e., the current reference area) is maintained. Then, when the control unit 46 determines the next exposure time and aperture amount in the condition determination step, these values ​​are determined based on the appropriate exposure amount of the current reference area (S034).

[0191] On the other hand, when the selected area is reselected, the control unit 46 implements a switching process, and reselects the selected area from the plurality of areas A2 and switches it according to the input operation of the user (S035). In addition, the control unit 46 implements a designation process in conjunction with the switching process, and redesignates the reference area in the designation process (S036). In addition, the control unit 46 determines the next exposure time and aperture value according to the appropriate exposure amount of the redesignated reference area in the subsequent condition determination process (S037).

[0192] In the video production flow according to the second embodiment, steps S025 to S037 ( S038 ) described above are repeatedly performed until the user issues an end instruction.

[0193] As described above, in the video production process according to the second embodiment, only when the selected area is switched, the reference area is allowed to be re-designated. This makes the quality of the selected video more stable than in the first embodiment where the reference area is changed every frame.

[0194] Note that the second embodiment is the same as the first embodiment except for the above-mentioned difference, and can obtain the same effects as those obtained in the first embodiment.

[0195] <<Third embodiment>>

[0196] refer to Fig.11 A third embodiment is described in which conditions such as exposure time are determined based on an appropriate exposure amount for a reference area immediately after a selection area is selected or reselected. Fig.11 This is an explanatory diagram of the video production flow according to the third embodiment. In the following, the third embodiment will be mainly described in terms of differences from the first embodiment.

[0197] according to Fig.10 and 11 It can be seen that each step in the video production flow according to the third embodiment is substantially the same as each step in the video production flow according to the second embodiment.

[0198] In the image production process involved in the third embodiment, the control unit 46 performs a designation process (S051, S056) immediately after the selection area is initially selected or immediately after the selection area is reselected. In the designation process, the control unit 46 designates the area A2 corresponding to the second area at that time (strictly speaking, the area A2 with the smallest appropriate exposure amount) as the reference area. Then, in the condition determination process (i.e., S052, S057) performed immediately after the designation process, conditions such as the exposure time are determined based on the appropriate exposure amount of the reference area designated in the designation process. Hereinafter, this condition determination process is referred to as the "first determination process".

[0199] On the other hand, in a frame where a selection area has been selected and the selection area has not been reselected (switched), the designation process is not performed. In the condition determination process (S054) in such a frame, conditions such as exposure time are determined based on the appropriate exposure amount of the reference area at the time when the selection area is selected or the time when the selection area is reselected, and the amount of change in the appropriate exposure amount of the current selection area. Hereinafter, such a condition determination process is referred to as a "second determination process".

[0200] In the first determination step, conditions such as exposure time are determined based on the appropriate exposure of the reference area at that point in time. In addition, the second determination step is implemented in the adjustment step after the first determination step is implemented. In the second determination step, the adjustment amount of the exposure in the adjustment step (specifically, the exposure when the first image is captured) is determined in consideration of the change over time of the appropriate exposure of the selected area at that point in time. In the adjustment step, the exposure when the first image is captured is adjusted based on the conditions such as exposure time determined in the first determination step and the adjustment amount determined in the second determination step.

[0201] refer to Figure 12 to Figure 15 , explain the above contents in detail.

[0202] Fig.12 The time-dependent changes in the appropriate exposure amount of each of the reference area and the selected area are shown. Fig.13 Indicates the adjusted exposure amount of the selected area when further adjustment is made based on the appropriate exposure amount of the reference area. Fig.14 The target exposure amount in the third embodiment is shown. Fig.15 The figure shows transitions of the exposure amounts after adjustment in the reference area and the selected area in the third embodiment.

[0203] In addition, Figure 12 to Figure 15 In each of the figures, the horizontal axis represents the elapsed time (specifically, the number of elapsed frames) from the start time of image recording, and the vertical axis represents the magnitude of the exposure amount.

[0204] Here, the “adjusted exposure amount” refers to an actual value that the exposure amount adjusted for each frame in the adjustment step finally reaches in each frame.

[0205] Furthermore, the "target exposure amount" is a target value that is expected to be achieved in each frame by adjusting the exposure amount for each frame in the adjustment process, and is set to a value corresponding to the appropriate exposure amount (for example, a value corresponding to the appropriate exposure amount).

[0206] As a specific example, assume that the appropriate exposure amounts for the reference area and the selected area are as follows: Fig.12 In this example, the appropriate exposure of the selected area (marked as EIt in the figure) is relatively stable, while the appropriate exposure of the reference area (marked as EIs in the figure) varies greatly. Fig.12 In the example shown, it is assumed that the selection area is reselected (switched) at a point in time when the elapsed time tc has elapsed.

[0207] In the above example, it is assumed that the exposure of the first image is adjusted for each frame. In each frame, the exposure is adjusted based on the appropriate exposure of the reference area. In this case, the exposure of the reference area and the selected area after adjustment is as follows: Fig.13The exposure amount of the reference area after adjustment (in the figure, ERs) varies greatly in the same manner as the appropriate exposure amount. The exposure amount of the selected area after adjustment (in the figure, ERt) shows the same change tendency as the appropriate exposure amount by changing the sensitivity (gain) of the pixel 42 corresponding to the selected area in such a way as to obtain the appropriate exposure amount.

[0208] However, the sensitivity (gain) for determining the exposure of the selected area after adjustment is affected by the exposure of the reference area after adjustment, in other words, the appropriate exposure of the reference area. Therefore, if the appropriate exposure of the reference area changes significantly, the sensitivity (gain) changes significantly, and thus, there is a possibility that the image of the selected area may have an increase or decrease in noise or other undesirable conditions.

[0209] In contrast, in the third embodiment, the first determination step is performed only by specifying the reference area immediately after the selection area is selected or immediately after the selection area is reselected. In the first determination step, the reference area is determined based on the appropriate exposure amount ( Fig.12 In the figure, marked as e1), to determine the exposure time and other conditions.

[0210] Furthermore, during the period after the first determination step (i.e., during the period when the selected area has been selected and has not been reselected), the second determination step is performed for each frame. In the second determination step of each frame, the exposure adjustment amount is determined based on the appropriate exposure amount e1 of the reference area when the selected area is selected or reselected, and the change in the appropriate exposure amount of the current selected area. Here, the "change in the appropriate exposure amount of the selected area" refers to the difference between the appropriate exposure amount of the selected area in the immediately previous frame and the appropriate exposure amount of the selected area in this frame, Fig.12 In the figure, it is marked with the symbol Δ.

[0211] As described above, the first determination step is performed, and then the second determination step is performed for each frame, thereby determining the target exposure amount in each frame. Fig.14 As shown in FIG. 1 , the target exposure (in the figure, marked as EG) at the time point tc when the selected area is reselected is set according to the appropriate exposure e1 of the reference area at that time point. The subsequent target exposure EG is determined as the appropriate exposure of the selected area changes. Specifically, only the difference Δ of the appropriate exposure changes in each frame.

[0212] Then, in each frame in the adjustment process, the exposure time and other conditions are determined so that the exposure amount when the first image is captured becomes the target exposure amount EG determined in each frame. Fig.15As shown in FIG. 1 , the exposure amount ERs of the reference area after adjustment becomes the exposure amount corresponding to the target exposure amount EG described above. Here, the target exposure amount EG reflects the change of the appropriate exposure amount of the selected area. Fig.15 As shown, the exposure amount ERs of the reference area after adjustment changes relatively stably.

[0213] Furthermore, in each frame of the adjustment process, the exposure amount of the selected area is adjusted according to the conditions such as the exposure time determined in each frame. To explain in more detail, in each frame, the sensitivity (gain) of the pixel 42 corresponding to the selected area is determined according to the conditions such as the exposure time determined in each frame and the appropriate exposure amount of the selected area in each frame. Fig.15 As shown in FIG. 1 , the exposure amount ERt of the selected area after adjustment becomes an exposure amount corresponding to the appropriate exposure amount in each frame.

[0214] Moreover, according to Fig.15 It can be seen that the sensitivity (gain) for determining the exposure amount of the selected area after adjustment is relatively stable. This is because the target exposure amount EG, which is the basis for determining the sensitivity, is stable. As a result, the increase or decrease of noise in the image of the selected area is suppressed.

[0215] As described above, in the third embodiment, the conditions such as the exposure time are determined based on the appropriate exposure of the reference area only after the selected area is selected or after the selected area is reselected. For the rest of the period, the exposure adjustment amount is determined based on the change in the appropriate exposure of the selected area. Thus, the image quality of the image of the selected area can be stabilized during the period when the selected area is not reselected (switched).

[0216] <<Fourth embodiment>>

[0217] refer to Fig.16 , a fourth embodiment is described in which the exposure amount is adjusted at a faster adjustment speed than usual just after the switching process is performed. Fig.16 It is an explanatory diagram of the video production process involved in the fourth embodiment.

[0218] according to Fig.16 It can be seen that steps S061 to S067 in the video production flow according to the fourth embodiment are the same as steps S001 to S007 in the video production flow according to the first embodiment.

[0219] Hereinafter, steps S068 to S074 in the video production flow according to the fourth embodiment will be described in detail. In the video production flow according to the fourth embodiment, when the switching process is performed, the subsequent steps are changed according to the elapsed time from the execution point of the switching process (S070).

[0220] Specifically, it is assumed that at a certain point in time (a certain frame), the control unit 46 implements a switching process to switch the selected area. In this case, the control unit 46 measures the elapsed time from the frame (strictly speaking, the number of frames that have passed) in step S070. Then, the control unit 46 implements the first recording process (S071) during the period from the implementation of the switching process immediately before to the first period. The first recording process is a process for recording the image of the selected area in the first period. The first period is determined as an arbitrary period, but is preferably determined to be within a range of 1 second.

[0221] Furthermore, when the current time point (current frame) is in the first period, the control unit 46 implements an adjustment process (S072) for adjusting the exposure amount of the selected area for the next frame. In this step S072, the exposure amount of the selected area is adjusted at a faster adjustment speed than usual based on the appropriate exposure amount at the time point. Specifically, when the aperture drive unit 23 is driven to change the aperture amount in order to adjust the exposure amount in the adjustment process, the drive speed of the aperture drive unit 23 is set to be faster than the usual drive speed.

[0222] The adjustment speed in the adjustment process in the first period can be arbitrarily determined as long as it is faster than the normal adjustment speed, but is preferably 1.5 times or more faster than the normal adjustment speed, and more preferably 2 times or more faster.

[0223] On the other hand, after the first period has passed since the switching process was performed immediately before, that is, when the current time point (current frame) is in the second period other than the first period, the control unit 46 performs the second recording process (S073). The second recording process is a process of recording the image of the selected area in the second period.

[0224] Furthermore, when the current time point (current frame) is in the second period, the control unit 46 implements an adjustment process (S074) for adjusting the exposure of the selected area for the next frame. In this step S074, the exposure of the selected area is adjusted at a normal adjustment speed based on the appropriate exposure at this time point. Specifically, in order to adjust the exposure, the aperture drive unit 23 is driven at a normal drive speed to change the aperture amount.

[0225] As described above, in the fourth embodiment, in the adjustment process performed in the first period, the adjustment speed of the exposure amount (specifically, the driving speed of the aperture drive unit 23) is set faster than the adjustment process performed in the second period. As a result, when the switching process is performed, the exposure amount of the selected area after the switching process can be quickly adjusted (optimized). As a result, the image of the selected area after the switching process can be obtained with an appropriate exposure amount and the image quality can be improved.

[0226] In the above example cited as a specific example of the fourth embodiment, the exposure of the selected area is adjusted in the adjustment process. However, in addition to the exposure, the conditions that should be quickly adjusted after the switching process is implemented may include, for example, the white balance of the image of the selected area and the focus of the photographing lens 14.

[0227] In the fourth embodiment, an adjustment process can be performed in each of the first period and the second period, and the adjustment process adjusts at least one of the exposure amount of the selected area, the white balance of the image of the selected area, and the focus. In this case, in the adjustment process performed in the first period, the adjustment speed of the condition (condition of the adjustment object) can be set faster than that of the adjustment process performed in the second period.

[0228] Specifically, when the aperture drive unit 23 is driven to change the aperture amount in order to adjust the exposure amount in the adjustment process, the driving speed of the aperture drive unit 23 in the adjustment process in the first period may be set faster than that in the adjustment process in the second period.

[0229] Furthermore, when the optical component 19 for focusing is moved for adjusting the focus in the adjustment process, the moving speed of the optical component 19 for focusing may be set faster in the adjustment process in the first period than in the adjustment process in the second period.

[0230] <<Fifth embodiment>>

[0231] refer to Fig.17 and Fig.18 A fifth embodiment is described in which, immediately after the switching process is performed, the exposure amount is preferably adjusted by changing parameters other than the aperture amount. Fig.17 This is an explanatory diagram of the video production process involved in the fifth embodiment. Fig.18 This shows an example of a change pattern specified by the adjustment data.

[0232] In addition, for the sake of explanation, Fig.18 Each numerical value of the change pattern shown is a prescribed value, and is different from the prescribed value in the actual change pattern.

[0233] The image production process involved in the fifth embodiment is substantially the same as the image production process involved in the fourth embodiment. That is, in the image production process involved in the fifth embodiment, various conditions are initially set, the first image is displayed as the image is captured, a plurality of areas A2 are set, and a selection area is selected from them (S081 to S084). Thereafter, the process after step S085 (steps S085 to S094) is performed based on the user's pressing operation of the release button 26 or the like.

[0234] Specifically, a first image of one frame is acquired, the cumulative value of RGB pixel values ​​is calculated for each of the plurality of regions A2, and an appropriate exposure is determined based on the cumulative value (S085-S087). Thereafter, the image of the selected region is recorded (S091, S093), and the exposure of the selected region is adjusted for the next frame (S092, S094).

[0235] Then, when the selected area is reselected, a switching process is performed to switch the selected area (S088, S089). The step after the switching process is performed is changed according to the elapsed time from the execution point of the switching process (S090).

[0236] Specifically, when the current time point (current frame) is in the first period described above, the control unit 46 implements the first recording process (S091). Also, when the current time point (current frame) is in the first period, the control unit 46 implements the first adjustment process and adjusts the exposure amount of the selected area for the next frame (S092).

[0237] On the other hand, when the current time point (current frame) is in the aforementioned second period, the control unit 46 implements the second recording process (S093). Furthermore, when the current time point (current frame) is in the second period, the control unit 46 implements the second adjustment process and adjusts the exposure amount of the selected area for the next frame (S094).

[0238] The above series of steps (ie, steps S085 to S094 ) are repeatedly performed until the user issues an end instruction ( S095 ).

[0239] The first adjustment process and the second adjustment process in the fifth embodiment are described. In each adjustment process, the exposure of the selected area is adjusted to the target exposure using the adjustment data based on the appropriate exposure at that point in time. To explain in more detail, in each adjustment process, the values ​​of multiple parameters including the exposure time (shutter speed), the sensitivity of the pixel 42 (ISO sensitivity), and the aperture (F value) are determined based on the adjustment data. Thereafter, the values ​​of each parameter are changed to the determined values, and the exposure of the selected area is adjusted to the target exposure.

[0240] The adjustment data is data that specifies the change pattern, and is prepared in the same amount as the change pattern and recorded in the internal memory 50 or buffer 56 of the imaging device body 12. The change pattern specifies the pattern (change tendency) of changing each value of a plurality of parameters in order to adjust the exposure to the target exposure.

[0241] Then, in the fifth embodiment, the adjustment data used for adjusting to the same target exposure amount are different between the first adjustment process and the second adjustment process. That is, the first adjustment data used in the first adjustment process and the second adjustment data used in the second adjustment process have different values ​​of the plurality of parameters prescribed for the same target exposure amount.

[0242] refer to Fig.18 , the differences between the first adjustment data and the second adjustment data are explained. Fig.18 The two adjustment data shown in FIG. 1 and FIG. 2 each define the change pattern of the aperture value, exposure time, and sensitivity for adjusting the exposure amount from the value ep to the target value et (target exposure amount). Fig.18 As shown, in the change pattern defined in the adjustment data, the value of each parameter at the adjustment start point and the value of each parameter at the adjustment end point are the same between two adjustment data.

[0243] On the other hand, in the first adjustment data ( Fig.18 In the change pattern prescribed by the upper side, the aperture amount gradually increases in stages. The exposure time, which is a parameter other than the aperture amount, increases sharply just after the adjustment starts, and then decreases as the aperture amount increases.

[0244] On the other hand, in the second adjustment data ( Fig.18 In the change pattern specified in the figure (shown at the bottom), the aperture value increases to the final value immediately after the adjustment starts, and then remains at the final value. During the readjustment period, the exposure time does not change and remains at a constant value.

[0245] As described above, in the change pattern defined by the first adjustment data, the degree of change (the amount of change per unit time) of the aperture amount with respect to the target exposure amount becomes smaller than that in the change pattern of the second adjustment data.

[0246] In other words, in the second adjustment step using the second adjustment data, priority is given to changing the aperture amount when adjusting the exposure amount, whereas in the first adjustment step using the first adjustment data, priority is given to changing parameters other than the aperture amount when adjusting the exposure amount.

[0247] As described above, in the fifth embodiment, when adjusting the exposure amount just after switching the selected area, the change of parameters other than the aperture amount (specifically, the exposure time) is given priority over the change of the aperture amount accompanying the mechanical drive. Since the exposure time, that is, the shutter speed of the electronic shutter is changed by digital processing, it can be changed faster than the aperture amount. Thus, the exposure amount of the selected area after switching can be quickly adjusted. In addition, since the slight change in brightness accompanying the sharp change in the aperture amount is suppressed, the influence caused by the slight change on the image quality can be suppressed. As a result, the image of the selected area after the switching process can be made of good quality.

[0248] In the above example, in the first adjustment step, the exposure time is changed with priority over the aperture value, but this is not limited to this. In the first adjustment step, the sensitivity (gain) of the pixel 42 corresponding to the selected area may be changed with priority over the aperture value.

[0249] Furthermore, as long as the structure includes the electronic neutral density filter 21, the plurality of parameters for adjusting the exposure amount may include the dimming degree of the electronic neutral density filter 21. In this case, the dimming degree of the electronic neutral density filter 21 (more directly, the voltage applied to the electronic neutral density filter 21) can be changed in priority to the aperture amount. Thus, the change of the aperture amount and the increase in the shutter speed can be suppressed.

[0250] <<Sixth embodiment>>

[0251] In the above five embodiments, the image is a dynamic image, that is, a collection of multiple frame images continuously captured at a constant frame rate. However, the recorded image is not limited to a dynamic image, and may also be a static image.

[0252] For example, the control unit 46 displays the first image as a live view image on the display 28, and sets a plurality of partial areas A2 in the shooting area A1 of the first image. Then, when the user selects one of the plurality of set areas A2 as a selected area, the control unit 46 extracts an image of the selected area and displays the image as shown in FIG. Figure 7 The display 28 displays the image. When a user inputs a recording instruction while the extracted image of the selected area is displayed, the control unit 46 records a still image file of the image of the selected area in a recording medium.

[0253] Furthermore, when the recorded image is a still image, the shutter speed that defines the exposure time may be the shutter speed of an electronic shutter or the shutter speed of the shutter 38 (focal plane shutter) which is a mechanical component.

[0254] <<Other implementation methods>>

[0255] The embodiment described above is a specific example given to explain the video production method according to one embodiment of the present invention in an easy-to-understand manner. This is just an example, and other embodiments are also conceivable.

[0256] For example, in the above-mentioned embodiment, the imaging device 10 including the electronic ND filter 21 is described, but an imaging device not including the electronic ND filter 21 may also be used.

[0257] Furthermore, in the above-mentioned embodiment, the imaging device 10 is provided with an autofocus function, but the present invention is not limited thereto, and may be an imaging device without an autofocus function. In other words, an imaging device that is not provided with a focus drive unit 22 and adjusts the focus only by operating the focus ring 16 (i.e., manually) may also be used.

[0258] Furthermore, in the above-mentioned embodiment, the camera 10 is used as an image production device to produce an image file, but the present invention is not limited thereto. For example, other devices connected to the camera by wire or wireless means, such as a camera controller or an external recorder, may also be used as an image production device. Then, an image file of the image captured by the camera can be produced by these devices.

[0259] Furthermore, in the above-mentioned embodiment, the imaging device is a digital camera, but it may also be a video camera, a mobile phone with an imaging optical system, a smartphone, a mobile terminal such as a tablet terminal, or the like.

[0260] Furthermore, the photographing lens may be a lens unit external to the imaging optical system of the mobile terminal.

[0261] Explanation of symbols

[0262] 10-shooting device, 12-shooting device body, 13-mount, 14-shooting lens, 16-focusing ring, 18-optical parts unit, 19-optical parts for focusing, 20-aperture, 21-electronic neutral density filter, 22-focusing drive unit, 23-aperture drive unit, 24-voltage application unit, 26-release button, 28-display, 30-first operation button, 32-second operation button, 34-third operation button, 36-touch panel, 38-shutter, 40-imaging element, 42-pixel, 44-analog signal processing circuit, 46-control unit, 47-controller, 48-image processing unit, 50-internal memory, 52-card slot, 54-memory card, 56-buffer, A0-unit area, A1-shooting area, A2-area, L1-optical axis, FR-area setting frame.

Claims

1. An image production method for producing an image document based on an image captured by a camera having a camera lens and an imaging element, the image production method comprising: A setting step of setting, in a current frame, a plurality of areas having a second field of view angle smaller than the first field of view angle in a shooting area of ​​a first image having a first field of view angle; a designation step of designating, among the first region and the second region included in the plurality of regions, the second region having a smaller appropriate exposure amount than the first region as a reference region; a condition determination step of determining, based on the appropriate exposure amount of the reference area in the current frame, an exposure time when capturing the first image in the next frame and an aperture amount with respect to incident light to the capturing lens; An adjustment step of capturing a first image in a next frame with the determined exposure time and aperture value; A selection step of selecting a selected area as a shooting area of ​​a recorded image from the plurality of areas in the next frame; a switching step of reselecting the selected area from the plurality of areas and switching the selected area after the selecting step in the next frame; and The production process records the image of the selected area before the switching process and the image of the selected area after the switching process to produce the image document.

2. The image production method according to claim 1, wherein: In the creation step, a dynamic image document as the image document is created by combining the image of the selected area before the switching step and the image of the selected area after the switching step.

3. The image production method according to claim 1 or 2, wherein: The specified step is performed when the switching step is performed, The condition determination step is performed every time the designation step is performed.

4. The image production method according to claim 1 or 2, further comprising: A sensitivity determination process, when the first area is selected as the selected area, determines the sensitivity of the pixel corresponding to the first area among the multiple pixels of the imaging element based on the appropriate exposure amount of the first area and the conditions determined by the condition determination process.

5. The image production method according to claim 4, wherein: In the sensitivity determination step, the sensitivity of the pixels corresponding to the first region is determined to be higher than the sensitivity of the pixels corresponding to the second region serving as the reference region.

6. The image production method according to claim 1 or 2, further comprising: an adjustment step of adjusting the exposure amount when shooting the first image over time by changing the condition over time, The condition determination process comprises: a first determining step of determining the condition based on the appropriate exposure amount of the reference area; and a second determination step of determining an adjustment amount of the exposure amount in the adjustment step based on the appropriate exposure amount of the selected area, The adjustment step is performed based on the condition determined in the first determination step and the adjustment amount determined in the second determination step.

7. The image production method according to claim 6, wherein: The first determination step is performed when the switching step is performed, The second determining step is performed in the adjusting step after the first determining step is performed.

8. The image production method according to claim 1, further comprising: a display step of displaying the first image on a display screen, In the setting step, the plurality of areas are set according to a setting operation performed by a user through the first image displayed on the display screen.

9. An image production method for producing an image document based on an image captured by a camera having a camera lens and an imaging element, the image production method comprising: A setting step of setting a plurality of areas having a second viewing angle smaller than the first viewing angle in a shooting area of ​​a first image having a first viewing angle; A selection step of selecting a selected area as a shooting area of ​​a recorded image from the plurality of areas; a switching step of reselecting the selected area from the plurality of areas and switching the selected area after the selecting step; a first recording step, which is performed after the switching step is performed and before a first period has passed, and records the image of the selected area during the first period; a second recording step, which is performed in a second period other than the first period, and records the image of the selected area in the second period; and an adjustment step of adjusting at least one of the exposure amount of the selected area, the white balance of the image of the selected area, and the focus of the photographing lens during each of the first period and the second period, In the adjustment process performed in the first period, the adjustment speed of the condition is faster than that of the adjustment process performed in the second period.

10. The image production method according to claim 9, wherein: In the adjustment step, the exposure amount is adjusted by driving an aperture driving unit that changes the aperture amount with respect to the incident light to the photographing lens. In the adjustment process performed in the first period, the driving speed of the aperture driving unit is faster than that of the adjustment process performed in the second period.

11. The image production method according to claim 9 or 10, wherein: In the adjustment step, the focus is adjusted by moving the focus optical component of the photographing lens. In the adjustment process performed in the first period, the moving speed of the focus optical component is faster than that in the adjustment process performed in the second period.

12. An image production method for producing an image document based on an image captured by a camera having a camera lens and an imaging element, the image production method comprising: A setting step of setting a plurality of areas having a second viewing angle smaller than the first viewing angle in a shooting area of ​​a first image having a first viewing angle; A selection step of selecting a selected area as a shooting area of ​​a recorded image from the plurality of areas; a switching step of reselecting the selected area and switching the selected area from the plurality of areas after the selecting step; a first recording step, which is performed after the switching step is performed and before a first period has passed, and records the image of the selected area during the first period; a second recording step, which is performed in a second period other than the first period, and records the image of the selected area in the second period; a first adjustment step, which is performed during the first period and uses first adjustment data to adjust the exposure of the selected area to a target exposure; and The second adjustment step is performed during the second period and uses the second adjustment data to adjust the exposure amount of the selected area to the target exposure amount. In the first adjustment step and the second adjustment step, the values ​​of a plurality of parameters including exposure time, sensitivity of pixels of the imaging element, and aperture amount with respect to incident light of the photographing lens are determined based on the first adjustment data or the second adjustment data, and the exposure amount is adjusted to the target exposure amount. The first adjustment data and the second adjustment data have different values ​​of the plurality of parameters defined for the target exposure.

13. The image production method according to claim 12, wherein: The first adjustment data and the second adjustment data define a change pattern when changing each value of the plurality of parameters in order to adjust the exposure amount to the target exposure amount. The first adjustment data defines the change pattern in which the degree of change of the aperture amount with respect to the target exposure amount is smaller than the change pattern in the second adjustment data.

14. The image production method according to claim 12 or 13, wherein: The camera device is provided with an electronic neutral density filter capable of electronically changing the dimming degree. The plurality of parameters include the dimming degree of the electronic dimming filter.

15. The image production method according to claim 1, wherein: The reference area in the current frame is an area having the smallest appropriate exposure amount among the plurality of areas, The exposure time and aperture value when capturing the first image in the next frame are determined according to the appropriate exposure in the reference area in the current frame.

Images