Video creation method

Abstract

To provide a video creation method capable of obtaining video having good image quality by extracting a part of captured video even when the position of the extracted area is changed.SOLUTION: A plurality of areas is set in a shooting area of first video, a selected area of recording video is selected from the plurality of areas, the selected area is reselected to switch the selected area, the video of the selected area in the first period is recorded from the switching of the selected area until the first period has elapsed, the video of the selected area in a second period is recorded in the second period other than the first period, the exposure amount of the selected area is adjusted to a target exposure amount in the first period, and the exposure amount of the selected area is adjusted to the target exposure amount in the second period. In the first adjustment step and the second adjustment step, values of a plurality of parameters are determined on the basis of the first adjustment data or the second adjustment data to adjust the exposure amount to the target exposure amount, and the values of the plurality of parameters differ from each other between the first adjustment data and the second adjustment data.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] An embodiment of the present invention relates to an image creating method for capturing an image using an image capturing device and creating an image file. [Background technology]

[0002] When acquiring images of multiple subjects in a scene, the image of the scene may be captured by a single image capture device, and the images of the subjects may be extracted from the captured images. Examples of such techniques include those described in Patent Documents 1 to 3.

[0003] Patent Document 1 discloses a technique for generating a cut-out image by cutting out a part of an original image captured by a single camera, and controlling the cut-out position to output the cut-out image so that the position of the subject changes.

[0004] Patent Document 2 discloses a technology that detects multiple objects based on a high-resolution wide-angle image captured by a single camera, cuts out an image of each detected object from the wide-angle image, and displays the images of the multiple objects in parallel on a monitor.

[0005] Patent Document 3 discloses a technique for specifying a number of arbitrary areas from an image captured by a single imaging device, and cutting out and outputting images corresponding to the specified number of arbitrary areas. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 11-220653 [Patent Document 2] JP 2004-194309 A [Patent Document 3] JP 2014-42357 A Summary of the Invention [Problem to be solved by the invention]

[0007] When extracting a portion of an image, the image quality of the extracted image is determined by the conditions at the time of shooting, such as the exposure, white balance, and lens focus. When these conditions are set to appropriate values, the extracted image will have good image quality.

[0008] On the other hand, the appropriate conditions for the extracted image are determined according to the position of the extracted region in the original image. Taking this into consideration, it is necessary to ensure good image quality for the extracted image even when the position of the extracted region changes. However, the techniques described in Patent Documents 1 to 3 do not optimize conditions such as exposure, white balance, and lens focus according to the position of the extracted image. Therefore, even if the techniques described in these documents are used, there is a risk that the image quality of the extracted image, particularly the image quality when the position of the extracted region changes, may not be properly ensured.

[0009] One embodiment of the present invention has been made in consideration of the above circumstances, and aims to provide an image creation method that can extract a portion of a captured image and obtain an image with good image quality even when the position of the extraction area is changed. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, an image creation method of one embodiment of the present invention is an image creation method that creates an image file based on an image captured by an imaging device equipped with a shooting lens and an image sensor, and is characterized by comprising: a setting process that sets multiple areas having a second angle of view smaller than the first angle of view within a shooting area of ​​a first image having a first angle of view; a designation process that designates, as a reference area, a second area having a lower appropriate exposure amount than the first area among the first and second areas included in the multiple areas; and a condition determination process that determines at least one condition among the exposure time when capturing the first image and the aperture amount for light incident on the shooting lens based on the appropriate exposure amount of the reference area.

[0011] In addition, a video creation method of one embodiment of the present invention may include a selection process for selecting a selection area, which is a shooting area of ​​a video to be recorded, from among a plurality of areas, a switching process for switching the selection area by reselecting the selection area from among the plurality of areas after the selection process, and a creation process for recording an image of the selection area before the switching process and an image of the selection area after the switching process to create a video file.

[0012] In the video creation method according to an embodiment of the present invention, the creation step may combine the video of the selected area before the switching step and the video of the selected area after the switching step to create a moving image file as a video file. In this case, a moving image file of video that maintains good image quality before and after the switching step can be obtained.

[0013] It is also preferable that the designation step is carried out when the switching step is carried out, and the condition determination step is carried out every time the designation step is carried out.

[0014] In addition, the image creation method of one embodiment of the present invention may further include a sensitivity determination process for determining, when the first region is selected as the selected region, the sensitivity of a pixel corresponding to the first region among a plurality of pixels of the imaging element based on an appropriate exposure amount for the first region and the conditions determined in the condition determination process. In the sensitivity determination step, it is preferable to determine the sensitivity of the pixels corresponding to the first region to be higher than the sensitivity of the pixels corresponding to the second region, which is the reference region.

[0015] Furthermore, the image creation method according to one embodiment of the present invention may further include an adjustment step of adjusting the exposure amount over time when capturing the first image by changing the conditions over time. In this case, the condition determination step may include a first determination step of determining the conditions based on the appropriate exposure amount of the reference region, and a second determination step of determining an adjustment amount of the exposure amount in the adjustment step based on the appropriate exposure amount of the selected region. Furthermore, the adjustment step may be performed based on the conditions determined in the first determination step and the adjustment amount determined in the second determination step. It is preferable to carry out the first determination step when the switching step is carried out, and to carry out the second determination step in the adjustment step after the first determination step is carried out.

[0016] The image creation method according to an embodiment of the present invention may further include a display step of displaying the first image on a display screen. In this case, the setting step may set a plurality of regions according to a setting operation performed by a user through the first image displayed on the display screen.

[0017] Moreover, according to one embodiment of the present invention, there is provided an image creation method for creating an image file based on an image captured by an image capture device equipped with a photographing lens and an image sensor, the image creation method including: a setting step for setting a plurality of areas having a second angle of view smaller than the first angle of view within a shooting area of ​​a first image having a first angle of view; a selection step for selecting a selected area from the plurality of areas as a shooting area of ​​the image to be recorded after the selection step; a switching step for reselecting the selected area from the plurality of areas and switching the selected area after the selection step; a first recording step that is performed during a first period from when the switching step is performed until a first period has elapsed and records an image of the selected area in the first period; a second recording step that is performed in a second period other than the first period and records an image of the selected area in the second period; and an adjustment step that adjusts at least one condition among the exposure amount of the selected area, the white balance of the image of the selected area, and the focus of the photographing lens during each of the first and second periods; and a faster adjustment speed of the conditions in the adjustment step performed during the first period than in the adjustment step performed during the second period.

[0018] Furthermore, in the above-described image creation method, in the adjustment process, the exposure amount is adjusted by driving an aperture drive unit that changes the aperture amount for light incident on the shooting lens, and it is more preferable that the drive speed of the aperture drive unit is faster in the adjustment process performed in the first period than in the adjustment process performed in the second period. Furthermore, in the above-described image creation method, it is more preferable that in the adjustment process, the focus is adjusted by moving a focusing optical component of the shooting lens, and that the moving speed of the focusing optical component is faster in the adjustment process performed in the first period than in the adjustment process performed in the second period.

[0019] According to one embodiment of the present invention, there is provided an image creation method for creating an image file based on an image captured by an image capture device including a photographing lens and an image sensor, the method including a setting step of setting a plurality of areas having a second angle of view smaller than the first angle of view within a capture area of ​​a first image having a first angle of view; a selection step of selecting a selected area from the plurality of areas as a capture area of ​​an image to be recorded; a switching step of reselecting the selected area and switching the selected area from the plurality of areas after the selection step; a first recording step which is performed during a period from when the switching step is performed until a first period has elapsed and records an image of the selected area in the first period; and a second recording step which is performed during a second period other than the first period and records the image of the selected area in the second period. the first adjustment step being carried out in a first period and using the first adjustment data to adjust the exposure of the selected region to the target exposure, and a second adjustment step being carried out in a second period and using the second adjustment data to adjust the exposure of the selected region to the target exposure, wherein in the first adjustment step and the second adjustment step, the values ​​of each of a plurality of parameters including the exposure time, the sensitivity of the pixels of the imaging element, and the aperture amount of light incident on the shooting lens are determined based on the first adjustment data or the second adjustment data to adjust the exposure to the target exposure, and an image creation method in which the values ​​of each of the plurality of parameters determined for the target exposure differ between the first adjustment data and the second adjustment data can also be realized.

[0020] In the above image creation method, the first adjustment data and the second adjustment data may define a change pattern for changing each of the multiple parameters to adjust the exposure to the target exposure. In this case, it is preferable that the first adjustment data defines a change pattern that changes the aperture amount to a smaller degree relative to the target exposure than the change pattern of the second adjustment data.

[0021] In the image creation method, the image capturing device may be provided with an electronic neutral density filter capable of electronically changing a degree of dimming. The parameters may include a degree of dimming of the electronic neutral density filter. [Brief description of the drawings]

[0022] [Figure 1] 1 is a perspective view showing an example of the appearance of an imaging device according to a first embodiment of the present invention. [Diagram 2] 1 is a rear view showing the rear side of an imaging device according to a first embodiment of the present invention. [Diagram 3] 1 is a block diagram showing a configuration of an imaging device according to a first embodiment of the present invention. [Figure 4] 4 is a diagram showing a correspondence relationship between pixels of an imaging element and an imaging area of ​​a first image. FIG. [Diagram 5] FIG. 11 is a diagram showing a procedure for setting an area of ​​an image to be extracted from a first image. [Figure 6] FIG. 11 is a diagram showing a state in which a plurality of areas of an image to be extracted from a first image are set. [Figure 7] FIG. 13 is a diagram showing an image of a selected area displayed on a display screen. [Figure 8] FIG. 2 is an explanatory diagram of a moving image file of recorded video. [Figure 9] FIG. 1 is a diagram showing a flow of a video production flow according to a first embodiment which is one embodiment of the present invention. [Figure 10] FIG. 11 is a diagram showing a flow of a video production flow according to a second embodiment which is one embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing a flow of a video production flow according to a third embodiment which is one embodiment of the present invention. [Figure 12] 13A and 13B are diagrams showing changes over time in the appropriate exposure amount for each of a reference area and a selected area. [Figure 13] 13A and 13B are diagrams showing the exposure amount after adjustment of a selected area when adjustment is continued based on the appropriate exposure amount of a reference area. [Figure 14] FIG. 11 is a diagram showing a target exposure amount in a third embodiment. [Figure 15] 13A to 13C are diagrams showing changes in the amount of exposure after adjustment of each of the reference area and the selected area in the third embodiment. [Figure 16] FIG. 13 is an explanatory diagram of a video production flow according to a fourth embodiment which is one embodiment of the present invention. [Figure 17] FIG. 13 is an explanatory diagram of a video production flow according to a fifth embodiment which is one embodiment of the present invention. [Figure 18] 13A and 13B are diagrams illustrating an example of a change pattern defined by adjustment data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, preferred embodiments (first to sixth embodiments) of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiment described below is merely an example given to facilitate understanding of one embodiment of the present invention, and does not limit the present invention. In other words, one embodiment of the present invention may be modified or improved from the embodiment described below without departing from the spirit of the invention. Furthermore, one embodiment of the present invention includes its equivalents.

[0024] <<First embodiment>> A first embodiment, which is one embodiment of the present invention, relates to a method of creating an image using a photographing device 10 shown in Figures 1 to 3. Figures 1 and 2 show the external appearance of the photographing device 10, with Figure 1 being a perspective view of the photographing device 10 and Figure 2 being a view of the photographing device 10 as seen from the rear side. Figure 3 is a block diagram showing the configuration of the photographing device 10.

[0025] [Basic configuration of imaging device] The image capturing device 10 is, for example, a digital camera used for capturing video. In the following description, unless otherwise specified, "video" refers to live video (live view image), that is, video captured in real time.

[0026] 1 and 2 is a lens-interchangeable digital camera, and includes a photographing device body 12 and a photographing lens 14. The photographing lens 14 is replaceably attached to a mount 13 of the photographing device body 12. However, the photographing device 10 is not limited to this, and may be a digital camera with an integrated lens.

[0027] (Photographic lens) As shown in FIG. 3, the photographing lens 14 includes an optical component unit 18, an aperture 20, an electronic neutral density filter 21, a focus driver 22, an aperture driver 23, and a voltage application unit 24. The optical component unit 18 has a plurality of lenses, including a focusing optical component 19 (focus lens). When the focusing optical component 19 moves in the direction of the optical axis L1, the focus of the photographing lens 14 changes.

[0028] There are two focus adjustment modes: manual focus mode and autofocus mode. In the manual focus mode, the user manually turns a focus ring 16 provided on the barrel of the photographic lens 14, which moves a focusing optical component 19 in the direction of an optical axis L1. In the autofocus mode, the control unit 46 drives the focus driver 22 to move the focus optical component 19 in the direction of the optical axis L1, and the focus is automatically adjusted to focus on a subject in the image. Note that the focus adjustment in the autofocus mode is performed, for example, when the user presses the release button 26 of the image capturing device body 12 halfway down, which is used as a trigger.

[0029] The drive motor of the focus drive unit 22 is constituted by an ultrasonic motor or the like, and is provided in the photographing device body 12 or in the photographing lens 14 .

[0030] The optical component unit 18 includes a wide-angle lens, an ultra-wide-angle lens, a 360-degree lens, an anamorphic lens, or the like. This allows the image capture device 10 to capture an image with a wide horizontal angle of view. Here, the maximum angle of view (hereinafter referred to as the first angle of view) when the image capture device 10 captures an image is determined according to the optical component unit 18 and the use of an image sensor 40 described below, and the image captured with the first angle of view corresponds to the "first image" in one embodiment of the present invention.

[0031] The imaging device 10 may include a plurality of optical component units 18 each having a different angle of view.

[0032] The diaphragm 20 is an optical component that is disposed in the optical path of the photographing lens 14 and is configured to have a variable aperture shape, and that adjusts the aperture amount (specifically, the aperture value or F-number) for the light incident on the photographing lens 14. The diaphragm 20 has an aperture shape that can be changed, for example, by mechanically driving a member that changes the size of the aperture. However, the diaphragm is not limited to this, and may be a diaphragm (physical diaphragm) whose aperture shape can be changed by driving a liquid crystal or an electrochromic element.

[0033] The opening shape of the diaphragm 20, that is, the aperture amount, is adjusted by a diaphragm driver .

[0034] The electronic neutral density filter 21 is an ND (Neutral Density) filter that is disposed in the optical path of the photographing lens 14 and can electronically change the degree of dimming. The electronic neutral density filter 21 is configured, for example, by mixing a light-absorbing dimming material into liquid crystal, and controls the degree of dimming (specifically, the light transmittance) by changing the inclination of the liquid crystal molecules.

[0035] The electronic neutral density filter 21 is not limited to a liquid crystal type electronic ND filter, and may be a precipitation type electronic ND filter that utilizes an oxidation-reduction reaction of a reactive substance (for example, silver chloride). As the electronic neutral density filter 21, for example, the electronic ND filter described in JP 2013-88596 A or JP 2019-68402 A can be used.

[0036] The degree of dimming of the electronic attenuation filter 21 is adjusted by changing the magnitude of the voltage (applied voltage) that the voltage application unit 24 applies to the electronic attenuation filter 21.

[0037] (imaging device body) 1 and 2, the photographing device body 12 includes an operation unit operated by a user. For example, a release button 26 is disposed on the top surface of the photographing device body 12. For example, when the user fully presses the release button 26, recording of an image photographed by the photographing device 10 or an image based on the photographed image is started. An example of an image based on the photographed image is an image of a selected area, which will be described later. Note that the user's release instruction (i.e., a recording instruction) may be input to the control unit 46 via a touch operation detection function of the display 28.

[0038] A display 28 constituting a display screen is disposed on the rear surface of the imaging device body 12, and a number of buttons 30, 32, and 34 are disposed around the display 28.

[0039] The display 28 is configured, for example, with an LCD (Liquid Crystal Display), an organic EL (Organic Electroluminescence) display, an LED (Light Emitting Diode) display, electronic paper, etc. On the display screen of the display 28, an image captured by the imaging device 10 or an image based on the captured image, etc. are displayed. Furthermore, on the display screen of the display 28, a selection menu regarding shooting conditions and the like, notification information for the user including warnings and the like, and playback images of images captured in the past are further displayed.

[0040] The display 28 has a function of detecting touch operations of the user's fingers. A transparent touch panel 36 is overlaid on the display 28 or incorporated inside the display 28. The touch panel 36 detects the contact position and displacement of the user's finger or stylus pen, and outputs a signal based on the detection result to a predetermined output destination. For example, assume that a user touches the touch panel 36 with two fingers close to each other and then performs an operation of widening the distance between the fingers (a so-called pinch-out operation). In this case, the touch panel 36 detects the positions of each of the two fingers at the start and end of the operation, and outputs a signal according to the detection result.

[0041] As shown in FIG. 3, within the housing of the photographing device main body 12, a shutter 38, an image sensor 40, an analog signal processing circuit 44, a control unit 46, an internal memory 50, a card slot 52, and a buffer 56 are provided.

[0042] The imaging element 40 is an image sensor and is configured with a solid-state imaging element such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor Image Sensor). The imaging element 40 has a plurality of pixels 42 arranged in a grid as shown in Fig. 4. Fig. 4 shows the correspondence between the pixels 42 of the imaging element 40 and the shooting area of ​​the first image.

[0043] Each pixel 42 has an on-chip microlens, a color filter, and a photodiode (photoelectric conversion element).

[0044] 4, each pixel 42 corresponds to one of the unit areas A0 constituting the image capturing area A1 of the image. The image capturing area A1 is an image capturing area when the image capturing device 10 captures the first image, and its angle of view corresponds to the first angle of view. The unit area A0 is an area forming the smallest unit in the image capturing area A1, and corresponds to the pixels 42 arranged vertically and horizontally in the image sensor 40. In other words, the first image captured in the shooting area A1 is composed of unit images the same number as the number of unit areas A0 (i.e., the number of pixels 42). In this specification, the number of unit images in the first image is conveniently referred to as the "number of pixels."

[0045] The image sensor 40 receives light from a subject that has passed through the photographic lens 14, converts the received light image into an electrical signal (image signal), and outputs the electrical signal. The image sensor 40 is also configured to be capable of performing an exposure operation using a so-called electronic shutter. In the following description, unless otherwise specified, "exposure" refers to exposure by the electronic shutter at the image sensor 40 while the shutter 38 is kept open. Also, "exposure time" refers to the time according to the shutter speed of the electronic shutter, or more precisely, the charge accumulation time.

[0046] 3, the analog signal processing circuit 44 reads out, for each pixel, an electrical signal (image signal) for one frame output from the imaging element 40. The analog signal processing circuit 44 amplifies the read-out image signal by an AGC (Auto Gain Controller), and performs signal processing such as correlated double sampling on the amplified signal. The processed signal is sent to a video processing unit 48 of the control unit 46.

[0047] The control unit 46 controls each unit of the image capturing device 10 and executes various processes related to the creation of a video file. The control unit 46 includes a controller 47 and a video processing unit 48, as shown in FIG.

[0048] The control unit 46 is composed of, for example, one or more processors, and is composed of, for example, a CPU (Central Processing Unit) and a control program. However, the present invention is not limited to this, and the above-mentioned processor may be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), an MPU (Micro-Processing Unit), or other ICs (Integrated Circuits), or may be composed of a combination of these. Furthermore, the above-mentioned processor may be configured such that the entire functions of the control unit 46 including the controller 47 and the image processing unit 48 are implemented in a single IC (Integrated Circuit) chip, as typified by a SoC (System on Chip) or the like. The hardware configuration of each of the processors described above may be realized by an electric circuit that combines circuit elements such as semiconductor elements.

[0049] The controller 47 generally controls the image capturing device 10 in accordance with a user's operation or a specified control pattern. For example, the controller 47 controls the image capturing element 40 and the analog signal processing circuit 44 so as to capture video (moving images) at a predetermined frame rate.

[0050] Moreover, the controller 47 determines shooting conditions according to the shooting environment, and controls each driving section, the image sensor 40, the analog signal processing circuit 44, and the image processing section 48 so that the shooting conditions become the determined conditions. The shooting conditions include the exposure amount, white balance, and focus of the shooting lens 14 when shooting an image. Furthermore, the controller 47 controls the image processor 48 so that the captured image or an image based on the captured image is recorded on a recording medium.

[0051] The video processing unit 48 converts the signal sent from the analog signal processing circuit 44 into digital image data, and then performs various processes on the digital image data, such as gamma correction, white balance correction, and scratch correction. The video processing unit 48 also compresses the processed digital image data in a compression format that complies with a predetermined standard.

[0052] During the shooting of the video, the video processor 48 generates compressed digital image data at a specific frame rate and acquires the video (strictly speaking, frame images) from the data. The video (frame images) acquired at this time correspond to the video shot at the first angle of view, i.e., the first video.

[0053] Furthermore, under the control of the controller 47, the image processing unit 48 executes various processes (for example, the extraction process described below) on the acquired image, and outputs the processed image to the display 28 frame by frame at a specific frame rate. Furthermore, the image processor 48 records the processed image on a recording medium and creates an image file. In this manner, the image processor 48 has a function of creating an image file, and the photographing device 10 having the image processor 48 is used as an image creating device.

[0054] In the following, unless otherwise specified, the respective operations and processes of the controller 47 and the video processing unit 48 will be described as the operation and process of the control unit 46. The process by the control unit 46 will be described in detail in a later section.

[0055] The internal memory 50 built into the photographing device body 12 and the memory card 54 removable from the photographing device body 12 via the card slot 52 are recording media, and video is recorded on the recording media by the control unit 46. The internal memory 50 and the memory card 54 are composed of flash memory, ferroelectric memory, or the like. Note that the internal memory 50 and the memory card 54 may be located outside the photographing device body 12, in which case the control unit 46 may record video on the external recording medium via a wired or wireless connection.

[0056] The buffer 56 functions as a work memory for the control unit 46, and is configured by, for example, a dynamic random access memory (DRAM), a ferroelectric memory, or the like.

[0057] [Processing by the control unit] The processes performed by the control unit 46 include a display process, an extraction process, a selection process, a switching process, a recording process, an exposure adjustment process, a focus adjustment process, and a white balance adjustment process. Each of the above-mentioned processes will be explained below individually.

[0058] (Display processing) The display process is a process for displaying various images on the display screen of the display 28. For example, when the control unit 46 starts the display process after the imaging device 10 is started, at that point in time, the first image captured by the imaging device 10 at the first angle of view, that is, the image captured in the imaging area A1 shown in FIG. Furthermore, when the extraction process is executed, the control unit 46 displays an image of the selected area, which will be described later, on the display screen in the display process. Furthermore, when the selected area is switched (i.e., when a switching step, which will be described later, is performed), the control unit 46 switches the image displayed on the display screen to the image of the selected area after the switching step.

[0059] The above-described execution operation of the display process by the control unit 46 corresponds to the display step. Note that in this embodiment, the image displayed on the display 28 by the display process is a live image (live view image) captured in real time at that time.

[0060] (Extraction process) The extraction process is a process of extracting a part of the first image. The extracted image is also called a cropped image, and is to be recorded in the recording process. In the extraction process, the user performs an operation for setting an area to be extracted in the shooting area of ​​the first video. This setting operation will be specifically described with reference to Fig. 5. Fig. 5 shows the procedure for setting the area of ​​the video to be extracted from the first video.

[0061] As shown in Fig. 5, the user performs a setting operation in a state where a first image (indicated as symbol P1 in Fig. 5) is displayed on the display screen of display 28. More specifically, when the user performs a predetermined button operation in a state where first image P1 is displayed, a rectangular area setting frame FR is displayed superimposed on first image P1 as shown in Fig. 5. An area A2 surrounded by this area setting frame FR becomes the area of ​​the image extracted from first image P1. The angle of view of the extracted image is smaller than the angle of view of first image P1, i.e., the first angle of view, and will be referred to as the second angle of view hereinafter.

[0062] The region setting frame FR is displayed with its position, size, and aspect ratio variable. For example, the user can change the display position of the region setting frame FR by touching one side of the region setting frame FR with a finger on the touch panel 36 and dragging the finger. The size and aspect ratio of the region setting frame FR can also be changed by touching the region setting frame FR with two fingers on the touch panel 36 and widening or narrowing the distance between the fingers. By changing the position, size, and aspect ratio of the region setting frame FR in this way, the position, size, and aspect ratio of the region of the image extracted from the first image P1 can be changed.

[0063] The above operations correspond to the setting operations by the user. In addition, by repeating the above series of operations multiple times, it is possible to set multiple areas A2 of the image to be extracted in the shooting area A1 of one first image P1, as shown in Fig. 6. Fig. 6 shows a state in which multiple areas A2 of the image to be extracted from the first image P1 are set.

[0064] When a plurality of regions A2 are set, the sizes (second angles of view) of the regions A2 may be the same for each region A2 or may differ for each region A2. Furthermore, the shape of each region A2 is not limited to a rectangle (rectangle), and may be a square, a parallelogram, a trapezoid, a rhombus, a circle, an ellipse, a triangle, a polygon having pentagons or more sides, or an indeterminate shape. Further, the number of areas A2 to be set is not particularly limited, but in this embodiment, it is assumed that there are two or more areas A2 as shown in FIG.

[0065] When the user repeats the setting operation multiple times, the control unit 46 accepts each setting operation and sets the number of areas A2 corresponding to the number of setting operations in the shooting area A1 of the first image P1. Such an operation of the control unit 46 corresponds to the setting step.

[0066] When the first image P1 is a high-quality image (for example, an image having 10 million or more pixels), the image of the second angle of view extracted from the first image P1 is an image of sufficiently high quality.

[0067] The number of pixels of the first image P1 is not particularly limited, but the lower limit is preferably 10 million or more, and more preferably 60 million or more. The upper limit of the number of pixels is preferably 1 billion or less, and more preferably 500 million or less. If the number of pixels exceeds the above lower limit, the visibility of the image of the second angle of view extracted from the first image P1 is guaranteed. If the number of pixels falls below the above upper limit, the amount of data of the first image P1 is reduced, and the processing speed by the control unit 46 is increased.

[0068] (Selection and switching processes) The selection process is a process of selecting a selection area in which an image to be recorded appears from among a plurality of areas A2 set in the shooting area A1 of the first image P1. The switching process is a process in which, after the selection process, a selected area is reselected from among the multiple areas A2, and the selected area is switched. In this embodiment, the number of selected areas in the selection process and the switching process is one, but this is not limited thereto, and two or more selected areas may be used.

[0069] The procedure of the selection process and the switching process will be described below. After the user sets the multiple areas A2 by performing the setting operation described above, he / she selects one of the areas A2 as the selected area. Thereafter, the user inputs the selected area A2 via the operation buttons 30, 32, 34 or the touch panel 36, etc., and the control unit 46 accepts the input operation. The control unit 46 determines the selected area from the multiple areas A2 based on the user's input operation. Such an operation by the control unit 46 corresponds to the selection process.

[0070] Furthermore, after the selection step, if the user reselects the selected area to another area A2 and performs an input operation again, the control unit 46 accepts the re-input operation and switches the selected area to another area A2 based on the re-input operation. Such an operation by the control unit 46 corresponds to the switching step. Note that the input operation (re-input operation) performed by the user when reselecting the selected area is not particularly limited. For example, the re-input operation may be an operation of tapping the area A2 to be selected on the display 28, or an operation of pressing down either the up / down button or the left / right button of the first operation button 30 which is a cross key.

[0071] As described above, in this embodiment, a selected area is selected from the multiple areas A2 in the selection step, and an image of the selected area is recorded as an image file. When the selection step is performed, the image of the selected area is displayed on the display screen of the display 28 in conjunction with the selection step, as shown in Fig. 7. Fig. 7 shows the image of the selected area displayed on the display screen.

[0072] Furthermore, by performing a switching process after the selection process, the selected area can be switched to change the image to be recorded from an image of one subject to an image of another subject. This allows individual images (more specifically, close-up images of each subject) of multiple subjects in the same scene (same location) to be simultaneously captured by one imaging device 10 without using multiple devices. Furthermore, the area A2 showing the video to be recorded can be switched in chronological order in response to a user operation, and video files showing the video before and after the switch can be easily obtained. Furthermore, in conjunction with the implementation of the switching process, the image displayed on the display screen of the display 28 is switched from the image of the selected area before the switching process to the image of the selected area after the switching process, allowing the user to confirm the switching of the selected area on the display screen.

[0073] (Recording Processing) The recording process is a process in which the control unit 46 records the video of the selected area on a recording medium and creates a video file (more specifically, a video file) related to the video. In the recording process, the operation of the control unit 46 to record the video of the selected area corresponds to the recording step. In the recording process, the operation of the control unit 46 to create a video file corresponds to the creation step. In this embodiment, the recording of a video and the creation of a video file are synonymous, and the creation process corresponds to the recording process.

[0074] Furthermore, when the control unit 46 carries out the switching process, it creates a video file by recording the video of the selected area before the switching process and the video of the selected area after the switching process. To explain in more detail, in the creation process, the control unit 46 creates a moving image file as a video file by combining the video (moving image) of the selected area before the switching process and the video (moving image) of the selected area after the switching process. This results in a moving image file of video in which the subject changes due to switching of the selected area, as shown in Fig. 8. Fig. 8 is an explanatory diagram of the moving image file of the recorded video.

[0075] (Exposure adjustment process) The exposure adjustment process is a process for adjusting the exposure of a selected region, and is executed by utilizing an AE (Automatic Exposure) function of the control unit 46. The amount of exposure is determined by a number of parameters including the exposure time (i.e., the shutter speed of the electronic shutter), the sensitivity (ISO sensitivity) of the pixels 42 of the image sensor 40, and the aperture amount (F-number) of the light incident on the photographing lens 14. The amount of exposure is adjusted by determining the value of each parameter and controlling the aperture driver 23, the image sensor 40, the image processor 48, etc. so that each parameter becomes the determined value. Here, the adjustment of the aperture 20 by the aperture driver 23 is performed using a known automatic light control (Automatic Light Control: ALC) function. Moreover, the adjustment of the shutter speed of the electronic shutter in the image sensor 40 is performed using a known electronic light control (Electronic Light Control: ELC) function.

[0076] Furthermore, if the photographing lens 14 is equipped with an electronic attenuation filter 21, the above-mentioned multiple parameters include the degree of dimming of the electronic attenuation filter 21. In this case, the value of the degree of dimming of the electronic attenuation filter 21 is determined, and the voltage application unit 24 is controlled so that the degree of dimming becomes the determined value, thereby adjusting the amount of exposure.

[0077] Incidentally, the appropriate value of the exposure amount of the selected region (appropriate exposure amount) varies depending on which region is the selected region among the multiple regions A2 set in the shooting region A1 of the first image P1.

[0078] To explain in more detail, when the first image P1 is shot, the amount of exposure at each point of the shooting area A1 including the multiple areas A2 varies depending on the subject and the environment at each point. That is, the appropriate amount of exposure for each area A2 in the shooting area A1 may vary depending on the position of the area A2, and is therefore determined for each area A2.

[0079] The appropriate exposure amount for each area A2 is determined by a known procedure. As an example, from an image signal output from a pixel 42 corresponding to a unit area A0 constituting a certain area A2 in the image sensor 40, the brightness of R (Red), G (Green), and B (Blue) (i.e., RGB pixel value) is integrated for that area A2. Here, the pixel 42 corresponding to the unit area A0 is the pixel 42 located at the same position as the unit area A0 in the positional relationship shown in FIG. 4. For example, the unit area A0 in the area A2 surrounded by a thick frame in FIG. 4 corresponds to the pixel 42 located within the range surrounded by the thick frame in the same figure. Then, the appropriate exposure amount when capturing an image of the area A2 is determined from the integrated value of the RGB pixel values ​​calculated for the above-mentioned area A2 (more specifically, the average value within the area A2).

[0080] The exposure amount of the selected area is adjusted based on the appropriate exposure amount determined in the above-mentioned manner. More specifically, the control unit 46 determines the values ​​of the exposure time (shutter speed) and the aperture value (F-number) based on the appropriate exposure amount of the predetermined area. This operation corresponds to the condition determination process by the control unit 46. The values ​​of each parameter determined in the condition determination process are values ​​that indicate the exposure conditions when capturing the first image including the image of the selected area. In this embodiment, both the exposure time and the aperture value are determined, but the present invention is not limited to this, and only one of the exposure time and the aperture value may be determined.

[0081] Furthermore, after determining the values ​​of the exposure time and the aperture, the control unit 46 determines the sensitivity (ISO sensitivity) value of the pixels 42 corresponding to the selected region according to the combination of these values. Strictly speaking, the gain (amplification ratio) for the output signal from the pixels 42 corresponding to the selected region is determined. This operation corresponds to the sensitivity determination step by the control unit 46. The gain may be, for example, an analog gain for an analog signal in the analog signal processing circuit 44 or the like, or a digital gain for a signal (data) after digital conversion in the video processing unit 48 or the like.

[0082] After the exposure time, aperture value, and sensitivity are determined, the control unit 46 controls the aperture driving unit 23, the image pickup device 40, the image processing unit 48, etc. so that these parameters become the determined values.

[0083] While the image is being recorded, the exposure adjustment process is periodically (for example, for each frame) repeated according to the above procedure. Each time the exposure adjustment process is performed, the exposure time, aperture value, and sensitivity are determined. Specifically, during shooting of a video, the integrated value of RGB pixel values ​​in a specific area of ​​the Nth frame image (N is a natural number) is calculated. Then, the appropriate exposure amount is calculated based on the calculated integrated value, and the exposure time and aperture amount when shooting the next frame image (i.e., the N+1th frame image) are determined based on the calculated appropriate exposure amount. In addition, the integrated value of the RGB pixel values ​​in the selected region in the Nth frame image is calculated, and the appropriate exposure amount for the selected region is calculated from the integrated value. Then, the sensitivity of the pixel 42 corresponding to the selected region in the Nth frame image is determined based on the calculated appropriate exposure amount, the exposure time and the aperture value at that time.

[0084] In this embodiment, the exposure time and the aperture change over time, and the exposure amount when capturing the first image is adjusted over time accordingly. Furthermore, the sensitivity of the pixels 42 corresponding to the selected region changes over time, so that the exposure amount of the selected region is adjusted over time. In this embodiment, "adjusting over time" means determining an adjustment amount for each frame for a parameter to be adjusted, and increasing or decreasing each parameter by that adjustment amount for each frame.

[0085] (Focus adjustment process) The focus adjustment process is a process in which, in the autofocus mode, the control unit 46 controls the focus driver 22 to automatically adjust the focus of the photographic lens 14. A known autofocus technique can be used for the focus adjustment in the autofocus mode.

[0086] (White balance adjustment processing) The white balance adjustment process is a process in which the control unit automatically adjusts the white balance of the image in the selected area. The white balance adjustment process is performed using a known AWB (Auto White Balance) function, for example, the technology described in Japanese Patent Application Laid-Open No. 2009-33410.

[0087] [Flow of video creation according to the first embodiment] The flow of video creation according to the first embodiment will be described with reference to Fig. 9. Fig. 9 is an explanatory diagram of the video creation flow according to the first embodiment. In the following description, the period in which one first video (frame image) is captured will be referred to as "one frame" for convenience.

[0088] In the video creation flow, the user first starts up the photographing device 10. After starting up the device, the control unit 46 performs initial settings for various conditions including photographing conditions and exposure conditions (S001). In this step S001, each of a number of parameters that determine the amount of exposure, such as the exposure time (shutter speed), aperture value (F value), and sensitivity (ISO sensitivity), is preset to an initial value.

[0089] Thereafter, image capturing at the first angle of view (i.e., capturing of the first image) is started by the image capturing device 10. In conjunction with this, a display step is carried out by the control unit 46, and the first image, which is a live image, is displayed on the display screen of the display 28 (S002).

[0090] The user performs the above-mentioned setting operation multiple times, and the control unit 46 accepts each setting operation and performs a setting step (S003). In this step S003, the control unit 46 sets multiple areas A2 in the shooting area A1 of the first image according to the setting operation performed by the user through the first image displayed on the display screen.

[0091] Furthermore, when the user selects a selection area from among the multiple areas A2 and inputs the selection result via the touch panel 36 or the like, the control unit 46 performs a selection step (S004). In this step S004, the control unit 46 selects a selection area from among the multiple areas A2 in response to the user's input operation.

[0092] Thereafter, when the user performs an operation such as pressing the release button 26 to start recording the video, the steps from step S005 onwards are carried out. First, in step S005, the control unit 46 acquires one frame of the first image (i.e., a frame image). In the next step S006, the control unit 46 calculates, for each of the multiple regions A2 set in step S003 in the acquired first image, an integrated value of RGB pixel values ​​of the image of each region A2. In the subsequent step S007, the control unit 46 specifies the appropriate exposure amount for each region A2 based on the integrated value of RGB pixel values ​​calculated for each region A2.

[0093] After performing step S007, the control unit 46 performs a sensitivity determination step to determine the sensitivity (more precisely, the gain) of the pixels 42 corresponding to the selected region in the image sensor 40 (S008). In the first (first) step S008 during the video creation flow, the sensitivity of the pixels 42 corresponding to the selected region is determined based on the appropriate exposure amount of the selected region, and the exposure time and aperture amount preset to initial values. Furthermore, in step S008, the control unit 46 adjusts the amount of exposure of the selected region by changing the gain so that the sensitivity of the pixels 42 corresponding to the selected region becomes the determined sensitivity.

[0094] Thereafter, the control unit 46 performs a recording step (creation step) to record the image of the selected area on the recording medium and starts creating a video file that is the image file (S009). In this step S009, the control unit 46 records the image of the selected area with the sensitivity (gain) determined in step S008. The control unit 46 also displays the image of the selected area on the display screen of the display 28.

[0095] Thereafter, the control unit 46 performs a designation step, and designates a predetermined area from among the multiple areas A2 set in step S003 as a reference area (S010). In this step S010, among the first and second areas included in the multiple areas A2, the second area having a smaller appropriate exposure amount than the first area is designated as the reference area. Here, the first and second areas are relative concepts, and by comparing the multiple areas A2, an area having a larger appropriate exposure amount corresponds to the first area, and an area having a smaller appropriate exposure amount corresponds to the second area.

[0096] In this embodiment, the area A2 having the smallest appropriate exposure amount among the multiple areas A2 is designated as the reference area. However, this is not limited to this, and an area A2 having a smaller appropriate exposure amount than any other area A2 may be designated as the reference area.

[0097] After designating the reference area, the control unit 46 executes a condition determination step to determine the exposure time and aperture amount when capturing the first video in the next frame (hereinafter, the next exposure time and aperture amount) (S011). In this step S011, the control unit 46 determines the next exposure time and aperture amount based on the appropriate exposure amount of the reference area.

[0098] Also, in step S011, the control unit 46 adjusts the exposure time and the aperture amount so that these values ​​become the values ​​determined in step S011. As a result, in the next frame, the first video (frame image) is shot with the exposure time and the aperture amount based on the appropriate exposure amount of the reference area specified immediately before.

[0099] Among the steps described above, the series of operations from step S005 onwards for acquiring the first image are repeated during the period in which the image is recorded. Therefore, when the exposure time and aperture value are determined in step S011, the first image is acquired with the above exposure time and aperture value in the frame immediately following (i.e., the second or subsequent step S005).

[0100] Furthermore, when the user selects a selection area once and then reselects it and performs an input operation again (S012), the control unit 46 executes a switching step (S013). In this step S013, the control unit 46 reselects and switches the selection area from among the multiple areas A2 in response to the user's input operation.

[0101] When step S013 is performed and the selected area is switched, the control unit 46 records the image of the selected area before the switching process and the image of the selected area after the switching process in step S009 in the frame immediately after the switching process to create a video file. More specifically, the control unit 46 combines the image of the selected area before the switching process and the image of the selected area after the switching process to create a moving image file as a video file.

[0102] Then, the control unit 46 repeats the above steps S005 to S013 until the user performs a predetermined operation to give an instruction to end the process (S014).

[0103] As described above, this embodiment is characterized in that the area with the smaller appropriate exposure amount among the multiple areas A2 is set as the reference area, and the next exposure time and aperture amount are determined based on the appropriate exposure amount of the reference area. This prevents excessive exposure (overexposure) when switching the selected image, and the image after switching can have good image quality.

[0104] To explain the above effect in more detail, when the selected area is switched, the exposure time and the aperture value are generally determined so that the exposure amount of the selected area after the switch is an appropriate exposure amount. On the other hand, among the multiple regions A2, the brightness of the region A2 other than the selected region (hereinafter, unselected region) may be greater than the brightness of the selected region. In such a case, if the exposure amount of the selected region is determined based on its appropriate exposure amount, the RGB pixel values ​​of the image of the unselected region may become saturated. As a result, the appropriate exposure amount of each region A2 may not be calculated correctly. In addition, when an unselected region (strictly speaking, an unselected region with an appropriate exposure amount smaller than that of the selected region) switches to a selected region, overexposure may occur.

[0105] In contrast, in this embodiment, the area A2 with a smaller appropriate exposure amount (strictly speaking, the area A2 with the smallest appropriate exposure amount) is used as the reference area, and the exposure time and the aperture value are determined based on the appropriate exposure amount of the reference area. This makes it possible to prevent problems such as overexposure described above, and to more appropriately determine conditions such as exposure time during video shooting.

[0106] In this embodiment, as described above, the above steps S005 to S013 are performed for each frame while the video is being recorded. Then, when the first area having a larger appropriate exposure amount is selected as the selected area from among the multiple areas A2, the exposure amount of the selected area is determined in the second and subsequent frames based on the conditions determined in the immediately preceding frame.

[0107] Specifically, in the sensitivity determination step performed in step 008 in the second or subsequent frames, the sensitivity (gain) of the pixels 42 corresponding to the selected region (i.e., the first region) is determined. At this time, the sensitivity is determined based on the appropriate exposure amount of the first region, which is the selected region at that time, and the exposure time and aperture amount determined in step S011 (i.e., the condition determination step) in the immediately preceding frame. In addition, in this embodiment, the above sensitivity is determined to be higher than the sensitivity of the pixels 42 corresponding to the second region, which is the reference region. As a result, the sensitivity of the pixels 42 corresponding to the selected region is appropriately determined, and the exposure amount of the first region, which is the selected region, is adjusted to be the appropriate exposure amount.

[0108] Note that there are other possible methods for increasing and optimizing the exposure of the selected region besides the above-mentioned method of increasing the sensitivity of the pixels 42 corresponding to the selected region. For example, the exposure of the selected region may be increased by performing frame synthesis on the image of the selected region. Frame synthesis is a signal processing method in which image signals of multiple frames that are consecutive on the time axis are added together to perform signal addition. However, if a moving subject is captured in the image to which frame synthesis is applied, the frame synthesis may result in an afterimage of the subject. Therefore, it is best to apply exposure adjustment by frame synthesis only to images in which the subject is not moving.

[0109] <<Second embodiment>> A second embodiment in which the reference region is re-designated only when the selected region is switched will be described with reference to Fig. 10. Fig. 10 is an explanatory diagram of a video creation flow according to the second embodiment. In the following, the second embodiment will be mainly described with respect to the points that are different from the first embodiment.

[0110] As can be seen by comparing FIGS. 9 and 10, the steps in the video production flow according to the second embodiment are generally common to the steps in the video production flow according to the first embodiment. Specifically, steps S021 to S029 in the video creation flow according to the second embodiment are common to steps S001 to S009 in the video creation flow according to the first embodiment. The video creation flow according to the second embodiment is also common to the video creation flow according to the first embodiment in that steps S025 and subsequent steps are repeated for each frame while the video is being recorded.

[0111] On the other hand, in the second embodiment, once the designation step is performed and the reference region is designated, the designation step is performed only when the switching step is performed thereafter. Also, in the second embodiment, the condition determination step is performed every time the designation step is performed. In the above points, the second embodiment differs from the first embodiment.

[0112] Regarding the video creation flow according to the second embodiment, the differences from the first embodiment will be described as follows. As shown in FIG. 10, in the second embodiment, it is determined whether a reference area has already been specified in each frame (S030).

[0113] If the reference area is not designated, the control unit 46 performs a designation step and designates a second area (specifically, an area with the smallest appropriate exposure amount) among the multiple areas A2 as the reference area (S031). After that, the control unit 46 performs a condition determination step and determines the next exposure time and aperture amount based on the appropriate exposure amount of the reference area designated in step S031 (S032). Normally, steps S031 and S032 are performed only immediately after the start of recording of the captured video (that is, the first frame).

[0114] If the reference area has already been specified, it is determined whether the user has once selected a selected area and then reselected it and performed an input operation again (S033). If the selected area has not been reselected, the reference area at that time (i.e., the current reference area) is maintained. Then, when the control unit 46 determines the next exposure time and aperture amount in the condition determination process, these values ​​are determined based on the appropriate exposure amount of the current reference area (S034).

[0115] On the other hand, when the selected area is reselected, the control unit 46 performs a switching process to reselect and switch the selected area from among the multiple areas A2 in response to an input operation by the user (S035). The control unit 46 also performs a designation process in conjunction with the switching process to redesignate the reference area in the designation process (S036). Furthermore, in the condition determination process immediately following, the control unit 46 determines the next exposure time and aperture value based on the appropriate exposure amount of the redesignated reference area (S037).

[0116] In the video production flow according to the second embodiment, the above-mentioned steps S025 to S037 are repeatedly carried out until the user issues an instruction to end the process (S038).

[0117] As described above, in the video creation flow according to the second embodiment, re-designation of the reference area is permitted only when the selected area is switched. This makes the image quality of the selected video more stable compared to the first embodiment, in which the reference area changes every frame. The second embodiment is common to the first embodiment except for the above-mentioned differences, and can provide the same effects as those provided by the first embodiment.

[0118] <<Third embodiment>> A third embodiment in which conditions such as exposure time are determined based on the appropriate exposure amount of the reference area immediately after the selection or reselection of the selected area is performed will be described with reference to Fig. 11. Fig. 11 is an explanatory diagram of an image creation flow according to the third embodiment. Note that the following mainly describes the aspects of the third embodiment that are different from the first embodiment.

[0119] As can be seen from FIGS. 10 and 11, the steps in the video production flow according to the third embodiment are substantially the same as the steps in the video production flow according to the second embodiment.

[0120] In the video creation flow according to the third embodiment, immediately after a selection area is selected for the first time or immediately after a selection area is reselected, the control unit 46 performs a designation step (S051, S056). In the designation step, the control unit 46 designates the area A2 that corresponds to the second area at that time (strictly speaking, the area A2 with the smallest appropriate exposure amount) as the reference area. Then, in a condition determination step (i.e., S052, S057) performed immediately after the designation step, conditions such as exposure time are determined based on the appropriate exposure amount of the reference area designated in the designation step. Hereinafter, such a condition determination step will be referred to as a "first determination step."

[0121] On the other hand, in a frame where a selected area has already been selected and there is no reselection (switching) of the selected area, the designation step is not performed. In the condition determination step (S054) in such a frame, conditions such as exposure time are determined based on the appropriate exposure amount of the reference area at the time when the selected area was selected or reselected and the amount of change in the appropriate exposure amount of the current selected area. Hereinafter, such a condition determination step will be referred to as the "second determination step."

[0122] In the first determination step, conditions such as exposure time are determined based on the appropriate exposure of the reference area at that time. In addition, in the adjustment step after the first determination step is performed, the second determination step is performed. In the second determination step, the adjustment amount of the exposure in the adjustment step (more specifically, the exposure at the time of shooting the first video) is determined taking into account the change over time in the appropriate exposure of the selected area at that time. In the adjustment step, the exposure at the time of shooting the first video is adjusted based on the conditions such as exposure time determined in the first determination step and the adjustment amount determined in the second determination step.

[0123] The above will be described in detail with reference to FIGS. Fig. 12 shows the change over time in the appropriate exposure amount for each of the reference area and the selected area. Fig. 13 shows the exposure amount after adjustment of the selected area when adjustment is continued based on the appropriate exposure amount for the reference area. Fig. 14 shows the target exposure amount in the third embodiment. Fig. 15 shows the transition of the exposure amount after adjustment of each of the reference area and the selected area in the third embodiment. In each of FIGS. 12 to 15, the horizontal axis indicates the time that has elapsed since the start of video recording (more specifically, the number of frames that have elapsed), and the vertical axis indicates the amount of exposure.

[0124] Here, the "adjusted exposure amount" refers to the actual value that the exposure amount finally reaches for each frame, which is adjusted for each frame in the adjustment process. In addition, the "target exposure amount" is the target value that is expected to be achieved in each frame by adjusting the exposure amount for each frame in the adjustment process, and is set to a value according to the appropriate exposure amount (for example, a value equivalent to the appropriate exposure amount).

[0125] As a specific example, consider a case in which the appropriate exposure amount of each of the reference area and the selected area changes as shown in Fig. 12. In this case, the appropriate exposure amount of the selected area (denoted as EIt in the figure) is relatively stable, whereas the appropriate exposure amount of the reference area (denoted as EIs in the figure) fluctuates greatly. Note that in the case shown in Fig. 12, it is assumed that the selected area is reselected (switched) at the point when the elapsed time is tc.

[0126] In the above case, when the exposure amount during the first video capture is adjusted for each frame, it is adjusted based on the appropriate exposure amount of the reference area for each frame. At this time, the exposure amount after adjustment of the reference area and the selected area changes as shown in FIG. 13. The exposure amount of the reference area after adjustment (indicated as ERs in the figure) fluctuates greatly like the appropriate exposure amount. The exposure amount of the selected area after adjustment (indicated as ERt in the figure) shows the same tendency of change as the appropriate exposure amount, by changing the sensitivity (gain) of the pixel 42 corresponding to the selected area so that it becomes the appropriate exposure amount.

[0127] However, the sensitivity (gain) that determines the exposure of the selected area after adjustment is affected by the exposure of the reference area after adjustment, in other words, the appropriate exposure of the reference area. Therefore, if the appropriate exposure of the reference area is significantly changed, the sensitivity (gain) also changes significantly, which may cause problems such as an increase or decrease in noise in the image of the selected area.

[0128] In contrast, in the third embodiment, the reference area is designated and the first determination step is performed only immediately after the selection or reselection of the selected area. In the first determination step, conditions such as exposure time are determined based on the appropriate exposure amount (denoted as e1 in FIG. 12) of the reference area at the time of selection or reselection of the selected area.

[0129] Furthermore, during the period following the first determination step (i.e., the period during which the selected area has already been selected and the selected area has not been reselected), the second determination step is performed for each frame. In the second determination step for each frame, the amount of exposure adjustment is determined based on the appropriate exposure e1 of the reference area at the time the selected area was selected or reselected, and the amount of change in the appropriate exposure of the current selected area. Here, the "amount of change in the appropriate exposure of the selected area" refers to the difference between the appropriate exposure of the selected area in the immediately preceding frame and the appropriate exposure of the selected area in the current frame, and is represented by the symbol Δ in FIG. 12.

[0130] As described above, the first determination step is performed, and thereafter, the second determination step is performed for each frame, thereby determining the target exposure amount for each frame. To explain in more detail, as shown in Fig. 14, the target exposure amount (indicated as EG in the figure) at the time tc when the selected area is reselected is set based on the appropriate exposure amount e1 of the reference area at that time. The target exposure amount EG thereafter is determined following the change in the appropriate exposure amount of the selected area, and specifically, changes by the difference Δ of the appropriate exposure amount for each frame.

[0131] Then, for each frame in the adjustment process, conditions such as exposure time are determined so that the exposure during first video shooting becomes the target exposure EG determined for each frame. As a result, the exposure ERs of the reference area after adjustment becomes the exposure according to the target exposure EG as shown in Fig. 15. Here, since the target exposure EG reflects the change in the appropriate exposure of the selected area, the exposure ERs of the reference area after adjustment becomes relatively stable as shown in Fig. 15.

[0132] In addition, in each frame during the adjustment process, the exposure amount of the selected region is adjusted based on conditions such as exposure time determined for each frame. More specifically, in each frame, the sensitivity (gain) of the pixel 42 corresponding to the selected region is determined according to conditions such as exposure time determined for each frame and the appropriate exposure amount of the selected region in each frame. As a result, the exposure amount ERt of the selected region after adjustment becomes the exposure amount equivalent to the appropriate exposure amount in each frame, as shown in FIG.

[0133] Furthermore, as can be seen from Figure 15, the sensitivity (gain) that determines the exposure of the selected area after adjustment is relatively stable. This is because the target exposure EG, which is the base for determining the sensitivity, is stable. As a result, problems such as increases and decreases in noise in the image of the selected area are suppressed.

[0134] As described above, in the third embodiment, conditions such as exposure time are determined based on the appropriate exposure of the reference area only immediately after the selection or reselection of the selected area. For other periods, the adjustment amount of the exposure is determined based on the change in the appropriate exposure of the selected area. This makes it possible to stabilize the image quality of the selected area while the selected area is not being reselected (switched).

[0135] <<Fourth embodiment>> A fourth embodiment in which the exposure amount is adjusted at a faster adjustment speed than normal immediately after the switching step is performed will be described with reference to Fig. 16. Fig. 16 is an explanatory diagram of an image creation flow according to the fourth embodiment.

[0136] As can be seen from FIG. 16, steps S061 to S067 in the video production flow according to the fourth embodiment are common to steps S001 to S007 in the video production flow according to the first embodiment.

[0137] Steps S068 to S074 of the video production flow according to the fourth embodiment will be described in detail below. In the video production flow according to the fourth embodiment, when a switching step is performed, the subsequent steps change depending on the time elapsed since the switching step was performed (S070).

[0138] More specifically, it is assumed that the control unit 46 performs a switching step at a certain point in time (a certain frame) to switch the selected area. In this case, in step S070, the control unit 46 measures the elapsed time from that frame (strictly speaking, the number of elapsed frames). Then, during the period from the previous switching step until the first period has elapsed, the control unit 46 performs a first recording step (S071). The first recording step is a step of recording the video of the selected area during the first period. The first period can be determined to be any period, but is preferably determined to be within a range of one second.

[0139] Furthermore, when the current time (current frame) is in the first period, the control unit 46 executes an adjustment step of adjusting the exposure amount of the selected region in preparation for the next frame (S072). In this step S072, the exposure amount of the selected region is adjusted at an adjustment speed faster than normal based on the appropriate exposure amount at that time. Specifically, in the adjustment step, when the aperture driving unit 23 is driven to change the aperture amount for adjusting the exposure amount, the driving speed of the aperture driving unit 23 is set faster than the normal driving speed. The adjustment speed in the adjustment process in the first period can be determined arbitrarily as long as it is faster than the normal adjustment speed, but it is preferably at least 1.5 times faster than the normal adjustment speed, and more preferably at least 2 times faster.

[0140] On the other hand, after the first period has elapsed since the previous switching step was performed, that is, when the current time (current frame) is in a second period other than the first period, the control unit 46 performs a second recording step (S073). The second recording step is a step of recording the video of the selected area in the second period.

[0141] Furthermore, when the current time (current frame) is in the second period, the control unit 46 executes an adjustment step of adjusting the exposure amount of the selected region in preparation for the next frame (S074). In this step S074, the exposure amount of the selected region is adjusted at a normal adjustment speed based on the appropriate exposure amount at that time. Specifically, the aperture drive unit 23 is driven at a normal drive speed to change the aperture amount in order to adjust the exposure amount.

[0142] As described above, in the fourth embodiment, the adjustment speed of the exposure amount (specifically, the drive speed of the aperture drive unit 23) is set faster in the adjustment process performed in the first period than in the adjustment process performed in the second period. This allows the exposure amount of the selected area after the switching process to be quickly adjusted (optimized) when the switching process is performed. As a result, an image of the selected area after the switching process is acquired with an appropriate exposure amount, and the image quality is improved.

[0143] In the above case given as a specific example of the fourth embodiment, the exposure amount of the selected area is adjusted in the adjustment process. However, there are other conditions that should be adjusted promptly after the switching process is performed, such as the white balance of the image of the selected area and the focus of the photographing lens 14, in addition to the exposure amount.

[0144] In the fourth embodiment, an adjustment process for adjusting at least one of the exposure amount of the selected area, the white balance of the image of the selected area, and the focus can be performed in each of the first and second periods. In this case, the adjustment speed of the condition (the condition to be adjusted) in the adjustment process performed in the first period may be faster than that in the adjustment process performed in the second period.

[0145] Specifically, when the aperture drive unit 23 is driven to change the aperture amount in order to adjust the amount of exposure during the adjustment process, the drive speed of the aperture drive unit 23 in the adjustment process in the first period may be made faster than that in the adjustment process in the second period. Furthermore, when the focusing optical component 19 is moved for focus adjustment in the adjustment process, the moving speed of the focusing optical component 19 in the adjustment process in the first period may be set faster than that in the adjustment process in the second period.

[0146] <<Fifth embodiment>> A fifth embodiment in which the amount of exposure is adjusted by preferentially changing parameters other than the aperture value immediately after the switching step is performed will be described with reference to Figures 17 and 18. Figure 17 is an explanatory diagram of a video creation flow according to the fifth embodiment. Figure 18 shows an example of a change pattern determined by adjustment data. It should be noted that the numerical values ​​of the change patterns shown in FIG. 18 are values ​​determined for the convenience of explanation, and differ from the values ​​defined in the actual change patterns.

[0147] The video creation flow according to the fifth embodiment is generally the same as that according to the fourth embodiment. That is, in the video creation flow according to the fifth embodiment, first, various conditions are initially set, a first video is displayed when video shooting starts, a plurality of areas A2 are set, and a selection area is selected from among them (S081 to S084). Then, the steps from step S085 onwards (steps S085 to S094) are carried out in response to the user pressing the release button 26 or the like.

[0148] Specifically, one frame of the first image is acquired, the integrated value of RGB pixel values ​​is calculated for each of the multiple areas A2, and the appropriate exposure amount is determined from the integrated value (S085 to S087). After that, the image of the selected area is recorded (S091, S093), and the exposure amount of the selected area is adjusted for the next frame (S092, S094).

[0149] Furthermore, when the selected region is reselected, a switching step is performed to switch the selected region (S088, S089). The step after the switching step is performed varies depending on the time that has elapsed since the switching step was performed (S090). Specifically, when the current time (current frame) is in the first period, the control unit 46 performs a first recording step (S091). When the current time (current frame) is in the first period, the control unit 46 performs a first adjustment step to adjust the exposure amount of the selected region in preparation for the next frame (S092).

[0150] On the other hand, if the current time (current frame) is in the second period, the control unit 46 performs a second recording step (S093). When the current time (current frame) is in the second period, the control unit 46 performs a second adjustment step to adjust the exposure amount of the selected region in preparation for the next frame (S094). The above series of steps (that is, steps S085 to S094) are repeatedly performed until the user issues an end instruction (S095).

[0151] The first and second adjustment steps in the fifth embodiment will be described. In each adjustment step, the exposure amount of the selected area is adjusted to a target exposure amount based on the appropriate exposure amount at that time using adjustment data. More specifically, in each adjustment step, the values ​​of a plurality of parameters including the exposure time (shutter speed), the sensitivity (ISO sensitivity) of the pixel 42, and the aperture value (F value) are determined based on the adjustment data. After that, the values ​​of the parameters are changed to the determined values, and the exposure amount of the selected area is adjusted to the target exposure amount.

[0152] The adjustment data is data that defines change patterns, is prepared in the same number as the change patterns, and is recorded in the internal memory 50 or buffer 56 of the photographing device body 12. The change patterns define patterns (changing tendencies) for changing the values ​​of a plurality of parameters in order to adjust the exposure amount to the target exposure amount.

[0153] In the fifth embodiment, the adjustment data used for adjustment to the same target exposure amount is different between the first and second adjustment steps, that is, the values ​​of the multiple parameters determined for the same target exposure amount are different between the first adjustment data used in the first adjustment step and the second adjustment data used in the second adjustment step.

[0154] The difference between the first adjustment data and the second adjustment data will be described with reference to Fig. 18. Both of the two adjustment data shown in Fig. 18 define change patterns for the aperture amount, exposure time, and sensitivity for adjusting the exposure amount from a value ep to a target value et (target exposure amount). In other words, as shown in Fig. 18, among the change patterns defined by the adjustment data, the value of each parameter at the start of adjustment and the value of each parameter at the end of adjustment are common between the two adjustment data.

[0155] On the other hand, in the change pattern determined by the first adjustment data (shown at the top in FIG. 18), the aperture value is gradually increased in stages. The exposure time, which is a parameter other than the aperture value, increases rapidly immediately after the start of adjustment, and then decreases in accordance with the increase in the aperture value. In contrast, in the change pattern determined by the second adjustment data (shown at the bottom in FIG. 18), the aperture value increases to the final value immediately after the start of adjustment and is then maintained at the final value. The exposure time is maintained at a constant value without change during the adjustment period.

[0156] As described above, in the change pattern defined in the first adjustment data, the change degree (amount of change per time) of the aperture amount relative to the target exposure amount is smaller than in the change pattern defined in the second adjustment data. In other words, in the second adjustment process using the second adjustment data, the aperture amount is changed with priority when adjusting the exposure amount, whereas in the first adjustment process using the first adjustment data, parameters other than the aperture amount are changed with priority when adjusting the exposure amount.

[0157] As described above, in the fifth embodiment, when adjusting the exposure amount immediately after the selected area is switched, priority is given to changing parameters other than the aperture amount (specifically, exposure time) rather than changing the aperture amount accompanied by mechanical drive. The exposure time, i.e., the shutter speed of the electronic shutter, can be changed by digital processing, and therefore can be changed more quickly than the aperture amount. This allows the exposure amount of the selected area after switching to be quickly adjusted. In addition, since minute fluctuations in brightness caused by a sudden change in the aperture amount are suppressed, it is possible to suppress the influence of the minute fluctuations on the image quality. As a result, the image of the selected area after the switching process can be made to have good image quality.

[0158] In the above case, the exposure time is changed in the first adjustment process with priority over the aperture amount, but this is not limited to this, and the sensitivity (gain) of the pixel 42 corresponding to the selected area may be changed in the first adjustment process with priority over the aperture amount. Furthermore, if the configuration includes the electronic attenuation filter 21, the multiple parameters for adjusting the exposure amount may include the degree of dimming of the electronic attenuation filter 21. In such a case, the degree of dimming of the electronic attenuation filter 21 (more directly, the voltage applied to the electronic attenuation filter 21) can be changed with priority over the aperture amount. This makes it possible to suppress changes in the aperture amount and increases in the shutter speed.

[0159] <<Sixth embodiment>> In the above-mentioned five embodiments, the video is a moving image, that is, a collection of a plurality of frame images captured continuously at a constant frame rate. However, the recorded video is not limited to a moving image, and may be a still image.

[0160] For example, the control unit 46 displays the first image on the display 28 as a through image, and sets a plurality of partial areas A2 in the shooting area A1 of the first image. Then, when the user selects one of the plurality of set areas A2 as a selected area, the control unit 46 extracts the image of the selected area and displays it on the display 28 as shown in Fig. 7. When the user inputs a recording instruction while the image of the extracted selected area is being displayed, the control unit 46 records a still image file of the image of the selected area on the recording medium.

[0161] When the video to be recorded is a still image, the shutter speed that defines the exposure time may be the shutter speed of an electronic shutter, or may be the shutter speed of the shutter 38 (focal plane shutter), which is a mechanical component.

[0162] <<Other embodiments>> The embodiment described above is a specific example given to clearly explain the video creation method of one embodiment of the present invention, and is merely one example, and other embodiments are possible. For example, in the above embodiment, the imaging device 10 is provided with the electronic neutral density filter 21, but the imaging device may not have the electronic neutral density filter 21. In the above embodiment, the photographing device 10 has an autofocus function, but the present invention is not limited to this, and the photographing device may not have an autofocus function. That is, a photographing device may be used that does not have the focus driving unit 22 and adjusts the focus only by operating the focus ring 16 (i.e., only manually).

[0163] In the above embodiment, the image capture device 10 creates a video file as an image creation device, but this is not limited to the above. For example, other devices connected to the image capture device by wire or wirelessly, such as a camera controller or an external recorder, may be used as the image creation device. These devices may then create a video file of the video captured by the image capture device.

[0164] Furthermore, in the above-described embodiment, the photographing device is a digital camera, but it may also be a video camera, a mobile phone with an imaging optical system, a smartphone, a tablet terminal, or other portable terminal. The imaging lens may be a lens unit that is externally attached to the imaging optical system of the above-mentioned mobile terminal. [Explanation of symbols]

[0165] 10. Imaging Device 12. Imaging device body 13. Mount 14. Shooting Lenses 16 Focus ring 18 Optical component unit 19 Focusing optics 20 apertures 21 Electronic neutral density filter 22 Focus drive unit 23 Aperture drive unit 24 Voltage application section 26 Release button 28 Display 30 First operation button 32 Second operation button 34 3rd operation button 36 Touch Panel 38 Shutter 40 Image sensor 42 pixels 44 Analog signal processing circuit 46 Control Unit 47 Controller 48 Video Processing Section 50 Internal Memory 52 Card Slots 54 Memory Card 56 Buffers A0 unit area A1 Shooting area A2 area L1 optical axis FR area setting frame

Claims

1. A video creation method for creating a video file based on a video captured by a photographing device having a photographing lens and an image sensor, comprising: A setting step in which a processor sets a plurality of areas having a second angle of view smaller than the first angle of view within a shooting area of ​​a first image having a first angle of view; a selection step in which the processor selects a selected area as a shooting area of ​​an image to be recorded from among the plurality of areas; a switching step of the processor, after the selecting step, reselecting the selected area from among the plurality of areas to switch the selected area; a first recording step, which is performed during a period from when the switching step is performed until a first period has elapsed, and in which the processor records an image of the selected area during the first period; a second recording step that is performed during a second period other than the first period, and in which the processor records an image of the selected area during the second period; a first adjustment step that is performed during the first period and in which the processor adjusts an exposure amount of the selected region to a target exposure amount using first adjustment data; a second adjustment step that is performed during the second period and in which the processor adjusts the exposure amount of the selected region to a target exposure amount using second adjustment data; In the first adjustment step and the second adjustment step, values ​​of a plurality of parameters including an exposure time, a sensitivity of a pixel of the image sensor, and an aperture amount of light incident on the photographing lens are determined based on the first adjustment data or the second adjustment data, and the exposure amount is adjusted to the target exposure amount; The image creating method, wherein the first adjustment data and the second adjustment data have mutually different values ​​for each of the plurality of parameters determined for the target exposure.

2. The video creation method according to claim 1 , further comprising a creation step in which the processor creates the video file by recording a video of the selected area before the switching step and a video of the selected area after the switching step.

3. 3. The video creating method according to claim 2, wherein in the creating step, a moving image file as the video file is created by combining the video of the selected area before the switching step and the video of the selected area after the switching step.

4. a designation step of the processor designating, as a reference area, a second area having a lower appropriate exposure amount than the first area, out of a first area and a second area included in the plurality of areas; The image creation method of claim 1, further comprising a condition determination step in which the processor determines at least one shooting condition among an exposure time when shooting the first image and an aperture value for light incident on the shooting lens based on the appropriate exposure amount of the reference area.

5. When the switching step is performed, the designation step is performed, The video production method according to claim 4 , wherein the condition determining step is performed each time the specifying step is performed.

6. 6. The image creation method of claim 4 or 5, further comprising a sensitivity determination step in which, when the first region is selected as the selected region, the processor determines the sensitivity of the pixel corresponding to the first region among a plurality of pixels of the imaging element based on the appropriate exposure amount of the first region and the conditions determined in the condition determination step.

7. 7. The image generating method according to claim 6, wherein in the sensitivity determining step, the sensitivity of the pixels corresponding to the first region is determined to be higher than the sensitivity of the pixels corresponding to the second region which is the reference region.

8. The method further includes an adjustment execution step of adjusting an exposure amount over time when capturing the first image by changing the capturing condition over time, The condition determining step includes: a first determination step of determining the photographing conditions based on the appropriate exposure amount of the reference area; a second determination step of determining an adjustment amount of the exposure amount in the adjustment execution step based on the appropriate exposure amount of the selected region, 8. The image creating method according to claim 4, wherein the adjustment execution step is performed based on the shooting conditions determined in the first determination step and the amount of adjustment determined in the second determination step.

9. When the switching step is performed, the first determination step is performed; 9. The video production method according to claim 8, wherein the second determination step is carried out in the adjustment execution step after the first determination step is carried out.

10. a display step of the processor displaying the first image on a display screen, The image creation method according to claim 1 , wherein in the setting step, the plurality of regions are set in accordance with a setting operation performed by a user through the first image displayed on the display screen.

11. a change pattern for changing each of the values ​​of the plurality of parameters in order to adjust the exposure amount to the target exposure amount is defined in the first adjustment data and the second adjustment data; 11. The image creation method according to claim 1, wherein the first adjustment data defines a change pattern in which the degree of change of the aperture amount relative to the target exposure amount is smaller than the change pattern of the second adjustment data.

12. The photographing device is provided with an electronic dimming filter capable of electronically changing a degree of dimming, 12. The image creation method according to claim 1, wherein the plurality of parameters includes the degree of dimming of the electronic neutral density filter.

13. A video production system having an imaging device and a processor, The processor, A plurality of areas having a second angle of view smaller than the first angle of view are set in a shooting area of ​​a first image having a first angle of view; accepting a selection of a selected area as a shooting area of ​​the video to be recorded from among the plurality of areas; After said selection, reselecting said selected area to switch said selected area among said plurality of areas; The switching is performed during a period from when the switching is performed until a first period has elapsed, and an image of the selected area during the first period is recorded; The method is carried out during a second period other than the first period, and an image of the selected area during the second period is recorded; adjusting an exposure amount of the selected region to a target exposure amount using first adjustment data, the adjustment being performed during the first period; adjusting the exposure amount of the selected region to a target exposure amount using second adjustment data, the adjustment being performed during the second period; In the first period and the second period, the exposure amount is adjusted to the target exposure amount based on the first adjustment data or the second adjustment data, for each value of a plurality of parameters including an exposure time, a pixel sensitivity, and an aperture amount of incident light; The first adjustment data and the second adjustment data have different values ​​of the plurality of parameters determined for the target exposure amount. Video production system.

Images