How to create a video

Abstract

To provide a video creation method capable of obtaining video having good image quality by extracting a part of captured video even when the position of the extracted area is changed.SOLUTION: A plurality of areas is set in a shooting area of first video, a selected area of recording video is selected from the plurality of areas, the selected area is reselected to switch the selected area, the video of the selected area in the first period is recorded from the switching of the selected area until the first period has elapsed, the video of the selected area in a second period is recorded in the second period other than the first period, the exposure amount of the selected area is adjusted to a target exposure amount in the first period, and the exposure amount of the selected area is adjusted to the target exposure amount in the second period. In the first adjustment step and the second adjustment step, values of a plurality of parameters are determined on the basis of the first adjustment data or the second adjustment data to adjust the exposure amount to the target exposure amount, and the values of the plurality of parameters differ from each other between the first adjustment data and the second adjustment data.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a video creation method for capturing video using a camera and creating a video file. [Background technology]

[0002] When capturing images of multiple subjects in a scene, the scene may be captured by a single camera, and the images of each subject may be extracted from the captured images. Examples of such techniques include those described in Patent Documents 1 to 3.

[0003] Patent Document 1 discloses a technique for generating a cutout image by cutting out a part of an original image captured by a single camera, controlling the cutout position, and outputting the cutout image so that the position of the subject changes.

[0004] Patent Document 2 discloses a technology that detects multiple objects based on high-resolution wide-angle images captured by a single camera, cuts out images of each detected object from the wide-angle image, and displays the images of the multiple objects in parallel on a monitor.

[0005] Patent Document 3 discloses a technology for specifying a plurality of arbitrary areas from an image captured by a single imaging device, and cutting out and outputting images corresponding to the specified plurality of arbitrary areas. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 11-220653 [Patent Document 2] Japanese Patent Application Laid-Open No. 2004-194309 [Patent Document 3] Japanese Patent Application Laid-Open No. 2014-42357 Summary of the Invention [Problem to be solved by the invention]

[0007] When extracting a portion of an image, the image quality of the extracted image is determined by the conditions at the time of shooting, such as exposure, white balance, and lens focus. When these conditions are set to appropriate values, the extracted image will have good image quality.

[0008] On the other hand, the appropriate conditions for the extracted image depend on the position of the extracted region in the original image. Taking this into consideration, it is necessary to ensure good image quality for the extracted image even when the position of the extracted region changes. However, the techniques described in Patent Documents 1 to 3 do not optimize conditions such as exposure, white balance, and lens focus according to the position of the extracted image. Therefore, even if the techniques described in these documents are used, there is a risk that the image quality of the extracted image, particularly when the position of the extracted area changes, may not be properly ensured.

[0009] One embodiment of the present invention has been made in consideration of the above circumstances, and aims to provide an image creation method that can extract a portion of a captured image and maintain good image quality even when the position of the extraction area is changed. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, one embodiment of the video creation method is a video creation method for creating a video file based on video captured by a photographing device equipped with a photographing lens and an image sensor, and is characterized by comprising: a setting process for setting multiple areas having a second angle of view smaller than the first angle of view within a photographing area of ​​a first video having a first angle of view; a designation process for designating, as a reference area, a second area having a lower appropriate exposure amount than the first area from among the first and second areas included in the multiple areas; and a condition determination process for determining, based on the appropriate exposure amount of the reference area, at least one condition from the exposure time when photographing the first video and the aperture amount for light incident on the photographing lens.

[0011] Furthermore, a video creation method according to one embodiment of the present invention may include a selection process for selecting a selected area, which is the shooting area of ​​the video to be recorded, from among a plurality of areas; a switching process for reselecting the selected area from among the plurality of areas after the selection process and switching the selected area; and a creation process for recording the video of the selected area before the switching process and the video of the selected area after the switching process to create a video file.

[0012] In the video creation method according to one embodiment of the present invention, the creation step may combine the video of the selected area before the switching step and the video of the selected area after the switching step to create a moving image file as a video file. In this case, a moving image file of video that maintains good image quality before and after the switching step can be obtained.

[0013] It is also preferable that the designation step is carried out when the switching step is carried out, and the condition determination step is carried out every time the designation step is carried out.

[0014] In addition, the image creation method of one embodiment of the present invention may further include a sensitivity determination step of determining, when the first region is selected as the selected region, the sensitivity of a pixel corresponding to the first region among the multiple pixels of the image sensor based on the appropriate exposure amount of the first region and the conditions determined in the condition determination step. In the sensitivity determination step, it is preferable to determine the sensitivity of the pixels corresponding to the first region so that it is higher than the sensitivity of the pixels corresponding to the second region, which is the reference region.

[0015] Furthermore, the image creation method according to one embodiment of the present invention may further include an adjustment step of adjusting the exposure amount over time when capturing the first image by changing the conditions over time. In this case, the condition determination step may include a first determination step of determining the conditions based on the appropriate exposure amount for the reference area, and a second determination step of determining the adjustment amount of the exposure amount in the adjustment step based on the appropriate exposure amount for the selected area. Furthermore, the adjustment step may be performed based on the conditions determined in the first determination step and the adjustment amount determined in the second determination step. It is preferable to carry out the first determining step when the switching step is carried out, and to carry out the second determining step in the adjustment step after the first determining step is carried out.

[0016] The image creation method according to an embodiment of the present invention may further include a display step of displaying the first image on a display screen. In this case, the setting step may set a plurality of areas in accordance with a setting operation performed by a user through the first image displayed on the display screen.

[0017] Furthermore, according to one embodiment of the present invention, there is provided a video creation method for creating a video file based on video captured by a photographing device equipped with a photographing lens and an image sensor, the method including: a setting step for setting a plurality of areas having a second angle of view smaller than the first angle of view within a photographing area of ​​a first video having a first angle of view; a selection step for selecting a selected area from the plurality of areas as a photographing area of ​​the video to be recorded; a switching step for reselecting the selected area from the plurality of areas after the selection step and switching the selected area; a first recording step that is carried out between the time the switching step is carried out and a first period has elapsed and that records video of the selected area in the first period; a second recording step that is carried out in a second period other than the first period and that records video of the selected area in the second period; and an adjustment step that adjusts at least one condition of the exposure of the selected area, the white balance of the video of the selected area, and the focus of the photographing lens during each of the first and second periods; and the adjustment speed of the conditions in the adjustment step carried out during the first period is faster than that in the adjustment step carried out during the second period.

[0018] Furthermore, in the above-described image creation method, in the adjustment process, the exposure amount is adjusted by driving an aperture drive unit that changes the aperture amount for light incident on the photographing lens, and it is more preferable that the drive speed of the aperture drive unit in the adjustment process performed in the first period is faster than that in the adjustment process performed in the second period. Furthermore, in the above-described image creation method, it is more preferable that in the adjustment process, the focus is adjusted by moving a focusing optical component of the photographing lens, and that the moving speed of the focusing optical component is faster in the adjustment process performed in the first period than in the adjustment process performed in the second period.

[0019] According to one embodiment of the present invention, there is provided a video creation method for creating a video file based on video captured by a photographing device having a photographing lens and an image sensor, the method including: a setting step for setting a plurality of areas having a second angle of view smaller than the first angle of view within a photographing area of ​​a first video having a first angle of view; a selection step for selecting a selected area from the plurality of areas as a photographing area of ​​the video to be recorded; a switching step for reselecting the selected area and switching the selected area from the plurality of areas after the selection step; a first recording step for recording the video of the selected area during the first period after the switching step is performed; and a second recording step for recording the video of the selected area during the second period, the first recording step being performed during a second period other than the first period. a first adjustment step, which is carried out in a first period, of adjusting the exposure amount of the selected region to a target exposure amount using the first adjustment data; and a second adjustment step, which is carried out in a second period, of adjusting the exposure amount of the selected region to the target exposure amount using the second adjustment data. In the first adjustment step and the second adjustment step, the values ​​of each of a plurality of parameters including the exposure time, the sensitivity of the pixels of the image sensor, and the aperture amount of incident light to the photographing lens are determined based on the first adjustment data or the second adjustment data, and the exposure amount is adjusted to the target exposure amount. An image creation method can also be realized in which the values ​​of each of the plurality of parameters determined for the target exposure amount differ between the first adjustment data and the second adjustment data.

[0020] In the above video production method, the first adjustment data and the second adjustment data may define a change pattern for changing each of the values ​​of the plurality of parameters to adjust the exposure amount to the target exposure amount. In this case, it is preferable that the first adjustment data defines a change pattern that changes the aperture amount to a smaller degree relative to the target exposure amount than the change pattern of the second adjustment data.

[0021] In the image creation method, the image capture device may be provided with an electronic neutral density filter capable of electronically changing the degree of dimming, and the plurality of parameters may include the degree of dimming of the electronic neutral density filter. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a perspective view showing an example of the appearance of an imaging device according to a first embodiment of the present invention. [Figure 2] 1 is a rear view showing the rear side of an imaging device according to a first embodiment of the present invention. [Figure 3] 1 is a block diagram showing the configuration of an imaging device according to a first embodiment of the present invention. [Figure 4] 4 is a diagram showing the correspondence between pixels of an imaging element and a shooting area of ​​a first video image. FIG. [Figure 5] FIG. 10 is a diagram showing a procedure for setting an area of ​​an image to be extracted from a first image. [Figure 6] FIG. 10 is a diagram showing how a plurality of areas of an image to be extracted from a first image are set. [Figure 7] FIG. 10 is a diagram showing an image of a selected area displayed on a display screen. [Figure 8] FIG. 2 is an explanatory diagram of a moving image file of recorded video. [Figure 9] FIG. 1 is a diagram showing a flow of a video creation flow according to a first embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing a flow of a video production flow according to a second embodiment of the present invention. [Figure 11] FIG. 10 is a diagram showing the flow of a video production flow according to a third embodiment which is one embodiment of the present invention. [Figure 12] 10A and 10B are diagrams showing changes over time in the appropriate exposure amounts of the reference area and the selected area. [Figure 13] FIG. 10 is a diagram showing the exposure amount after adjustment of the selected area when adjustment is continued based on the appropriate exposure amount of the reference area. [Figure 14] FIG. 11 is a diagram showing a target exposure amount in a third embodiment. [Figure 15] 13A and 13B are diagrams showing changes in exposure amount after adjustment of each of the reference area and the selected area in the third embodiment. [Figure 16] FIG. 10 is an explanatory diagram of a video production flow according to a fourth embodiment which is one embodiment of the present invention. [Figure 17] FIG. 10 is an explanatory diagram of a video production flow according to a fifth embodiment which is one embodiment of the present invention. [Figure 18] FIG. 10 is a diagram illustrating an example of a change pattern defined by adjustment data. DETAILED DESCRIPTION OF THE INVENTION

[0023] Preferred embodiments (first to sixth embodiments) of the present invention will now be described in detail with reference to the accompanying drawings. However, the embodiment described below is merely an example given to facilitate understanding of one embodiment of the present invention, and does not limit the present invention. In other words, one embodiment of the present invention may be modified or improved from the embodiment described below without departing from the spirit of the invention. Furthermore, one embodiment of the present invention includes equivalents thereof.

[0024] <<First Embodiment>> A first embodiment of the present invention relates to a method of creating an image using a photographing device 10 shown in Figures 1 to 3. Figures 1 and 2 show the appearance of the photographing device 10, with Figure 1 being a perspective view of the photographing device 10 and Figure 2 being a view of the photographing device 10 as seen from the rear side. Figure 3 is a block diagram showing the configuration of the photographing device 10.

[0025] [Basic configuration of imaging device] The image capturing device 10 is, for example, a digital camera used for capturing video. In the following description, unless otherwise specified, "video" refers to live video (live view image), that is, video captured in real time.

[0026] 1 and 2 is a digital camera with an interchangeable lens, and includes a camera body 12 and a photographic lens 14. The photographic lens 14 is interchangeably attached to a mount 13 of the camera body 12. However, the present invention is not limited to this, and the camera body 10 may also be a digital camera with an integrated lens.

[0027] (photographic lens) As shown in FIG. 3, the photographing lens 14 includes an optical component unit 18, an aperture 20, an electronic neutral density filter 21, a focus driver 22, an aperture driver 23, and a voltage application unit 24. The optical component unit 18 has a plurality of lenses, including a focusing optical component 19 (focus lens). When the focusing optical component 19 moves in the direction of the optical axis L1, the focus of the photographing lens 14 changes.

[0028] There are two focus adjustment modes: manual focus mode and autofocus mode. In manual focus mode, the user manually turns a focus ring 16 provided on the lens barrel of the photographic lens 14, which moves a focusing optical component 19 along the optical axis L1. In the autofocus mode, the control unit 46 drives the focus drive unit 22 to move the focus optical component 19 in the direction of the optical axis L1, and the focus is automatically adjusted to focus on the subject in the image. Note that the focus adjustment in the autofocus mode is triggered, for example, when the user half-presses the release button 26 of the image capturing device body 12.

[0029] The drive motor of the focus drive unit 22 is configured by an ultrasonic motor or the like, and is provided in the image taking device body 12 or the image taking lens 14 .

[0030] The optical component unit 18 includes a wide-angle lens, an ultra-wide-angle lens, a 360-degree lens, an anamorphic lens, or the like. This enables the image capture device 10 to capture video with a wide horizontal angle of view. Here, the maximum angle of view (hereinafter referred to as the first angle of view) when the image capture device 10 captures video is determined depending on the use of the optical component unit 18 and the image sensor 40 (described below), and the video captured at the first angle of view corresponds to the "first video" in one embodiment of the present invention.

[0031] The imaging device 10 may include a plurality of optical component units 18 each having a different angle of view.

[0032] The diaphragm 20 is an optical component that is disposed in the optical path of the photographing lens 14, has a variable aperture shape, and adjusts the aperture amount (specifically, the aperture value or F-number) for light incident on the photographing lens 14. The diaphragm 20 has an aperture shape that can be changed, for example, by mechanically driving a member that changes the size of the aperture. However, the diaphragm is not limited to this, and may be a diaphragm (physical diaphragm) whose aperture shape can be changed by driving a liquid crystal or an electrochromic element.

[0033] The opening shape of the diaphragm 20, that is, the aperture amount, is adjusted by a diaphragm driver .

[0034] The electronic neutral density filter 21 is an ND (Neutral Density) filter that is placed in the optical path of the photographing lens 14 and can electronically change the degree of dimming. The electronic neutral density filter 21 is made, for example, by mixing a light-absorbing dimming material into liquid crystal, and controls the degree of dimming (specifically, light transmittance) by changing the tilt of the liquid crystal molecules.

[0035] The electronic neutral density filter 21 is not limited to a liquid crystal type electronic ND filter, but may be a precipitation type electronic ND filter that utilizes an oxidation-reduction reaction of a reactive substance (for example, silver chloride). As the electronic neutral density filter 21, for example, the electronic ND filters described in Japanese Patent Application Laid-Open No. 2013-88596 or Japanese Patent Application Laid-Open No. 2019-68402 can be used.

[0036] The degree of dimming of the electronic attenuation filter 21 is adjusted by changing the magnitude of the voltage (applied voltage) that the voltage application unit 24 applies to the electronic attenuation filter 21.

[0037] (imaging device body) 1 and 2, the photographing device body 12 includes an operation unit that is operated by the user. For example, a release button 26 is disposed on the top surface of the photographing device body 12. For example, when the user fully presses the release button 26, recording of an image captured by the photographing device 10 or an image based on the captured image begins. An example of an image based on the captured image is an image of a selected area, which will be described later. The user's release instruction (i.e., a recording instruction) may be input to the control unit 46 via a touch operation detection function of the display 28.

[0038] A display 28 constituting a display screen is disposed on the rear surface of the imaging device body 12, and a plurality of buttons 30, 32, and 34 are disposed around the display 28.

[0039] The display 28 is configured by, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electroluminescence) display, an LED (Light Emitting Diode) display, electronic paper, etc. Images captured by the image capturing device 10 or images based on the captured images are displayed on the display screen of the display 28. Furthermore, the display screen of the display 28 also displays a selection menu relating to the shooting conditions, notification information for the user including warnings, and playback images of previously acquired images.

[0040] The display 28 has a function for detecting touch operations of the user's fingers. A transparent touch panel 36 is overlaid on the display 28 or incorporated inside the display 28. The touch panel 36 detects the contact position and displacement of the user's finger or stylus pen, and outputs a signal based on the detection result to a predetermined output destination. For example, suppose that a user touches the touch panel 36 with two fingers that are close to each other and then widens the space between the fingers (a so-called pinch-out operation). In this case, the touch panel 36 detects the positions of each of the two fingers at the start and end of the operation and outputs a signal according to the detection result.

[0041] As shown in FIG. 3, the housing of the photographing device main body 12 contains a shutter 38, an image sensor 40, an analog signal processing circuit 44, a control unit 46, an internal memory 50, a card slot 52, and a buffer 56.

[0042] The imaging element 40 is an image sensor and is configured by a solid-state imaging element such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor Image Sensor). The imaging element 40 has a plurality of pixels 42 arranged in a grid as shown in Fig. 4. Fig. 4 shows the correspondence between the pixels 42 of the imaging element 40 and the shooting area of ​​the first video.

[0043] Each pixel 42 has an on-chip microlens, a color filter, and a photodiode (photoelectric conversion element).

[0044] 4, each pixel 42 corresponds to one of the unit areas A0 that make up the image capturing area A1. The image capturing area A1 is the capturing area when the image capturing device 10 captures the first image, and its angle of view corresponds to the first angle of view. The unit area A0 is the smallest unit of the image capturing area A1, and corresponds to the pixels 42 that are arranged vertically and horizontally on the image sensor 40. In other words, the first image captured in the shooting area A1 is composed of unit images of the same number as the number of unit areas A0 (i.e., the number of pixels 42). In this specification, the number of unit images in the first image is conveniently referred to as the "number of pixels."

[0045] The image sensor 40 receives light from a subject that has passed through the photographic lens 14, converts the received light image into an electrical signal (image signal), and outputs the electrical signal. The image sensor 40 is also configured to be capable of performing an exposure operation using a so-called electronic shutter. In the following explanation, unless otherwise specified, "exposure" means exposure by the electronic shutter at the image sensor 40 while the shutter 38 is kept open. Furthermore, "exposure time" means the time corresponding to the shutter speed of the electronic shutter, or more precisely, the charge accumulation time.

[0046] 3, the analog signal processing circuit 44 reads out, for each pixel, an electrical signal (image signal) for one frame output from the image sensor 40. The analog signal processing circuit 44 amplifies the read-out image signal using an AGC (Auto Gain Controller) and performs signal processing such as correlated double sampling on the amplified signal. The processed signal is sent to a video processing unit 48 in the control unit 46.

[0047] The control unit 46 controls each unit of the image capturing device 10 and executes various processes related to the creation of video files. The control unit 46 includes a controller 47 and a video processing unit 48 as shown in FIG.

[0048] The control unit 46 is composed of, for example, one or more processors, and is composed of, for example, a CPU (Central Processing Unit) and a control program. However, the present invention is not limited to this, and the processor may be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), an MPU (Micro-Processing Unit), or other ICs (Integrated Circuits), or may be composed of a combination of these. The above processor may also be configured such that the functions of the entire control unit 46 including the controller 47 and the video processing unit 48 are implemented in a single IC (Integrated Circuit) chip, as typified by an SoC (System on Chip). The hardware configuration of each of the processors described above may be realized by an electric circuit that combines circuit elements such as semiconductor elements.

[0049] The controller 47 comprehensively controls the image capturing device 10 in accordance with a user's operation or a predetermined control pattern. For example, the controller 47 controls the image capturing element 40 and the analog signal processing circuit 44 so as to capture video (moving images) at a predetermined frame rate.

[0050] The controller 47 also determines the shooting conditions according to the shooting environment, and controls the various driving units, the image sensor 40, the analog signal processing circuit 44, and the video processing unit 48 so that the shooting conditions are the determined conditions. The shooting conditions include the exposure amount, white balance, and focus of the shooting lens 14 when shooting an image. Furthermore, the controller 47 controls the video processing unit 48 so that the captured video or the video based on the captured video is recorded on a recording medium.

[0051] The video processing unit 48 converts the signal sent from the analog signal processing circuit 44 into digital image data, and then performs various processes on the digital image data, such as gamma correction, white balance correction, and defect correction. The video processing unit 48 also compresses the processed digital image data in a compression format that complies with a predetermined standard.

[0052] Then, during video capture, video processing unit 48 generates compressed digital image data at a specific frame rate and acquires video (strictly speaking, frame images) from that data. The video (frame images) acquired at this time correspond to video captured at the first angle of view, i.e., the first video.

[0053] Furthermore, under the control of the controller 47, the video processing unit 48 performs various processes (for example, extraction processes described below) on the acquired video, and outputs the processed video to the display 28 frame by frame at a specific frame rate. Furthermore, the video processing unit 48 records the processed video on a recording medium and creates a video file. In this way, the video processing unit 48 has a function for creating a video file, and the imaging device 10 having the video processing unit 48 is used as a video creation device.

[0054] In the following, unless otherwise specified, the operations and processes of the controller 47 and the video processing unit 48 will be described as the operations and processes of the control unit 46. The processes by the control unit 46 will be described in detail later.

[0055] The internal memory 50 built into the photographing device main body 12 and the memory card 54 that can be attached to and detached from the photographing device main body 12 via the card slot 52 are recording media, and images are recorded on the recording media by the control unit 46. The internal memory 50 and the memory card 54 are configured with flash memory, ferroelectric memory, or the like. Note that the internal memory 50 and the memory card 54 may be located outside the photographing device main body 12, in which case the control unit 46 may record images on the external recording media via a wired or wireless connection.

[0056] The buffer 56 functions as a work memory for the control unit 46, and is configured by, for example, a DRAM (Dynamic Random Access Memory), a ferroelectric memory, or the like.

[0057] [Processing by the control unit] The processing by the control unit 46 includes display processing, extraction processing, selection processing, switching processing, recording processing, exposure amount adjustment processing, focus adjustment processing, and white balance adjustment processing. Each of the above processes will be explained below individually.

[0058] (Display processing) The display process is a process of displaying various images on the display screen of the display 28. For example, when the control unit 46 starts the display process after the image capturing device 10 is started, the first image captured by the image capturing device 10 at the first angle of view, that is, the image captured in the image capturing area A1 shown in FIG. Furthermore, when the extraction process is executed, the control unit 46 displays an image of the selected area (described later) on the display screen in the display process. Furthermore, when the selected area is switched (i.e., when the switching step (described later) is performed), the control unit 46 switches the image displayed on the display screen to the image of the selected area after the switching step.

[0059] The above-described execution operation of the display process by the control unit 46 corresponds to the display step. In the present embodiment, the image displayed on the display 28 by the display process is a live image (live view image) captured in real time at that time.

[0060] (Extraction process) The extraction process is a process of extracting a portion of the first video. The extracted video is also called a cropped video, and is the recording target in the recording process. During the extraction process, the user performs an operation to set the area to be extracted from the shooting area of ​​the first video. This setting operation will be specifically described with reference to Fig. 5. Fig. 5 shows the procedure for setting the area of ​​the video to be extracted from the first video.

[0061] As shown in Fig. 5, the user performs a setting operation while a first image (denoted by symbol P1 in Fig. 5) is displayed on the display screen of display 28. Specifically, when the user operates a predetermined button while the first image P1 is displayed, a rectangular area setting frame FR is displayed superimposed on the first image P1, as shown in Fig. 5. An area A2 surrounded by this area setting frame FR is the area of ​​the image extracted from the first image P1. The angle of view of the extracted image is smaller than the angle of view of the first image P1, i.e., the first angle of view, and will be referred to as the second angle of view hereinafter.

[0062] The region setting frame FR is displayed with its position, size, and aspect ratio variable. For example, the user can change the display position of the region setting frame FR by touching one side of the region setting frame FR with a finger on the touch panel 36 and dragging the finger. The user can also change the size and aspect ratio of the region setting frame FR by touching the region setting frame FR with two fingers on the touch panel 36 and widening or narrowing the space between the fingers. By changing the position, size, and aspect ratio of the region setting frame FR in this way, the position, size, and aspect ratio of the region of the video extracted from the first video P1 can be changed.

[0063] The above operations correspond to the setting operations by the user. Furthermore, by repeating the above series of operations multiple times, it is possible to set multiple areas A2 of the image to be extracted within the shooting area A1 of one first image P1, as shown in Figure 6. Figure 6 shows how multiple areas A2 of the image to be extracted from the first image P1 have been set.

[0064] When a plurality of areas A2 are set, the sizes (second angles of view) of the areas A2 may be the same among the areas A2, or may be different among the areas A2. Furthermore, the shape of each region A2 is not limited to a rectangle (oblong), but may be a square, a parallelogram, a trapezoid, a rhombus, a circle or an ellipse, a triangle or a polygon with pentagons or more sides, or an irregular shape. The number of areas A2 to be set is not particularly limited, but in this embodiment, it is assumed to be two or more as shown in FIG.

[0065] When the user repeats the setting operation multiple times, the control unit 46 accepts each setting operation and sets the number of areas A2 corresponding to the number of setting operations in the shooting area A1 of the first image P1. Such an operation of the control unit 46 corresponds to the setting step.

[0066] When the first image P1 is a high-quality image (for example, an image having 10 million or more pixels), the image at the second angle of view extracted from the first image P1 is an image with sufficiently high quality.

[0067] The number of pixels in the first image P1 is not particularly limited, but the lower limit is preferably 10 million or more, and more preferably 60 million or more. The upper limit is preferably 1 billion or less, and more preferably 500 million or less. If the number of pixels exceeds the lower limit, the visibility of the image of the second angle of view extracted from the first image P1 is ensured. If the number of pixels is below the upper limit, the amount of data in the first image P1 is reduced, and processing by the control unit 46 is accelerated.

[0068] (Selection and switching processes) The selection process is a process of selecting a selection area in which an image to be recorded appears from among a plurality of areas A2 set in the shooting area A1 of the first image P1. The switching process is a process in which, after the selection process, a selected area is reselected from among the multiple areas A2 and the selected area is switched. In this embodiment, the number of regions that are selected in the selection process and the switching process is one, but this is not limitative, and two or more regions may be selected.

[0069] To explain the procedures of the selection process and the switching process, the user sets multiple areas A2 by performing the setting operation described above, and then selects one of the areas A2 as the selected area. The user then inputs the selected area A2 using the operation buttons 30, 32, 34 or the touch panel 36, and the control unit 46 accepts this input operation. The control unit 46 determines the selected area from the multiple areas A2 based on the user's input operation. Such an operation by the control unit 46 corresponds to the selection step.

[0070] Furthermore, after the selection step, if the user reselects the selected area to another area A2 and performs an input operation again, the control unit 46 accepts the re-input operation and switches the previously selected area to another area A2 based on the re-input operation. Such an operation by the control unit 46 corresponds to the switching step. The input operation (re-input operation) performed by the user when reselecting the selected area is not particularly limited. For example, the re-input operation may be an operation of tapping the area A2 to be selected on the display 28, or an operation of pressing down either the up / down button or the left / right button of the first operation button 30, which is a cross key.

[0071] As described above, in this embodiment, a selected area is selected from among multiple areas A2 in the selection step, and an image of the selected area is recorded as a video file. Furthermore, when the selection step is performed, an image of the selected area is displayed on the display screen of display 28 in conjunction with the selection step, as shown in Figure 7. Figure 7 shows the image of the selected area displayed on the display screen.

[0072] Furthermore, by performing a switching process after the selection process, the selected area can be switched to change the image to be recorded from an image of one subject to an image of another subject. This allows individual images (more specifically, close-up images of each subject) of multiple subjects in the same scene (same location) to be simultaneously captured by a single image capture device 10 without using multiple devices. Furthermore, the area A2 showing the video to be recorded can be switched in chronological order in response to a user operation, and video files showing the video before and after the switch can be easily obtained. Furthermore, in conjunction with the implementation of the switching process, the image displayed on the display screen of display 28 switches from the image of the selected area before the switching process to the image of the selected area after the switching process, allowing the user to confirm the switching of the selected area on the display screen.

[0073] (Recording Processing) The recording process is a process in which the control unit 46 records the video of the selected area on a recording medium and creates a video file (more specifically, a video file) related to that video. In the recording process, the operation of the control unit 46 to record the video of the selected area corresponds to the recording step. In addition, in the recording process, the operation of the control unit 46 to create a video file corresponds to the creation step. In this embodiment, recording of a video and creating a video file are synonymous, and the creating process corresponds to the recording process.

[0074] Furthermore, when the switching process is performed, the control unit 46 creates a video file by recording the video of the selected area before the switching process and the video of the selected area after the switching process. More specifically, in the creation process, the control unit 46 combines the video (moving image) of the selected area before the switching process and the video (moving image) of the selected area after the switching process to create a moving image file as a video file. This results in a moving image file of video in which the subject changes when the selected area is switched, as shown in FIG. 8. FIG. 8 is an explanatory diagram of the moving image file of the recorded video.

[0075] (Exposure adjustment processing) The exposure amount adjustment process is a process for adjusting the exposure amount of the selected region, and is executed by utilizing the AE (Automatic Exposure) function of the control unit 46. The exposure amount is determined by a number of parameters, including the exposure time (i.e., the shutter speed of the electronic shutter), the sensitivity (ISO sensitivity) of the pixels 42 of the image sensor 40, and the aperture size (F-number) of light incident on the photographic lens 14. The exposure amount is adjusted by determining the values ​​of each parameter and controlling the aperture driver 23, the image sensor 40, the video processor 48, etc. so that each parameter reaches the determined value. Here, the adjustment of the aperture 20 by the aperture driver 23 is performed using a known automatic light control (ALC) function. Furthermore, the adjustment of the shutter speed of the electronic shutter in the image sensor 40 is performed using a known electronic light control (ELC) function.

[0076] Furthermore, if the photographing lens 14 is equipped with an electronic neutral density filter 21, the above-mentioned multiple parameters include the degree of dimming of the electronic neutral density filter 21. In this case, the value of the degree of dimming of the electronic neutral density filter 21 is determined, and the voltage application unit 24 is controlled so that the degree of dimming becomes the determined value, thereby adjusting the exposure amount.

[0077] Incidentally, the appropriate value of the exposure amount (appropriate exposure amount) of the selected area varies depending on which area is the selected area among the multiple areas A2 set in the shooting area A1 of the first image P1.

[0078] More specifically, when the first image P1 is captured, the amount of exposure at each location in the capture area A1, which includes multiple areas A2, varies depending on the subject and environment at each location. That is, the appropriate amount of exposure for each area A2 in the capture area A1 can vary depending on the position of that area A2, and is therefore determined for each area A2.

[0079] The appropriate exposure amount for each region A2 is determined by a known procedure. For example, the brightness of R (Red), G (Green), and B (Blue) (i.e., RGB pixel values) for a certain region A2 is integrated from image signals output from pixels 42 corresponding to a unit region A0 constituting the region A2 in the image sensor 40. Here, the pixel 42 corresponding to the unit region A0 is the pixel 42 located at the same position as the unit region A0 in the positional relationship shown in FIG. 4. For example, the unit region A0 in the region A2 enclosed by a bold frame in FIG. 4 corresponds to the pixel 42 located within the range enclosed by the bold frame. The appropriate exposure amount for capturing an image of the region A2 is then determined from the integrated value of the RGB pixel values ​​calculated for the region A2 (more specifically, the average value within the region A2).

[0080] The exposure amount of the selected area is adjusted based on the appropriate exposure amount determined in the above-described manner. Specifically, the control unit 46 determines the values ​​of the exposure time (shutter speed) and aperture value (F-number) based on the appropriate exposure amount of the predetermined area. This operation corresponds to the condition determination process by the control unit 46. The values ​​of each parameter determined in the condition determination process are values ​​that indicate the exposure conditions when capturing the first video including the video of the selected area. In this embodiment, both the exposure time and the aperture value are determined, but this is not limiting, and only one of the exposure time and the aperture value may be determined.

[0081] Furthermore, after determining the values ​​of the exposure time and aperture value, the control unit 46 determines the sensitivity (ISO sensitivity) value of the pixels 42 corresponding to the selected area according to the combination of these values. Strictly speaking, the gain (amplification ratio) for the output signal from the pixels 42 corresponding to the selected area is determined. This operation corresponds to the sensitivity determination step by the control unit 46. The gain may be, for example, an analog gain for an analog signal in the analog signal processing circuit 44 or the like, or a digital gain for a signal (data) after digital conversion in the video processing unit 48 or the like.

[0082] After the exposure time, aperture value, and sensitivity are determined, the control unit 46 controls the aperture driving unit 23, the image pickup device 40, the image processing unit 48, etc. so that these parameters become the determined values.

[0083] While the video is being recorded, the exposure adjustment process is repeated periodically (for example, for each frame) using the above procedure. Each time the exposure adjustment process is performed, the exposure time, aperture value, and sensitivity are determined. Specifically, during video capture, the integrated value of RGB pixel values ​​within a predetermined area in the Nth frame image (N is a natural number) is calculated, and then the appropriate exposure amount is calculated based on the calculated integrated value, and the exposure time and aperture value for capturing the next frame image (i.e., the N+1th frame image) are determined based on the calculated appropriate exposure amount. The integrated value of the RGB pixel values ​​in the selected region in the Nth frame image is then calculated, and the appropriate exposure amount for the selected region is calculated from the calculated integrated value.Then, the sensitivity of the pixel 42 corresponding to the selected region in the Nth frame image is determined based on the calculated appropriate exposure amount, the exposure time and aperture value at that time.

[0084] In this embodiment, the exposure time and aperture change over time, and accordingly, the exposure amount when capturing the first video is adjusted over time. Furthermore, the sensitivity of the pixels 42 corresponding to the selected region changes over time, and as a result, the exposure amount of the selected region is adjusted over time. In this embodiment, "adjusting over time" means determining an adjustment amount for each parameter to be adjusted for each frame, and increasing or decreasing each parameter by that adjustment amount for each frame.

[0085] (Focus adjustment process) The focus adjustment process is a process in which, in autofocus mode, the control unit 46 controls the focus driver 22 to automatically adjust the focus of the photographic lens 14. Known autofocus techniques can be used for focus adjustment in autofocus mode.

[0086] (White balance adjustment processing) The white balance adjustment process is a process in which the control unit automatically adjusts the white balance of the image in the selected area. The white balance adjustment process is performed using a known AWB (Auto White Balance) function, such as the technology described in Japanese Patent Application Laid-Open No. 2009-33410.

[0087] [Video creation flow according to the first embodiment] The flow of video creation according to the first embodiment will be described with reference to Fig. 9. Fig. 9 is an explanatory diagram of the video creation flow according to the first embodiment. In the following description, the period for capturing one first video (frame image) will be referred to as "one frame" for convenience.

[0088] In the video creation flow, the user first starts up the photographing device 10. After starting up the device, the control unit 46 performs initial settings for various conditions, including photographing conditions and exposure conditions (S001). In this step S001, each of a plurality of parameters that determine the amount of exposure, such as exposure time (shutter speed), aperture value (F-number), and sensitivity (ISO sensitivity), is preset to an initial value.

[0089] Thereafter, image capturing at the first angle of view (i.e., capturing of the first video) is started by the image capturing device 10. In conjunction with this, a display step is carried out by the control unit 46, and the first video, which is a live video, is displayed on the display screen of the display 28 (S002).

[0090] The user performs the setting operation described above multiple times, and the control unit 46 accepts each setting operation and performs the setting step (S003). In this step S003, the control unit 46 sets multiple areas A2 in the shooting area A1 of the first image in accordance with the setting operation performed by the user through the first image displayed on the display screen.

[0091] Furthermore, when the user selects a selection area from among the multiple areas A2 and inputs the selection result via the touch panel 36 or the like, the control unit 46 performs a selection step (S004). In this step S004, the control unit 46 selects a selection area from among the multiple areas A2 in response to the user's input operation.

[0092] Thereafter, when the user performs an operation such as pressing the release button 26 to start recording the video, the processes from step S005 onwards are carried out. First, in step S005, the control unit 46 acquires one frame of the first image (i.e., a frame image). In the next step S006, the control unit 46 calculates, for each of the multiple regions A2 set in step S003 in the acquired first image, an integrated value of the RGB pixel values ​​of the image of each region A2. Then, in step S007, the control unit 46 determines the appropriate exposure amount for each region A2 based on the integrated value of the RGB pixel values ​​calculated for each region A2.

[0093] After performing step S007, the control unit 46 performs a sensitivity determination step to determine the sensitivity (more precisely, the gain) of the pixels 42 corresponding to the selected region in the image sensor 40 (S008). In the first (first) step S008 during the video creation flow, the sensitivity of the pixels 42 corresponding to the selected region is determined based on the appropriate exposure amount for the selected region and the exposure time and aperture value preset to initial values. In step S008, the control unit 46 adjusts the exposure amount of the selected region by changing the gain so that the sensitivity of the pixels 42 corresponding to the selected region becomes the determined sensitivity.

[0094] Thereafter, the control unit 46 performs a recording step (creation step) to record the video of the selected area on a recording medium and starts creating a video file that is the video file (S009). In this step S009, the control unit 46 records the video of the selected area with the sensitivity (gain) determined in step S008. The control unit 46 also displays the video of the selected area on the display screen of the display 28.

[0095] Thereafter, the control unit 46 performs a designation step, and designates a predetermined area from the plurality of areas A2 set in step S003 as a reference area (S010). In this step S010, of the first and second areas included in the plurality of areas A2, the second area having a lower appropriate exposure amount than the first area is designated as the reference area. Here, the first area and the second area are relative concepts, and by comparing the plurality of areas A2, the area having a higher appropriate exposure amount corresponds to the first area, and the area having a lower appropriate exposure amount corresponds to the second area.

[0096] In this embodiment, the area A2 with the smallest appropriate exposure amount among the multiple areas A2 is designated as the reference area. However, this is not limited to this, and an area A2 with an appropriate exposure amount smaller than any of the other areas A2 may be designated as the reference area.

[0097] After designating the reference area, the control unit 46 executes a condition determination step to determine the exposure time and aperture value when capturing the first video in the next frame (hereinafter referred to as the next exposure time and aperture value) (S011). In this step S011, the control unit 46 determines the next exposure time and aperture value based on the appropriate exposure value of the reference area.

[0098] Furthermore, in step S011, the control unit 46 adjusts the exposure time and aperture value so that these values ​​become the values ​​determined in step S011. As a result, in the next frame, the first video (frame image) is captured with the exposure time and aperture value based on the appropriate exposure amount of the reference area specified immediately before.

[0099] Of the steps described above, the series of operations from step S005 onwards for acquiring the first image are repeatedly performed while the image is being recorded. Therefore, once the exposure time and aperture value are determined in step S011, the first image is acquired with the above exposure time and aperture value in the frame immediately following (i.e., the second or subsequent step S005).

[0100] Furthermore, if the user selects a selection area once and then reselects it and performs an input operation again (S012), the control unit 46 performs a switching step (S013). In this step S013, the control unit 46 reselects and switches the selection area from among the multiple areas A2 in response to the user's input operation.

[0101] When step S013 is performed and the selected area is switched, the control unit 46 records the image of the selected area before the switching process and the image of the selected area after the switching process in step S009 in the frame immediately after that to create a video file. More specifically, the control unit 46 combines the image of the selected area before the switching process and the image of the selected area after the switching process to create a moving image file as a video file.

[0102] Then, the control unit 46 repeats the above steps S005 to S013 until the user performs a predetermined operation to give an instruction to end the process (S014).

[0103] As described above, this embodiment is characterized in that the area with the lowest appropriate exposure amount among the multiple areas A2 is set as the reference area, and the next exposure time and aperture value are determined based on the appropriate exposure amount of the reference area. This prevents excessive exposure (overexposure) when switching the selected image, and ensures that the image after switching has good image quality.

[0104] To explain the above effect in more detail, when the selected area is switched, the exposure time and aperture value are generally determined so that the exposure amount of the selected area after the switch is an appropriate exposure amount. On the other hand, among the multiple regions A2, the brightness of the regions A2 other than the selected region (hereinafter referred to as unselected regions) may be greater than the brightness of the selected region. In such a case, if the exposure amount of the selected region is determined based on the appropriate exposure amount, the RGB pixel values ​​of the image of the unselected region may become saturated. As a result, the appropriate exposure amount of each region A2 may not be calculated correctly. Furthermore, when an unselected region (strictly speaking, an unselected region with an appropriate exposure amount smaller than that of the selected region) is switched to a selected region, overexposure may occur.

[0105] In contrast, in this embodiment, the area A2 with the smaller appropriate exposure amount (strictly speaking, the area A2 with the smallest appropriate exposure amount) is used as the reference area, and the exposure time and aperture value are determined based on the appropriate exposure amount of the reference area. This makes it possible to prevent problems such as excessive exposure described above, and to more appropriately determine conditions such as exposure time during video shooting.

[0106] In this embodiment, as described above, steps S005 to S013 are performed for each frame while the video is being recorded. When a first area A2 having a larger appropriate exposure amount is selected as the selected area, the exposure amount for the selected area is determined for the second and subsequent frames based on the conditions determined for the immediately preceding frame.

[0107] Specifically, in the sensitivity determination step performed in step S008 for the second or subsequent frame, the sensitivity (gain) of the pixels 42 corresponding to the selected region (i.e., the first region) is determined. At this time, the sensitivity is determined based on the appropriate exposure amount of the first region, which is the selected region at that time, and the exposure time and aperture value determined in step S011 (i.e., the condition determination step) for the immediately preceding frame. Furthermore, in this embodiment, the above sensitivity is determined to be higher than the sensitivity of the pixels 42 corresponding to the second region, which is the reference region. As a result, the sensitivity of the pixels 42 corresponding to the selected region is appropriately determined, and the exposure amount of the first region, which is the selected region, is adjusted to be the appropriate exposure amount.

[0108] Note that methods for increasing and optimizing the exposure of the selected region are not limited to the method of increasing the sensitivity of the pixels 42 corresponding to the selected region as described above. For example, the exposure of the selected region may be increased by performing frame synthesis on the image of the selected region. Frame synthesis is a signal processing method that adds together image signals from multiple frames that are consecutive on the time axis. However, if a moving subject is captured in the video to be frame-combined, the frame-combining process may result in an afterimage of the subject. Therefore, it is best to only apply frame-combining exposure adjustment to video in which the subject is not moving.

[0109] <<Second embodiment>> A second embodiment in which the reference area is re-designated only when the selected area is switched will be described with reference to Fig. 10. Fig. 10 is an explanatory diagram of a video creation flow according to the second embodiment. In the following, the second embodiment will be mainly described with respect to the differences from the first embodiment.

[0110] As can be seen from a comparison between FIGS. 9 and 10, the steps in the video production flow according to the second embodiment are generally common to the steps in the video production flow according to the first embodiment. Specifically, steps S021 to S029 in the video production flow according to the second embodiment are common to steps S001 to S009 in the video production flow according to the first embodiment. The video production flow according to the second embodiment is also common to the video production flow according to the first embodiment in that step S025 and subsequent steps are repeated for each frame while the video is being recorded.

[0111] On the other hand, in the second embodiment, once the designation step is performed and the reference region is designated, the designation step is performed only when the switching step is performed. Also, in the second embodiment, the condition determination step is performed each time the designation step is performed. In the above points, the second embodiment differs from the first embodiment.

[0112] To explain the differences between the video creation flow of the second embodiment and the first embodiment, as shown in Figure 10, in the second embodiment, it is determined whether a reference area has already been specified in each frame (S030).

[0113] If the reference area is not designated, the control unit 46 performs a designation step to designate a second area (specifically, an area with the smallest appropriate exposure amount) among the multiple areas A2 as the reference area (S031). Thereafter, the control unit 46 performs a condition determination step to determine the next exposure time and aperture value based on the appropriate exposure amount of the reference area designated in step S031 (S032). Normally, steps S031 and S032 are performed only immediately after the start of recording of the captured video (that is, the first frame).

[0114] If a reference area has already been specified, it is determined whether the user has selected a selected area once and then reselected it and performed an input operation again (S033). If a selected area has not been reselected, the reference area at that time (i.e., the current reference area) is maintained. Then, when determining the next exposure time and aperture value in the condition determination step, the control unit 46 determines these values ​​based on the appropriate exposure amount for the current reference area (S034).

[0115] On the other hand, if the selected area is reselected, the control unit 46 performs a switching step, reselecting and switching the selected area from among the multiple areas A2 in response to a user's input operation (S035). The control unit 46 also performs a designation step in conjunction with the switching step, redesignating the reference area in the designation step (S036). Furthermore, the control unit 46 determines the next exposure time and aperture value based on the appropriate exposure amount for the redesignated reference area in the condition determination step immediately following (S037).

[0116] In the video production flow according to the second embodiment, the above steps S025 to S037 are repeatedly performed until the user issues an instruction to end the process (S038).

[0117] As described above, in the video creation flow according to the second embodiment, re-designation of the reference area is permitted only when the selected area is switched. This makes the image quality of the selected video more stable compared to the first embodiment, in which the reference area changes every frame. The second embodiment is common to the first embodiment except for the above-mentioned differences, and can provide the same effects as those obtained in the first embodiment.

[0118] <<Third Embodiment>> A third embodiment in which conditions such as exposure time are determined based on the appropriate exposure amount of the reference area immediately after selection or reselection of a selected area will be described with reference to Fig. 11. Fig. 11 is an explanatory diagram of a video creation flow according to the third embodiment. Note that the following description will mainly focus on the differences between the first embodiment and the third embodiment.

[0119] As can be seen from FIGS. 10 and 11, the steps in the video production flow according to the third embodiment are substantially the same as the steps in the video production flow according to the second embodiment.

[0120] In the video production flow according to the third embodiment, immediately after a selected area is selected for the first time or immediately after a selected area is reselected, the control unit 46 performs a designation step (S051, S056). In the designation step, the control unit 46 designates an area A2 that corresponds to the second area at that time (strictly speaking, an area A2 with the smallest appropriate exposure amount) as a reference area. Then, in a condition determination step (i.e., S052, S057) performed immediately after the designation step, conditions such as exposure time are determined based on the appropriate exposure amount of the reference area designated in the designation step. Hereinafter, this condition determination step will be referred to as a "first determination step."

[0121] On the other hand, in frames where a selected area has already been selected and there is no reselection (switching) of the selected area, the designation step is not performed. In the condition determination step (S054) for such frames, conditions such as exposure time are determined based on the appropriate exposure amount for the reference area at the time the selected area was selected or reselected and the amount of change in the appropriate exposure amount for the current selected area. Hereinafter, this condition determination step will be referred to as the "second determination step."

[0122] In the first determination step, conditions such as exposure time are determined based on the appropriate exposure amount of the reference area at that time. Furthermore, in the adjustment step after the first determination step is performed, a second determination step is performed. In the second determination step, the adjustment amount of the exposure amount in the adjustment step (more specifically, the exposure amount during first video shooting) is determined taking into account changes over time in the appropriate exposure amount of the selected area at that time. In the adjustment step, the exposure amount during first video shooting is adjusted based on the conditions such as exposure time determined in the first determination step and the adjustment amount determined in the second determination step.

[0123] The above will be described in detail with reference to FIGS. Fig. 12 shows the change over time in the appropriate exposure amount for each of the reference area and the selected area. Fig. 13 shows the exposure amount after adjustment for the selected area when adjustment is continued based on the appropriate exposure amount for the reference area. Fig. 14 shows the target exposure amount in the third embodiment. Fig. 15 shows the change over time in the exposure amount after adjustment for each of the reference area and the selected area in the third embodiment. 12 to 15, the horizontal axis indicates the time elapsed from the start of video recording (more specifically, the number of frames elapsed), and the vertical axis indicates the amount of exposure.

[0124] Here, the "adjusted exposure amount" refers to the actual value that is finally reached for each frame of the exposure amount that is adjusted for each frame in the adjustment process. In addition, the "target exposure amount" is the target value that is expected to be achieved for each frame of the exposure amount adjusted for each frame in the adjustment process, and is set to a value according to the appropriate exposure amount (for example, a value equivalent to the appropriate exposure amount).

[0125] As a specific example, consider a case in which the appropriate exposure amounts for the reference area and the selected area change as shown in Figure 12. In this case, the appropriate exposure amount for the selected area (denoted as EIt in the figure) is relatively stable, while the appropriate exposure amount for the reference area (denoted as EIs in the figure) fluctuates greatly. In the case shown in Figure 12, it is assumed that the selected area is reselected (switched) at the point when the elapsed time is tc.

[0126] In the above case, when the exposure amount during the first video capture is adjusted for each frame, it is adjusted based on the appropriate exposure amount for the reference area for each frame. In this case, the exposure amounts after adjustment for the reference area and the selected area change as shown in FIG. 13. The exposure amount of the reference area after adjustment (denoted as ERs in the figure) fluctuates greatly, just like the appropriate exposure amount. The exposure amount of the selected area after adjustment (denoted as ERt in the figure) shows the same trend of change as the appropriate exposure amount, as the sensitivity (gain) of the pixels 42 corresponding to the selected area is changed so that it becomes the appropriate exposure amount.

[0127] However, the sensitivity (gain) that determines the exposure amount of the selected area after adjustment is affected by the exposure amount of the reference area after adjustment, in other words, the appropriate exposure amount of the reference area. Therefore, if the appropriate exposure amount of the reference area is changed significantly, the sensitivity (gain) also changes significantly, which may cause problems such as an increase or decrease in noise in the image of the selected area.

[0128] In contrast, in the third embodiment, a reference area is designated and the first determination step is performed only immediately after the selection or reselection of the selected area. In the first determination step, conditions such as exposure time are determined based on the appropriate exposure amount (denoted as e1 in FIG. 12) of the reference area at the time of selection or reselection of the selected area.

[0129] Furthermore, during the period following the first determination step (i.e., the period during which the selected area has already been selected and the selected area has not yet been reselected), the second determination step is performed for each frame. In the second determination step for each frame, the exposure adjustment amount is determined based on the appropriate exposure e1 of the reference area at the time the selected area was selected or reselected, and the amount of change in the appropriate exposure of the current selected area. Here, the "amount of change in the appropriate exposure of the selected area" refers to the difference between the appropriate exposure of the selected area in the immediately preceding frame and the appropriate exposure of the selected area in the current frame, and is represented by the symbol Δ in Figure 12.

[0130] As described above, the first determination step is performed, and thereafter, the second determination step is performed for each frame, thereby determining the target exposure for each frame. More specifically, as shown in Figure 14, the target exposure (denoted as EG in the figure) at time tc when the selected region is reselected is set based on the appropriate exposure e1 of the reference region at that time. The target exposure EG thereafter is determined in accordance with changes in the appropriate exposure for the selected region, and specifically, changes by the difference Δ in the appropriate exposure for each frame.

[0131] Then, for each frame in the adjustment process, conditions such as exposure time are determined so that the exposure during first video shooting matches the target exposure EG determined for each frame. As a result, the exposure ERs of the reference area after adjustment matches the target exposure EG, as shown in Figure 15. Here, the target exposure EG reflects changes in the appropriate exposure of the selected area, so the exposure ERs of the reference area after adjustment remains relatively stable, as shown in Figure 15.

[0132] Furthermore, for each frame during the adjustment process, the exposure amount of the selected region is adjusted based on conditions such as the exposure time determined for each frame. More specifically, for each frame, the sensitivity (gain) of the pixels 42 corresponding to the selected region is determined based on conditions such as the exposure time determined for each frame and the appropriate exposure amount of the selected region for each frame. As a result, the exposure amount ERt of the selected region after adjustment becomes the exposure amount corresponding to the appropriate exposure amount for each frame, as shown in FIG. 15.

[0133] Furthermore, as can be seen from Figure 15, the sensitivity (gain) that determines the exposure of the selected area after adjustment is relatively stable. This is because the target exposure EG, which is the base for determining the sensitivity, is stable. As a result, problems such as increases and decreases in noise in the image of the selected area are suppressed.

[0134] As explained above, in the third embodiment, conditions such as exposure time are determined based on the appropriate exposure amount for the reference area only immediately after the selection or reselection of the selected area. For other periods, the exposure adjustment amount is determined based on changes in the appropriate exposure amount for the selected area. This makes it possible to stabilize the image quality of the selected area while the selected area is not being reselected (switched).

[0135] <<Fourth Embodiment>> A fourth embodiment in which the exposure amount is adjusted at a faster adjustment speed than normal immediately after the switching step is performed will be described with reference to Fig. 16. Fig. 16 is an explanatory diagram of an image creation flow according to the fourth embodiment.

[0136] As can be seen from FIG. 16, steps S061 to S067 in the video production flow according to the fourth embodiment are common to steps S001 to S007 in the video production flow according to the first embodiment.

[0137] Steps S068 to S074 of the video production flow according to the fourth embodiment will be described in detail below. In the video production flow according to the fourth embodiment, when a switching step is performed, the subsequent steps change depending on the time elapsed since the switching step was performed (S070).

[0138] More specifically, suppose that the control unit 46 performs a switching step at a certain point in time (a certain frame) to switch the selected area. In this case, in step S070, the control unit 46 measures the elapsed time from that frame (strictly speaking, the number of elapsed frames). Then, during the first period from the previous switching step (S071), the control unit 46 performs a first recording step. The first recording step is a step of recording the video of the selected area during the first period. The first period can be set to any period, but is preferably set to within one second.

[0139] Furthermore, when the current time (current frame) is in the first period, the control unit 46 performs an adjustment step of adjusting the exposure amount of the selected region in preparation for the next frame (S072). In this step S072, the exposure amount of the selected region is adjusted at a faster adjustment speed than normal based on the appropriate exposure amount at that time. Specifically, in the adjustment step, when the aperture drive unit 23 is driven to change the aperture amount for exposure amount adjustment, the drive speed of the aperture drive unit 23 is set faster than the normal drive speed. The adjustment speed in the adjustment process in the first period can be determined arbitrarily as long as it is faster than the normal adjustment speed, but it is preferably at least 1.5 times faster than the normal adjustment speed, and more preferably at least 2 times faster.

[0140] On the other hand, after the first period has elapsed since the previous switching step was performed, that is, if the current time point (current frame) is in a second period other than the first period, the control unit 46 performs a second recording step (S073). The second recording step is a step of recording the video of the selected area in the second period.

[0141] Furthermore, when the current time point (current frame) is in the second period, the control unit 46 performs an adjustment step of adjusting the exposure amount of the selected region in preparation for the next frame (S074). In this step S074, the exposure amount of the selected region is adjusted at a normal adjustment speed based on the appropriate exposure amount at that time. Specifically, the aperture drive unit 23 is driven at a normal drive speed to change the aperture amount in order to adjust the exposure amount.

[0142] As described above, in the fourth embodiment, the exposure adjustment speed (specifically, the drive speed of the aperture drive unit 23) is set faster in the adjustment process performed in the first period than in the adjustment process performed in the second period. This allows the exposure of the selected area after the switching process to be quickly adjusted (optimized). As a result, an image of the selected area after the switching process is acquired with an appropriate exposure, resulting in good image quality.

[0143] In the above-mentioned case given as a specific example of the fourth embodiment, the exposure amount of the selected area is adjusted in the adjustment process. However, there are other conditions that should be adjusted promptly after the switching process is performed, such as the white balance of the image of the selected area and the focus of the photographing lens 14, in addition to the exposure amount.

[0144] In the fourth embodiment, an adjustment process for adjusting at least one of the exposure amount of the selected area, the white balance of the image of the selected area, and the focus can be performed in each of the first period and the second period. In this case, the adjustment speed of the condition (the condition to be adjusted) in the adjustment process performed in the first period may be faster than that in the adjustment process performed in the second period.

[0145] Specifically, when the aperture drive unit 23 is driven to change the aperture amount to adjust the exposure amount during the adjustment process, the drive speed of the aperture drive unit 23 during the adjustment process in the first period should be faster than that during the adjustment process in the second period. Furthermore, when the focusing optical component 19 is moved for focus adjustment in the adjustment process, the moving speed of the focusing optical component 19 in the adjustment process in the first period may be set faster than in the adjustment process in the second period.

[0146] <<Fifth Embodiment>> A fifth embodiment in which the exposure amount is adjusted by preferentially changing parameters other than the aperture value immediately after the switching step is performed will be described with reference to Figures 17 and 18. Figure 17 is an explanatory diagram of a video creation flow according to the fifth embodiment. Figure 18 shows an example of a change pattern determined by the adjustment data. It should be noted that the numerical values ​​of the change patterns shown in FIG. 18 are values ​​determined for the convenience of explanation, and are different from the values ​​defined in the actual change patterns.

[0147] The video creation flow according to the fifth embodiment is generally the same as that according to the fourth embodiment. That is, in the video creation flow according to the fifth embodiment, first, various conditions are initially set, a first video is displayed when video shooting starts, multiple areas A2 are set, and a selection area is selected from among them (S081 to S084). Thereafter, the steps from step S085 onwards (steps S085 to S094) are carried out in response to a user pressing the release button 26 or the like.

[0148] Specifically, one frame of the first image is acquired, the integrated value of the RGB pixel values ​​for each of the multiple areas A2 is calculated, and the appropriate exposure amount is determined from the integrated value (S085 to S087). After that, the image of the selected area is recorded (S091, S093), and the exposure amount of the selected area is adjusted in preparation for the next frame (S092, S094).

[0149] Furthermore, if the selected area is reselected, a switching step is performed to switch the selected area (S088, S089). The step after the switching step is performed varies depending on the time elapsed since the switching step was performed (S090). Specifically, if the current time (current frame) is in the first period, the control unit 46 performs a first recording step (S091). Also, if the current time (current frame) is in the first period, the control unit 46 performs a first adjustment step to adjust the exposure amount of the selected region in preparation for the next frame (S092).

[0150] On the other hand, if the current time (current frame) is in the second period, the control unit 46 performs a second recording step (S093). Also, when the current time (current frame) is in the second period, the control unit 46 performs a second adjustment step to adjust the exposure amount of the selected region in preparation for the next frame (S094). The above series of steps (that is, steps S085 to S094) are repeatedly performed until the user issues an instruction to end the process (S095).

[0151] The first and second adjustment steps in the fifth embodiment will be described. In each adjustment step, the exposure amount of the selected region is adjusted to a target exposure amount using adjustment data based on the appropriate exposure amount at that time. More specifically, in each adjustment step, the values ​​of multiple parameters, including exposure time (shutter speed), sensitivity of pixel 42 (ISO sensitivity), and aperture value (F-number), are determined based on the adjustment data. Then, the values ​​of each parameter are changed to the determined values, and the exposure amount of the selected region is adjusted to the target exposure amount.

[0152] The adjustment data is data that defines change patterns, and is prepared in the same number as the change patterns, and is recorded in the internal memory 50 or buffer 56 of the image capturing device main body 12. The change patterns define patterns (change trends) for changing the values ​​of multiple parameters in order to adjust the exposure amount to the target exposure amount.

[0153] In the fifth embodiment, the adjustment data used to adjust to the same target exposure amount is different between the first and second adjustment steps, i.e., the values ​​of the multiple parameters determined for the same target exposure amount are different between the first adjustment data used in the first adjustment step and the second adjustment data used in the second adjustment step.

[0154] The difference between the first adjustment data and the second adjustment data will be described with reference to Fig. 18. Both of the two adjustment data shown in Fig. 18 define change patterns for the aperture value, exposure time, and sensitivity for adjusting the exposure amount from a value ep to a target value et (target exposure amount). In other words, as shown in Fig. 18, among the change patterns defined by the adjustment data, the value of each parameter at the start of adjustment and the value of each parameter at the end of adjustment are common between the two adjustment data.

[0155] On the other hand, in the change pattern determined by the first adjustment data (shown at the top in Fig. 18), the aperture value is gradually increased in stages. The exposure time, which is a parameter other than the aperture value, increases rapidly immediately after the start of adjustment and then decreases as the aperture value increases. In contrast, in the change pattern determined by the second adjustment data (shown at the bottom in Figure 18), the aperture value increases to the final value immediately after the start of adjustment and is then maintained at the final value. The exposure time remains constant throughout the adjustment period.

[0156] As described above, in the change pattern defined in the first adjustment data, the degree of change (amount of change per time) of the aperture amount relative to the target exposure amount is smaller than in the change pattern defined in the second adjustment data. In other words, in the second adjustment process using the second adjustment data, the aperture value is changed preferentially when adjusting the exposure amount, whereas in the first adjustment process using the first adjustment data, parameters other than the aperture value are changed preferentially when adjusting the exposure amount.

[0157] As described above, in the fifth embodiment, when adjusting the exposure amount immediately after switching the selected area, priority is given to changing parameters other than the aperture amount (specifically, exposure time) rather than changing the aperture amount, which requires mechanical operation. The exposure time, i.e., the shutter speed of the electronic shutter, can be changed by digital processing, so it can be changed more quickly than the aperture amount. This allows the exposure amount of the selected area after switching to be quickly adjusted. Furthermore, since minute fluctuations in brightness that accompany sudden changes in aperture amount are suppressed, the effects of these minute fluctuations on the image quality can be suppressed. As a result, the image of the selected area after the switching process can have good image quality.

[0158] In the above case, the exposure time is changed in the first adjustment process with priority over the aperture amount, but this is not limited to this, and the sensitivity (gain) of the pixel 42 corresponding to the selected area may be changed in the first adjustment process with priority over the aperture amount. Furthermore, if the configuration includes the electronic neutral density filter 21, the multiple parameters for adjusting the exposure amount may include the degree of dimming of the electronic neutral density filter 21. In such a case, the degree of dimming of the electronic neutral density filter 21 (more directly, the voltage applied to the electronic neutral density filter 21) can be changed with priority over the aperture size. This makes it possible to suppress changes in aperture size and increases in shutter speed.

[0159] <<Sixth Embodiment>> In the five embodiments described above, the video is a moving image, that is, a collection of multiple frame images captured continuously at a constant frame rate. However, the video to be recorded is not limited to a moving image, and may be a still image.

[0160] For example, the control unit 46 displays the first video as a through image on the display 28, and sets a plurality of partial areas A2 in the shooting area A1 of the first video. Then, when the user selects one of the plurality of set areas A2 as a selected area, the control unit 46 extracts the video of the selected area and displays it on the display 28 as shown in Fig. 7. When the user inputs a recording instruction while the video of the extracted selected area is being displayed, the control unit 46 records a still image file of the video of the selected area on a recording medium.

[0161] When the video to be recorded is a still image, the shutter speed that defines the exposure time may be the shutter speed of an electronic shutter, or may be the shutter speed of the mechanical component shutter 38 (focal plane shutter).

[0162] <<Other embodiments>> The embodiment described above is a specific example given to clearly explain the video creation method of one embodiment of the present invention, and is merely an example, and other embodiments may be considered. For example, in the above-described embodiment, the image capturing device 10 is provided with the electronic neutral density filter 21, but the image capturing device may not have the electronic neutral density filter 21. Furthermore, in the above-described embodiment, the photographing device 10 has an autofocus function, but this is not limiting, and the photographing device may not have an autofocus function. That is, a photographing device may be used that does not have a focus driver 22 and adjusts the focus only by operating the focus ring 16 (i.e., only manually).

[0163] In the above embodiment, the image capture device 10 acts as a video creation device to create a video file, but this is not limiting. For example, other devices connected to the image capture device via a wired or wireless connection, such as a camera controller or an external recorder, may be used as the video creation device. These devices may then create a video file of the video captured by the image capture device.

[0164] Furthermore, in the above-described embodiment, the photographing device is a digital camera, but it may also be a video camera, a mobile phone with an imaging optical system, a smartphone, a tablet terminal, or other portable terminal. The imaging lens may be a lens unit that is externally attached to the imaging optical system of the mobile terminal described above. [Explanation of symbols]

[0165] 10 Imaging equipment 12 Imaging device body 13 Mount 14. Photographic Lens 16 Focus ring 18 Optical Component Unit 19 Focusing optics 20 apertures 21 Electronic neutral density filter 22 Focus drive unit 23 Aperture drive unit 24 Voltage application section 26 Release button 28 Display 30 First operation button 32 Second operation button 34 Third operation button 36 Touch Panel 38 Shutter 40 Image sensor 42 pixels 44 Analog signal processing circuit 46 Control Unit 47 Controller 48 Video Processing Unit 50 internal memory 52 card slots 54 Memory Card 56 buffers A0 unit area A1 Shooting Area A2 area L1 optical axis FR area setting frame

Claims

1. A video creation method for creating a video file based on a video captured by a photographing device equipped with a photographing lens and an image sensor, comprising: a setting step in which a processor sets a plurality of areas having a second angle of view smaller than the first angle of view within a shooting area of ​​a first image having a first angle of view; a selection step in which the processor selects a selected area as a shooting area of ​​an image to be recorded from among the plurality of areas; a switching step in which the processor, after the selecting step, reselects the selected area from among the plurality of areas to switch the selected area; a first recording step, which is performed during a period from when the switching step is performed until a first period has elapsed, in which the processor records an image of the selected area during the first period; a second recording step that is performed during a second period other than the first period, and in which the processor records an image of the selected area during the second period; a first adjustment step, which is carried out during the first period, in which the processor adjusts the exposure amount of the selected region to a target exposure amount using first adjustment data; a second adjustment step, which is carried out during the second period, in which the processor adjusts the exposure amount of the selected region to a target exposure amount using second adjustment data; In the first adjustment step and the second adjustment step, the values ​​of a plurality of parameters including an exposure time, sensitivity of pixels included in the image sensor, and an aperture amount of incident light to the photographing lens are determined based on the first adjustment data or the second adjustment data, and the exposure amount is adjusted to the target exposure amount; The image creating method, wherein the first adjustment data and the second adjustment data have mutually different values ​​for each of the plurality of parameters determined for the target exposure.

2. The video creation method according to claim 1 , further comprising a creation step in which the processor creates the video file by recording the video of the selected area before the switching step and the video of the selected area after the switching step.

3. 3. The video creating method according to claim 2, wherein the creating step combines the video of the selected area before the switching step and the video of the selected area after the switching step to create a moving image file as the video file.

4. a designation step in which the processor designates, as a reference area, a second area having a lower appropriate exposure amount than the first area, from among a first area and a second area included in the plurality of areas; The image creation method of claim 1, further comprising a condition determination step in which the processor determines at least one of the shooting conditions, namely, the exposure time when shooting the first image and the aperture value for the light incident on the shooting lens, based on the appropriate exposure amount of the reference area.

5. When the switching step is performed, the designation step is performed, The video production method according to claim 4 , wherein the condition determining step is performed each time the specifying step is performed.

6. 6. The image creation method of claim 4, further comprising a sensitivity determination step in which, when the first region is selected as the selected region, the processor determines the sensitivity of the pixel corresponding to the first region among the plurality of pixels of the imaging element based on the appropriate exposure amount of the first region and the conditions determined in the condition determination step.

7. 7. The image creating method according to claim 6, wherein the sensitivity determining step determines the sensitivity of the pixels corresponding to the first region to be higher than the sensitivity of the pixels corresponding to the second region, which is the reference region.

8. The method further includes an adjustment execution step of adjusting an exposure amount over time when capturing the first video by changing the capturing conditions over time, The condition determining step includes: a first determination step of determining the photographing conditions based on the appropriate exposure amount of the reference area; a second determination step of determining an adjustment amount of the exposure amount in the adjustment execution step based on the appropriate exposure amount of the selected region, The video creation method according to claim 4 , wherein the adjustment execution step is performed based on the shooting conditions determined in the first determination step and the adjustment amount determined in the second determination step.

9. When the switching step is performed, the first determination step is performed; The video production method according to claim 8 , wherein the second determining step is performed in the adjustment executing step after the first determining step is performed.

10. a display step of the processor displaying the first video on a display screen, The image creation method according to any one of claims 1 to 9, wherein the setting step sets the plurality of areas in accordance with a setting operation performed by a user through the first image displayed on the display screen.

11. a change pattern for changing each of the values ​​of the plurality of parameters in order to adjust the exposure amount to the target exposure amount is defined in the first adjustment data and the second adjustment data; 11. The image creation method according to claim 1, wherein the first adjustment data defines a change pattern that changes the aperture amount to a smaller degree relative to the target exposure amount than the change pattern of the second adjustment data.

12. the photographing device is provided with an electronic neutral density filter capable of electronically changing the degree of light control; The image creation method according to claim 1 , wherein the plurality of parameters includes the degree of dimming of the electronic neutral density filter.

13. A video production system having an imaging device and a processor, The processor: a plurality of areas having a second angle of view smaller than the first angle of view are set within a shooting area of ​​a first image having a first angle of view; Accepting a selection of a selected area as a shooting area of ​​the video to be recorded from among the plurality of areas; After said selection, reselecting said selected area to switch said selected area among said plurality of areas; The switching is performed during a period from when the switching is performed until a first period has elapsed, and an image of the selected area during the first period is recorded; The method is carried out during a second period other than the first period, and an image of the selected area during the second period is recorded; adjusting the exposure amount of the selected region to a target exposure amount using first adjustment data, the adjustment being carried out during the first period; adjusting the exposure amount of the selected region to a target exposure amount using second adjustment data, which is carried out during the second period; During the first period and the second period, the exposure amount is adjusted to the target exposure amount based on the first adjustment data or the second adjustment data, for each value of a plurality of parameters including an exposure time, a pixel sensitivity, and an aperture amount for incident light; The first adjustment data and the second adjustment data have different values ​​for each of the plurality of parameters determined for the target exposure amount. Video production system.

Images