Imaging apparatus, control device, control method, program, and storage medium

Abstract

To enable photographing at a desired angle of view, while reducing fluctuation or suppressing a moving subject blur.SOLUTION: An imaging apparatus includes: an imaging part for capturing an image of a subject by an imaging optical system capable of optical zooming; an acquisition part for acquiring information on the fluctuation of the image captured by the imaging part; an image processing part for executing correction processing for correcting the fluctuation of the image on the basis of the information on the fluctuation of the image and electronic zoom processing for enlarging or reducing the image captured by the imaging part; and a determination part for determining a ratio between an optical zoom magnification of the imaging optical system and an electronic zoom magnification of the image processing part on the basis of a set zoom magnification and the information on the fluctuation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control device, a control method, a program, and a storage medium. [Background technology]

[0002] In use cases of surveillance cameras, such as port surveillance and infrastructure monitoring, it is known that when taking telephoto shots of ships or aircraft, for example, fluctuations in the subject image caused by uneven changes in the refractive index of the atmosphere (such as heat haze) can reduce the visibility of the subject.In response to this, a method has been known that reduces fluctuations by smoothing the image in the time direction, taking advantage of the fact that the displacement of the subject image due to fluctuations can be approximated by a normal distribution based on a predetermined position.

[0003] Furthermore, since fluctuations change depending on the environment, time, etc., when reducing fluctuations using time-direction smoothing processing, it is necessary to change the strength of the time-direction smoothing processing according to the degree of fluctuation. For example, Patent Document 1 discloses a technology that detects the degree of fluctuation using an input image and corrects fluctuations by averaging frame images in the time direction according to the detected degree of fluctuation. The technology disclosed in Patent Document 1 makes it possible to appropriately correct fluctuations even when the degree of fluctuation changes.

[0004] Furthermore, according to Patent Document 2, the likelihood of heat haze occurring increases when the focal length is long. Therefore, by applying the technology disclosed in Patent Document 2, image capture is performed while suppressing the use of telephoto focal lengths equal to or greater than a predetermined value, and it is possible to prevent image quality degradation due to heat haze. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] WO15 / 132826 [Patent Document 2] Patent Publication No. 2014-191307 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method of smoothing an image in the time direction has the drawback that, if the smoothing time is extended, a moving subject becomes blurred (hereinafter referred to as motion blur). Furthermore, with the conventional technology disclosed in the above-mentioned Patent Document 2, the focal length is limited, making it impossible to use the telephoto angle of view, and therefore it is not possible to capture an image with a desired angle of view when using a telephoto lens.

[0007] An object of the present invention is to enable shooting at a desired angle of view while reducing fluctuation or suppressing blurring of moving objects. [Means for solving the problem]

[0008] In order to solve the above problem, an imaging device according to one aspect of the present invention is an imaging device characterized by having an imaging unit that captures an image of a subject using an imaging optical system capable of optical zoom, an acquisition unit that acquires information regarding fluctuations in the image captured by the imaging unit, an image processing unit that performs a correction process that corrects the fluctuations in the image based on the information regarding the image fluctuation, and an electronic zoom process that enlarges or reduces the image captured by the imaging unit, and a determination unit that determines the ratio between the optical zoom magnification of the imaging optical system and the electronic zoom magnification of the image processing unit based on a set zoom magnification and the information regarding the fluctuations. [Effects of the Invention]

[0009] According to the present invention, it is possible to capture an image at a desired angle of view while reducing fluctuation or suppressing blurring caused by moving objects. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing the configuration of an imaging apparatus. [Figure 2] FIG. 2 is a diagram showing the functional configuration of the imaging apparatus. [Figure 3] 4 is a flowchart showing a processing operation of the imaging apparatus. [Figure 4] FIG. 10 is a diagram showing an example of fluctuation. [Figure 5] FIG. 10 is a diagram showing an example of fluctuation information. [Figure 6] FIG. 10 is a diagram showing an example of fluctuation reduction. [Figure 7] 10A and 10B are diagrams showing image fluctuations in each zoom method. [Figure 8] 5A to 5C are diagrams showing blurring of an image caused by a moving object in each zoom method. [Figure 9] FIG. 2 is a diagram showing the functional configuration of the imaging apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, a configuration may be made by appropriately combining parts of each embodiment described below.

[0012] <Embodiment 1> (Device configuration) Fig. 1 is a diagram illustrating the device configuration of an imaging device according to this embodiment. The imaging device 100 includes a lens 101, an imaging element 102, an image processing unit 103, a CPU (Central Processing Unit) 104, a RAM (Random Access Memory) 105, and a ROM (Read Only Memory) 106. The blocks shown in Fig. 1 are electrically connected via a bus 107, and the lens 101 and the imaging element 102 receive control signals from the CPU 104, for example.

[0013] The lens 101 is an imaging optical system that collects light from a subject and forms an optical image of the subject on the imaging surface of the image sensor 102. The lens 101 includes, for example, a zoom lens and a focus lens, and the positions of the lenses are adjusted by motors operating in response to control signals from the CPU 104, thereby changing the optical zoom magnification and focal length. In other words, the lens 101 is an imaging optical system that is capable of optical zoom.

[0014] The image sensor 102 is a photoelectric conversion element, such as a CMOS (Complementary Metal Oxide Semiconductor) sensor, that converts an optical image of a subject into an electrical signal. The image sensor 102 may further convert the converted electrical signal from an analog signal to a digital signal.

[0015] The image processing unit 103 is an image processor that performs various image processing on the image signal output from the image sensor 102. Examples of image processing that is performed include fluctuation correction processing that corrects fluctuation in the image and electronic zoom processing that enlarges or reduces the image. The fluctuation correction processing is image processing that reduces fluctuation in the optical image caused by uneven changes in the refractive index of the atmosphere (such as heat haze). Specific processing of the fluctuation correction processing will be described later.

[0016] The CPU 104 is a central processing unit that controls the image capture device 100. The RAM 105 provides a work area used by the CPU 104 when it executes processing. The RAM 105 also functions as a frame memory and a buffer memory. The ROM 106 stores programs and image data used by the CPU 104 to control the image capture device 100.

[0017] (Functional configuration) FIG. 2 is a diagram showing the functional configuration of the imaging device according to this embodiment. Of the blocks shown in FIG. 2, parts that are common to the device configuration are assigned the same reference numerals, and descriptions thereof will be omitted. Furthermore, of the blocks shown in FIG. 2, functions that are realized by software can be realized by executing a program for providing each function by the CPU 104 or the image processing unit 103. Note that the configuration of each block shown in FIG. 2 is an example, and multiple blocks may constitute one function, or any block may be divided into blocks that perform multiple functions. Alternatively, any block may be realized by an external device separate from the imaging device.

[0018] The image capturing apparatus 100 includes a fluctuation information acquisition unit 201 , a fluctuation correction unit 202 , a zoom ratio determination unit 203 , a zoom magnification acquisition unit 204 , an electronic zoom unit 205 , and an optical zoom control unit 206 .

[0019] An image of a subject captured by a lens 101 (imaging optical system) capable of optical zoom is captured by an image sensor 102, which is an imaging unit. The image captured by the imaging unit is image data or an image signal made up of multiple pixels, and includes multiple color information. The imaging surface of the image sensor 102 has color filters corresponding to the colors red (Red: R), green (Green: G), and blue (Blue: B). Color information is generated by converting the amount of light transmitted through the color filters into an electrical signal. Note that in this embodiment, it is assumed that the image contains fluctuations in the subject image caused by uneven changes in the refractive index of the atmosphere (such as heat haze).

[0020] An image captured by the imaging unit is used as an input image, and the image is subjected to jitter correction processing by the jitter correction unit 202 and electronic zoom processing by the electronic zoom unit 205, and output as an output image. Note that the functions of the jitter correction unit 202 and the electronic zoom unit 205 can be realized by the image processing unit 103.

[0021] The imaging device according to this embodiment determines the ratio between the optical zoom magnification of the imaging optical system and the electronic zoom magnification of the image processing unit 103 based on information about fluctuations in the input image. This aims to reduce fluctuations and obtain an image with a desired zoom magnification (angle of view).

[0022] The fluctuation information acquisition unit 201 acquires information about fluctuation of the input image. The information about fluctuation refers to information indicating the amount of fluctuation of the image, the degree of fluctuation from small to large, or the strength of the correction process to reduce the fluctuation by the fluctuation correction unit 202. In other words, the fluctuation information acquisition unit 201 acquires at least one of the amount of fluctuation of the image, the degree of fluctuation, and the strength of the correction process. The amount of fluctuation or the degree of fluctuation can be calculated from the difference between the input image and an image captured before the input image. The strength of the correction process is acquired as a strength set in advance by the user or the like.

[0023] The fluctuation correction unit 202 performs a correction process to reduce the fluctuation of the image based on the information about the fluctuation acquired from the fluctuation information acquisition unit 201, and outputs the image with reduced fluctuation to the electronic zoom unit 205. That is, the image processing unit 103 performs an electronic zoom process after performing the correction process.

[0024] The correction process uses smoothing (averaging) process in the time direction (frame direction), such as a simple moving average or a weighted moving average between multiple frame images. Based on the information about the fluctuation, the fluctuation correction unit 202 increases the strength of the correction process the stronger the degree of fluctuation contained in the input image or the greater the amount of fluctuation. More specifically, the stronger the strength of the correction process, the greater the number of frame images used in the smoothing process in the time direction (frame direction). Alternatively, the stronger the strength of the correction process, the longer the smoothing of frame images captured over a longer period of time. If the fluctuation information acquisition unit 201 has acquired the strength of the correction process as information about the fluctuation, the correction process is performed based on the strength of the correction process.

[0025] The zoom ratio determination unit 203 acquires information about fluctuation acquired by the fluctuation information acquisition unit 201 and the system zoom magnification S acquired by the zoom magnification acquisition unit 204. The system zoom magnification S is the overall zoom magnification of the entire image capture device 100, and determines the angle of view when capturing an image with the image capture device 100. The system zoom magnification S is a zoom magnification that can be set in advance by the user. An input unit (not shown) accepts input of the zoom magnification from the user. The input unit may be a network interface or the like, and may accept a zoom magnification instruction input to an external information processing device via a network. Alternatively, a UI (User Interface) may be provided in the image capture device 100, and the instruction may be input directly thereto.

[0026] The system zoom magnification S is equal to the product of the electronic zoom magnification D used in the electronic zoom unit 205 and the optical zoom magnification L used in the optical zoom control unit 206. The zoom ratio determination unit 203 then determines the zoom distribution ratio R based on the information related to fluctuation. The zoom distribution ratio R is the ratio between the electronic zoom magnification D and the optical zoom magnification L. The zoom ratio determination unit 203 calculates the zoom distribution ratio R based on the information related to fluctuation so that the proportion of the electronic zoom magnification D increases as the degree or amount of fluctuation increases. The zoom distribution ratio R can be determined, for example, by referring to an LUT (Look Up Table) that receives the magnitude of the amount of fluctuation as input and outputs the zoom distribution ratio R.

[0027] Furthermore, the zoom ratio determination unit 203 calculates the electronic zoom magnification D and the optical zoom magnification L based on the system zoom magnification S and the zoom distribution ratio R. The electronic zoom magnification D is output to the electronic zoom unit 205, and the optical zoom magnification L is output to the optical zoom control unit 206. If the zoom distribution ratio R is the electronic zoom magnification D divided by the optical zoom magnification L, the electronic zoom magnification D and the optical zoom magnification L can be calculated based on the system zoom magnification S and the zoom distribution ratio R as follows:

[0028]

number

[0029] The zoom magnification acquisition unit 204 acquires the system zoom magnification S and outputs it to the zoom ratio determination unit 203 .

[0030] The electronic zoom unit 205 acquires the jitter-reduced image from the jitter correction unit 202 and the electronic zoom magnification D from the zoom ratio determination unit 203. Then, based on the electronic zoom magnification D, it performs electronic zoom processing on the jitter-reduced image and outputs the electronically zoomed image as an output image. The electronic zoom processing changes the angle of view of the output image by, for example, cutting out a part of the jitter-reduced image based on the electronic zoom magnification D.

[0031] The optical zoom control unit 206 acquires the optical zoom magnification L from the zoom ratio determination unit 203, and outputs a zoom lens drive control value to the lens 101 based on the optical zoom magnification L so as to achieve an angle of view represented by the optical zoom magnification L. The zoom lens drive control value is, for example, a value indicating the position of the zoom lens, a focal length, a drive control value of a motor that moves the lens, etc.

[0032] The lens 101 is a lens with a variable focal length, which acquires a zoom lens drive control value from the optical zoom control unit 206 and moves the zoom lens based on the zoom lens drive control value. The lens 101 also focuses light within a predetermined angle of view depending on the position of the focus lens or zoom lens, and forms a subject image on the imaging surface of the imaging element 102.

[0033] (Operation description) 2 is a flowchart showing the flow of processing of the imaging apparatus according to this embodiment. The operations shown in this flowchart are realized by the CPU 104 loading and executing a program stored in a storage medium such as the ROM 106.

[0034] In S01, the optical zoom control unit 206 drives the lens 101 based on a lens control value based on the current optical zoom magnification L, changes the focal length of the lens 101, and changes the angle of view.

[0035] In S02, the electronic zoom unit 205 performs electronic zoom processing to change the angle of view by performing image cropping processing based on the current electronic zoom magnification D.

[0036] In S03, the fluctuation information acquisition unit 201 acquires the amount of fluctuation of the input image as information related to fluctuation.

[0037] In S04, the zoom magnification acquisition unit 204 acquires the system zoom magnification S, which is the zoom magnification set for the entire image capture device.

[0038] In S05, the zoom ratio determination unit 203 determines whether the amount of fluctuation is large based on the acquired information about the fluctuation. Specifically, it determines whether the amount of fluctuation is larger than a predetermined threshold. If the degree of fluctuation / strength of the correction process (three levels: low, medium, and high) has been acquired as information about the fluctuation, it only needs to determine whether the degree of fluctuation / strength of the correction process is medium or higher. If it is determined that the amount of fluctuation is large, the process proceeds to S06. If it is determined that the amount of fluctuation is small, the process ends.

[0039] In S06, based on the acquired information on fluctuation, the zoom ratio determination unit 203 calculates the zoom distribution ratio R, which is the ratio between the electronic zoom magnification D and the optical zoom magnification L. Based on the information on fluctuation, the zoom ratio determination unit 203 calculates the zoom distribution ratio R such that the proportion of the electronic zoom magnification D increases as the amount of fluctuation increases.

[0040] In S07, the zoom ratio determination unit 203 calculates the optical zoom magnification L based on the system zoom magnification S and the zoom distribution ratio R.

[0041] In S08, the zoom ratio determination unit 203 calculates the electronic zoom magnification D based on the system zoom magnification S and the zoom distribution ratio R.

[0042] The optical zoom magnification L calculated in S07 and the electronic zoom magnification D calculated in S08 are applied to the next frame and subsequent frames.

[0043] (Information about fluctuations and how to obtain it) The information on fluctuations acquired by the fluctuation information acquisition unit 201 will be described in more detail below with reference to FIGS.

[0044] First, the effect of fluctuation on an input image will be explained using FIG. 3. FIG. 3(a) shows an example of an image obtained by capturing a still subject without fluctuation. FIG. 3(b) shows an example of an image obtained by capturing a still subject with fluctuation. As shown in FIGS. 3(a) and 3(b), even when capturing a still subject, if there is fluctuation, a phenomenon occurs in which what is originally a straight line becomes distorted. FIG. 3(c) is a diagram showing the fluctuation of pixel values ​​in the time direction at coordinate P in FIG. 3(a) and coordinate Q in FIG. 3(b). The fluctuation of coordinate P is shown by a solid line, and the fluctuation of coordinate Q is shown by a dashed line. As shown in FIG. 3(c), when there is no fluctuation (solid line), the pixel value is constant (no fluctuation), but when there is fluctuation (dashed line), the pixel value fluctuates over time. Therefore, when there is fluctuation, a phenomenon occurs in which pixel values ​​fluctuate over time, even for an originally still subject, as if it were a moving subject.

[0045] Next, an example of a method by which the fluctuation information acquisition unit 201 acquires information about fluctuations from an input image will be described with reference to FIG. 4. Similar to FIG. 3(c), FIG. 4 shows fluctuations in pixel values ​​in the time direction at a predetermined coordinate, with the degree of fluctuation increasing in the order of (a), (b), and (c), with (c) being the largest. The interval at which the input image is acquired is denoted by Δt, the time at which the predetermined input image is acquired is denoted by t1, and the time Δt after t1 is denoted by t2. For each of FIG. 4(a), (b), and (c), the absolute value of the difference between the pixel value at t1 and the pixel value at t2 is calculated; the greater the degree of fluctuation, the greater the absolute value of the difference. By repeatedly performing this process for multiple coordinates of the input image, the fluctuation information acquisition unit 201 acquires information about fluctuations.

[0046] (Regarding correction processing) The correction process performed by the fluctuation correction unit 202 will be described in more detail below with reference to FIG. 5. The fluctuation correction unit 202 performs correction processing on the input captured image based on information about fluctuations acquired from the fluctuation information acquisition unit 201. An example of the correction processing performed by the fluctuation correction unit 202 will be described with reference to FIG. 5. FIG. 5 shows fluctuations in pixel values ​​in the time direction at a predetermined coordinate, similar to FIG. 3(c). As the correction processing, a time-direction smoothing process such as a simple moving average or a weighted moving average is used. FIG. 5(a) shows fluctuations in pixel values ​​when time-direction smoothing processing is not applied, while FIGS. 5(b) and 5(c) show fluctuations in pixel values ​​when time-direction smoothing processing is applied. Because the pixel value displacement due to fluctuations is approximated to a normal distribution based on a predetermined position, it can be reduced by time-direction smoothing processing. FIG. 5(c) shows an example of the effect when the time-direction smoothing processing (correction processing) is stronger than that of FIG. 5(b). Methods for changing the intensity of fluctuation reduction include changing the number of frames to be smoothed and changing the weighting of each frame to be smoothed.

[0047] (Effects of the invention) The control of the imaging device according to this embodiment and its effects will be described below. FIG. 6 shows the area where fluctuation occurs when the angle of view is changed (zoomed) by different methods. 1001 represents an image captured at a wide-angle angle of view, and is an example of a scene in which subjects at various distances are included within the angle of view. In FIG. 6, the area where fluctuation occurs is indicated by the hatched area. Area 1002 surrounded by a dashed line does not exhibit fluctuation at the wide-angle angle of view shown in 1001. 1003 and 1004 represent images captured when the angle of view is changed to the telephoto side and the angle of view is changed to area 1002. 1003 represents the angle of view when the angle of view is changed (zoomed) by lengthening the focal length using optical zoom, while 1004 represents the angle of view when the angle of view is changed (enlarged) by cropping an image without changing the focal length using electronic zoom. As mentioned above, a long focal length increases the likelihood of fluctuation (heat haze). Therefore, no fluctuation occurs in 1002 when shooting at a wide angle, but fluctuation occurs in the shaded area in 1003 when the focal length is increased by optical zoom. On the other hand, in 1004, when the angle of view is changed by cropping the image without changing the focal length (electronic zoom), it is possible to enlarge the image without fluctuation.

[0048] In this embodiment, the zoom ratio determination unit 203 controls the ratio of the electronic zoom magnification to the optical zoom magnification based on information about jitter so that the greater the degree or amount of jitter, the greater the ratio of the electronic zoom magnification to the optical zoom magnification. That is, the greater the amount of jitter, the smaller the optical zoom magnification and the shorter the focal length, reducing the likelihood of jitter occurring. Then, the electronic zoom magnification is increased by the amount that the optical zoom magnification is reduced. As a result, it becomes possible to prevent jitter and capture an image with a desired angle of view.

[0049] Next, the effect of the imaging device of this embodiment will be described from another perspective. FIG. 7 shows the occurrence of shaking and motion blur when the angle of view is changed (zoomed) by different methods. 2001 represents an image captured at a wide-angle angle of view, and 2003 and 2004 represent cases where the angle of view is changed to 2002 by zooming. 2003 represents the angle of view when zooming by lengthening the focal length using optical zoom, and 2004 represents the angle of view when an image is cropped and enlarged using electronic zoom without changing the focal length. 2003 also represents the occurrence of motion blur, while 2004 represents the absence of motion blur. The area 2002 surrounded by a dashed line is the area including the subject of interest, and is assumed to be experiencing shaking. In other words, FIG. 7 shows a case where zooming is performed on a subject of interest experiencing shaking.

[0050] Here, consider a case where correction processing is performed based on information about fluctuation. For example, the correction processing is performed based on the ratio of the fluctuation-occurring area to the angle of view. The larger the fluctuation-occurring area, the greater the fluctuation amount is determined, and the stronger the correction processing (smoothing processing in the time direction) is. In this embodiment, since information about fluctuation (fluctuation amount) is acquired in a process prior to electronic zooming, even in the case of 2004, in which the angle of view is changed by cropping the image, the fluctuation amount is acquired based on 2001. Therefore, even if the crop size of the image (electronic zoom magnification) changes, the fluctuation amount does not change, and the fluctuation amounts of 2001 and 2004 are the same. The greater the fluctuation amount, the stronger the correction processing and the greater the motion blur. However, since 2001 and 2004 have the same fluctuation amount, the degree of motion blur is also the same. In other words, motion blur can be maintained even if the angle of view is changed (enlarged) by electronic zooming. On the other hand, in this embodiment, information about the fluctuation (amount of fluctuation) is acquired in processing subsequent to the optical zoom, and since 2003 changes the angle of view due to the optical zoom, the amount of fluctuation is acquired based on 2003. Since the ratio of the fluctuation occurrence area to the angle of view in 2003 is larger than the angle of view 2001 before zooming, it is determined that the amount of fluctuation is large, and stronger correction processing is applied. The stronger the correction processing, the greater the motion blur, so 2003 will have more motion blur than 2001 or 2004. As a result, if the subject of interest is moving, motion blur is more likely to occur in 2003 enlarged by optical zoom compared to 2004 enlarged by electronic zoom when compared at the same angle of view. In other words, if you want to zoom in on a subject of interest, you can use electronic zoom to zoom while preventing motion blur.

[0051] In this embodiment, the zoom ratio determination unit 203 controls the ratio of the electronic zoom magnification to the optical zoom magnification based on information about the jitter so that the greater the degree or amount of jitter, the greater the ratio. That is, if the amount of jitter is large, the optical zoom magnification is reduced and the electronic zoom magnification is increased. By reducing the optical zoom magnification, the area ratio of the jitter region included in the angle of view becomes smaller, thereby reducing the amount of jitter and weakening the strength of the correction process (frame smoothing process in the time direction), thereby suppressing blurring of moving objects. As a result, it becomes possible to capture an image by enlarging it to a desired angle of view while preventing blurring of moving objects.

[0052] The effect of the imaging device of this embodiment will be further described from another perspective. In the imaging device of this embodiment, the electronic zoom unit 205 is arranged after the jitter correction unit 202. Therefore, electronic zoom processing is performed on an image after jitter has been reduced. Regardless of whether optical zoom or electronic zoom is used, when an image is enlarged, the displacement of the subject image due to jitter also increases, which may make the jitter more noticeable. In contrast, in this embodiment, as described above, electronic zoom processing is performed on an image after jitter has been reduced. In other words, since electronic zoom processing is performed after jitter has been reduced in advance, it is possible to prevent jitter from being emphasized by electronic zoom processing. On the other hand, optical zoom processing is performed in the lens, and is therefore performed at the very front of the imaging processing system of this embodiment. In other words, since an image is enlarged by optical zoom without jitter reduction, there is a risk that jitter will be emphasized.

[0053] For these reasons, in this embodiment, electronic zoom is more effective at preventing the emphasis of jitter that accompanies zoom than optical zoom. When the degree of jitter is large, the imaging device of this embodiment increases the ratio of electronic zoom magnification that does not emphasize jitter, making it possible to capture an image with a desired enlarged angle of view while suppressing jitter.

[0054] Note that the zoom ratio determination unit 203 calculated the zoom distribution ratio R based on the information about fluctuation so that the proportion of the electronic zoom magnification D increases as the degree or amount of fluctuation increases, but this is not limiting. For example, the zoom distribution ratio R may be calculated so that the proportion of the electronic zoom magnification D increases as the system zoom magnification S increases. By doing so, it is possible to prevent fluctuation from becoming too large during high-magnification zoom shooting.

[0055] (Variation) In this embodiment, an image capture device that determines the ratio between the optical zoom magnification and the electronic zoom magnification based on information about fluctuation has been described, but the function of determining the zoom distribution ratio R in the image capture device according to embodiment 1 can also be implemented by an external control device. That is, it is implemented as a control device for controlling an image capture device that has an image capture optical system capable of optical zoom and an image processing unit that performs electronic zoom processing to enlarge or reduce a captured image.

[0056] In this case, the control device may have an acquisition unit that acquires information about image fluctuation from the imaging device, a determination unit that determines the zoom distribution ratio R, and an output unit that outputs the determined zoom distribution ratio R to the imaging device. Preferably, the control device has an input unit that receives the system zoom magnification S.

[0057] <Embodiment 2> An imaging device according to this embodiment will be described below with reference to Fig. 8. In this embodiment, information about fluctuation is acquired from an output image, and the information about fluctuation and the electronic zoom magnification and optical zoom magnification calculated based on the acquired information about fluctuation are fed back to the next frame onward to perform correction processing and zoom control.

[0058] 8 is a diagram showing the configuration of an imaging device according to an embodiment of the present invention. Note that a description of functional units that are the same as those in the first embodiment will be omitted, and only functional units that have functions different from those in the first embodiment will be described below.

[0059] The fluctuation information acquisition unit 201 acquires information about fluctuation from the output image, and feeds back the information about fluctuation to the fluctuation correction unit 202 and outputs it to the zoom ratio determination unit 203. The information about fluctuation is the degree or amount of fluctuation contained in the input image. Alternatively, the information about fluctuation is the strength of the correction process.

[0060] The fluctuation correction unit 202 performs correction processing on the input image based on the information about fluctuation fed back from the fluctuation information acquisition unit 201 , and outputs the image with reduced fluctuation to the electronic zoom unit 205 .

[0061] The zoom ratio determination unit 203 acquires information about fluctuation from the fluctuation information acquisition unit 201 and the system zoom magnification S from the zoom magnification acquisition unit 204. Then, based on the information about fluctuation, it calculates the electronic zoom magnification D and the optical zoom magnification L. Then, it feeds back the electronic zoom magnification D to the electronic zoom unit 205 and the optical zoom magnification L to the optical zoom control unit 206.

[0062] The electronic zoom unit 205 acquires the jitter-reduced image from the jitter correction unit 202 and the electronic zoom magnification D from the zoom ratio determination unit 203. Then, based on the electronic zoom magnification D, it performs electronic zoom processing on the jitter-reduced image and outputs the electronically zoomed image as an output image. The electronic zoom processing may be performed, for example, by cutting out a portion of the jitter-reduced image based on the electronic zoom magnification D.

[0063] With the above configuration, the imaging device of this embodiment calculates the ratio between the electronic zoom magnification D and the optical zoom magnification L based on information about fluctuations acquired from the output image. The imaging device then calculates the electronic zoom magnification D and the optical zoom magnification L and feeds them back to the optical zoom control unit and the electronic zoom unit. By performing this in this manner, zoom control can be performed that further optimizes the strength of the correction process and the ratio between the electronic zoom magnification D and the optical zoom magnification L, making it possible to capture an image with a desired angle of view while reducing fluctuations.

[0064] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-described first embodiment. This program is supplied to a system or device via a network or a computer-readable storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0065] 100 Imaging device 101 Lens 102 Image sensor 103 Image processing section 201 Fluctuation Information Acquisition Unit 202 Fluctuation correction unit 203 Zoom ratio determination unit 204 Zoom magnification acquisition unit 205 Electronic zoom section 206 Optical zoom control section

Claims

1. an imaging unit that captures an image of a subject using an imaging optical system capable of optical zoom; an acquisition unit that acquires information about fluctuations in the image captured by the imaging unit; an image processing unit that performs a correction process to correct fluctuations in the image based on information about the fluctuations in the image, and an electronic zoom process to enlarge or reduce the image captured by the imaging unit; a determination unit that determines a ratio between an optical zoom magnification of the imaging optical system and an electronic zoom magnification of the image processing unit based on a set zoom magnification and information about the fluctuation; An imaging device comprising:

2. 2. The imaging device according to claim 1, wherein the information about the fluctuation includes at least one of an amount of fluctuation in the image and a strength of the correction process.

3. 3. The imaging device according to claim 2, wherein the determination unit determines the ratio such that the greater the amount of fluctuation, the greater the ratio of the electronic zoom magnification.

4. 3. The imaging device according to claim 2, wherein the determination unit determines the ratio such that the stronger the strength of the correction processing, the larger the ratio of the electronic zoom magnification.

5. The imaging device according to claim 1 , wherein the determination unit increases the electronic zoom magnification as the set zoom magnification increases.

6. 3. The imaging device according to claim 2, wherein the strength of the correction process is determined based on the amount of fluctuation in the image.

7. 2. The imaging device according to claim 1, wherein the determination unit determines the optical zoom magnification and the electronic zoom magnification so that a product of the optical zoom magnification and the electronic zoom magnification matches the set zoom magnification.

8. further comprising an input unit that accepts input of a zoom magnification; 2. The imaging device according to claim 1, wherein the determination unit determines the optical zoom magnification and the electronic zoom magnification based on the zoom magnification and the ratio inputted to the input unit.

9. The imaging device according to claim 1 , wherein the acquisition unit acquires the information about the fluctuation from an image before electronic zoom processing is performed by the image processing unit.

10. 2. The imaging device according to claim 1, wherein the image processing unit executes the electronic zoom process after executing the correction process.

11. A control device for controlling an imaging device having an imaging optical system capable of optical zoom and an image processing unit that performs electronic zoom processing to enlarge or reduce a captured image, an acquisition unit that acquires information about fluctuations in an image captured by the imaging device; a determination unit that determines a ratio between an optical zoom magnification of the imaging optical system and an electronic zoom magnification of the image processing unit based on a set zoom magnification and information about the fluctuation; an output unit that outputs the ratio between the optical zoom magnification and the electronic zoom magnification determined by the determination unit to the imaging device.

12. an imaging step of capturing an image of a subject using an imaging optical system capable of optical zoom; acquiring information about fluctuations in the image captured in the imaging step; an image processing step of executing a correction process for correcting fluctuations in the image based on information about the fluctuations in the image, and an electronic zoom process for enlarging or reducing the image captured in the imaging step; a determining step of determining a ratio between the optical zoom magnification of the imaging optical system and the electronic zoom magnification of the image processing step based on a set zoom magnification and information about the fluctuation; A control method comprising:

13. A program for causing a computer to execute the control method according to claim 12.

14. A computer-readable storage medium storing the program according to claim 13.

Images