Control device, imaging device, control method, and control program

Abstract

To provide a control apparatus, an imaging apparatus, a control method, and a control program capable of improving followability of a focus position with respect to a subject.SOLUTION: A system control unit 11, on the basis of image data of a first frame and image data of a second frame among a plurality of frames obtained by an imaging apparatus 100, predicts distance information concerning the distance between the imaging apparatus 100 and a subject 301 in a third frame after the first frame and the second frame. The system control unit 11 sets a setting possible range being a range of a focus position capable of setting with respect to the third frame, on the basis of the state of the subject 301.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a control device, an imaging device, a control method, and a control program.

Background Art

[0002] Patent Document 1 describes a focus control device that widens a movement permission range for permitting movement of a focus lens as the difference between the position of the focus lens before being moved based on a phase difference amount and the target in-focus position of the focus lens based on the phase difference amount becomes larger. Patent Document 2 describes an imaging device that selects a frame rate of a through-image suitable for the moving speed of a subject and controls an imaging element or the like at the frame rate to capture the through-image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

[0004] One embodiment according to the technology of the present disclosure provides a control device, an imaging device, a control method, and a control program capable of improving the followability of a focus position with respect to a subject.

Means for Solving the Problems

[0005] A control device according to one aspect of the present invention includes a processor and is a control device capable of controlling an imaging device. The processor predicts distance information regarding the distance between the imaging device and a subject in a third frame after the first frame and the second frame based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, and sets a settable range, which is a range of the focus position of imaging by the imaging device that can be set for the third frame, based on the state of the subject.

[0006] A control method according to one aspect of the present invention is a control method by a control device including a processor and capable of controlling an imaging device. The processor predicts distance information regarding the distance between the imaging device and a subject in a third frame after the first frame and the second frame based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, and sets a settable range, which is a range of the focus position of imaging by the imaging device that can be set for the third frame, based on the state of the subject.

[0007] A control program according to one aspect of the present invention is a control program of a control device including a processor and capable of controlling an imaging device. The control program causes the processor to execute a process of predicting distance information regarding the distance between the imaging device and a subject in a third frame after the first frame and the second frame based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, and setting a settable range, which is a range of the focus position of imaging by the imaging device that can be set for the third frame, based on the state of the subject.

Advantages of the Invention

[0008] According to the present invention, it is possible to provide a control device, an imaging device, a control method, and a control program that can improve the followability of the focus position with respect to a subject.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0011] <Configuration of the imaging device 100 which is an embodiment of the imaging device of the present invention> FIG. 1 is a diagram showing the configuration of the imaging device 100 which is an embodiment of the imaging device of the present invention.

[0012] The imaging device 100 shown in FIG. 1 is a digital camera including a lens device 40 having an imaging lens 1, a diaphragm 2, a lens control unit 4, a lens driving unit 8, and a diaphragm driving unit 9, and a main body unit 100A. The main body unit 100A includes an imaging unit 50, a system control unit 11, an operation unit 14, a display device 22, a memory 16 including a RAM (Random Access Memory) and a ROM (Read only memory), a memory control unit 15 for controlling data recording to the memory 16 and data reading from the memory 16, a digital signal processing unit 17, and an external memory control unit 20 for controlling data recording to a recording medium 21 and data reading from the recording medium 21.

[0013] The lens device 40 may be detachable from the main body 100A or may be integrated with the main body 100A. The imaging lens 1 includes a focus lens or the like that is movable in the optical axis direction. This focus lens is a lens for adjusting the focus of the imaging optical system including the imaging lens 1 and the aperture 2, and is composed of a single lens or a plurality of lenses. By moving the focus lens in the optical axis direction, the position of the principal point of the focus lens changes along the optical axis direction, and the focus position on the subject side is changed. Note that as the focus lens, a liquid lens whose position of the principal point in the optical axis direction can be changed by electrical control may be used.

[0014] The lens control unit 4 of the lens device 40 is configured to be communicable with the system control unit 11 of the main body 100A by wire or wirelessly. The lens control unit 4 controls the focus lens included in the imaging lens 1 via the lens drive unit 8 to change the position of the principal point of the focus lens or controls the aperture value of the aperture 2 via the aperture drive unit 9 according to a command from the system control unit 11.

[0015] The imaging unit 50 includes an image sensor 5 that images a subject through an imaging optical system including the imaging lens 1 and the aperture 2, and an image sensor drive unit 10 that drives the image sensor 5.

[0016] The image sensor 5 has an imaging surface 60 (see FIG. 2) on which a plurality of pixels 61 are two-dimensionally arranged, and converts a subject image formed on this imaging surface 60 by the imaging optical system into pixel signals by these plurality of pixels 61 and outputs them. As the image sensor 5, a CMOS (complementary metal-oxide-semiconductor) image sensor is preferably used. Hereinafter, it will be described on the assumption that the image sensor 5 is a CMOS image sensor.

[0017] The system control unit 11 that comprehensively controls the entire electrical control system of the imaging device 100 drives the image sensor 5 via the image sensor drive unit 10 and outputs a subject image imaged through the imaging optical system of the lens device 40 as an image signal.

[0018] The imaging element driving unit 10 generates a driving signal based on a command from the system control unit 11 and supplies this driving signal to the imaging element 5, thereby driving the imaging element 5. The hardware configuration of the imaging element driving unit 10 is an electric circuit configured by combining circuit elements such as semiconductor elements.

[0019] An instruction signal from the user is input to the system control unit 11 through the operation unit 14. The operation unit 14 includes a touch panel integrated with a display surface 22b to be described later, various buttons, and the like.

[0020] The system control unit 11 comprehensively controls the entire imaging device 100, and its hardware structure is various processors that execute a program including an imaging control program to perform processing. The program executed by the system control unit 11 is stored in the ROM of the memory 16.

[0021] Examples of the various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes a program to perform various processes; a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing; or a dedicated electric circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration specifically designed to execute specific processes. More specifically, the structure of these various processors is an electric circuit formed by combining circuit elements such as semiconductor elements.

[0022] The system control unit 11 may be configured by one of the various processors, or may be configured by a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0023] The display device 22 includes a display surface 22b composed of an organic EL (Electro Luminescence) panel, a liquid crystal panel, or the like, and a display controller 22a that controls the display on the display surface 22b.

[0024] The memory control unit 15, the digital signal processing unit 17, the external memory control unit 20, and the display controller 22a are interconnected by a control bus 24 and a data bus 25 and are controlled by commands from the system control unit 11.

[0025] <Schematic configuration of the imaging device 5 shown in FIG. 1> FIG. 2 is a plan schematic view showing the schematic configuration of the imaging device 5 shown in FIG. 1. The imaging device 5 has an imaging surface 60 in which a plurality of pixels 61 are two-dimensionally arranged in a row direction X and a column direction Y orthogonal thereto. Among the plurality of pixels 61, there are a distance measurement pixel 61b that receives one of a pair of light beams that have passed through two different portions arranged in the row direction X of the pupil region of the imaging optical system and detects a signal corresponding to the received light amount, a distance measurement pixel 61c that receives the other of the pair of light beams and detects a signal corresponding to the received light amount, and a normal pixel 61a that receives both of the pair of light beams and detects a signal corresponding to the received light amount.

[0026] In the example of FIG. 2, on the imaging surface 60, there are a pixel line 62 in which a plurality of normal pixels 61a are arranged in the row direction X, and a pixel line 63 in which the distance measurement pixel 61b and the distance measurement pixel 61c are alternately arranged in the row direction X. The pixel line 63 only needs to include a plurality of pairs of the distance measurement pixel 61b and the distance measurement pixel 61c, and may include a normal pixel 61a in addition to this pair. Hereinafter, when not distinguishing between the pixel line 62 and the pixel line 63, it is simply referred to as a pixel line. The imaging device 5 further includes a drive circuit 64 that drives the pixels 61 arranged on the imaging surface 60, and a signal processing circuit 65 that processes pixel signals read from each pixel 61 of each pixel line arranged on the imaging surface 60 to a signal line.

[0027] Hereinafter, in FIG. 2, the end on one end side (the upper side in the figure) in the column direction Y of the imaging surface 60 is referred to as the upper end, and the end on the other end side (the lower side in the figure) in the column direction Y of the imaging surface 60 is referred to as the lower end.

[0028] Based on the signal from the pixel element driving unit 10, the driving circuit 64 drives each pixel line independently, and performs reset (discharge of the charge accumulated in the photoelectric conversion element) of each pixel 61 included in each pixel line, and reads out the pixel signal corresponding to the charge accumulated in the photoelectric conversion element of each pixel 61 to the signal line, etc.

[0029] The signal processing circuit 65 performs correlated double sampling processing on the pixel signal read out from each pixel 61 of the pixel line to the signal line, converts the pixel signal after the correlated double sampling processing into a digital signal, and outputs it to the data bus 25 (see FIG. 1). The signal processing circuit 65 is controlled by the pixel element driving unit 10.

[0030] The digital signal processing unit 17 performs signal processing such as demosaicing processing and gamma correction processing on the pixel signal group output from the pixel element 5 to the data bus 25 to generate imaging image data.

[0031] The imaging device 100 is equipped with a continuous shooting mode in which a plurality of imaging image data are continuously generated in response to one imaging instruction and recorded on the recording medium 21.

[0032] In the continuous shooting mode, the system control unit 11 drives the pixel element 5 by the rolling shutter method by the pixel element driving unit 10 to image the subject. The driving in the rolling shutter method includes rolling reset driving and rolling readout driving. The rolling reset driving is a driving that sequentially performs the process of resetting each pixel 61 of the pixel line and starting the exposure of each pixel 61 while changing the pixel line. The rolling readout driving is a driving that sequentially reads out signals from each pixel 61 of the exposed pixel line and ends the exposure of that pixel line while changing the pixel line.

[0033] In the continuous shooting mode, when the system control unit 11 receives a shooting instruction, it continuously performs recording imaging control for recording captured image data, displaying a live view image on the display surface 22b, and outputting a pixel signal for distance measurement from the imaging device 5. Further, the system control unit 11 performs at least once display imaging control for displaying a live view image on the display surface 22b and outputting a pixel signal for distance measurement from the imaging device 5 between each of the multiple recording imaging controls.

[0034] The distance measurement is, for example, distance measurement using the phase difference method used in phase difference AF (Auto Focus). For example, the distance measurement performs a correlation operation between a first pixel signal group output from each distance measurement pixel 61b included in the same pixel line 63 and a second pixel signal group output from each distance measurement pixel 61c, and based on the result of the correlation operation, derives the driving amount of the focus lens necessary to focus on the target subject.

[0035] The correlation operation is a process of calculating the area S[d] surrounded by two data waveforms when the data waveform composed of the first pixel signal group and the data waveform composed of the second pixel signal group are shifted by a shift amount d, by changing the shift amount d with a plurality of values.

[0036] FIG. 3 is a diagram showing an example of a state where the subject has moved out of the AF area. FIG. 4 is a diagram showing an example of a state where an obstacle has entered in front of the subject within the AF area. In FIGS. 3 and 4, the image 300 is a captured image (for example, a through image) obtained by the imaging device 100.

[0037] In the examples of FIGS. 3 and 4, the subject 301 and the background 302 are reflected in the image 300. The subject 301 is the subject of imaging by the imaging device 100, and in the examples of FIGS. 3 and 4, it is a running person. The background 302 is the background (for example, sky, building, etc.) on the back side of the subject 301 as viewed from the imaging device 100.

[0038] In addition, a frame line indicating the currently effective AF area 303 among the AF areas that can be used by the imaging device 100 is superimposed on the image 300. In the examples of FIGS. 3 and 4, the AF area 303 is one AF area located at the center of the imaging element 5.

[0039] For example, when trying to image a moving subject 301 with the imaging device 100, if the subject 301 is moving horizontally as viewed from the imaging device 100, as shown in FIG. 3, the subject 301 may move out of the AF area 303. In this case, since distance measurement of the background 302 is performed in the AF area 303, if the focus position is set using the distance measurement result of the AF area 303 as it is, the background 302 will be in focus, resulting in a so-called back-focus state.

[0040] Alternatively, as shown in FIG. 4, an obstacle 401 may enter in front of the subject 301 in the AF area 303. In this case, since distance measurement of the obstacle 401 is performed in the AF area 303, if the focus position is set using the distance measurement result of the AF area 303 as it is, the obstacle 401 in front of the subject 301 will be in focus, resulting in a so-called front-focus state.

[0041] <Distance measurement results continuously over time by the imaging device 100> FIG. 5 is a diagram showing an example of distance measurement results continuously over time by the imaging device 100. In FIG. 5, the horizontal axis represents time. The times t1, t2,... on the horizontal axis are the times of consecutive imaging frames. The vertical axis represents the distance from the imaging device 100, and the higher up, the closer it is to the imaging device 100.

[0042] The change in subject position 510 is the change in the actual distance of the subject 301 from the imaging device 100 over time. In the example of FIG. 5, the subject 301 initially has a constant distance from the imaging device 100, and then approaches the imaging device 100 while gradually accelerating.

[0043] The distance measurement results p1 to p11 are the results of distance measurement performed using the AF area 303 at times t1 to t11 respectively. In the distance measurement results p1, p3, p5 to p8, p10, p11, the distance of the subject 301 is correctly measured. In the distance measurement result p2, as shown in FIG. 3, the subject 301 has moved out of the AF area 303, and the distance of the background 302 is measured. In the distance measurement result p4, as shown in FIG. 4, an obstacle 401 has entered in front of the subject 301 in the AF area 303, and the distance of the obstacle 401 is measured.

[0044] The system control unit 11 has a subject holding function for avoiding, for example, setting a focus position that focuses on the background 302 based on the distance measurement result p2, or setting a focus position that focuses on the obstacle 401 based on the distance measurement result p4. Specifically, the system control unit 11 sets a range in which the focus position can be set with reference to the current focus position, and performs control not to set a focus position outside the settable range.

[0045] Here, if the settable range is too wide, the distance measurement result p2 of the background 302 or the distance measurement result p4 of the obstacle 401 will be included in the settable range, and the background 302 or the obstacle 401 will be in focus. On the other hand, if the settable range is too narrow, there will be a problem that when the user intentionally moves the AF area 303 from the subject 301 to another subject, the subject 301 remains in focus, or due to sudden acceleration of the subject 301, etc., the distance measurement result of the subject 301 goes out of the settable range and the focus does not follow the subject 301. The system control unit 11 solves these problems by dynamically changing the settable range.

[0046] <Focus setting process by the system control unit 11> FIG. 6 is a flowchart showing an example of the focus setting process by the system control unit 11. The system control unit 11 sets the focus position of imaging by the imaging unit 50 for each frame of consecutive imaging by, for example, the process shown in FIG. 6.

[0047] Here, during the process of FIG. 6, the most recent past frame is set as the second frame, the frame before the second frame (for example, the immediately previous frame) is set as the first frame, and the frame after the second frame (for example, the immediately following frame) is set as the third frame. That is, in time series, they are acquired by the imaging unit 50 in the order of the first frame, the second frame, and the third frame. In this case, the system control unit 11 sets the focus position in the third frame by the process of FIG. 6.

[0048] First, the system control unit 11 acquires the distance measurement results of the first frame and the second frame based on the image data of the past first frame and second frame (step S61). The distance measurement results of the first frame and the second frame are, for example, those acquired based on the image data of the first frame and the second frame by distance measurement using the phase difference method as described above.

[0049] Next, the system control unit 11 determines the state of the subject 301, etc., and sets a settable range within which the focus position in the third frame can be set based on the determination result (step S62). The setting of the settable range in step S62 will be described later (see, for example, FIG. 7).

[0050] Next, the system control unit 11 tentatively calculates the focus position in the third frame based on the distance measurement results of the first frame and the second frame acquired in step S61 (step S63). In step S63, for example, the system control unit 11 predicts the distance measurement result of the third frame by linear prediction or the like based on the distance measurement results of the first frame and the second frame, and calculates the focus position corresponding to the predicted distance measurement result. The focus position tentatively calculated in step S63 is the focus position predicted to be in focus on the subject 301, and is an example of distance information regarding the distance between the imaging device 100 and the subject 301.

[0051] Next, the system control unit 11 determines whether the focus position temporarily calculated in step S63 is included in the settable range set in step S62 (step S64). If the temporarily calculated focus position is included in the settable range (step S64: Yes), the system control unit 11 sets the temporarily calculated focus position as the target focus position in the third frame (step S65).

[0052] In step S64, if the temporarily calculated focus position is not included in the settable range (step S64: No), the system control unit 11 sets the target focus position in the third frame within the settable range (step S66). For example, the system control unit 11 sets the target focus position set for the immediately preceding second frame or first frame as the target focus position in the third frame. Alternatively, the system control unit 11 may set the focus position corresponding to the ranging result predicted by linear prediction or the like with reference to frames before the first frame and the second frame as the target focus position in the third frame.

[0053] Next, the system control unit 11 controls the lens device 40 so that the focus position of the imaging lens 1 at the time of imaging the third frame becomes the target focus position set in step S65 or step S66 (step S67), and ends a series of processes. The control in step S67 is performed, for example, by the system control unit 11 outputting a control signal to the lens control unit 4.

[0054] <Setting of the settable range by the system control unit 11> FIG. 7 is a flowchart showing an example of setting the settable range by the system control unit 11. In step S62 shown in FIG. 6, the system control unit 11 sets the settable range by, for example, the process shown in FIG. 7.

[0055] First, the system control unit 11 calculates the current depth magnification (step S701). The depth magnification is the ratio of the change amount between frames of the distance between the imaging device 100 and the subject 301 to the depth of field, and is an example of movement distance information that is information regarding the change amount of the distance between the imaging device 100 and the subject 301.

[0056] For example, the system control unit 11 calculates the current depth magnification by the following formula (1). In the following formula (1), the AF drive amount is the change amount (absolute value) between frames of the distance between the imaging device 100 and the subject 301. Specifically, the AF drive amount is a value obtained by converting the difference between the distance measurement result in the first frame and the focus position in the second frame into the drive amount of the focus lens of the imaging lens 1. The depth of field is calculated based on the aperture value (F value) of the aperture 2, the focal length of the imaging lens 1, and the distance between the imaging device 100 and the subject 301. The distance between the imaging device 100 and the subject 301 is, for example, the focus position in the second frame.

[0057] Depth magnification = AF drive amount ÷ Depth of field...(1)

[0058] Next, the system control unit 11 determines whether the depth magnification calculated in step S701 is continuously less than a predetermined stop determination threshold value (step S702). For example, the system control unit 11 acquires N depth magnifications calculated in step S701 in the recent past, including the depth magnification calculated in the most recent step S701, and makes the determination in step S702 based on whether each of the acquired N depth magnifications is less than the stop determination threshold value. N is a natural number of 3 or more. However, in order to distinguish from the state where the subject is changed, which will be described later, N in step S702 is set to a sufficiently large number (for example, 4 or more).

[0059] In step S702, when the depth magnification is continuously less than the first threshold value (step S702: Yes), the system control unit 11 determines that the state of the subject 301 is the "stopped" state (step S703). The "stopped" state means that the distance between the imaging device 100 and the subject 301 is substantially constant, such as when the subject 301 stops at one place or when the subject 301 moves on a circumference centered on the imaging device 100.

[0060] In this case, the system control unit 11 sets the settable range in the third frame to the minimum value (step S704) and ends a series of processes. Setting the settable range in the third frame to the minimum value means, for example, setting a settable range with a predetermined minimum width centered on the focus position in the immediately preceding second frame as the settable range in the third frame. The minimum width is a width greater than 0.

[0061] In step S702, when the depth magnification is not continuously less than the first threshold value (step S702: No), the system control unit 11 determines whether the depth magnification is continuously increasing (step S705). For example, the system control unit 11 acquires N depth magnifications calculated in step S701 in the most recent past, including the depth magnification calculated in the most recent step S701, and makes the determination in step S705 based on whether the acquired N depth magnifications are getting higher as they are newer.

[0062] In step S705, when the depth magnification is continuously increasing (step S705: Yes), the system control unit 11 determines that the state of the subject 301 is the "accelerated" state (step S706). The "accelerated" state means that the amount of change in the distance between the imaging device 100 and the subject 301 is increasing, such as when the subject 301 starts running so as to approach the imaging device 100 or move away from the imaging device 100.

[0063] In this case, the system control unit 11 expands the settable range in the third frame (step S707) and ends the series of processes. Expanding the settable range in the third frame means, for example, setting, as the settable range in the third frame, a settable range wider than the current settable range (for example, the settable range set for the second frame) centered on the focus position in the immediately preceding second frame.

[0064] In step S705, when the depth magnification has not been continuously increasing (step S705: No), the system control unit 11 determines whether the depth magnification has been continuously decreasing (step S708). For example, the system control unit 11 acquires N depth magnifications calculated in step S701 in the most recent past, including the depth magnification calculated in the most recent step S701, and makes the determination in step S708 based on whether the acquired N depth magnifications are lower as they are newer.

[0065] In step S708, when the depth magnification has been continuously decreasing (step S708: Yes), the system control unit 11 determines that the state of the subject 301 is the "deceleration" state (step S709). The "deceleration" state is a state in which the amount of change in the distance between the imaging device 100 and the subject 301 is decreasing, such as a state in which the subject 301 that was running towards or away from the imaging device 100 has started walking.

[0066] In this case, the system control unit 11 sets the settable range in the third frame to the default value (step S710) and ends the series of processes. Setting the settable range in the third frame to the default value means, for example, setting, as the settable range in the third frame, a settable range with a predetermined default width centered on the focus position in the immediately preceding second frame. The default width of the settable range is wider than the minimum width of the settable range described above.

[0067] In step S708, when the depth magnification is not continuously decreasing (step S708: No), the system control unit 11 determines whether the depth magnification is continuously constant (step S711). For example, the system control unit 11 acquires N depth magnifications calculated in step S701 in the most recent past, including the depth magnification calculated in the most recent step S701, and makes the determination in step S705 based on whether the difference between each of the temporally consecutive depth magnifications is equal to or less than a predetermined value.

[0068] In step S711, when the depth magnification is continuously constant (step S711: Yes), the system control unit 11 determines that the state of the subject 301 is in a "constant speed" state (step S712). The "constant speed" state is a state in which the amount of change in the distance between the imaging device 100 and the subject 301 is constant, such as a state in which the subject 301 is running or walking so as to approach the imaging device 100 or move away from the imaging device 100. In this case, the system control unit 11 sets the configurable range in the third frame to the default value (step S713) and ends the series of processes.

[0069] In step S711, when the depth magnification is not continuously constant (step S711: No), the system control unit 11 determines whether the distance measurement result has changed in the reverse direction (step S714). For example, the system control unit 11 acquires three distance measurement results within a certain period in the past going back from the present. Assuming these three distance measurement results are the first distance measurement result, the second distance measurement result, and the third distance measurement result in order from the oldest, the system control unit 11 determines that the distance measurement result has changed in the reverse direction when the second distance measurement result has increased by a predetermined value or more from the first distance measurement result and the third distance measurement result has decreased by a predetermined value or more from the second distance measurement result, or when the second distance measurement result has decreased by a predetermined value or more from the first distance measurement result and the third distance measurement result has increased by a predetermined value or more from the second distance measurement result.

[0070] In step S714, when the distance measurement result changes in the reverse direction (step S714: Yes), the system control unit 11 determines that it is in the "background distance measurement" state or the "obstacle distance measurement" state (step S715). In this case, the system control unit 11 sets the configurable range in the third frame to the default value (step S716) and ends the series of processes.

[0071] In step S715, when the second distance measurement result increases by a predetermined value or more from the first distance measurement result and the third distance measurement result decreases by a predetermined value or more from the second distance measurement result, the system control unit 11 determines that it is in the "background distance measurement" state. The "background distance measurement" state is a state where, for example, the subject 301 has moved out of the AF area 303, and the distance measurement of the background 302 instead of the subject 301 has been performed, but then the subject 301 has re-entered the AF area 303 and the distance measurement of the subject 301 is being performed.

[0072] Also, in step S715, when the second distance measurement result decreases by a predetermined value or more from the first distance measurement result and the third distance measurement result increases by a predetermined value or more from the second distance measurement result, the system control unit 11 determines that it is in the "obstacle distance measurement" state. The "obstacle distance measurement" state is a state where, for example, an obstacle 401 in front of the subject 301 as seen from the imaging device 100 has entered the AF area 303, and the distance measurement of the obstacle 401 instead of the subject 301 has been performed, but then the obstacle 401 has moved out of the AF area 303 and the distance measurement of the subject 301 is being performed again.

[0073] In step S714, if the distance measurement result has not changed in the reverse direction (step S714: No), the system control unit 11 determines whether the distance measurement result has been constant continuously after the change (step S717). For example, the system control unit 11 acquires three distance measurement results within a certain past period going back from the present. Assuming these four distance measurement results are the first distance measurement result, the second distance measurement result, the third distance measurement result, and the fourth distance measurement result in chronological order from the oldest, the system control unit 11 determines that the distance measurement result has been constant continuously after the change when the difference between the first distance measurement result and the second distance measurement result is greater than or equal to a predetermined value, and the difference between the second distance measurement result and the third distance measurement result is less than or equal to the predetermined value, and the difference between the third distance measurement result and the fourth distance measurement result is less than or equal to the predetermined value.

[0074] In step S717, if the distance measurement result has been constant continuously after the change (step S717: Yes), the system control unit 11 determines that the user has intentionally changed the subject from subject 301 to another subject (step S718). In this case, the system control unit 11 expands the settable range in the third frame (step S719) and ends the series of processes.

[0075] In step S717, if the distance measurement result has not been constant continuously after the change (step S717: No), the system control unit 11 sets the settable range in the third frame to the default value (step S720) and ends the series of processes.

[0076] Note that in steps S710, S713, S716, and S720, since subject 301 is in a moving state, the settable range in the immediately preceding (e.g., the immediately preceding second frame) is the default value or an expanded value from the default value. Therefore, by setting the settable range to the default value in steps S710, S713, S716, and S720, the settable range will be maintained or reduced.

[0077] As shown in FIG. 7, when the depth magnification (movement distance information) in a plurality of frames obtained by the imaging unit 50 is continuously less than a predetermined value, the system control unit 11 determines that it is in the "stop" state and sets the settable range to the minimum value. Thereby, when the subject 301 moves out of the AF area 303 and the distance measurement of the background 302 is performed, or when the obstacle 401 enters the AF area 303 and the distance measurement of the obstacle 401 is performed, it can be accurately detected. For this reason, it is possible to suppress the background 302 and the obstacle 401 from being in focus, and improve the followability of the focus position with respect to the subject 301.

[0078] Further, when the depth magnification (movement distance information) in a plurality of frames obtained by the imaging unit 50 is continuously increasing, the system control unit 11 determines that it is in the "acceleration" state and expands the settable range. Thereby, in a situation where the movement of the subject 301 fluctuates and the distance between the imaging device 100 and the subject 301 is difficult to predict, the settable range is expanded, and it is possible to suppress the focus position temporarily calculated based on the distance measurement results of the first frame and the second frame from deviating from the settable range due to the acceleration of the subject 301. For this reason, it is possible to improve the followability of the focus position with respect to the subject 301.

[0079] Further, when the depth magnification (movement distance information) in a plurality of frames is continuously decreasing, the system control unit 11 determines that it is in the "deceleration" state and maintains or reduces the settable range. Thereby, in a situation where the subject 301 is approaching a stop state, the settable range is maintained or reduced, and when the subject 301 moves out of the AF area 303 and the distance measurement of the background 302 is performed, or when the obstacle 401 enters the AF area 303 and the distance measurement of the obstacle 401 is performed, it can be accurately detected. For this reason, it is possible to suppress the background 302 and the obstacle 401 from being in focus, and improve the followability of the focus position with respect to the subject 301.

[0080] Further, when the difference in depth magnification (movement distance information) in a plurality of frames is within a predetermined range, the system control unit 11 determines that it is in a "constant speed" state and maintains or reduces the settable range. As a result, in a situation where the distance between the imaging device 100 and the subject 301 is easy to predict, the settable range is maintained or reduced. When the subject 301 moves out of the AF area 303 and the distance measurement of the background 302 is performed, or when an obstacle 401 enters the AF area 303 and the distance measurement of the obstacle 401 is performed, it is possible to accurately detect such cases. Therefore, it is possible to suppress the background 302 and the obstacle 401 from being out of focus and improve the followability of the focus position with respect to the subject 301.

[0081] Further, when the change direction of the distance measurement results (distance information) in a plurality of frames switches to the reverse direction, the system control unit 11 determines that it is in a "background distance measurement" state or an "obstacle distance measurement" state and maintains or reduces the settable range. As a result, in a situation where the distance measurement of the background 302 or the obstacle 401 is temporarily performed but the distance measurement of the subject 301 is currently being performed, the settable range is maintained or reduced. When the subject 301 moves out of the AF area 303 and the distance measurement of the background 302 is performed, or when an obstacle 401 enters the AF area 303 and the distance measurement of the obstacle 401 is performed, it is possible to accurately detect such cases. Therefore, it is possible to suppress the background 302 and the obstacle 401 from being out of focus and improve the followability of the focus position with respect to the subject 301.

[0082] Further, when the distance measurement results (distance information) in a plurality of frames change by a predetermined value or more and then remain within a predetermined range continuously, the system control unit 11 determines that the subject has been intentionally changed and expands the settable range. As a result, in a situation where the subject has been changed and it is necessary to focus on the new subject, the settable range is expanded, and it is possible to suppress continuing to focus on the subject 301 before the change even though the subject has been changed.

[0083] Also, in the process shown in FIG. 7, the system control unit 11 may not use the distance measurement results that are outside the settable range for determining the state of the subject 301 and the like. Thereby, for example, it is possible to suppress erroneously determining the state of the subject 301 using the distance measurement results of the background 302 or the obstacle 401.

[0084] Note that steps S703, S706, S709, S712, S715, and S718 are processes for the system control unit 11 to recognize each state such as the "stop" state. However, the system control unit 11 may shift to any of steps S704, S707, S710, S713, S716, and S719 according to the determination results of steps S702, S705, S708, S711, S714, and S717 and set the settable range. Therefore, it may be a process in which steps S703, S706, S709, S712, S715, and S718 are omitted.

[0085] <Minimization of the Settable Range in the "Stop" State> FIG. 8 is a diagram showing an example of the minimization of the settable range in the "stop" state. The settable range 810 is the settable range set by step S62 in FIG. 6 (specifically, the process in FIG. 7). The settable range 810 is defined by, for example, a lower limit value 811 and an upper limit value 812. That is, the range from the lower limit value 811 or more and the upper limit value 812 or less is the settable range 810.

[0086] In the example of FIG. 9, the distance between the imaging device 100 and the subject 301 is constant. In this case, the system control unit 11 determines that the state of the subject 301 is the "stop" state and sets the settable range 810 to the minimum value. Thereby, it is possible to suppress the background 302 or the obstacle 401 from being in focus and improve the followability of the focus position with respect to the subject 301.

[0087] <Expansion of the Settable Range in the "Acceleration" State> FIG. 9 is a diagram showing an example of expanding the settable range in the "acceleration" state. In the example of FIG. 9, the subject 301 is approaching the imaging device 100 while accelerating. In this case, the system control unit 11 determines that the state of the subject 301 is in the "acceleration" state and expands the settable range 810. Thereby, it is possible to suppress the distance measurement result of the subject 301 approaching while accelerating from deviating from the settable range 810, and improve the followability of the focus position with respect to the subject 301.

[0088] As described above, based on the image data of the first frame and the image data of the second frame among the plurality of frames obtained by the imaging device 100 (imaging unit 50), the system control unit 11 predicts (temporarily calculates) the focus position in the third frame after the first frame and the second frame (see, for example, FIG. 6).

[0089] Also, the system control unit 11 sets a settable range, which is the range of the focus position that can be set for the third frame, based on the state of the subject 301. Specifically, the system control unit 11 determines the state of the subject 301 based on the depth magnification (movement distance information) in the plurality of frames (see, for example, FIG. 7).

[0090] Then, the system control unit 11 sets the focus position for imaging in the third frame based on the predicted focus position and the set settable range (see, for example, FIG. 6).

[0091] <Control of Focus Position When Determined to be in the "Obstacle Distance Measurement" State> FIG. 10 is a diagram showing an example of the control of the focus position when it is determined to be in the "obstacle distance measurement" state. For example, in a situation where the distance between the imaging device 100 and the subject 301 is changing, as in the example shown in FIG. 10, temporarily, an obstacle 401 enters in front of the subject 301 in the AF area 303, and the distance measurement of the obstacle 401 is performed. In this case, the system control unit 11 determines that it is in the "obstacle distance measurement" state based on, for example, the distance measurement results p3 to p5.

[0092] At this time, if the system control unit 11 maintains the focus position at the time of the distance measurement result p3 before the subject 301 is hidden by the obstacle 401, since the subject 301 moves while being hidden by the obstacle 401, when the obstacle 401 between the imaging device 100 and the subject 301 disappears, it becomes impossible to focus on the subject 301.

[0093] On the other hand, the system control unit 11 predicts the movement of the subject 301 based on the distance measurement results p1 to p3 before the distance measurement result p4 which is the distance measurement result of the obstacle 401, and based on the prediction result, the system control unit 11 moves the focus position even while the subject 301 is hidden by the obstacle 401, so that when the obstacle 401 between the imaging device 100 and the subject 301 disappears, the focus can be in a state close to the subject 301.

[0094] <False determination when the distance between the obstacle 401 and the subject 301 is short> FIG. 11 is a diagram showing an example of false determination when the distance between the obstacle 401 and the subject 301 is short. For example, as in the example of FIG. 11, when the distance between the obstacle 401 and the subject 301 is short, if the normal settable range 810 is set, in a situation where the obstacle 401 and the subject 301 intersect, it is likely to be falsely determined as the "acceleration" state at the timing when the front obstacle 401 enters the AF area 303. That is, it is likely to be falsely determined that the position of the subject 301 changes as shown by the dotted arrow in FIG. 11. The process for suppressing this false determination will be described with reference to FIG. 12.

[0095] <Another example of setting the settable range 810 by the system control unit 11> FIG. 12 is a flowchart showing another example of setting the settable range 810 by the system control unit 11. In step S62 shown in FIG. 6, the system control unit 11 reduces the settable range by the process shown in FIG. 12, for example.

[0096] Steps S1201 to S1220 shown in FIG. 12 are the same as steps S701 to S720 shown in FIG. 7. However, in step S1213 when it is determined that the state is "constant speed", the system control unit 11 reduces the settable range 810 based on the depth width (step S1213). The depth width is the fluctuation width of the depth magnification (driving depth width).

[0097] For example, the system control unit 11 acquires N depth magnifications calculated in step S1201 in the most recent past, including the depth magnification calculated in the most recent step S1201, and calculates the fluctuation width (for example, the difference between the minimum value and the maximum value) of the N depth magnifications as the depth width. Then, the smaller the calculated depth width, that is, the closer the subject 301 is to a complete constant speed state, the narrower the system control unit 11 makes the settable range 810. The reduction of the settable range 810 by the process of FIG. 12 will be described with reference to FIG. 13.

[0098] <Reduction of the settable range 810 based on the depth width> FIG. 13 is a diagram showing an example of the reduction of the settable range 810 based on the depth width. In the same situation as FIG. 10, the system control unit 11 calculates the depth width based on, for example, the distance measurement results p1 to p3, and reduces the settable range 810 based on the calculated depth width. As a result, the distance measurement result p4, which is the distance measurement result of the obstacle 401, falls outside the settable range 810 and is not used for determining the state of the subject 301 or the like. Thereby, for example, in the situation shown in FIG. 11, it is possible to suppress misjudgment as an "acceleration" state.

[0099] <Distance measurement results by a plurality of AF areas of the imaging device 100> FIG. 14 is a diagram showing an example of the distance measurement results by a plurality of AF areas of the imaging device 100. In the above, the process based on the distance measurement result based on one AF area 303 has been described, but the imaging device 100 may have a plurality of AF areas. In this case, a plurality of distance measurement results can be obtained for each frame.

[0100] In the example of FIG. 14, ranging results p11, p12, p13 are obtained at time t1, ranging results p21, p22 are obtained at time t2, ranging results p31, p32, p33 are obtained at time t3, and ranging results p41, p42, p43 are obtained at time t4.

[0101] For example, in predictive AF that captures an image while predicting the future movement of a moving subject, there is a problem of determining which subject is the main subject. To address such a problem, a general digital camera has a mechanism that allows a user to custom-select which AF area, such as foreground priority, center priority, or auto, to use preferentially. As shown in FIGS. 15 to 20 below, the imaging device 100 has a mechanism for automatically performing such main subject selection to some extent.

[0102] FIG. 15 is a diagram showing an example of a plurality of AF areas included in the imaging device 100. For example, the imaging device 100 may have AF areas 151 to 159. The AF areas 151 to 159 are arranged in a 3×3 matrix, and the AF area 155 is the central area. The AF area 155 is an example of a first area near the center among the AF areas 151 to 159. The AF areas 151 to 154 and 156 to 159 are examples of a second area around the first area.

[0103] The subject 150 is a subject to be imaged by the imaging device 100 and is a flying bird in the example of FIG. 15. Since the distance between the bird, which is the subject 150, and the imaging device 100 varies depending on the part, the ranging results of each of the AF areas 151 to 159 are different from each other.

[0104] The system control unit 11 sets some of the AF areas 151 to 159 as priority areas, and performs the above-described various processes (for example, the process in FIG. 6) using the distance measurement results of the set priority areas as the distance measurement results for that frame. Alternatively, the system control unit 11 performs the above-described various processes (for example, the process in FIG. 6) using the distance measurement result calculated by preferentially considering the distance measurement results of the priority areas as the distance measurement result for that frame.

[0105] In this case, among the distance measurement results of each of the AF areas 151 to 159, it is conceivable to use the distance measurement result with the smallest distance from the imaging device 100. However, if there are tree branches between the subject 150 and the imaging device 100 and any of those tree branches enters one of the AF areas 151 to 159, the focus will be on that tree branch. Also, for example, when the subject 150 is a person standing on the ground, if the ground between the subject 150 and the imaging device 100 enters one of the AF areas 151 to 159, the focus will be on that ground.

[0106] Also, among the distance measurement results of each of the AF areas 151 to 159, it is conceivable to use the central distance or the like of the distance band in which many distance measurement results are distributed, but there are problems such as front focus or back focus.

[0107] <Priority Area Setting Process by System Control Unit 11> FIG. 16 is a flowchart showing an example of the priority area setting process by the system control unit 11. The system control unit 11 sets a priority area from among the AF areas 151 to 159 for each frame of consecutive imaging, for example, by the process shown in FIG. 16.

[0108] First, the system control unit 11 measures the distances of AF areas 151 to 159 (step S161). Next, the system control unit 11 determines whether the distance measurement result of the AF area 155, which is the central area, is the closest based on the distance measurement results of the AF areas 151 to 159 obtained in step S161 (step S162). Specifically, the system control unit 11 determines whether the distance measurement result of the AF area 155 is the closest by checking whether the distance measurement result of the AF area 155 indicates the closest distance among the distance measurement results of the AF areas 151 to 159.

[0109] In step S162, if the distance measurement result of the AF area 155 is the closest (step S162: Yes), the system control unit 11 sets the AF area 155 as the priority area (step S163) and ends the series of processes.

[0110] In step S162, if the distance measurement result of the AF area 155 is not the closest (step S162: No), the system control unit 11 determines whether the AF area 155, which is the central area, is the rear pin and whether there is a surrounding area among the AF areas 151 to 154 and 156 to 159 whose distance measurement result is close to the previous distance measurement result (for example, the distance measurement result of the previous priority area) (step S164). The current distance measurement result is the distance measurement result of the second frame (for example, the most recent frame in the past). The previous distance measurement result is the distance measurement result of the first frame (for example, the frame immediately before the second frame).

[0111] In step S164, for example, the system control unit 11 determines whether the AF area 155 is the rear pin by checking whether the distance measurement result of the AF area 155 is greater than a predetermined value based on the reference value based on the distance measurement results of the AF areas 151 to 154 and 156 to 159. The reference value based on the distance measurement results of the AF areas 151 to 154 and 156 to 159 may be, for example, the average value of the distance measurement results of the AF areas 151 to 154 and 156 to 159, or the minimum value of the distance measurement results of the AF areas 151 to 154 and 156 to 159.

[0112] Further, for each of the AF areas 151 to 154 and 156 to 159, the system control unit 11 determines whether the current distance measurement result is close to the previous distance measurement result (for example, the distance measurement result of the previous priority area) based on whether the difference between the current distance measurement result and the previous distance measurement result is less than a predetermined value.

[0113] In step S164, when the AF area 155 is the rear pin and there is a surrounding area where a distance measurement result close to the previous distance measurement result is obtained, the corresponding surrounding area is set as the priority area (step S165), and a series of processes is terminated. Thereby, when the AF area 155 is the rear pin and cannot be used, it is possible to set, as the priority area, a surrounding area where a distance measurement result close to the previous distance measurement result is obtained, that is, a surrounding area where the same part as the previous priority area in the subject 150 is likely to be included. When there are a plurality of surrounding areas where distance measurement results close to the previous distance measurement result are obtained, the system control unit 11 sets, as the priority area, the surrounding area where the distance measurement result closest to the previous distance measurement result is obtained.

[0114] In step S164, when the AF area 155 is not the rear pin or there is no surrounding area where a distance measurement result close to the previous distance measurement result is obtained (step S164: Yes), the system control unit 11 determines whether the current distance measurement result of the AF area 155, which is the central area, is close to the previous distance measurement result (for example, the distance measurement result of the previous priority area), and whether there is a surrounding area among the AF areas 151 to 154 and 156 to 159 whose distance measurement result is close to that of the AF area 155 (step S166).

[0115] In step S166, for example, the system control unit 11 determines whether the current distance measurement result of the AF area 155 is close to the previous distance measurement result (for example, the distance measurement result of the previous priority area) based on whether the difference between the previous distance measurement result (for example, the distance measurement result of the previous priority area) and the current distance measurement result of the AF area 155 is less than a predetermined value. Further, the system control unit 11 uses each of the AF areas 151 to 154 and 156 to 159 as a target peripheral area, and determines whether the distance measurement result is close to the AF area 155 based on whether the difference between the current distance measurement result of the AF area 155 and the current distance measurement result of the target peripheral area is less than a predetermined value.

[0116] In step S166, when the current distance measurement result of the AF area 155 is close to the previous distance measurement result and there is a peripheral area where the distance measurement result is close to the AF area 155 (step S166: Yes), the system control unit 11 sets the corresponding peripheral area as the priority area (step S167) and ends the series of processes. Thereby, when the AF area 155 is not the closest but is close to the previous distance measurement result, a peripheral area where the distance measurement result is close to the AF area 155 can be set as the priority area. When there are a plurality of peripheral areas where the distance measurement result is close to the AF area 155, the system control unit 11 sets the peripheral area where the distance measurement result is closest to the AF area 155 as the priority area.

[0117] In step S166, when the current distance measurement result of the AF area 155 is not close to the previous distance measurement result or there is no peripheral area where the distance measurement result is close to the AF area 155 (step S166: No), the system control unit 11 sets the AF area 155, which is the central area, as the priority area (step S168) and ends the series of processes.

[0118] For example, in the process shown in FIG. 6, the system control unit 11 uses the distance measurement result of the priority area set according to FIG. 17 among the distance measurement results of each of the AF areas 151 to 159 as the distance measurement result of the target frame. Alternatively, in the process shown in FIG. 6, the system control unit 11 may use, as the distance measurement result of the target frame, a distance measurement result calculated by preferentially considering the distance measurement result of the priority area.

[0119] <Acquisition of ranging results in the priority area> Figures 17 to 20 are diagrams showing an example of the acquisition of ranging results in the priority area. In Figure 17, the ranging result p1 at time t1 is the ranging result of the priority area selected from among the AF areas 151 to 159 by the process of Figure 16 for time t1. The ranging result p2 at time t2 is the ranging result of the priority area selected from among the AF areas 151 to 159 by the process of Figure 16 for time t2. The ranging result p3 at time t3 is the ranging result of the priority area selected from among the AF areas 151 to 159 by the process of Figure 16 for time t3.

[0120] Figures 18 to 20 show an example of the positional relationship between the AF areas 151 to 159 and the subject 150 at times t1 to t3, respectively. In Figures 18 to 20, the AF areas set as the priority area among the AF areas 151 to 159 are indicated by thick lines.

[0121] As shown in Figure 18, at time t1, the portion of the subject 150 closest to the imaging device 100 (for example, the beak of a bird) is within the AF area 155. In this case, in the process of Figure 16, the AF area 155, which is the central area, is determined to be the closest, and the AF area 155 is set as the priority area by step S163. In this case, the ranging result p1 at time t1 in Figure 17 is the ranging result of the AF area 155 at time t1.

[0122] As shown in Figure 19, at time t2, the subject 150 is not within the AF area 155 and the AF area 155 is in a back focus state. Also, assume that the ranging result of the AF area 151 at time t2 and the measurement result at time t1 (the ranging result at time t1 of the AF area 155 set as the priority area at time t1) are close. In this case, in the process of Figure 16, the AF area 151 is set as the priority area by step S165. In this case, the ranging result p1 at time t2 in Figure 17 is the ranging result of the AF area 151 at time t2.

[0123] As shown in FIG. 20, assume that at time t3, the distance measurement result of AF area 155 is close to the measurement result at time t2 (the distance measurement result at time t2 of AF area 151 set as the priority area at time t2). Also assume that the distance measurement result of AF area 158 is close to the distance measurement result of AF area 155. In this case, in the process of FIG. 16, AF area 158 is set as the priority area by step S167. In this case, the distance measurement result p3 at time t3 in FIG. 17 is the distance measurement result of AF area 158 at time t3.

[0124] As shown in FIGS. 14 to 20, the system control unit 11 sets the target focus position of the third frame based on each distance measurement result of the current time (the second frame) by AF areas 151 to 159 and each distance measurement result of the previous time (the first frame) by AF areas 151 to 159. Thereby, using the past time-series distance measurement results, it is possible to set the target focus position of the third frame using an appropriate distance measurement result among the distance measurement results of AF areas 151 to 159. For this reason, the followability of the focus position with respect to the subject 150 can be improved.

[0125] The system control unit 11 sets the target focus position of the third frame based on the comparison between the distance measurement result (distance information) of AF area 155 (the first area near the center) among AF areas 151 to 159 (multiple areas) and the distance measurement results of AF areas 151 to 154, 156 to 159 (the second area at the periphery) among AF areas 151 to 159. Thereby, according to the relationship with the distance measurement result of AF area 155, which is often adjusted to the subject 150, it is possible to set the target focus position of the third frame using an appropriate distance measurement result among the distance measurement results of AF areas 151 to 154, 156 to 159. For this reason, the followability of the focus position with respect to the subject 150 can be improved.

[0126] (Modification Example 1) As an example of the movement distance information, which is information regarding the amount of change in the distance between the imaging device 100 and the subject, the depth magnification calculated by the above formula (1) has been described. However, the movement distance information is not limited to the above depth magnification. For example, the movement distance information may be the AF driving amount in the above formula (1).

[0127] Alternatively, the movement distance information may be the angle formed by the combined vector of the AF driving amount vector and the depth of field vector and the depth of field vector. The AF driving amount vector is a vector indicating the AF driving amount in the above formula (1). The depth of field vector is a vector indicating the depth of field in the above formula (1). This restricts the range in which the movement distance information can be obtained to a certain range, facilitating the design of thresholds and the like when determining the state of the subject 301 and the like.

[0128] (Modification 2) In the above-described embodiment, the case where the phase difference method is used as the distance measurement method (AF method) has been described. However, as the distance measurement method, a configuration using the contrast method used in contrast AF may also be used. Further, as the distance measurement method, a configuration using a hybrid method that combines the phase difference method and the contrast method may also be used.

[0129] (Modification 3) The imaging device of the present invention is not limited to the imaging device 100 mainly used for imaging, and can also be applied to various information terminals having an imaging function, such as smartphones, tablet terminals, and notebook personal computers. Next, the configuration of a smartphone 200, which is another embodiment of the imaging device of the present invention, will be described.

[0130] <Appearance of Smartphone 200> FIG. 21 shows the appearance of the smartphone 200. The smartphone 200 shown in FIG. 21 has a flat plate-shaped housing 201, and is provided with a display input unit 204 in which a display panel 202 as a display unit and an operation panel 203 as an input unit are integrated on one surface of the housing 201.

[0131] The housing 201 also includes a speaker 205, a microphone 206, an operation unit 207, and a camera unit 208. Note that the configuration of the housing 201 is not limited to this. For example, a configuration in which the display unit and the input unit are independent, or a configuration having a folding structure or a slide mechanism can also be adopted.

[0132] <Configuration of the smartphone 200> FIG. 22 is a block diagram showing the configuration of the smartphone 200.

[0133] As shown in FIG. 22, the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a storage unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) reception unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0134] In addition, as the main functions of the smartphone 200, it has a wireless communication function for performing mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0135] The wireless communication unit 210 performs wireless communication with the base station device BS included in the mobile communication network NW according to the instructions of the main control unit 220. Using this wireless communication, transmission and reception of various file data such as voice data and image data, electronic mail data, and reception of web data or streaming data are performed.

[0136] The display input unit 204 is a so-called touch panel that displays images (still images and moving images) or character information, etc. under the control of the main control unit 220 to visually transmit information to the user, and detects user operations on the displayed information. It includes a display panel 202 and an operation panel 203.

[0137] The display panel 202 uses an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), etc. as a display device.

[0138] The operation panel 203 is a device that is placed on the display surface of the display panel 202 so as to be visible, and detects one or more coordinates operated by a user's finger or a stylus. When this device is operated by a user's finger or a stylus, a detection signal generated due to the operation is output to the main control unit 220. Next, the main control unit 220 detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0139] As shown in FIG. 22, the display panel 202 and the operation panel 203 of the smartphone 200 exemplified as one embodiment of the imaging device of the present invention are integrated to form a display input unit 204, but the operation panel 203 is arranged so as to completely cover the display panel 202.

[0140] When such an arrangement is adopted, the operation panel 203 may also have a function of detecting user operations for areas outside the display panel 202. In other words, the operation panel 203 may include a detection area for the overlapping portion overlapping the display panel 202 (hereinafter referred to as the display area) and a detection area for the outer edge portion that does not overlap the other display panel 202 (hereinafter referred to as the non-display area).

[0141] Note that the size of the display area and the size of the display panel 202 may be completely the same, but it is not necessarily required that the two match. Also, the operation panel 203 may include two sensitive areas, an outer edge portion and an inner portion other than that. Furthermore, the width of the outer edge portion is appropriately designed according to the size of the housing 201 and the like.

[0142] Furthermore, as the position detection method adopted in the operation panel 203, there are a matrix switch method, a resistive film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, a capacitance method, etc., and any method can be adopted.

[0143] The communication unit 211 includes a speaker 205 or a microphone 206, converts the user's voice input through the microphone 206 into voice data that can be processed by the main control unit 220, and outputs it to the main control unit 220, or decodes the voice data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0144] Also, as shown in FIG. 21, for example, the speaker 205 can be mounted on the same surface as the surface where the display input unit 204 is provided, and the microphone 206 can be mounted on the side surface of the housing 201.

[0145] The operation unit 207 is a hardware key using a key switch or the like, and receives an instruction from the user. For example, as shown in FIG. 21, the operation unit 207 is mounted on the side surface of the housing 201 of the smartphone 200, and when pressed with a finger or the like, it turns on, and when the finger is released, it becomes an off state by the restoring force of a spring or the like. It is a push-button type switch.

[0146] The storage unit 212 stores the control program and control data of the main control unit 220, application software, address data associating the name or phone number of the communication partner, etc., the data of the sent and received e-mails, the Web data downloaded by Web browsing, the downloaded content data, and also temporarily stores streaming data, etc. The storage unit 212 is composed of an internal storage unit 217 built in the smartphone and an external storage unit 218 having a detachable external memory slot.

[0147] Note that each of the internal storage unit 217 and the external storage unit 218 that constitute the storage unit 212 is realized using a storage medium such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., MicroSD (registered trademark) memory, etc.), a RAM (Random Access Memory), or a ROM (Read Only Memory).

[0148] The external input / output unit 213 serves as an interface with all external devices connected to the smartphone 200 and is for directly or indirectly connecting to other external devices by communication etc. (e.g., Universal Serial Bus (USB), IEEE (Institute of Electrical and Electronics Engineers) 1394, Bluetooth (registered trademark), RFID (RadioFrequency Identification), infrared communication (Infrared Data Association: IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee (registered trademark), etc.) or a network (e.g., Ethernet (registered trademark), wireless LAN (Local Area Network), etc.).

[0149] Examples of external devices connected to the smartphone 200 include wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards connected via a card socket, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video devices connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video devices, wired / wirelessly connected smartphones, wired / wirelessly connected personal computers, wired / wirelessly connected personal computers, earphones, and the like.

[0150] The external input / output unit 213 can transmit data received from such external devices to each component inside the smartphone 200, or can transmit data inside the smartphone 200 to the external devices.

[0151] The GNSS receiver unit 214 receives GNSS signals transmitted from GNSS satellites ST1 to STn according to the instructions of the main control unit 220, executes positioning calculation processing based on the received multiple GNSS signals, and detects the position of the smartphone 200 consisting of latitude, longitude, and altitude. When the GNSS receiver unit 214 can obtain position information from the wireless communication unit 210 or the external input / output unit 213 (for example, wireless LAN), it can also detect the position using the position information.

[0152] The motion sensor unit 215 includes, for example, a three-axis acceleration sensor and the like, and detects the physical movement of the smartphone 200 according to the instructions of the main control unit 220. By detecting the physical movement of the smartphone 200, the moving direction or acceleration of the smartphone 200 is detected. Such detection results are output to the main control unit 220.

[0153] The power supply unit 216 supplies the power stored in a battery (not shown) to each part of the smartphone 200 according to the instructions of the main control unit 220.

[0154] The main control unit 220 includes a microprocessor and operates according to the control program and control data stored in the storage unit 212, and comprehensively controls each part of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. In addition, the main control unit 220 has a mobile communication control function for controlling each part of the communication system and an application processing function to perform voice communication or data communication through the wireless communication unit 210.

[0155] The application processing function is realized by the main control unit 220 operating according to the application software stored in the storage unit 212. Examples of the application processing function include an infrared communication function for controlling the external input / output unit 213 to perform data communication with a counter device, an e-mail function for sending and receiving e-mails, or a web browsing function for browsing web pages.

[0156] In addition, the main control unit 220 has an image processing function such as displaying a video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0157] The image processing function refers to the function in which the main control unit 220 decodes the above-mentioned image data, performs image processing on the decoding result, and displays the image on the display input unit 204.

[0158] Furthermore, the main control unit 220 executes display control for the display panel 202 and operation detection control for detecting user operations through the operation unit 207 and the operation panel 203.

[0159] By executing display control, the main control unit 220 displays software keys such as icons or scroll bars for starting application software, or displays a window for creating an e-mail. Note that the scroll bar is a software key for receiving an instruction to move the display portion of an image that is too large to fit within the display area of the display panel 202.

[0160] Also, by executing operation detection control, the main control unit 220 detects a user operation through the operation unit 207, receives an operation on the icon and an input of a character string to the input field of the window through the operation panel 203, or receives a scroll request for a display image through the scroll bar.

[0161] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is an overlapping portion (display area) that overlaps the display panel 202 or an outer edge portion (non-display area) that does not overlap the other display panel 202, and has a touch panel control function for controlling the sensitive area of the operation panel 203 or the display position of the software key.

[0162] Also, the main control unit 220 can detect a gesture operation on the operation panel 203 and execute a preset function according to the detected gesture operation.

[0163] The gesture operation means an operation of drawing a locus with a finger or the like, designating a plurality of positions simultaneously, or combining these to draw a locus for at least one of the plurality of positions, rather than a conventional simple touch operation.

[0164] The camera unit 208 includes the imaging unit 50 in the imaging apparatus 100 shown in FIG. 1.

[0165] The captured image data generated by the camera unit 208 can be stored in the storage unit 212 or output through the external input / output unit 213 or the wireless communication unit 210.

[0166] In the smartphone 200 shown in FIG. 21, the camera unit 208 is mounted on the same surface as the display input unit 204. However, the mounting position of the camera unit 208 is not limited to this, and it may be mounted on the back surface of the display input unit 204.

[0167] Also, the camera unit 208 can be used for various functions of the smartphone 200. For example, an image acquired by the camera unit 208 can be displayed on the display panel 202, or the image of the camera unit 208 can be used as one of the operation inputs on the operation panel 203.

[0168] Also, when the GNSS receiver unit 214 detects a position, it is also possible to detect the position with reference to an image from the camera unit 208. Furthermore, with reference to an image from the camera unit 208, it is possible to determine the optical axis direction of the camera unit 208 of the smartphone 200 or the current usage environment without using a three-axis acceleration sensor or in combination with a three-axis acceleration sensor. Of course, an image from the camera unit 208 can also be used within application software.

[0169] In addition, position information acquired by the GNSS receiver unit 214, voice information acquired by the microphone 206 (which may be converted into text information by a main control unit or the like), attitude information acquired by the motion sensor unit 215, etc. can be added to still image or moving image data and stored in the storage unit 212, or output through the external input / output unit 213 or the wireless communication unit 210.

[0170] Even in the smartphone 200 having the above-described configuration, similar to the imaging device 100, it is possible to improve the followability of the focus position with respect to the subject.

[0171] As described above, the following matters are disclosed in this specification.

[0172] (1) A control device including a processor and capable of controlling an imaging device, wherein the processor, based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, predicts distance information regarding the distance between the imaging device and the subject in a third frame after the first frame and the second frame, and sets a settable range, which is a range of the focus position of imaging by the imaging device that can be set for the third frame, based on the state of the subject. Control device.

[0173] (2) The control device according to (1), wherein the processor, sets the focus position in the third frame based on the set distance information in the third frame and the settable range. Control device.

[0174] (3) The control device according to (1) or (2), wherein the processor, acquires movement distance information, which is information regarding the amount of change in the distance between the imaging device and the subject, based on the image data of the plurality of frames, and sets the settable range for the third frame based on the movement distance information in the plurality of frames. Control device.

[0175] (4) The control device according to (3), wherein the processor, sets the settable range for the third frame based on the difference in the movement distance information in the plurality of frames. Control device.

[0176] (5) The control device according to (3) or (4), wherein the processor, When the movement distance information in the plurality of frames increases, expand the settable range. Control device.

[0177] (6) The control device according to any one of (3) to (5), wherein the processor When the movement distance information in the plurality of frames decreases, maintain or reduce the settable range. Control device.

[0178] (7) The control device according to any one of (3) to (6), wherein the processor When the difference in the movement distance information in the plurality of frames is within a predetermined range, maintain or reduce the settable range. Control device.

[0179] (8) The control device according to any one of (3) to (7), wherein the processor When the change direction of the distance information in the plurality of frames switches to the reverse direction, maintain or reduce the settable range. Control device.

[0180] (9) The control device according to any one of (3) to (8), wherein the processor When the distance information in the plurality of frames is within a predetermined range after changing by a predetermined value or more, expand the settable range. Control device.

[0181] (10) The control device according to any one of (3) to (9), wherein the movement distance information is the ratio of the amount of change between frames of the distance between the imaging device and the subject to the depth of field of the subject. Control device.

[0182] (11) The control device according to any one of (1) to (10), wherein the processor acquires a plurality of distance information corresponding to a plurality of areas of an image represented by the image data, and predicts the distance information of the third frame based on the plurality of distance information of the first frame and the plurality of distance information of the second frame, Control device.

[0183] (12) The control device according to (11), wherein the processor predicts the distance information of the third frame based on a comparison between the distance information of a first area near the center among the plurality of areas and the distance information of a second area around the first area among the plurality of areas, Control device.

[0184] (13) An imaging device comprising the control device according to any one of (1) to (12), Imaging device.

[0185] (14) A control method by a control device including a processor and capable of controlling an imaging device, wherein the processor, predicts distance information regarding the distance between the imaging device and a subject in a third frame after the first frame and the second frame based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, sets a settable range that is a range of a focus position of imaging by the imaging device that can be set for the third frame based on the state of the subject, Control method.

[0186] (15) A control program for a control device including a processor and capable of controlling an imaging device, wherein the processor, Based on the image data of the first frame and the image data of the second frame among the plurality of frames obtained by the imaging device, predict distance information regarding the distance between the imaging device and the subject in the third frame after the first frame and the second frame. Based on the state of the subject, set a settable range that is the range of the focus position of imaging by the imaging device that can be set for the third frame. A control program for executing processing.

Industrial Applicability

[0187] The present invention is particularly applicable to a digital camera or the like and is highly convenient and effective.

Explanation of Signs

[0188] 1 Imaging lens 4 Lens control unit 5 Image sensor 8 Lens drive unit 9 Drive unit 10 Image sensor drive unit 11 System control unit 14,207 Operation unit 15 Memory control unit 16 Memory 17 Digital signal processing unit 20 External memory control unit 21 Recording medium 22 Display device 22a Display controller 22b Display surface 24 Control bus 25 Data bus 40 Lens device 50 Imaging unit 60 Imaging surface 61 Pixel 61a Normal pixel 61b,61c Distance measurement pixels 62,63 Pixel lines 64 Drive circuit 65 Signal processing circuit 100 Imaging device 100A main body 150, 301 object to be written 151~159, 303 AF area 200 smartphone 201 housing 202 display panel 203 operation panel 204 display input unit 205 speaker 206 microphone 208 camera unit 210 wireless communication unit 211 call unit 212 memory unit 213 external input / output unit 214 GNSS receiver unit 215 motion sensor unit 216 power supply unit 217 internal memory unit 218 external memory unit 220 main control unit 300 image 302 background 401 obstacle 510 change in object position 810 settable range 811 lower limit value 812 upper limit value ST1~STn GNSS satellites

Claims

1. A control device comprising a processor and capable of controlling an imaging device, wherein the processor: predicts distance information regarding the distance between the imaging device and a subject in a third frame after the first frame and the second frame, based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device; sets a settable range, which is a range of the focus position of imaging by the imaging device that can be set for the third frame, based on the state of the subject; sets the focus position in the third frame within the set settable range, based on the predicted distance information in the third frame; A control device.

2. The control device according to claim 1, wherein the processor: acquires movement distance information, which is information regarding the amount of change in the distance between the imaging device and the subject, based on the image data of the plurality of frames; sets the settable range for the third frame based on the movement distance information in the plurality of frames; A control device.

3. The control device according to claim 2, wherein the processor: sets the settable range for the third frame based on the difference in the movement distance information in the plurality of frames; A control device.

4. The control device according to claim 2 or 3, wherein the processor: expands the settable range when the movement distance information in the plurality of frames is increasing; A control device.

5. The control device according to any one of claims 2 to 4, wherein the processor: maintains or reduces the settable range when the movement distance information in the plurality of frames is decreasing; A control device.

6. The control device according to any one of claims 2 to 5, wherein the processor: maintains or reduces the settable range when the difference in the movement distance information in the plurality of frames is within a predetermined range; A control device.

7. The control device according to any one of claims 2 to 6, wherein the processor: maintains or reduces the settable range when the change direction of the distance information in the plurality of frames has switched to the reverse direction; A control device.

8. The control device according to any one of claims 2 to 7, wherein the processor: When the distance information in the plurality of frames is within a predetermined range after changing by a predetermined value or more, expand the settable range. Control device.

9. The control device according to any one of claims 2 to 8, The movement distance information is the ratio between the amount of change between frames of the distance between the imaging device and the subject and the depth of field. Control device.

10. The control device according to any one of claims 1 to 9, The processor acquires a plurality of distance information corresponding to a plurality of areas of an image represented by the image data, and predicts the distance information of the third frame based on the plurality of distance information of the first frame and the plurality of distance information of the second frame. Control device.

11. The control device according to claim 10, The processor predicts the distance information of the third frame based on a comparison between the distance information of a first area near the center among the plurality of areas and the distance information of a second area around the first area among the plurality of areas. Control device.

12. An imaging device comprising the control device according to any one of claims 1 to 11. Imaging device.

13. A control method by a control device including a processor and capable of controlling an imaging device, wherein the processor predicts distance information regarding the distance between the imaging device and the subject in a third frame after the first frame and the second frame based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, sets a settable range that is the range of the focus position of imaging by the imaging device that can be set for the third frame based on the state of the subject, sets the focus position in the third frame within the set settable range based on the predicted distance information in the third frame. Control method.

14. A control program for a control device including a processor and capable of controlling an imaging device, wherein the processor predicts distance information regarding the distance between the imaging device and the subject in a third frame after the first frame and the second frame based on the image data of the first frame and the image data of the second frame among a plurality of frames obtained by the imaging device, Based on the state of the subject, set a configurable range, which is the range of the focus position of imaging by the imaging device that can be set for the third frame. Set the focus position in the third frame based on the predicted distance information in the third frame within the set configurable range. A control program for executing the process.

Images