Control device, imaging apparatus, control method, and program

Abstract

To provide a control device that can prevent loss of a photographing opportunity.SOLUTION: A control device has selection means that selects, of subjects included in an image, a main subject to be focused, and setting means that sets a depth of field. When the subjects overlap each other in a depth direction, the setting means performs first setting of setting a first depth of field to a second depth of field that is deeper than the first depth of field, and when the main subject is switched after the first setting is performed, performs second setting of setting the second depth of field to the first depth of field.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a control device.

Background Art

[0002] Conventionally, a technique has been proposed for photographing a subject according to the intention of a photographer by setting the subject along the intention as an AF target and bringing it into a focused state when a plurality of subjects are detected. Patent Document 1 discloses a configuration in which when a plurality of moving subjects are detected, a subject located at the head in the moving direction (leading subject) is determined as the main subject.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The configuration of Patent Document 1 is mainly assumed to be used for photographing in a sports scene such as competition. However, in such a scene, even if the leading subject is set as the main subject, it may not always be possible to photograph in a state where the leading subject is in focus. For example, when a second subject overtakes a first subject that was in focus and the second subject becomes the new leading subject, there is a possibility that the second subject is not in focus immediately after overtaking. Also, even after setting the second subject as the AF target as the main subject, the camera cannot be brought into a focused state until it performs calculations and moves the lens. In particular, in a sports scene such as competition, suitable shooting opportunities are limited, but the continued state of not being able to focus on the leading subject further reduces the shooting opportunities.

[0005] An object of the present invention is to provide a control device capable of suppressing loss of shooting opportunities.

Means for Solving the Problems

[0006] As one aspect of the present invention, a control device includes a selection means for selecting a main subject to be focused on among the subjects included in an image, and a setting means for setting a depth of field. When the subjects overlap in the depth direction, the setting means performs a first setting for setting a first depth of field to a second depth of field deeper than the first depth of field. When the main subject is switched after the first setting is performed, the setting means performs a second setting for setting the second depth of field to the first depth of field.

Advantages of the Invention

[0007] According to the present invention, it is possible to provide a control device capable of suppressing the loss of a shooting opportunity.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant explanations are omitted.

[0010] FIG. 1 is a configuration diagram of a camera system (hereinafter referred to as a camera) according to an embodiment of the present invention. The camera has an interchangeable lens 100 and a camera body (imaging device) 120. In the present embodiment, the camera body 120 is a mirrorless camera which is an example of an imaging device. The interchangeable lens 100 is configured to be attachable to the camera body 120.

[0011] The interchangeable lens 100 includes a photographing lens unit 101 and a microcomputer for lens system control (hereinafter referred to as a lens control unit) 111. The photographing lens unit 101 includes a main photographing optical system 102, a diaphragm 103 for adjusting the amount of light, and a focus lens group 104 for adjusting the focus. The lens control unit 111 includes a diaphragm control unit 112 for controlling the diaphragm 103 and a focus lens control unit 113 for controlling the focus lens group 104. The focus lens control unit 113 drives the focus lens group 104 in the optical axis direction of the photographing lens unit 101 based on the focus lens drive information acquired from the camera body 120, and performs focus adjustment of the camera.

[0012] Note that the focus lens group 104 may have a plurality of focus lenses or may have only one focus lens. Further, although the interchangeable lens 100 is shown as a single-focus lens for simplification of the drawing in the present embodiment, it may be a lens (zoom lens) whose focal length can be changed. When the interchangeable lens 100 is a zoom lens, the lens control unit 111 acquires focal length information from an encoder that detects the zoom lens position. Further, when the interchangeable lens 100 has a shake correction function, the lens control unit 111 controls a shift lens group for shake correction or the like.

[0013] The camera body 120 has a function of detecting a subject in an image and selecting a subject (leading subject) located at the head of the moving direction as a main subject to be focused. The subject is, for example, a face of a person, a human body, a specific animal such as a dog or a cat, and a vehicle such as an automobile or a motorcycle.

[0014] The shutter 121 is used for exposure control. The imaging device 122 is composed of a CMOS (Complementary Metal Oxide Semiconductor) sensor or the like and acquires an image. Also, the imaging device 122 transmits an imaging signal to the analog signal processing circuit 123. The analog signal processing circuit 123 processes the imaging signal acquired from the imaging device 122 and then transmits the processed imaging signal to the camera signal processing circuit 124 as an image. In the present embodiment, an imaging means for acquiring an image is constituted by the imaging device 122 and the analog signal processing circuit 123.

[0015] The microcomputer for camera system control (hereinafter referred to as the camera control unit) 131 is a control device that controls the entire camera. For example, the camera control unit 131 drives and controls a motor for driving the shutter (not shown) to drive the shutter 121. The memory card 125 is a recording medium for recording the captured image. The pressed state of the release switch 181 operated by the photographer is sent to the camera control unit 131, and the captured image is stored in the memory card 125 according to this state.

[0016] The image display unit 171 includes a display device such as a liquid crystal panel (LCD) and displays the scene (landscape) that the photographer is about to photograph with the camera or the captured image. The touch panel 172 is an operation unit that allows the photographer to specify coordinates on the image display unit 171 using a finger or a touch pen. The touch panel 172 may be integrally configured with the image display unit 171. For example, the touch panel 172 may be configured so as not to interfere with the display of the image display unit 171 and incorporated inside the image display unit 171. At this time, the input coordinates on the touch panel 172 and the display coordinates on the image display unit 171 are associated with each other. Thereby, a GUI (Graphical User Interface) can be configured as if the photographer can directly operate the screen displayed on the image display unit 171. The operation state of the touch panel 172 is managed by the camera control unit 131.

[0017] The camera body 120 is provided with a mount contact portion 161, which is a communication terminal for communicating with the interchangeable lens 100, on the mount surface with the interchangeable lens 100. Further, the interchangeable lens 100 is provided with a mount contact portion 114, which is a communication terminal for communicating with the camera body 120, on the mount surface with the camera body 120.

[0018] The lens control unit 111 and the camera control unit 131 control the communication so as to perform serial communication at a predetermined timing via the mount contact portions 114 and 161. Through this communication, focus lens drive information, aperture drive information, etc. are sent from the camera control unit 131 to the lens control unit 111, and optical information such as the focal length is sent from the lens control unit 111 to the camera control unit 131.

[0019] The camera signal processing circuit 124 includes a subject detection unit 141. The subject detection unit 141 detects a subject from an image and transmits detection information (information regarding the subject) of the subject to the camera control unit 131.

[0020] The camera control unit 131 includes a subject number counting unit (acquisition means) 150, a time-series correlation processing unit (comparison means) 151, an AF target setting unit (selection means) 152, a focus detection unit 153, and a depth of field setting unit (setting means) 154. The subject number counting unit 150 counts (acquires) the number of subjects included in the image using the detection information from the subject detection unit 141. The time-series correlation processing unit 151 compares the detection information from the subject detection unit 141 before and after in time (the first timing and the second timing after the first timing) to determine whether the same object is detected. The AF target setting unit 152 notifies the focus detection unit 153 that the specified area is the AF target. In this embodiment, the AF target setting unit 152 uses the detection information from the subject detection unit 141 to determine (select) the area of the subject (leading subject) located at the head of the moving direction among the subjects included in the image as the AF target. As a method for determining the leading subject, for example, a method using a motion vector or a method using an algorithm learned by machine learning for the leading subject may be used. The focus detection unit 153 performs focus detection processing based on the image signal corresponding to the main subject notified by the AF target setting unit 152. The focus detection processing is executed, for example, by a phase difference detection method or a contrast detection method. In the case of the phase difference detection method, as the focus detection processing, a calculation of the image shift amount calculated by correlating a pair of image signals having a parallax, or a processing of further converting the image shift amount into a defocus amount and calculating it is performed. The defocus amount can be further converted into the focus lens driving amount in consideration of the sensitivity when the interchangeable lens 100 is driven. The depth of field setting unit 154 determines (sets) the width of the depth of field based on the outputs of the subject number counting unit 150, the time-series correlation processing unit 151, and the focus detection unit 153, and performs a process of converting it into the aperture amount.

[0021] The camera control unit 131 transmits the focus lens driving amount detected by the focus detection unit 153 and the aperture amount calculated by the depth of field setting unit 154 to the lens control unit 111. The focus lens control unit 113 controls the focus lens group 104 based on the focus lens driving information received from the camera control unit 131. The aperture control unit 112 controls the aperture 103 based on the aperture amount received from the camera control unit 131.

[0022] Hereinafter, the operation of the camera according to this embodiment will be described. First, the operation from when the camera acquires an image until it determines the focusing target and the depth of field and controls the interchangeable lens 100 will be described. FIG. 2 is a flowchart showing the operation from image acquisition to control of the interchangeable lens 100.

[0023] In step S201, the analog signal processing circuit 123 processes the imaging signal captured from the imaging device 122 and sends it as an image to the camera signal processing circuit 124.

[0024] In step S202, the subject detection unit 141 detects a subject from the acquired image. The detection information is sent to the camera control unit 131.

[0025] In step S203, the camera control unit 131 determines whether a valid subject has been detected by the subject detection unit 141 using the detection information from the subject detection unit 141. If the camera control unit 131 determines that a valid subject has been detected, it executes the process of step S204, and if it determines otherwise, it ends this flow.

[0026] In step S204 (selection step), the AF target setting unit 152 estimates (selects) the leading subject and sets the area of the leading subject as the AF target.

[0027] In step S205, the camera control unit 131 determines whether the depth of field has not been extended (set to the first depth of field). If the camera control unit 131 determines that the depth of field has not been extended, it executes the process of step S206. On the other hand, if the camera control unit 131 determines that the depth of field has been extended (set to the second depth of field deeper than the first depth of field), it executes the process of step S207. Note that if it is the frame in which the flow of FIG. 2 is executed for the first time, it is determined that the depth of field has not been extended.

[0028] In step S206, the camera control unit 131 executes the determination process for depth of field extension. After the execution of the process of this step, the depth of focus and the depth of field are determined.

[0029] In step S207, the camera control unit 131 executes the determination process for depth of field restoration. After the execution of the process of this step, the depth of focus and the depth of field are determined.

[0030] In step S208, the aperture control unit 112 controls the aperture 103 according to the set depth of field.

[0031] In step S209, the focus lens control unit 113 controls the focus lens group 104 according to the set AF target and the depth of focus.

[0032] Next, with reference to FIG. 3, the determination process for depth of field extension in step S206 will be described. The determination process for depth of field extension is a process of determining whether to extend the depth of field when the depth of field has not been extended, and determining the depth of focus and the depth of field. FIG. 3 is a flowchart showing the determination process for depth of field extension.

[0033] In step S301 (acquisition step), the subject number counting unit 150 counts (acquires) the number of subjects included in the image.

[0034] In step S302, the camera control unit 131 determines whether the number of subjects counted in step S301 has decreased compared to the number of subjects in the previous frame. If the camera control unit 131 determines that the number of subjects counted in step S301 has decreased compared to the number of subjects in the previous frame, it executes the process of step S303; if it determines otherwise, it executes the process of step S305. Note that when the frame in which the flow of FIG. 3 is executed for the first time, the determination is made assuming that the number of subjects in the previous frame is 0. Also, in the present embodiment, the number of subjects is used to determine whether the subject is hidden (overlapped) in the depth direction and cannot be seen, but the present invention is not limited to this. Other methods may be used as long as it is possible to determine whether the subject is hidden in the depth direction.

[0035] In step S303, the time-series correlation processing unit 151 compares the detection information from the subject detection unit 141 in the current frame (second timing) with the detection information from the subject detection unit 141 in the previous frame (first timing).

[0036] In step S304, the camera control unit 131 determines, using the result obtained in step S303, whether the subject that disappeared in the current frame was present (located) within a certain range (predetermined area) on the end side of the image in the previous frame. If the camera control unit 131 determines that the subject that disappeared in the current frame was present within a certain range on the end side of the image in the previous frame, it regards it as disappearance due to frame out and executes the process of step S305. If the camera control unit 131 determines that the subject that disappeared in the current frame was not present within a certain range on the end side of the image in the previous frame, it regards it as disappearance due to being blocked and executes the process of step S307.

[0037] In step S305, the camera control unit 131 sets the focusing depth so as to focus on the depth at which the leading subject exists.

[0038] In step S306 (setting step), the depth of field setting unit 154 sets the depth of field to the depth of field of a normal width (first depth of field).

[0039] In step S307, the camera control unit 131 records, as the subject to be adjusted for the depth of field (hereinafter referred to as the target subject), the subject among the disappeared subjects that is closest to the leading subject in the previous frame.

[0040] In step S308, the camera control unit 131 sets the in-focus depth based on the distance between the camera and the leading subject and the distance between the camera and the target subject. The distance is estimated using the defocus amount of each subject obtained from the focus detection unit 153. The in-focus depth is set in the middle of the leading subject and the target subject for the purpose of not bringing the out-of-focus area into focus, minimizing the driving amount of the lens, and accelerating the completion of the process, and minimizing the expansion width of the depth of field.

[0041] In step S309 (setting step), the depth of field setting unit 154 performs a first setting to set the depth of field to the depth of field of a width in which the leading subject and the target subject fit (second depth of field) at the in-focus depth set in step S308.

[0042] In step S310, the camera control unit 131 records the current time measured in the camera. The recorded time is used in the determination process for depth of field restoration.

[0043] Next, with reference to FIG. 4, the determination process for depth of field restoration in step S207 will be described. The determination process for depth of field restoration is a process of determining whether to restore the depth of field when the depth of field is expanded, and determining the in-focus depth and the depth of field. FIG. 4 is a flowchart showing the determination process for depth of field restoration.

[0044] In step S401, the camera control unit 131 determines whether the leading subject estimated in step S204 has switched from the leading subject in the previous frame. If the camera control unit 131 determines that the leading subject has switched, it executes the process of step S403. If it determines that the leading subject has not switched, it executes the process of step S402.

[0045] In step S402, the camera control unit 131 determines whether a predetermined time has elapsed since the depth of field has been expanded. The elapsed time is the difference between the time recorded in step S310 and the time when step S402 is being executed. If the camera control unit 131 determines that the predetermined time has elapsed, it executes the process of step S403. If it determines that the predetermined time has not elapsed, it executes the process of step S407.

[0046] In step S403, the camera control unit 131 sets the focus depth so as to focus on the depth at which the leading subject exists.

[0047] In step S404, the camera control unit 131 determines whether the leading subject is in focus at the current focus lens position with the width of the original depth of field with respect to the focus depth set in step S403. If the camera control unit 131 determines that it is in focus, it executes the process of step S405. If it determines otherwise, it executes the process of step S406.

[0048] In step S405 (setting step), the depth of field setting unit 154 performs a second setting to set the depth of field to the original width of the depth of field (first depth of field).

[0049] In step S406 (setting step), the depth of field setting unit 154 sets the depth of field to the width of the depth of field with the expansion maintained (second depth of field). That is, the width with the depth of field expanded is maintained.

[0050] The processes from step S404 to step S406 are measures for maintaining the in-focus state until the focus lens group 104 is driven to reach the position corresponding to the depth of focus set in step S405 after the switching of the leading subject.

[0051] In step S407, the camera control unit 131 sets the depth of focus to be in the middle between the leading subject and the target subject.

[0052] In step S408 (setting step), the depth of field setting unit 154 sets the depth of field at the depth of focus set in step S308 to the depth of field (second depth of field) in which the leading subject and the target subject are included.

[0053] As described above, in the determination process of depth of field restoration, the decrease in the number of subjects is regarded as a sign of the switching of the leading subject, and by expanding the depth of field in advance, the state of being in focus on the new leading subject is achieved even immediately after the switching of the leading subject.

[0054] Hereinafter, the setting of the depth of field will be described using specific examples. FIG. 5 is a diagram showing the situation before expanding the depth of field, and shows a situation in which a subject A, which is the leading subject, and a subject B behind the subject A are detected. In FIG. 5, the depth of field is the normal depth of field (the first depth of field) without being expanded, and since it is focused on the subject A, the subject B is out of focus. FIG. 6 is a diagram showing the situation after the number of subjects decreases and the depth of field is expanded, and shows a situation in which the subject B is blocked by the subject A and not detected from the situation of FIG. 5. In FIG. 6, since the number of subjects has decreased, it is considered that there is a sign of switching of the leading subject, and the depth of field is expanded so that the subject B that has been detected so far is included in the focusing range (set to the second depth of field). At this time, by positioning the focusing depth in the middle of the subject A and the subject B, the expansion width of the depth of field is minimized. FIG. 7 is a diagram showing the situation immediately after the leading subject has switched in the state where the depth of field is expanded, and shows the situation immediately after the subject B has overtaken the subject A and newly become the leading subject from the situation of FIG. 6. In FIG. 7, since it is immediately after the leading subject has switched, the driving of the focus lens group 104 is not in time, and although the focusing depth cannot be adjusted to the position of the subject B, since the depth of field has been expanded in advance, it is in a focused state.

[0055] As described above, according to the configuration of the present embodiment, when using the function of switching the AF target to the leading subject in photographing a sports scene such as competition, it is possible to bring the camera into a state of being focused on the new leading subject even immediately after the leading subject has switched. Thereby, the loss of the photographing opportunity can be suppressed, and a large number of suitable photographing opportunities can be obtained.

[0056] In the present embodiment, the detection information of the subjects in the previous frame and the current frame is correlated to estimate the disappeared subject, but other means may be used. For example, means for tracking a plurality of subjects may be used, and the disappeared object and position may be estimated from the tracking result.

[0057] Also, in this embodiment, when a predetermined time has elapsed after expanding the depth of field, and the decrease in the number of subjects cannot be regarded as a sign of switching of the leading subject, a process of returning the depth of field to its original state is performed. However, instead of the passage of time, other information that can be estimated that the switching of the leading subject does not occur may be used. For example, it may be estimated that the switching of the leading subject does not occur using the number of frames after expanding the depth of field. [Other Embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0058] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) Selection means for selecting a main subject to be focused on from among the subjects included in the image, and setting means for setting the depth of field, wherein when the subjects overlap in the depth direction, the setting means performs a first setting of setting a first depth of field to a second depth of field deeper than the first depth of field, and when the switching of the main subject is performed after the first setting, the setting means performs a second setting of setting the second depth of field to the first depth of field. A control device characterized by this. (Configuration 2) further comprising comparison means for comparing information about the subject acquired from a first image at a first timing with information about the subject acquired from a second image at a second timing after the first timing, wherein when the number of subjects at the second timing is less than the number of subjects at the first timing, and the subject not detected from the second image exists within a predetermined region on the end side of the first image, the setting means does not perform the first setting. The control device according to Configuration 1, characterized by this. (Configuration 3) comparison means for comparing information about the subject obtained from the first image at the first timing and information about the subject obtained from the second image at a second timing after the first timing; The setting means determines the width of the second depth of field in accordance with the distance to the subject not detected from the second image when the number of subjects at the second timing is smaller than the number of subjects at the first timing, according to the control device of configuration 1 or 2. (Configuration 4) The setting means sets the depth of focus when the number of subjects decreases, and performs the first setting according to the depth of focus, according to the control device of any one of configurations 1 to 3. (Configuration 5) The setting means performs the second setting when a predetermined time has elapsed after performing the first setting, according to the control device of any one of configurations 1 to 4. (Configuration 6) The setting means performs the second setting when the main subject is switched after performing the first setting and the main subject after the switch is in a state of being in focus in the first depth of field, according to the control device of any one of configurations 1 to 5. (Configuration 7) The selection means selects the subject at the head of the moving direction as the main subject, according to the control device of any one of configurations 1 to 6. (Configuration 8) An imaging device comprising the control device of any one of configurations 1 to 7, and imaging means for acquiring an image. (Method 1) a selection step of selecting a main subject to be focused on among the subjects included in the image; and a setting step of setting the depth of field. In the setting step, when the subjects overlap in the depth direction, a first setting is performed to set the first depth of field to a second depth of field deeper than the first depth of field. When the main subject is switched after the first setting is performed, a second setting is performed to set the second depth of field to the first depth of field. A control method characterized by this. (Configuration 9) A program characterized by causing a computer to execute the control method described in Method 1.

[0059] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

Explanation of Signs

[0060] 131 Microcomputer for camera system control (control device) 152 AF target setting unit (selection means) 154 Depth of field setting unit (setting means)

Claims

1. selection means for selecting a main subject to be focused on from among the subjects included in the image; and setting means for setting the depth of field; wherein the setting means performs a first setting for setting a first depth of field to a second depth of field deeper than the first depth of field when the subjects overlap in the depth direction, and when the main subject is switched after performing the first setting, performs a second setting for setting the second depth of field to the first depth of field. A control device characterized by this.

2. further comprising comparison means for comparing information about the subject obtained from a first image at a first timing with information about the subject obtained from a second image at a second timing after the first timing; wherein the setting means does not perform the first setting when the number of subjects at the second timing is less than the number of subjects at the first timing and the subject not detected from the second image exists within a predetermined region on the end side of the first image. The control device according to claim 1, characterized by this.

3. further comprising comparison means for comparing information about the subject obtained from a first image at a first timing with information about the subject obtained from a second image at a second timing after the first timing; wherein the setting means determines the width of the second depth of field according to the distance to the subject not detected from the second image when the number of subjects at the second timing is less than the number of subjects at the first timing. The control device according to claim 1 or 2, characterized by this.

4. wherein the setting means sets the focusing depth and performs the first setting according to the focusing depth when the number of subjects decreases. The control device according to claim 1 or 2, characterized by this.

5. wherein the setting means performs the second setting when a predetermined time has elapsed after performing the first setting. The control device according to claim 1 or 2, characterized by this.

6. wherein the setting means performs the second setting when the main subject is switched after performing the first setting and the main subject after switching is in a focused state at the first depth of field. The control device according to claim 1 or 2, characterized by this.

7. wherein the selection means selects the subject at the head of the moving direction as the main subject. The control device according to claim 1 or 2, characterized by this.

8. An imaging apparatus comprising: the control device according to claim 1 or 2; and imaging means for acquiring an image.

9. A control method comprising: a selection step of selecting a main subject to be focused on from among subjects included in an image; and a setting step of setting a depth of field, wherein in the setting step, when the subjects overlap in the depth direction, a first setting is performed to set a first depth of field to a second depth of field deeper than the first depth of field, and when the main subject is switched after the first setting is performed, a second setting is performed to set the second depth of field to the first depth of field.

10. A program causing a computer to execute the control method according to claim 9.

Images