Focusing device, imaging device, focusing method, and program

Abstract

To provide a focus adjustment device that allows for appropriate setting of autofocus operation during video recording. [Solution] The focus adjustment device includes a detection means for detecting the movement of a subject in an image obtained by imaging the subject, and a setting means for setting the focus adjustment parameter. The setting means sets the parameter to a first parameter if the movement of the subject is not a specific movement, and sets the parameter to a second parameter different from the first parameter if the movement of the subject is a specific movement.

Description

Technical Field

[0001] The present invention relates to a focus adjustment device.

Background Art

[0002] In recent years, in a video shooting scene, it has been required to change the operation settings of AF (auto focus), such as the tracking characteristics and speed, depending on whether the focus needs to be switched immediately or not. Patent Document 1 discloses a configuration for changing voice recognition conditions based on the voice of the user speaking and the pre-registered voice.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the configuration of Patent Document 1, in order to change the voice recognition conditions, it is necessary for the user to utter a specific voice, and this voice will be recorded during video shooting.

[0005] The present invention can provide a focus adjustment device capable of appropriately setting the operation settings of auto focus during video shooting.

Means for Solving the Problems

[0006] A focus adjustment device as one aspect of the present invention comprises a detection means for detecting the movement of a subject in an image obtained by imaging the subject, and a setting means for setting a focus adjustment parameter, wherein the setting means sets the parameter to a first parameter if the movement of the subject is not a specific movement, and sets the parameter to a second parameter different from the first parameter if the movement of the subject is the specific movement. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a focus adjustment device that can appropriately set the autofocus operation settings during video recording. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram of an imaging device having a focus adjustment device according to an embodiment of the present invention. [Figure 2] This diagram shows the menu screen for specifying AF (Automatic Focus) operation settings. [Figure 3] This is a flowchart showing the method of adjusting focus. [Figure 4] This flowchart shows the method for registering subject movements. [Figure 5] This figure shows an example of an input image used when registering subject movements. [Figure 6] This is an example of how the monitor display looks when registered operation settings are configured. [Figure 7] This diagram shows the timing for setting the AF (Automatic Focus) operation settings. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings. In each drawing, the same reference numeral is used for identical components, and redundant descriptions are omitted. (Configuration of the focusing mechanism) Figure 1 is a block diagram of an imaging device 100 having a focus adjustment device according to an embodiment of the present invention. The imaging device 100 is a digital still camera or video camera capable of photographing a subject and recording video and still image data onto various media such as tape, solid memory, optical disk, and magnetic disk, but the present invention is not limited to these. Each unit within the imaging device 100 is connected via a bus 160. Each unit is controlled by a CPU 151 (central processing unit).

[0010] The lens unit 101 comprises a fixed 1-group lens 102, a zoom lens 111, an aperture 103, a fixed 3-group lens 121, and a focus lens 131.

[0011] The aperture control unit 105 adjusts the aperture diameter of the aperture 103 by driving the aperture 103 via the aperture motor 104 (AM) in accordance with the command from the CPU 151, thereby adjusting the amount of light during shooting.

[0012] The zoom control unit 113 changes the focal length by driving the zoom lens 111 via the zoom motor 112 (ZM).

[0013] The focus control unit 133 determines the amount to drive the focus motor 132 (FM) based on the amount of deviation (defocus amount) in the focus direction of the lens unit 101. The focus control unit 133 also controls the focus adjustment state by driving the focus lens 131 via the focus motor 132 (FM). Autofocus (AF) control is realized by controlling the movement of the focus lens 131 by the focus control unit 133 and the focus motor 132.

[0014] The focus lens 131 is a lens used for adjusting the focus, and although it is simply shown as a single lens in Figure 1, it is usually composed of multiple lenses.

[0015] The subject image (optical image) formed on the imaging surface of the imaging device 141 via the lens unit 101 is converted into an electrical signal (imaging signal) by the imaging device 141. The imaging device 141 is a photoelectric conversion element that performs photoelectric conversion of the subject image into an electrical signal. On the imaging surface of the imaging device 141, light receiving elements with m pixels in the horizontal direction and n pixels in the vertical direction are arranged. The electrical signal output from the imaging device 141 is arranged as an image signal (image data) by the imaging signal processing unit 142, and an image of the imaging surface can be acquired.

[0016] The image data output from the imaging signal processing unit 142 is sent to the imaging control unit 143 and temporarily stored in the RAM 154 (random access memory). The image data stored in the RAM 154 is compressed by the image compression / decompression unit 153 and then recorded on the image recording medium 157. Also, the image data stored in the RAM 154 is sent to the image processing unit 152. The image processing unit 152 performs processing such as reducing / enlarging the image data to an optimal size and calculating the similarity between image data. By appropriately sending the image data processed to the optimal size to the monitor display (display means) 150 for display, preview image display and through image display can be performed. In addition, the subject detection result of the subject detection unit (detection means) 161 can also be superimposed and displayed on the image data. Also, by using the RAM 154 as a ring buffer, a plurality of image data acquired within a predetermined period, the detection results of the subject detection unit 161 for each image data, etc. can be buffered.

[0017] The operation switch 156 is an input interface including a touch panel, buttons, etc., and various operations can be performed by selecting and operating various function icons displayed on the monitor display 150.

[0018] The CPU 151 determines the accumulation time of the imaging device 141 and the set value of the gain when outputting from the imaging device 141 to the imaging signal processing unit 142 based on an instruction from the user via the operation switch 156 or the magnitude of the pixel signal of the image data stored in the RAM 154. The imaging control unit 143 receives an instruction of the accumulation time and the set value of the gain from the CPU 151 and controls the imaging device 141.

[0019] The subject detection unit 161 determines an area where a predetermined subject exists using the image signal. The subject is detected based on a dictionary with a teacher for targets such as people and animals created in advance. As a detection method by learning, there is a method called a convolutional neural network (hereinafter abbreviated as CNN). In CNN, inference processing is executed based on the image signal and dictionary data which is a processing parameter, and the dictionary data is generated by learning processing based on teacher data in advance.

[0020] The focus control unit 133 controls AF for a specific subject area.

[0021] The aperture control unit 105 controls the exposure using the luminance value of a specific subject area.

[0022] The image processing unit 152 executes gamma correction, white balance processing, etc. based on the subject area.

[0023] The monitor display 150 displays an image, a subject detection result, etc. in a rectangle or the like.

[0024] The flash memory (storage means) 155 records a control program necessary for the operation of the imaging device 100, parameters used for the operation of each unit, etc. When the imaging device 100 is activated by a user operation (when shifting from the power-off state to the power-on state), the control program and parameters stored in the flash memory 155 are read into the RAM 154. The CPU 151 controls the operation of the imaging device 100 according to the control program and parameters loaded into the RAM 154.

[0025] The user can operate a menu screen, as shown in Figure 2, displayed on the monitor display 150, to specify (set) the AF operation settings (parameters) controlled by the focus control unit 133. In Figure 2, the AF operation settings are AF speed and AF tracking characteristics (sensitivity). The values ​​set by the user are recorded in the flash memory 155.

[0026] The subject motion registration unit (registration means) 162 records (registers) the subject motion (specific motion) set by the user in the flash memory 155. The subject motion registration unit 162 also records the AF operation setting value corresponding to the registered subject motion in the flash memory 155. Furthermore, the subject motion registration unit 162 may also record characteristic objects of the shooting scene corresponding to the registered subject motion in the flash memory 155.

[0027] The subject motion determination unit (determination means) 163 determines whether the subject motion in the image signal is a subject motion registered by the subject motion registration unit 162.

[0028] The AF operation setting determination unit (setting means) 164 determines whether to switch to the AF operation setting corresponding to the subject operation when it determines that a subject operation registered by the subject operation determination unit 163 exists in the image signal, and reflects this in the control of the focus control unit 133.

[0029] The defocus calculation unit 165 calculates the amount of defocus from the imaging device 100 for any region in the image. The defocus amount may be calculated for a single point, or it may be calculated at equal intervals across the entire image and output as a defocus map. The generated defocus information is recorded in the RAM 154 and referenced by the image processing unit 152. The focus control unit 133 calculates the focus drive amount based on the defocus information and performs focus adjustment.

[0030] In this embodiment, the focus adjustment device is comprised of a subject detection unit 161 and an AF operation setting determination unit 164. The focus adjustment device may also include at least one of a flash memory 155, a subject motion registration unit 162, and a subject motion determination unit 163. (Processing flow) The focus adjustment method of this embodiment will be described below with reference to Figure 3. Figure 3 is a flowchart of the focus adjustment method. The AF operation setting is assumed to be set to the default setting, which is the standard operation setting (first parameter).

[0031] In step S101, the subject detection unit 161 acquires the input image from the imaging control unit 143.

[0032] In step S102 (detection step), the subject detection unit 161 performs subject detection on the input image. In this embodiment, the subject detection unit 161 outputs the head region and torso region of the person as rectangles. The subject detection unit 161 also detects each joint point of the person and detects the person's posture and movement based on each joint point. Furthermore, the subject detection unit 161 can detect the shooting scene in the input image. In this embodiment, the subject detection unit 161 detects the shooting scene by detecting characteristic objects of the shooting scene (such as a soccer ball in a soccer scene, and a knife and cutting board in a cooking scene).

[0033] Now, with reference to Figure 4, the subject motion registration method of this embodiment will be described. Figure 4 is a flowchart of the subject registration method. The subject motion registration method is performed before shooting.

[0034] In step S201, the subject motion registration unit 162 acquires the input image. At this time, the subject is performing the motion (specific motion) that you want to register, as shown in Figure 5. In Figure 5(A), person 501 is dribbling a soccer ball 502, and in Figure 5(B), person 501 is waving their hand.

[0035] In step S202, the subject motion registration unit 162 detects a specific motion. The subject detection unit 161 detects each joint point of the person and detects the person's posture and movement. At this time, the unit may also detect characteristics of the person to identify the same person.

[0036] In step S203, the subject motion registration unit 162 detects the shooting scene. In this embodiment, the subject motion registration unit 162 detects the shooting scene by detecting characteristic objects in the shooting scene (such as a soccer ball in a soccer scene, and a knife and cutting board in a cooking scene).

[0037] In step S204, the subject motion registration unit 162 determines whether it has detected a specific motion and shooting scene. If the subject motion registration unit 162 determines that it has detected a specific motion and shooting scene, it executes the process in step S205; otherwise, it executes the process in step S206.

[0038] In step S205, the subject motion registration unit 162 first displays a menu screen as shown in Figure 2 on the monitor display 150. Next, the subject motion registration unit 162 acquires (registers) the registered operation setting (second parameter), which is the AF operation setting corresponding to the specific operation detected in step S202 selected by the user. The subject motion registration unit 162 also records the acquired AF registered operation setting in the flash memory 155.

[0039] In step S206, the subject motion registration unit 162 registers the specific motion detected in step S202 and the shooting scene detected in step S203, and records them in the flash memory 155. The information recorded in the flash memory 155 in steps S205 and S206 is linked and managed by the subject motion registration unit 162.

[0040] In step S207, the subject motion registration unit 162 displays information that was not detected on the monitor display 150 (error display). It may also display information indicating the reason why registration failed.

[0041] The above describes the method for registering subject actions. Note that specific actions can be changed. Here, "change" includes updating or adding specific actions, and deleting at least one registered specific action. By configuring specific actions to be changeable, it is possible to improve usability. In addition, although the shooting scene was also registered in this embodiment, if the decision of whether or not to change the AF operation setting is made based only on the specific action, the process of detecting and recording the shooting scene may be omitted.

[0042] Let's return to the flowchart in Figure 3 and continue the explanation.

[0043] In step S103, the subject motion determination unit 163 determines whether the subject's motion in the input image is a specific motion registered in the subject motion registration unit 162. The subject motion determination unit 163 makes this determination based on the similarity of the posture and movement of the person detected by the subject detection unit 161. If the subject motion determination unit 163 determines that the subject's motion is a specific motion, it executes the process in step S104; otherwise, it executes the process in step S105. In addition to the posture and movement of the person, the subject motion registration unit 162 may also make the determination using the similarity of characteristic objects in the shooting scene or the matching of person feature information to determine whether the person is the same as a registered person.

[0044] In step S104 (setting step), the AF operation setting determination unit 164 sets the AF operation setting to the registered operation setting. At this time, the user may be notified that the registered operation setting has been set by a notification means. For example, the monitor display 150 may function as a notification means. In this case, as shown in Figure 6, an icon 601 indicating that the registered operation setting has been set can be displayed on the monitor display 150 superimposed on the input image. Alternatively, an LED or other light-emitting element may function as a notification means. In this case, the light-emitting element may be configured to emit light of a specific color or to blink while the registered operation setting is set.

[0045] In step S105 (setting step), the AF operation setting determination unit 164 sets the AF operation setting to the standard operation setting.

[0046] In step S106, the focus control unit 133 adjusts the focus based on the AF operation settings set in step S104 or step S105.

[0047] Note that the transition period for switching to the registered action setting may differ depending on whether the focus is being adjusted to a different subject or to the same subject. A specific example is explained with reference to Figure 7. Figure 7(A) shows an example of switching to a different subject. At time T(a), when the subject's action is determined not to be a specific action, the standard action setting is set, and at time T(b), when the subject's action is determined to be a specific action, the registered action setting is set. Then, at time T(c), when the subject's action is determined not to be a specific action, it is set back to the standard action setting. Figure 7(B) shows a transition to the same subject, for example, when only the upper body of a person is initially photographed, and then the focus is adjusted to the hand along with the registered hand movement. Initially, similar to Figure 7(A), the registered action setting is set at time T(b), when the subject's action is determined to be a specific action, and the focus is adjusted to the hand (first area). Subsequently, if it is determined at time T(c) that the subject's movement is not a specific movement, and the camera is to focus on the same person's face (second area) again, the camera will not set to the standard operation setting at time T(c), but will set to the standard operation setting at time T(d) after the camera has finished focusing on the person's face.

[0048] The configuration described above enables focus adjustment that allows you to appropriately switch AF operation settings according to the shooting scene during video recording. (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0049] This embodiment includes the following configurations and methods. (Composition 1) A detection means for detecting the movement of the subject in an image obtained by capturing the subject, It has setting means for setting parameters for focus adjustment, The setting means is characterized in that, if the movement of the subject is not a specific movement, the parameter is set to a first parameter, and if the movement of the subject is a specific movement, the parameter is set to a second parameter different from the first parameter. (Configuration 2) The detection means is capable of detecting the shooting scene in the image, The focus adjustment device according to Configuration 1, characterized in that the setting means sets the parameter to the first parameter when the subject's movement is the specific movement and the shooting scene is not the specific shooting scene, and sets the parameter to the second parameter when the subject's movement is the specific movement and the shooting scene is the specific shooting scene. (Composition 3) The detection means is capable of detecting a person in the image, The focusing device according to configuration 1 or 2, characterized in that the setting means sets the parameter to the first parameter when the subject's movement is the specific movement and the person is not the specific person, and sets the parameter to the second parameter when the subject's movement is the specific movement and the person is the specific person. (Composition 4) The setting means is characterized in that, when the movement of the subject is the specific movement and the image is in focus on a first region of the subject, the time taken to set the parameter from the second parameter to the first parameter when switching to a state where the movement of the subject is not the specific movement and the image is in focus on a second region different from the first region of the subject is longer than the time taken when switching to a state where the movement of the subject is not the specific movement and the image is in focus on a subject different from the subject. This is the focus adjustment device according to any one of configurations 1 to 3. (Composition 5) A focus adjustment device according to any one of configurations 1 to 4, further comprising a notification means for notifying that the aforementioned parameter is set to a second parameter. (Composition 6) The notification means is a display means for displaying the image, The focus adjustment device according to configuration 5, characterized in that when the parameter is set to the second parameter, the display means displays an icon superimposed on the image to indicate that the parameter is set to the second parameter. (Composition 7) A focus adjustment device according to any one of configurations 1 to 6, further comprising a registration means for registering the aforementioned specific operation. (Composition 8) The focus adjustment device according to configuration 7, characterized in that the registration means can change the specific operation. (Composition 9) A focus adjustment device according to any one of configurations 1 to 8, further comprising a storage means for storing the aforementioned specific operation. (Composition 10) A focus adjustment device according to any one of configurations 1 to 9, further comprising a determination means for determining whether the movement of the subject is the specific movement. (Composition 11) The focusing device according to any one of configurations 1 to 10, characterized in that the parameter includes at least one of speed and sensitivity. (Composition 12) A focusing device described in any one of configurations 1 to 11, An imaging device characterized by having an image sensor. (Method 1) A detection step for detecting the movement of the subject in an image obtained by capturing the subject, It includes a setting step for setting parameters for adjusting focus, A method for adjusting focus, characterized in that, in the setting step, if the movement of the subject is not a specific movement, the parameter is set to a first parameter, and if the movement of the subject is a specific movement, the parameter is set to a second parameter different from the first parameter. (Composition 13) A program characterized by causing a computer to execute the focus adjustment method described in Method 1.

[0050] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of symbols]

[0051] 161 Subject detection unit (detection means) 164 AF operation setting determination unit (setting means)

Claims

1. A detection means for detecting the movement of the subject in an image obtained by capturing the subject, It has setting means for setting parameters for focus adjustment, The setting means is characterized in that, if the movement of the subject is not a specific movement, the parameter is set to a first parameter, and if the movement of the subject is a specific movement, the parameter is set to a second parameter different from the first parameter.

2. The detection means is capable of detecting the shooting scene in the image, The focus adjustment device according to claim 1, characterized in that the setting means sets the parameter to the first parameter when the subject's movement is the specific movement and the shooting scene is not the specific shooting scene, and sets the parameter to the second parameter when the subject's movement is the specific movement and the shooting scene is the specific shooting scene.

3. The detection means is capable of detecting a person in the image, The focusing device according to claim 1, wherein the setting means sets the parameter to the first parameter when the action of the subject is the specific action and the person is not the specific person, and sets the parameter to the second parameter when the action of the subject is the specific action and the person is the specific person.

4. The focus adjustment device according to claim 1, characterized in that the setting means sets the time taken to set the parameter from the second parameter to the first parameter when switching to a state in which the subject's movement is not the specific movement and a second region different from the first region of the subject is in focus, when the subject's movement is the specific movement and a first region of the subject is in focus in the image, to be longer than the time taken when switching to a state in which the subject's movement is not the specific movement and a different subject is in focus in the image.

5. The focus adjustment device according to any one of claims 1 to 4, further comprising a notification means for notifying that the aforementioned parameter is set to a second parameter.

6. The notification means is a display means for displaying the image, The focus adjustment device according to claim 5, characterized in that when the parameter is set to the second parameter, the display means displays an icon superimposed on the image to indicate that the parameter is set to the second parameter.

7. The focus adjustment device according to any one of claims 1 to 4, further comprising registration means for registering the aforementioned specific operation.

8. The focus adjustment device according to claim 7, characterized in that the registration means can change the specific operation.

9. The focus adjustment device according to any one of claims 1 to 4, further comprising a storage means for storing the aforementioned specific operation.

10. The focus adjustment device according to any one of claims 1 to 4, further comprising determination means for determining whether the movement of the subject is the specific movement.

11. The focusing device according to any one of claims 1 to 4, characterized in that the parameter includes at least one of speed and sensitivity.

12. A focus adjustment device according to any one of claims 1 to 4, An imaging device characterized by having an image sensor.

13. A detection step for detecting the movement of the subject in an image obtained by capturing the subject, It includes a setting step for setting parameters for adjusting focus, A method for adjusting focus, characterized in that, in the setting step, if the movement of the subject is not a specific movement, the parameter is set to a first parameter, and if the movement of the subject is a specific movement, the parameter is set to a second parameter different from the first parameter.

14. A program characterized by causing a computer to execute the focus adjustment method described in claim 13.

Images