Control apparatus, imaging apparatus, control method, storage medium, and program product

Abstract

The invention provides a control apparatus, an imaging apparatus, a control method, a storage medium, and a program product. The control apparatus performs autofocus control on a predetermined subject selected based on a user operation for changing an in-focus state of a subject included in an image, and performs control to superimpose and display first information for indicating that the predetermined subject is a subject of the autofocus control on the predetermined subject. The autofocus control can be switched to any autofocus control among a plurality of autofocus controls including a tracking AF that always keeps a subject of the autofocus control in an in-focus state. When the auto focus control is set to track AF, the control apparatus performs tracking AF on the predetermined subject after the user operation ends.

Description

Technical Field

[0001] This invention relates to control devices, camera devices, control methods, storage media, and program products, and particularly to focus control. Background Technology

[0002] Japanese Patent 5474528 describes a method for determining the area closest to focus as the main subject area after a user's manual focus operation has ended, and continuously performing autofocus (AF) control on that main subject area. In Japanese Patent 5474528, when a detected subject, such as a person, is present at the focus position, AF control is performed on that person. When multiple people are present, AF control is performed on the person determined according to a priority order. When no person is detected, AF control is performed within the focus position area based on a range of the same color.

[0003] When shooting as a video production style, users can operate the lens with their left hand (focus, zoom, aperture, etc.) and the camera with their right hand (pan, tilt, and start / stop recording, etc.). In this case, when the user manually focuses the lens with their left hand, they can determine the area they intend to capture and select the main subject.

[0004] In Japanese Patent 5474528, the user cannot inspect the subject that is the object of AF control during manual focus operation. Because the AF frame is not displayed on the main subject determined by the camera, it is difficult for the user to inspect the main subject after the manual focus operation is completed and determine which subject to apply AF to. Summary of the Invention

[0005] The present invention was made in view of the above-mentioned problems and has realized a technique that makes it easy to control the main subject as the object of AF control selected by the user.

[0006] To address the aforementioned problems, the present invention provides a control device comprising: a display control unit for controlling the display of a captured image on a display unit; and a control unit for performing autofocus control on a predetermined subject selected based on a user operation for changing the focus state of a subject included in the image, wherein the display control unit controls the display to overlay and display first information indicating that the predetermined subject is the object of the autofocus control on the predetermined subject, the autofocus control being switchable to any of a plurality of autofocus controls including tracking AF, the tracking AF ensuring that the subject to which the autofocus control is located is always in focus, and, when the autofocus control is set to the tracking AF, the control unit performs the tracking AF on the predetermined subject after the user operation ends.

[0007] To address the aforementioned problems, the present invention provides a camera device comprising: a camera unit for capturing an image of a subject formed by an optical system; and a control device as described above.

[0008] To address the aforementioned problems, the present invention provides a control method for a control device, the control method comprising the following steps: performing autofocus control on a predetermined subject selected based on a user operation for changing the focus state of a subject included in a captured image; and controlling the display to overlay first information indicating that the predetermined subject is the object of the autofocus control on the predetermined subject.

[0009] To address the aforementioned problems, the present invention provides a computer-readable storage medium storing a program for enabling a computer to function as a control device as described above.

[0010] To address the aforementioned problems, the present invention provides a computer program product for enabling a computer to function as a control device as described above.

[0011] According to the present invention, the main subject, which is the object of AF control selected by the user operation, can be easily grasped.

[0012] Further features of the invention will become apparent from the following description of typical embodiments (with reference to the accompanying drawings). Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating the structure of a camera device according to this embodiment.

[0014] Figures 2A to 2C This is a diagram illustrating a method for setting a ranging area on a camera screen according to this embodiment.

[0015] Figure 3This is a diagram illustrating the AF control mode setting menu at the end of the focusing operation according to this embodiment.

[0016] Figure 4A and Figure 4B This is a flowchart illustrating the focusing control process according to this embodiment.

[0017] Figure 5 This is a flowchart illustrating the focusing mode determination process according to this embodiment.

[0018] Figure 6 This is a flowchart illustrating the AF frame determination process during subject detection according to this embodiment.

[0019] Figure 7 This is a flowchart illustrating the AF frame determination process when the subject is not detected according to this embodiment.

[0020] Figure 8 This is a flowchart illustrating the focus threshold setting process when the subject is not detected according to this embodiment.

[0021] Figure 9 This is a flowchart illustrating the AF frame display process during subject detection according to this embodiment.

[0022] Figure 10 This is a flowchart illustrating the AF frame display process when the subject is not detected according to this embodiment.

[0023] Figures 11A to 11D This is a diagram illustrating a subject region extraction method when the subject is not detected according to this embodiment.

[0024] Figure 12 This is a diagram illustrating the focus threshold setting method when the subject is not being detected according to this embodiment.

[0025] Figures 13A to 13C This is a diagram illustrating the AF frame display method during subject detection according to this embodiment.

[0026] Figures 14A to 14D This is a diagram illustrating the AF frame display method when the subject is not detected according to this embodiment. Detailed Implementation

[0027] The embodiments will be described in detail below with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. Several features are described in the embodiments, but this is not a limitation requiring all such features, and multiple such features can be appropriately combined. Furthermore, in the drawings, the same reference numerals are assigned to the same or similar structures, and redundant descriptions are omitted.

[0028] The following will describe an example of applying the camera device of the present invention to a digital camera, but the camera device can be applied to other camera devices such as digital still cameras.

[0029] Equipment Structure

[0030] First, refer to Figure 1 The hardware structure of the camera device according to this embodiment will be explained.

[0031] Figure 1 This is a block diagram illustrating the hardware structure of a camera device according to this embodiment.

[0032] The imaging device 100 includes an imaging optical system 150 for forming an image of the subject as light from the subject. The imaging optical system 150 includes a fixed first lens group 101, a zoom lens 102, an aperture 103, a fixed second lens group 104, and a focus compensation lens 105. The camera controller (hereinafter referred to as the camera microcomputer) 115 includes a focus position detection unit 119, a focus determination unit 120, a focus control unit 121, and a zoom control unit 122.

[0033] The zoom lens 102 moves along the optical axis and performs a zoom operation. The focus compensation lens (hereinafter referred to as the focusing lens) 105 has functions for correcting the movement of the focal plane accompanying the zoom operation and for adjusting the focus state. The imaging optical system 150 includes a focusing lens operation unit 124 (hereinafter referred to as the focusing ring). The focusing ring 124 is an operation member that can rotate around the optical axis of the imaging optical system 150 and can be operated by the user. When the user operates the focusing ring 124 in shooting mode, the focusing lens 105 is moved along the optical axis by the focusing lens drive unit 112 according to the amount of operation of the focusing ring 124. Note that information such as the driving amount, driving speed, and driving direction of the focusing lens 105 corresponding to the amount and direction of rotation of the focusing ring 124 (which is the operation information of the focusing ring 124) is output to the camera microcomputer 115. The imaging optical system 150 also includes a control ring operation unit (hereinafter referred to as the control ring) 125. The control ring 125 is an operating component that can rotate around the optical axis of the imaging optical system 150 and can be operated by the user. When the user operates the control ring 125, the detection of the subject or the shooting settings (ISO / aperture / gain, etc.) are switched according to the operation information of the control ring 125. Note that information such as the amount and direction of rotation of the control ring 125, which is the operation information of the control ring 125, is output to the camera microcomputer 115.

[0034] The imaging unit 106 includes an image sensor (which includes a photoelectric conversion element such as a CCD or CMOS, and peripheral circuitry for driving the photoelectric conversion element and reading signals), and performs photoelectric conversion on the subject image formed by light passing through the imaging optics system 150 on the imaging surface and outputs it as an electrical signal. The image sensor includes a microlens for pupil segmentation of the exit pupil of the imaging optics system 150, and the imaging device 100 performs autofocus (AF) control based on a pair of image signals obtained from light beams passing through different pupil regions using the imaging surface phase difference method.

[0035] The CSD / AGC circuit 107 includes: a CDS circuit for performing correlated double sampling processing to remove reset noise (low frequency) included in the camera signal output from the camera unit 106; and an AGC circuit for amplifying the camera signal output from the camera unit 106 and controlling it to a certain level.

[0036] The image signal processing unit 108 performs image processing on the camera signal output from the CDS / AGC circuit 107 to generate image data.

[0037] The display unit 109 includes a display device such as a liquid crystal panel or an organic EL, and displays image data output from the image signal processing unit 108, as well as setting information during shooting and an AF frame for indicating the focus detection area when AF control is started, as described later.

[0038] The recording unit 110 encodes the image data output from the image signal processing unit 108 using a motion picture compression method such as MP4 format and records the image data as a motion picture file on a recording medium, or decodes a motion picture file read from the recording medium. The recording medium is a magnetic tape, optical disc, or semiconductor memory, etc. Note that the camera device 100 according to this embodiment can generate not only motion picture data, but also still image data and motion picture data synthesized with audio data.

[0039] The zoom lens drive unit 111 includes an actuator, such as a stepper motor, DC motor, vibrating motor, and voice coil motor, for moving the zoom lens 102 in the optical axis direction. The focusing lens drive unit 112 includes an actuator, such as a stepper motor, DC motor, vibrating motor, and voice coil motor, for moving the focusing lens 105 in the optical axis direction.

[0040] Phase difference AF gate 113 allows the first and second focus detection signals of pixels within the ranging area set by the ranging area setting unit 118 used for phase difference AF (described later) to pass through. These pixels correspond to the focus detection area within the AF frame detected by the camera surface phase difference AF method. Here, the ranging area is also referred to as the focus detection area. Note that although camera surface phase difference AF will be described in this embodiment, the present invention is not limited to camera surface phase difference AF, and other focus detection methods can be used, as long as information related to the distance to the subject can be obtained. For example, the time-of-flight (ToF) method is considered, in which distance is calculated by emitting light (infrared light or laser) and measuring the time until the light hits the subject and reflects back. A method is considered in which a radio frequency identification (RFID) tag or an ultra-wideband (UWB) tag can be attached to the subject, and an antenna can receive signals from the tag to indicate the location.

[0041] The phase difference AF signal processing unit 114 performs correlation calculations based on the two focus detection signals that have passed through the phase difference AF gate 113, calculates the image offset for each focus detection area, and outputs the image offset to the focus position detection unit 119 of the camera microcomputer 115.

[0042] The camera microcomputer 115 includes a processor (CPU) for performing computational and control processing of the imaging device 100, non-volatile memory (ROM) for storing programs executed by the processor, and working memory (RAM) loaded with programs read from the non-volatile memory, constants, and variables for executing the program. The camera microcomputer 115 controls the components of the imaging device 100 by loading the program stored in the ROM into the RAM and executing the program.

[0043] Memory 123 is a flash memory that is externally attached to expand the memory area of ​​the camera microcomputer 115.

[0044] The focus position detection unit 119 calculates the defocus amount, which is the offset between the current focus position and the focus position on the imaging surface of the imaging unit 106, based on the image offset calculated by the phase difference AF signal processing unit 114 using the imaging surface phase difference AF method. Then, the focus position detection unit 119 outputs the defocus amount as a ranging result to the focus control unit 121. The focus position detection unit 119 converts the defocus amount into a driving amount for the focusing lens 105 and outputs this driving amount to the focus determination unit 120.

[0045] The focus determination unit 120 calculates control information for driving the focus lens 105 based on the driving amount of the focus lens 105 received from the focus position detection unit 119, and outputs the control information to the focus control unit 121.

[0046] The focusing control unit 121 controls the focusing lens drive unit 112 to drive the focusing lens 105 based on the control information of the focusing lens 105 received from the focusing judgment unit 120, and performs AF control.

[0047] The zoom control unit 122 controls the zoom lens drive unit 111 to drive the zoom lens 102 based on the drive amount and drive speed of the zoom lens 102 corresponding to the user's zoom operation, and performs zoom control.

[0048] Note that the AF method according to this embodiment is not limited to the camera plane phase difference AF method, and other AF methods can be applied.

[0049] Figure 1 The TV-AF gate 116 and TV-AF signal processing unit 117 are components used to implement TV-AF methods other than the camera plane phase difference AF method.

[0050] TV-AF gate 116 outputs to TV-AF signal processing unit 117 only the camera signal of the focus detection area used for focus detection using the TV-AF method from the camera signals of all pixels output from CDS / AGC circuit 107.

[0051] The TV-AF signal processing unit 117 applies known methods, such as using a filter to extract specific frequency components, to the camera signal of the focus detection area (AF frame in this embodiment) of TV-AF output from the TV-AF gate 116, and generates a TV-AF evaluation value with the highest image sharpness (contrast) at the focus position.

[0052] The TV-AF evaluation value is output to the focus determination unit 120. The TV-AF evaluation value is output to the focus control unit 121. The focus control unit 121 controls the focus lens drive unit 112 to drive the focus lens 105 based on the TV-AF evaluation value, and performs AF control using the TV-AF method.

[0053] The ranging region setting unit 118 used for phase difference AF sets the pixel group in all pixels of the imaging unit 106 that reads the first focus detection signal and the second focus detection signal input to the phase difference AF gate 113. The ranging region setting unit 118 can variably set the focus detection region for reading the focus detection signal used for focus detection using the phase difference AF method according to the subject detection state.

[0054] The subject region detection unit 126 performs known subject detection processing on the camera signal output from the CDS / AGC circuit 107, and detects the subject present in the camera frame corresponding to the image data generated by the image signal processing unit 108. According to this embodiment, the subject detection processing is facial detection processing for detecting facial regions of a person. For facial region detection processing, for example, there is a method for extracting skin color regions from the gradient colors of each pixel represented by the image data, and detecting the face based on the matching degree with a pre-prepared facial contour plate. In addition, there are methods for facial detection by extracting facial feature points such as eyes, nose, and mouth by referring to known pattern recognition techniques. This embodiment is not limited to a specific subject detection processing, and any method can be used. The subject being detected is not limited to a person or face, and can be an animal, insect, or vehicle, etc., other than a person or face.

[0055] Note that the camera device 100 according to this embodiment can switch to any of a plurality of focusing modes, including AF mode, MF mode, and MF / AF switching mode. AF mode is used to automatically adjust the focus of the subject (main subject) to a focusing state via AF control. MF mode is used by the user to manually adjust the focus ring 124 to bring the desired subject (main subject) into focus. MF / AF switching mode is a mode that automatically switches to AF mode after the operation in MF mode ends, and continues to perform AF control with the subject that has entered focus state through manual focusing operation as the main subject.

[0056] How to set up the distance measurement area

[0057] Next, refer to Figures 2A to 2C This section will explain the method for setting the ranging region using the ranging region setting unit 118 according to this embodiment.

[0058] Figures 2A to 2C An example of the ranging area set by the ranging area setting unit 118 is shown.

[0059] Figure 2A An example is shown of a ranging region 201 set on the camera frame 200 corresponding to the image data generated by the image signal processing unit 108. The ranging region 201 includes multiple rectangular regions arranged in two dimensions. Note that... Figure 2A This example illustrates the ranging area when no subject is detected on the camera frame 200, and regardless of whether the focusing mode of the camera device 100 is AF mode, MF mode, or MF / AF switching mode, the ranging area 201 remains the same. Figure 2A The arrangement is as shown in the example.

[0060] Figure 2B Example of ranging areas 202 to 204 during subject detection. For each of the subjects 205 to 207, multiple two-dimensional ranging areas corresponding to the center and size of the face are set. Note that... Figure 2B Examples of ranging areas are shown when a subject is detected, and examples of the arrangement are shown when the focusing mode of the camera device 100 is AF mode or MF / AF switching mode. The ranging area is as follows: Figure 2B The arrangement shown is intended to reduce focus blur in the AF control of each detected subject.

[0061] Figure 2C Example of a ranging area 208 during subject detection. Figure 2C In the example, multiple two-dimensional ranging regions are set for all detected subjects 209 to 211. Note that... Figure 2C The example shows the ranging area when a subject is detected, and also illustrates the arrangement when the focusing mode of the camera device 100 is MF mode. The ranging area is as follows... Figure 2C The illustrated arrangement allows the camera device 100 to efficiently target the subject aimed at by the user through manual focus, and this arrangement is consistent with... Figure 2A The state is similar.

[0062] In this way, by changing the ranging range according to the detection state of the subject and the focusing mode, a subject that matches the user's intention can be specified, and the focus state on the subject can be maintained in the AF control.

[0063] The focusing control unit 121 is based on the Figures 2A to 2C AF control is performed using at least one or more of the multiple ranging results obtained from the illustrated ranging areas 201 to 204.

[0064] AF control mode

[0065] Next, the AF control mode will be explained.

[0066] The camera device 100 according to this embodiment can execute the following multiple AF control modes. The camera device 100 according to this embodiment can be changed to the following first AF control mode, second AF control mode, or third AF control mode by user operation.

[0067] • First AF control mode: Center and closest distance priority AF

[0068] • Second AF control mode: Facial AF

[0069] • Third AF control mode: Tracking AF

[0070] In the "First AF Control Mode: Center and Closest Distance Priority AF", AF control is performed when the subject is prioritized as the closest to the center of the image and the subject along the optical axis. The focus control unit 121 determines whether a subject is located as the closest to the center of the image and the subject along the optical axis based on the subject detection results detected by the subject area detection unit 126. Then, when the distance measurement result of the subject located as the closest to the center and the subject along the optical axis is available, AF control is performed based on the distance measurement result of the subject located as the closest to the center and the subject along the optical axis. As a result, the subjects that appear to be closest to the center of the image and the subject along the optical axis are focused one after another, thus making the AF control highly versatile and enabling AF in various scenarios.

[0071] In the "Second AF Control Mode: Face AF", the position and size of the face area of ​​the person as the main subject are determined based on the subject detection results detected by the subject area detection unit 126. AF control is performed while tracking the main subject, ensuring that the distance measurement results of the face area are always in focus. When multiple faces are present in the image, the priority face is, for example, the face closest to the center of the image and having the largest size. Therefore, the camera device 100 automatically determines the priority order of subjects and performs AF control based on the subject detection results in the image, which is effective when a subject such as a face is detected.

[0072] Exclusively control "First AF Control Mode: Center and Closest Distance Priority AF" and "Second AF Control Mode: Face AF", and when there is no subject detected in "Second AF Control Mode: Face AF", the mode is switched to "First AF Control Mode: Center and Closest Distance Priority AF".

[0073] In the "Third AF Control Mode: Tracking AF," AF control is performed while tracking the main subject selected by the user from multiple detected subjects within the camera frame to maintain focus. For example, tracking AF is applied to the main subject so that the distance measurement results of the area corresponding to the user-specified position are always in focus. For instance, the user specifies a position on the camera frame, and the position and size of the subject are determined based on the color information of the specified position and the distance measurement results, or the position and size of a face are determined based on the detection results of a face selected by the user. Thus, AF control continues only for the subject the user is aiming at, and the camera 100 does not change the subject, making AF control highly dedicated.

[0074] "First AF control mode: center and closest distance priority AF" and "Third AF control mode: tracking AF" are exclusive controls, and in "Third AF control mode: tracking AF", when there is no subject to be detected, the control mode is switched to "First AF control mode: center and closest distance priority AF".

[0075] Since AF control is known, its detailed description will be omitted.

[0076] Menu settings

[0077] Next, refer to Figure 3 This section will explain the menu settings items according to this embodiment.

[0078] The menu according to this embodiment includes a menu name 301. Menu name 301 is "AF control at the end of MF operation", and AF control can be selected after the manual focus operation ends.

[0079] According to this embodiment, menu 301 includes three options 302, 303, and 304. The first option 302 is a setting to "Do not perform tracking AF". When the first option 302 is selected, tracking AF will not begin after the manual focusing operation ends. Specifically, when the AF mode is set to "First AF Control Mode: Center and Closest Distance Priority AF" or "Second AF Control Mode: Face AF", the same AF control is maintained even after the manual focusing operation ends, or the operation of the focus ring 124 is restricted.

[0080] Option 303 is the setting for "Perform Tracking AF (Do not display tracking candidate box)". When option 303 is selected, tracking AF begins after the manual focus operation ends. Specifically, when the AF mode is set to "First AF Control Mode: Center and Closest Distance Priority AF" or "Second AF Control Mode: Face AF", the AF mode is switched to "Third AF Control Mode: Tracking AF" after the manual focus operation ends.

[0081] In the second option 303, if the user operates the focus ring 124 to change to any of the multiple detected subjects having a distance difference relative to the imaging optical system 150, tracking AF begins for the subject after the manual focus operation ends. This eliminates the need for the user to perform operations to change the AF control mode.

[0082] The third option 304 is the setting for "Perform Tracking AF (Display Tracking Candidate Box)". In the third option 304, the switching of the AF control mode after the manual focus operation ends is the same as the switching in the second option 303. During and after the manual focus operation, a tracking candidate box is displayed on the main subject selected by the user using the focus ring 124 for tracking. (See later reference...) Figures 14A to 14D The explanation is provided, but the tracking candidate box is displayed in a different format than the tracking box used to indicate that tracking AF will be performed on the selected main subject after the manual focus operation ends. In the third option 304, the user can check whether the subject is in general focus while operating the focus ring 124. In this way, by presenting the subject as the object of tracking AF after the manual focus operation ends to the user in advance during the manual focus operation, shooting failures can be reduced.

[0083] Note that the following will refer to Figure 9 To explain the details of option 302, option 303, and option 304.

[0084] Methods for determining the main subject from multiple detected subjects

[0085] In this embodiment, the primary subject that is the tracking target of the tracking AF can be determined from the plurality of subjects detected by the subject region detection unit 126 by the following first subject determination method A or second subject determination method B.

[0086] First subject determination method A: Method of changing the priority of the subject

[0087] Second subject determination method B: Method of changing the focus state of the subject

[0088] "A: Method for changing the priority of a subject" is as follows: This method assigns priority to multiple detection subjects based on their position and size, and determines the primary subject by allowing the user to change the priority. The user can change the priority order of the subjects and determine the primary subject by selecting detection subjects using touch operation (not shown), cross-key operation, or operation input of the control ring 125.

[0089] "B: The method of changing the focus state of the subject" is a method of determining the subject that is in general focus from multiple detected subjects by using the focus ring 124 to change the focus state of the subject.

[0090] According to this embodiment, the camera device 100 can operate the first subject determination method A or the second subject determination method B through the following different operating components.

[0091] • First subject determination method A: Control loop 125

[0092] Second subject determination method B: Focus ring 124

[0093] Focusing ring 124 and control ring 125

[0094] In the camera device 100 according to this embodiment, such as Figure 1 As illustrated, the camera optical system 150 includes a focusing ring 124 and a control ring 125.

[0095] When switching the main subject via the focus ring 124, the camera device 100 of this embodiment determines that the subject in a roughly focused state is the subject that the user is aiming at and performs tracking AF. However, when there are multiple detected subjects at approximately the same distance relative to the camera optical system 150, and the multiple subjects are in a roughly focused state, it becomes difficult to determine the subject that the user is aiming at.

[0096] Therefore, in this embodiment, when multiple subjects are detected at approximately the same distance, the primary subject can be switched by operating the control ring 125 instead of the focus ring 124. Thus, for example, similar to other operating components (cross-pad and touch operation), the AF frame can be moved to the right or left of the subject displaying the current AF frame, depending on the rotation direction of the control ring 125.

[0097] In this embodiment, when switching between multiple detection subjects with distance differences, the operation of the focusing ring 124 is recommended, and when switching between subjects without distance differences, the operation of the control ring 125 is recommended.

[0098] This has two advantages.

[0099] The first advantage is that the user can select the subject (subject with a distance difference or no distance difference) using only their left hand. In cases where multiple subjects are known to exist at approximately the same distance, the user would need to use their right hand to operate the directional pad or similar controls to select the subject. In this embodiment, since the camera optical system 150 includes two operating components—a focusing ring 124 and a control ring 125—the user can select the subject (subject with a distance difference or no distance difference) simply by operating the operating components of the camera optical system 150, which are held in the left hand.

[0100] The second advantage is that users can control the rack focus speed when switching subjects by operating the recommended focus ring 124 when selecting subjects with a distance difference, thus improving the quality of moving images. On the other hand, when selecting subjects without a distance difference, the quality of the rack focus is not required, so switching subjects simply by operating the control ring 125 will not affect the quality of moving images.

[0101] Focusing control processing

[0102] Next, refer to Figures 4A to 10 This will explain the focusing control process according to this embodiment.

[0103] According to this embodiment, the focus control processing is achieved by loading the program stored in the ROM into the RAM and executing the program through the camera microcomputer 115, and controlling each component as the focus control unit 121.

[0104] First, refer to Figure 4A and Figure 4B The main flow of the focusing control process according to this embodiment will be explained.

[0105] Figure 4A and Figure 4B The processing is performed repeatedly according to a predetermined period (e.g., the readout period of the camera signal of the camera unit 106).

[0106] In step S400, the focusing control unit 121 obtains the lens information of the imaging optical system 150 and initiates the process in step S401. The lens information includes focal length information, subject distance information, aperture information, operation information of the focusing ring 124, and operation information of the control ring 125.

[0107] In step S401, the focusing control unit 121 obtains the reference value from the user. Figure 3 The menu screen displays information related to "AF control at the end of the focusing operation," and the process proceeds to step S402.

[0108] In step S402, the focusing control unit 121 sets a reference. Figures 2A to 2C The ranging area is defined, and the process proceeds to step S403.

[0109] In step S403, the focus control unit 121 obtains the ranging result from the focus position detection unit 119 and proceeds to step S404. In this embodiment, it is assumed that the ranging result is obtained... Figures 2A to 2C The results for all distance measurement areas set in the diagram.

[0110] In step S404, the focus control unit 121 determines whether the setting of "AF control after MF operation ends" obtained in step S401 is "tracking". If the setting is "not tracking", the process proceeds to step S405. If the setting is "tracking", the process proceeds to step S408.

[0111] In step S405, the focus control unit 121 performs processing to determine the AF frame and proceeds to step S406. When a face is detected, the focus control unit 121 uses the ranging area for the face as the AF frame for face AF. When no face is detected, the focus control unit 121 sets the ranging area closest to the center of the image and the optical axis as the AF frame if a ranging result that is closest to the center of the image and the optical axis can be obtained; otherwise, the focus control unit 121 predicts the position of the subject based on the current ranging result and the ranging area is the AF frame.

[0112] In step S406, the focus control unit 121 performs processing for displaying the AF frame, and the processing proceeds to step S407. If a face is detected, the focus control unit 121 displays the AF frame on the face; otherwise, the center and nearest-distance priority AF of the AF frame are not displayed.

[0113] In step S407, the focusing control unit 121 performs AF control processing, returns the processing to step S400, and repeats the processing. The focusing control unit 121 outputs control information for driving the focusing lens 105 to the focusing lens drive unit 112 based on the ranging result of the AF frame determined in step S405.

[0114] In step S408, the focus control unit 121 determines whether the focus mode is "AF mode," "MF mode," or "MF / AF switching mode," and proceeds to step S409. (See below for further details.) Figure 5 To explain the details of this process.

[0115] In step S409, the focus control unit 121 determines whether the focus mode is "AF mode". If the focus mode is AF mode, the process proceeds to step S410; otherwise, the process proceeds to step S415.

[0116] In step S410, the focus control unit 121 determines whether it has received an instruction from the control loop 125 to change the main subject, and if the instruction has been received, it causes the process to proceed to step S411; otherwise, it causes the process to proceed to step S412.

[0117] In step S411, the focus control unit 121 performs processing to change the subject, and the processing proceeds to step S412. The user can notify the camera microcomputer 115 of a subject change command by rotating the control ring 125. When multiple subjects are detected by the subject area detection unit 126, and when a subject change command is given from the control ring 125, the focus control unit 121 changes the main subject according to the command.

[0118] For example, in Figure 2B When the control ring 125 is rotated to the right, and the face currently displaying the AF frame is the subject 207 being detected, the AF frame moves to the subject 206 being located on the right. When the control ring 125 is rotated to the left, and the face currently displaying the AF frame is the subject 207 being detected, the AF frame moves to the subject 205 being located on the left side of the face displaying the AF frame.

[0119] In step S412, the focus control unit 121 performs processing to determine the AF frame and proceeds to step S413. When tracking AF is being performed, the focus control unit 121 sets the ranging area of ​​the tracked object as the AF frame. When tracking AF is not being performed, and when a ranging result closest to the center of the image frame in the optical axis direction can be obtained, the focus control unit 121 sets the ranging area closest to the center of the image frame in the optical axis direction as the AF frame; otherwise, the focus control unit 121 predicts the position of the subject based on the current ranging result and sets the ranging area as the AF frame.

[0120] In step S413, the focus control unit 121 performs processing for displaying the AF frame and proceeds to step S414. When tracking AF is being performed, the focus control unit 121 displays the AF frame in the ranging area of ​​the tracked object; otherwise, center and closest distance priority AF is used and the AF frame is not displayed.

[0121] In step S414, the focusing control unit 121 performs AF control processing, returns the processing to step S400, and repeats the processing. Based on the ranging result of the AF frame determined in step S412, the focusing control unit 121 outputs control information for driving the focusing lens to the focusing lens drive unit 112.

[0122] In step S415, the focus control unit 121 determines whether there is a detected subject in the approximate focus range, and if there is a detected subject, the process proceeds to step S416; otherwise, the process proceeds to step S419.

[0123] In step S416, the focus control unit 121 performs AF frame determination processing during subject detection and proceeds to step S417. (See below for further details.) Figure 6 To explain the details of this process.

[0124] In step S417, the focus control unit 121 performs AF frame display processing during subject detection and proceeds to step S418. (See below for further details.) Figure 9 as well as Figures 13A to 13C To explain the details of this process.

[0125] In step S418, the focusing control unit 121 performs manual focusing control processing, returns the processing to step S400, and repeats the processing. The focusing control unit 121 outputs control information for driving the focusing lens to the focusing lens drive unit 112 based on the operation amount of the focusing ring 124 or the focusing operation amount using a remote controller (not shown).

[0126] In step S419, the focus control unit 121 performs AF frame determination processing when the subject is not detected, and then proceeds to step S420. (See below for further details.) Figure 7 and Figure 8 To explain the details of this process.

[0127] In step S420, the focus control unit 121 performs AF frame display processing when the subject is not detected, and then proceeds to step S418. (See below for further details.) Figure 10 and Figures 14A to 14D To explain the details of this process.

[0128] Focusing mode detection and processing

[0129] Next, refer to Figure 5 To explain Figure 4A The focus mode determination process in step S408.

[0130] In step S501, the focusing control unit 121 determines whether to perform a manual focusing operation using the focusing ring 124 or a manual focusing operation to drive the focusing lens 105 using a remote control (not shown). If a manual focusing operation is performed, the focusing control unit 121 causes the process to proceed to step S502; otherwise, the process proceeds to step S503.

[0131] In step S502, the focusing control unit 121 sets the focusing mode to "MF mode" and ends the process.

[0132] In step S503, the focus control unit 121 determines whether a predetermined time has elapsed since the end of the manual focus operation. If the predetermined time has elapsed, the process proceeds to step S504; otherwise, the process proceeds to step S505. In this embodiment, the predetermined fixed time is set to, for example, 0.6 seconds.

[0133] In step S504, the focus control unit 121 determines that the manual focus operation has ended, sets the focus mode to "AF mode", and ends the process.

[0134] In step S505, the focus control unit 121 determines that a manual focus operation is in progress, sets the focus mode to "MF / AF switching mode" as preparation for switching from MF mode to AF mode, and ends the process.

[0135] There are also methods for switching between AF and MF before manual focusing. However, in this embodiment, in order to seamlessly switch between AF and MF, the focusing mode is changed to AF mode based on whether the manual focusing operation has ended.

[0136] AF frame determines processing

[0137] Next, refer to Figure 6 To explain Figure 4B The AF frame determination process during subject detection in step S416.

[0138] In step S601, the focus control unit 121 determines whether there are multiple detected subjects within the approximate focus range. If there are no multiple detected subjects within the approximate focus range, the focus control unit 121 proceeds to step S603 and sets an AF frame for one detected subject within the approximate focus range. If there are multiple detected subjects within the approximate focus range, the focus control unit 121 proceeds to step S602 and sets an AF frame for the detected subject positioned as being closest to the center of the image frame in the optical axis direction. Note that the method for determining the AF frame is not limited to... Figure 6 For example, when there are multiple detected subjects in the approximate focus range, the AF frame can be set for the detected subject that is closest to the focus state or the detected subject that is closest to the center of the image.

[0139] Next, refer to Figure 7 To explain Figure 4B The AF frame determination process in step S419 when the subject is not detected.

[0140] In step S700, the focusing control unit 121 performs processing for setting a first focus threshold and a second focus threshold, and then proceeds to step S701. The first focus threshold is used to set a first focus threshold and a second focus threshold. Figure 4A The second focus threshold is a threshold used to extract focus information of the subject area from the ranging result obtained in step S403. The second focus threshold is a threshold used to set focus information of a representative area from the main subject area extracted based on the first focus threshold. The representative area is the area set when the user changes the focus of the subject using the focus ring 124, and is, for example, set as an AF frame for AF control after the manual focus operation ends. During the manual focus operation, the representative area is set... Figure 4B The tracking candidate box at the start of AF in step S420. (See below for reference.) Figure 8 This section details the methods for setting the first and second focus thresholds.

[0141] In step S701, the focus control unit 121 performs processing to extract the subject area based on the first focus threshold set in step S700, and then proceeds to step S702. Here, reference will be made to... Figure 11A This section explains the method for extracting the subject region when the subject is not being detected, according to this embodiment. Figure 11A In the example, subjects 1100 to 1104 are objects or the ground. Objects 1100 and 1101 are positioned at approximately the same distance relative to the camera optical system 150. Objects 1102 and 1103 are positioned as closest to the center of the image and the optical axis when viewed from object 1100, and the ground 1104 is positioned at approximately the same distance. Then, the first focus information 1105 is... Figure 4A The first focus information obtained in step S403 is determined based on the first focus threshold set in step S700 in the ranging result. Subject region 1106 is the extracted subject region, and the extraction method is, for example, to extract it as a subject region when each focus information 1105 is adjacent. The extraction method can be a method for extracting subject regions by color or brightness, etc. When multiple objects (such as object 1100 and object 1101, etc.) exist within the range of the first focus threshold, if the focus information 1105 is not adjacent (such as subject region 1106 and subject region 1107, etc.), it can be extracted as another subject region.

[0142] In step S702, the focus control unit 121 determines whether a non-subject area exists within the subject area extracted in step S701. If a non-subject area is determined to exist, the process proceeds to step S703; otherwise, the process proceeds to step S704. Here, reference will be made to... Figure 11B This will explain the method for determining the non-subject area according to this embodiment. Figure 11B Examples of states include: From Figure 11AFrom the point of focus, the user operates the focus ring 124 to be closest to the center of the image frame and the nearest distance along the optical axis, and the objects 1102, 1103, and the ground 1104 are in focus. Region 1108 is the area containing the first focus information when the focus ring 124 is operated, and the objects 1102, 1103, and the ground 1104 are extracted as a subject region. In this way, for example, when the subject regions of different objects are continuous with the ground region in between, these subject regions are extracted as a single subject region. In order to extract this single subject region 1108 as a separate subject region, when a portion of the ground region of the subject region is larger than a predetermined proportion relative to the image frame, as in region 1109, that portion is determined to be a non-subject region and is removed. This non-subject region is, for example, a ground region continuous with the subject region, and can be a wall or ceiling, etc.

[0143] In step S703, the focus control unit 121 performs processing to remove non-subject areas, and the processing proceeds to step S704. Here, reference will be made to... Figure 11C This embodiment describes a method for removing non-subject areas. Figure 11C Example from Figure 11B The state of removing non-subject areas 1109 determined in step S702 from the subject area 1108. By removing non-subject areas 1109 from the subject area 1108, an adjacent subject area 1108 is divided into non-adjacent subject areas 1110 and 1111, and can therefore be extracted as separate subject areas. In this way, by removing non-subject areas, the extraction accuracy of the subject area is improved, and non-subject areas are not selected in the AF frame determination process.

[0144] In step S704, the focus control unit 121 performs a process for setting a reference point from the subject area where the non-subject area has been removed in step S703, and then proceeds to step S705. Here, reference will be made to... Figure 11CThis embodiment describes a method for setting a reference point from a subject area after the removal of non-subject areas. First, a primary subject area is determined based on the remaining subject area 1110 and subject area 1111 after the removal of non-subject areas in step S703. In the method for determining the primary subject area, when multiple subject areas are extracted, for example, the largest subject area is determined as the primary subject area, or the subject area at the center of the frame is determined as the primary subject area. When only one subject area is extracted, that subject area is determined as the primary subject area. When no subject area is extracted, the primary subject area is not determined. Next, after determining the primary subject area, a reference point 1112 is set from the primary subject area. The reference point 1112 can be, for example, set as the center of gravity of the primary subject area or the position closest to the center of the image frame in the optical axis direction.

[0145] In step S705, the focus control unit 121 sets a representative area based on the reference point of the subject determined in step S704 and the second focus threshold set in step S700, and then proceeds to step S706. Here, reference will be made to... Figure 11D The method for setting a representative region according to this embodiment will be explained. Region 1113 is second focus information determined based on the second focus threshold set in step S700. Region 1114 is the representative region. First, since the representative region 1114 is selected from the second focus information 1113, which is the focus range, when the representative region 1114 is set as the AF frame in step S707, AF control can start from the focus position in the focus range. Therefore, when starting AF control after the manual focusing operation ends, it is not necessary to consider the continuity between the user's focusing ring 124 operation speed and the AF control speed, and the change in focus position is small, so the quality of the moving image is not degraded. Next, the method for selecting the representative region 1114 from the second focus information 1113 will be explained. The second focus information may be, for example, a region including a large amount of background around the object 1102, such as focus information 1113, or a region included in the object 1103, such as focus information 1116. Therefore, when selecting the representative region 1114 from the second focus information, there is a possibility that the region outside the object 1102 (which is the main subject region set in step S704) may be selected. To avoid this, when selecting the representative region 1114 from the second focus information 1113, the reference point set in step S704 is set in the region with the shortest distance in the xy plane. In this way, by setting the region with the shortest distance relative to the reference point and included in the second focus information as the representative region, regions different from the main subject region will not be set as representative regions.

[0146] In step S706, the focusing control unit 121 determines whether the representative area obtained in step S705 exists, and if the representative area exists, the process proceeds to step S707; otherwise, the process proceeds to step S709.

[0147] In step S707, the focus control unit 121 sets the representative area as the AF frame and causes the process to proceed to step S708.

[0148] In step S709, the focus control unit 121 sets the AF frame when the represented area does not exist as described in step S706, and proceeds to step S710. If a face is detected, the focus control unit 121 sets the ranging area for the face as the AF frame and performs face AF. If no face is detected, the focus control unit 121 sets the ranging area as the AF frame if a ranging result with the closest distance to the center of the image frame in the optical axis direction exists; otherwise, the focus control unit 121 predicts the position of the subject based on the current ranging result and sets the ranging area as the AF frame.

[0149] In step S708, the focus control unit 121 sets the AF control mode to "tracking AF" and initiates processing. Figure 4B Step S420 in the process.

[0150] In step S710, the focus control unit 121 sets the AF control mode to "Face AF / Center and Closest Distance Priority AF" and initiates processing. Figure 4B Step S420 in the process.

[0151] Setting and processing of the first and second focus thresholds

[0152] Next, refer to Figure 8 To explain Figure 7 The setting process of the first focus threshold and the second focus threshold in step S700.

[0153] In step S800, the focusing control unit 121 performs operations based on... Figure 4A The lens information obtained in step S400 is used to set the upper and lower limits of the width as the first focus threshold, and the process proceeds to step S801. Here, reference will be made to... Figure 12 This section explains the method for setting the width of the first focus threshold according to this embodiment. Figure 12In the diagram, the vertical axis indicates the depth of field, and 0 indicates the focus position; the larger the number, the farther away from the focus position. FΔ is the unit of focus, and in this embodiment, the depth of field range from -1FΔ to 1FΔ is set as the focus range. Note that the focus range represents the range of depth of field that appears to be in focus, and is the range added to the focus position (0FΔ) by the front focus distance (closest distance side) and the rear focus distance (infinity distance side).

[0154] Region 1200 indicates the range of the first focus threshold, and regions 1201 and 1202 indicate the area outside the range of the first focus threshold. Regions 1203 to 1205 indicate objects, indicating that objects 1203 and 1204 are within the range of the first focus threshold, and that object 1205 is outside the range of the first focus threshold. In this embodiment, the first focus threshold is set in the range of -1FΔ to 3FΔ, and is configured to include not only the focus range (-1FΔ to 1FΔ) but also the non-focus range (1FΔ to 3FΔ). Note that in this embodiment, the first focus threshold is the depth of field, but it can be a value based on the subject distance. In this way, by setting a first focus threshold that includes the non-focus range, for example, it can be... Figure 7 In step S701, objects such as object 1203, which are slightly off-focus but have a large presence, are extracted as large subject areas. By extracting large subject areas, the performance is improved. Figure 7 In step S704, the possibility of selecting the main subject is determined. On the other hand, since the object extracted by magnifying the subject area is selected as the main subject area, it is possible to prevent the selection of objects such as object 1204 that are within the focus range but are not desired to be selected as the main subject (e.g., small objects such as stones). By setting a first focus threshold that includes the outside of the focus range, even if the focus position slightly shifts from the focus range when the user focuses on the object using the focus ring 124, the object can be extracted as a large subject area. On the other hand, when the first focus threshold is set to the focus range (-1FΔ to 1FΔ) and the subject area is extracted, the size of the extracted subject area becomes smaller. For this reason, when the focus position slightly shifts from the object when the user ends the operation of the focus ring 124, the relative size of the subject area of ​​the object changes, and there is a possibility that a ground area or other small subject area at approximately the same distance will be selected as the main subject area.

[0155] In step S801, the focus control unit 121 determines the operating direction of the focus ring 124. When the focus ring 124 is operated to the infinite distance side in the optical axis direction, the focus control unit 121 causes the process to proceed to step S802, and when the focus ring 124 is operated to be closest to the center of the image frame in the optical axis direction, the process proceeds to step S805.

[0156] In step S802, the focus control unit 121 sets the first focus threshold based on the operating direction (infinity distance side) of the focus ring 124 in step S801 and the width of the first focus threshold set in step S800. Here, reference will be made to... Figure 12 The method for setting a first focus threshold according to this embodiment will be explained. Arrow 1206 indicates the operating direction of the focusing ring 124. Region 1200 indicates the range of the first focus threshold. Figure 12 In the example, the range 1200 of the first focus threshold is divided into a range on the closest distance side and a range on the infinity side, with the focus position (0FΔ) as the boundary, and the proportion of the threshold on the infinity side (which is the operation direction 1206 of the focus ring 124) is set to be greater than the proportion of the threshold on the closest distance side. In this way, by increasing the proportion of the range of the first focus threshold on the operation direction 1206 of the focus ring 124, objects located on the closest distance side relative to the focus position when the user ends the operation of the focus ring 124 can be excluded from the options. On the other hand, when AF control is performed on the main subject without excluding objects on the closest distance side, there is a possibility that the focus position changes in the direction opposite to the operation direction 1206 of the focus ring 124. Therefore, by increasing the proportion of the first focus threshold on the operation direction 1206 of the focus ring 124, the subject area can be extracted significantly without the focus position changing in the direction opposite to the operation direction 1206 of the focus ring 124.

[0157] In step S805, the focus control unit 121 sets the first focus threshold based on the operating direction (closest distance side) of the focus ring 124 in step S801 and the width of the first focus threshold set in step S800. In this case, since the operating direction of the focus ring 124 is on the closest distance side, the range of the first focus threshold is set such that the proportion on the closest distance side is greater than the proportion on the infinite distance side.

[0158] In step S803, the focusing control unit 121 determines whether the operation of the focusing ring 124 has ended. If it determines that the operation of the focusing ring 124 has ended, the process proceeds to step S804; otherwise, the process proceeds to step S806. The focusing control unit 121 determines that the operation of the focusing ring 124 has ended if the operating speed of the focusing ring 124 is decelerating, and determines that the focusing ring is currently operating if the operating speed of the focusing ring 124 is accelerating or constant.

[0159] In step S804, if it is determined in step S803 that the operation of the focusing ring 124 has ended, the focusing control unit 121 performs processing for setting the second focus threshold and initiates the processing. Figure 7 Step S701. In this case, the second focus threshold is set, for example, to a value smaller than the first focus threshold set in step S800. This is because, when the user ends the operation of the focus ring 124, an operation for focusing on the subject is performed, and it is expected that the focus position will not change after the user ends the operation of the focus ring 124. Therefore, by... Figure 7 The representative area set in step S705 is set within the focus range, so the change in focus position after the operation of focus ring 124 ends is smaller, and AF control can be performed on the main subject without reducing the quality of the moving image.

[0160] In step S806, if it is determined in step S803 that the focusing ring 124 is being operated, the focusing control unit 121 performs processing for setting the second focus threshold and initiates the processing. Figure 7 Step S701. In this case, the second focus threshold is set to the same value as the first focus threshold set in step S800. During the operation of the focus ring 124, it is not necessary to set a representative area from the focus range, and the reference point set in step S704 is set as the representative area. Thus, for example, when the reference point is the center of gravity of the subject area, it is possible to... Figure 4B In step S420, the tracking candidate box at the start of AF is displayed at the center of gravity of the subject area. Figure 4B If the position of the tracking candidate box at the start of AF, as shown in step S420, is set to the center of gravity of the subject area, the user can easily grasp the starting position of AF after the operation of the focus ring 124 ends.

[0161] AF frame display processing

[0162] Next, refer to Figure 9 To explain Figure 4B The AF frame display processing during subject detection in step S417.

[0163] In step S901, the focus control unit 121 determines the focus mode, and if the mode is "MF / AF switching mode", the process proceeds to step S902; otherwise, the process proceeds to step S903.

[0164] In step S902, when the user has finished operating the focusing ring 124, the focusing control unit 121 targets the focus ring 124. Figure 4B Step S416 sets the AF frame to detect the subject and displays it. Figure 13A The illustrated face tracking bounding box. In Figure 13A In the example, tracking box 1302 is shown, which indicates the tracking for... Figure 4B In step S416, the detection subject 1301 of the AF frame is set to perform tracking AF.

[0165] In step S903, the focus control unit 121 determines whether the tracking candidate frame display setting is valid. If the tracking candidate frame display setting is valid, the process proceeds to step S904. If the tracking candidate frame display setting is invalid, the process proceeds to step S905.

[0166] In step S904, the focus control unit 121 detects that the user is operating the focus ring 124 and... Figure 4B Step S416 sets the AF frame to detect the subject display. Figure 13B The illustrated face bounding box 1304 and tracking candidate bounding box 1305 are shown. Figure 13B In the example, Figure 4B In step S416, a face frame 1304 for indicating the detected subjects within the approximate focus range is displayed on the AF frame detection subject 1303. At the primary subject designated by the user-operated control ring 125 among the multiple detected subjects 1303 within the approximate focus range, a tracking candidate frame 1305 for indicating tracking candidates is displayed.

[0167] In step S905, the focusing control unit 121 targets the focus control unit 121 in the following steps: Figure 4B Step S416 sets the AF frame to detect the subject display. Figure 13C The illustrated face frame 1304. Figure 13C In the example, Figure 4B In step S416, a face frame 1304 for indicating the approximate focus range of the detected subject is displayed on the AF frame detection subject 1303. Since the tracking candidate frame display setting is invalid, the tracking candidate frame 1305 is not displayed on the primary subject designated by the user-operated control ring 125 among the multiple detected subjects 1303.

[0168] Note that the tracking box, face box, and tracking candidate box are displayed in different forms so that users can identify them.

[0169] exist Figure 6 In step S601, if there are multiple subjects to be detected within the approximate focus range, such as Figure 13B and Figure 13C As illustrated, icon 1306 is displayed. This prompts the user to operate control ring 125 and informs the user that the main subject cannot be changed using focus ring 124. Instead of using icon 1306, the notification can be obtained by... Figure 6 In step S602, the tracking candidate frame is continuously displayed at the AF frame position determined, and the switching of the subject is restricted to notify the user.

[0170] Next, refer to Figure 10 To explain Figure 4B The AF frame display processing in step S420 when the subject is not detected.

[0171] In step S1001, the focus control unit 121 determines the focus mode, and if the mode is "MF / AF switching mode", the process proceeds to step S1002; otherwise, the process proceeds to step S1005.

[0172] In step S1002, the focus control unit 121 determines whether the AF control mode is "tracking AF". If the AF control mode is "tracking AF", the process proceeds to step S1003; otherwise, the process proceeds to step S1004.

[0173] In step S1003, when the user has finished operating the focusing ring 124, the focusing control unit 121... Figure 4B In step S418, the tracking frame is displayed on the main subject specified by the user using manual focus operation.

[0174] In step S1004, the focusing control unit 121 does not display the tracking frame.

[0175] In step S1005, the focus control unit 121 determines whether the tracking candidate frame display setting is valid. If the tracking candidate frame display setting is valid, the process proceeds to step S1006. If the tracking candidate frame display setting is invalid, the process proceeds to step S1008.

[0176] In step S1006, the focus control unit 121 determines whether the AF control mode is "tracking AF". If the AF control mode is "tracking AF", the process proceeds to step S1007; otherwise, the process proceeds to step S1008.

[0177] In step S1007, the focus control unit 121 displays a signal indicating that the user is operating the focus ring 124 and... Figure 4B In step S419, the subject of the AF frame is set to be the tracking candidate frame of the tracking object.

[0178] In step S1008, the focusing control unit 121 does not display the tracking frame.

[0179] Figures 14A to 14D This is a diagram illustrating the AF frame display method when the subject is not detected according to this embodiment.

[0180] Figures 14A to 14D This example illustrates the changes in the display state of the AF frame from before the operation of the focus ring 124 until the operation of the focus ring 124 and after the operation of the focus ring 124. Figures 14A to 14D The illustrated subjects 1401 to 1403 each indicate a subject that was not detected by the subject area detection unit 126, and it is assumed that subject 1401 is on the infinite distance side, subject 1402 is closer to the nearest distance side than subject 1401, and subject 1403 is closer to the nearest distance side than subject 1402.

[0181] Figure 14A Example of a camera view before the focus ring 124 is operated. Figure 14A In the example, because no subject was detected and AF mode was set, therefore... Figure 2A The AF box is not displayed.

[0182] Figure 14B Example of video footage during the operation of focus ring 124. Figure 14B In the example, through Figure 4B The AF frame determination process in step S419 sets the AF frame in the subject 1402 and displays a tracking candidate frame 1404 on the subject 1402 at the start of tracking to indicate the candidate tracking object.

[0183] Figure 14C Examples in pairs and Figure 14B The video footage captured during manual focus adjustment using the focus ring 124 when the subject is closer to the nearest distance than the subject in the image. Figure 14B Similarly, in Figure 14C In China, through Figure 4B The AF frame determination process in step S419 sets the AF frame in the subject 1403 and displays the tracking candidate frame 1404 overlaid on the subject 1403.

[0184] Note that when not detected Figures 14A to 14C In the state of the subject, it can be like Figure 13B and Figure 13C That would display icon 1306 to inform the user that the main subject can be changed using focus ring 124.

[0185] Figure 14D Example of the video footage after the focus ring 124 operation is completed. Figure 14D In the example, the focus mode is "MF / AF switching mode," and the AF control after the MF operation ends is "tracking AF," with an overlay display indicating what will be focused on. Figure 14C The subject 1403, which is already a candidate for tracking, is the tracking frame 1405 for performing tracking AF.

[0186] Note that in this embodiment, an example is shown where the tracking candidate frame 1404 is a square frame with dashed lines and the tracking frame 1405 is a square frame with solid lines. However, the invention is not limited to this, and the shape, color, and size of the frames can be changed. For example, the tracking candidate frame 1404 can be an orange double keyframe, and its size can be displayed as a fixed size, while the tracking frame 1405 can be a white double rectangle, and its size can vary depending on the size of the subject.

[0187] According to this embodiment, by overlaying the tracking frame and the tracking candidate frame at the AF frame position set by the manual focusing operation when the subject is not being detected, the user can easily grasp the subject being tracked during and after the manual focusing operation.

[0188] Other embodiments

[0189] Embodiments of the present invention can also be implemented by providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0190] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be accorded the broadest interpretation to cover all such modifications and equivalent constructions and functions.

Claims

1. A control device, comprising: The display control unit is used to control the display of captured images on the display unit; as well as A control unit is configured to perform automatic focus control on a predetermined subject selected based on a user operation to change the focus state of a subject included in the image. The display control unit controls the display to overlay first information indicating that the predetermined subject is the object of the autofocus control onto the predetermined subject. The autofocus control can switch to any of a plurality of autofocus controls, including tracking AF, which ensures that the subject being focused is always in focus. When the autofocus control is set to the tracking AF, the control unit performs the tracking AF on the predetermined subject after the user operation ends.

2. The control device according to claim 1, wherein, The display control unit controls the display to overlay the first information onto the predetermined subject until the user operation ends.

3. The control device according to claim 1, further comprising: The setting unit is used to set whether to perform the tracking AF on the predetermined subject after the user operation ends. When the setting unit configures the device to perform the tracking AF, the control unit performs the tracking AF on the predetermined subject after the user operation ends.

4. The control device according to any one of claims 1 to 3, further comprising: A detection unit is used to detect a subject from the captured image. In the event that the detection unit does not detect a subject, the display control unit performs control to display the first information in a first form on the predetermined subject selected by the user operation during the user operation.

5. The control device according to claim 4, wherein, The display control unit controls the display to display the first information in a second form, different from the first form, on the predetermined subject selected by the user operation after the user operation ends.

6. The control device according to any one of claims 1 to 3, wherein, The first information is information used to indicate that the predetermined subject is the object of the tracking AF.

7. The control device according to claim 3, wherein, When the tracking autofocus is set to perform on the predetermined subject after the user operation ends, it is possible to set whether to display the first information, and The display control unit controls the display to overlay the first information onto the predetermined subject when the display is set to show the first information.

8. The control device according to any one of claims 1 to 3, wherein, When the autofocus control is not set to the tracking AF, the display control unit controls the display not to display the first information on the predetermined subject.

9. The control device according to any one of claims 1 to 3, 5 and 7, wherein, The first piece of information is the frame surrounding the subject.

10. The control device according to any one of claims 1 to 3, further comprising: A detection unit is used to detect a subject from the captured image. In the case where the detection unit detects multiple subjects and displays the first information, the display control unit controls the display to overlay second information in a form different from the first information onto the multiple subjects during the user operation, and to overlay the first information together with the second information onto a subject selected from the multiple subjects in a first form.

11. The control device according to claim 10, wherein, The display control unit controls the display to display the first information in a second form, different from the first form, on the selected subject from the plurality of subjects after the user operation ends.

12. The control device according to claim 10, wherein, The plurality of subjects includes subjects having the same focusing distance.

13. The control device according to claim 11 or 12, wherein, The second piece of information is the frame surrounding the subject.

14. The control device according to claim 4, wherein, If the detection unit does not detect a subject, the control unit extracts the subject region based on a first threshold using focus information obtained from the image in response to the user's operation, and determines the predetermined subject by setting a representative region from the subject region based on a second threshold.

15. The control device according to claim 14, wherein, The first threshold is set such that when the user operation is on the infinity side in the optical axis direction, the range of the infinity side is greater than the range of the nearest distance side. The first threshold is set such that, when the user operation is on the nearest distance side in the optical axis direction, the range of the nearest distance side is greater than the range of the infinite distance side. The second threshold is set to the same value as the first threshold until the user operation ends. After the user operation ends, the second threshold is set to a value smaller than the first threshold.

16. A camera device, comprising: The camera unit is used to capture images of the subject formed by the optical system; as well as The control device according to any one of claims 1 to 15.

17. A control method for a control device, comprising the following steps: Automatic focus control is performed on a predetermined subject selected based on user operations to change the focus state of the subject included in the captured image; as well as Control is performed to overlay and display first information indicating that the predetermined subject is the object of the autofocus control on the predetermined subject.

18. A computer-readable storage medium storing a program for causing a computer to function as a control device according to any one of claims 1 to 15.

19. A computer program product for causing a computer to function as a control device according to any one of claims 1 to 15.

Images