Camera device and camera device control method

Abstract

This invention provides a camera device and a control method for the camera device. A camera control unit detects a subject using a first method and a second method different from the first method. Based on the detection results obtained using the first method and the detection results obtained using the second method, it displays candidate correction positions for correcting subject blur on an LCD. The camera control unit and the lens control unit control the correction of subject blur by driving a shift lens based on the correction positions specified from the candidates.

Description

Technical Field

[0001] This invention relates to a camera device and a method for controlling the camera device. Background Technology

[0002] Recent cameras have been equipped with a function called Eye Autofocus (hereinafter referred to as AF), which detects the eyes of a person or animal and focuses on those eyes. As with portrait photography (where accurately focusing on the eyes is crucial), there is a position that the user wants to focus on based on the composition and scene. Furthermore, it is necessary to suppress blur at the position where the user wants to focus.

[0003] This section briefly describes the types of blur that occur in an image. There are two types of blur: camera shake caused by hand or body movement and subject blur caused by moving subjects. In subject blur, the amount of blur varies depending on the position of the same subject. Therefore, in correcting subject blur, any point of the same subject is determined to be corrected. Thus, a mechanism is needed that allows the position of the subject blur to be corrected to be displayed on the user interface to inform the photographer of that position. Japanese Patent Application Laid-Open No. 2018-180336 discloses a method for displaying subject blur by using the position of the ranging point frame of the AF for a camera optical system. According to Japanese Patent Application Laid-Open No. 2018-180336, a vector frame for correcting the subject blur position is determined based on detected vector information, and when driven at a shutter speed set to the vector value of the determined vector frame, the display changes to indicate whether the driving amount falls within the driving area of ​​the optical correction system.

[0004] However, the location where the camera automatically corrects for subject blur does not always match the location the photographer expects to correct for subject blur. Summary of the Invention

[0005] The present invention provides a camera device that enables the correction of subject blur at the subject blur correction position expected by the photographer.

[0006] An imaging device according to the present invention includes: a first detection unit configured to detect a subject; a second detection unit configured to detect the subject in a method different from that used by the first detection unit to detect the subject; a display unit configured to display candidates for correcting subject blur based on each of the detection results from the first detection unit and the second detection unit; and a control unit configured to control the correction of subject blur based on the correction positions specified from the candidates.

[0007] The present invention discloses a control method for a camera device, the control method comprising: a first detection step for detecting a subject; a second detection step for detecting the subject using a method different from that of the first detection step; a display step for displaying candidates for correcting subject blur based on the detection results of the first detection step and the detection results of the second detection step; and a control step for controlling the correction of subject blur based on the correction positions specified from the candidates.

[0008] Other features of the invention will become clear from the following description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0009] Figure 1 This is a diagram showing the configuration of the camera device.

[0010] Figure 2 This is a diagram illustrating an example configuration of camera shake correction control.

[0011] Figure 3 This is the overall flowchart of the panning control.

[0012] Figure 4 This is a diagram used to describe the subject vector detection method.

[0013] Figure 5 It is a diagram used to describe the angle between the camera device and the subject.

[0014] Figure 6A and Figure 6B It is a diagram used to describe the relationship between the angle between the camera device and the subject and the angular velocity of the subject, as well as the relationship between the angle and the angular acceleration of the subject.

[0015] Figure 7 This is a diagram showing a display example of candidate locations for subject blur correction.

[0016] Figure 8 This is a flowchart showing the position of the subject blur correction.

[0017] Figure 9 It is a diagram used to describe the calculation of the centroid.

[0018] Figure 10A and Figure 10B It is a diagram used to describe the position of subject blur correction based on the center of gravity of the subject. Detailed Implementation

[0019] Reference Figure 1 The configuration of the camera device according to this embodiment is described. Figure 1This diagram illustrates the configuration of a camera device. The camera device has a camera body 130 and an interchangeable lens 100 as a lens assembly. The interchangeable lens 100 is detachable from the camera body 130. Furthermore, although this embodiment describes an example of an interchangeable lens type camera device where the interchangeable lens 100 is detachable from the camera body 130, the camera device is not limited to this, and a lens-integrated camera device that integrates the lens with the camera body can also be employed.

[0020] First, the configuration of the interchangeable lens 100 will be described. The interchangeable lens 100 has a shooting lens unit 101, a position detection unit 105, an encoder 106, an angular velocity detection unit 111, a lens control unit 112, a driver 113, an amplifier circuit (AMP) 114, and a mounting contact unit 115.

[0021] The imaging lens unit 101 includes an imaging optical system 102, a zoom lens group (hereinafter referred to as a zoom lens) 103, and a shift lens group (hereinafter referred to as a shift lens) 104. The imaging optical system 102 guides the subject light to the image sensor 132 via the zoom lens 103 and the shift lens 104. The zoom lens 103 is a lens group whose focal length can be changed. The shift lens 104 is a correction component for optically correcting the blur (image blur) of the captured image caused by shake applied to the imaging device. Specifically, when the lens control unit 112 controls the driver 113 to drive the shift lens 104 in a direction perpendicular to the optical axis, image blur is corrected.

[0022] The position detection unit 105 detects the position of the shift lens 104. The position detection unit 105 is, for example, a Hall element. The encoder 106 is a zoom encoder that detects the position of the zoom lens 103. Additionally, the angular velocity detection unit 111 detects jitter applied to the imaging device and outputs an angular velocity signal as a jitter detection signal. The angular velocity detection unit 111 is, for example, a gyroscope sensor.

[0023] Additionally, driver 113 drives shift lens 104. Amplifier circuit 114 amplifies the signal (position detection signal) indicating the position of shift lens 104 detected by position detection unit 105 and outputs the signal to lens control unit 112. Mounting contact unit 115 relays communication between replaceable lens 100 and camera body 130.

[0024] Additionally, the lens control unit 112 controls the entire interchangeable lens 100. The lens control unit 112 is, for example, a central processing unit (CPU). The lens control unit 112 includes a camera shake correction control unit 121 and a panning control unit 122. The camera shake correction control unit 121 performs camera shake correction control, which is a control for correcting image blur caused by camera shake. The panning control unit 122 performs panning assist control. Panning assist control is a control that uses a corrective optics system to correct image blur (subject blur) of the subject caused by the deviation between the movement of the subject during panning and the shooting direction of the camera device. Although a shift lens 104 is used as the corrective optics system in this embodiment, an image sensor can be used as the corrective optics system. The panning control unit 122 uses the subject angular velocity calculated by the camera control unit 141 to calculate the drive amount of the shift lens 104 to be used when correcting subject blur.

[0025] Although the lens control unit 112 performs focusing lens control, aperture control, camera shake correction control, and panning assist control, its description is omitted for simplicity of the accompanying drawings. Furthermore, to perform camera shake correction control, for example, shake about two orthogonal axes (such as shake in the vertical and horizontal directions) is detected and corrected, and the configuration for processing about one axis is similar to the configuration for processing about the other axis. Therefore, the configuration about one axis will be described below.

[0026] Next, the configuration of the camera body 130 will be described. The camera body 130 includes a shutter 131, an image sensor 132, an analog signal processing circuit (AFE) 133, and a camera signal processing circuit 134. The shutter 131 adjusts the amount of light to be received by the image sensor 132. The image sensor 132 is, for example, a CMOS sensor, and photoelectrically converts the light of the subject to output a signal for capturing an image. "CMOS" is an abbreviation for "Complementary Metal-Oxide-Semiconductor". The AFE 133 converts the analog signal output by the image sensor 132 into a digital signal and outputs the digital signal to the camera signal processing circuit 134. The camera signal processing circuit 134 performs predetermined processing on the signal output by the AFE 133. The camera signal processing circuit 134 includes a motion vector detection unit 145 that detects motion vectors from captured images. The motion vector detection unit 145 detects the amount of motion in images from different frames acquired by the image sensor 132 as motion vectors. Furthermore, in this embodiment, a mirrorless camera is assumed, but it can also be applied to a single-lens reflex camera with a mirror. In a single-lens reflex camera, the motion vector detection unit 145 can detect motion vectors from the video signal of the metering sensor.

[0027] The camera body 130 also includes a timing generator (TG) 135, an operation unit 136, a memory card 139, and a display panel (hereinafter referred to as LCD) 140. "LCD" 140 is an abbreviation for "Liquid Crystal Display". The TG 135 sets the operating timing of the image sensor 132 and AFE 133. The operation unit 136 receives user input. The operation unit 136 has a power switch, a release switch, etc. The memory card 139 contains recorded video (captured images). The LCD 140 displays captured images, various information, etc. The LCD 140 also has a touch panel function, which enables touch operation using electrostatic methods, pressure-sensitive methods, etc., and receives user input.

[0028] The camera body 130 also includes a driver 137, a shutter drive motor 138, an angular velocity detection unit 171, a distance detection unit 181, a mounting contact unit 144, a camera control unit 141, and a memory 142. The driver 137 drives the shutter drive motor 138. The shutter drive motor 138 drives the shutter 131. The angular velocity detection unit 171 detects camera shake applied to the imaging device and outputs an angular velocity signal (shake detection signal) to the camera control unit 141. The distance detection unit 181 detects the distance to the subject. The mounting contact unit 144 relays communication between the camera body 130 and the interchangeable lens 100. In addition to various setting information, arithmetic operation information, etc., the memory 142 also temporarily stores image data.

[0029] The camera control unit 141 includes a CPU and controls the entire camera body 130. For example, the camera control unit 141 calculates the angular velocity of the subject (subject angular velocity) to be used to correct image blur based on the motion vector obtained from the captured image by controlling the camera signal processing circuit 134. Furthermore, the camera control unit 141 communicates various information (e.g., information for controlling the subject angular velocity for panning assistance, etc.) with the lens control unit 112 of the interchangeable lens 100 via mounting contact units 115 and 144.

[0030] The camera control unit 141 includes a shutter control unit 151, a subject shake correction calculation unit 152, a shutter speed calculation unit 153, and a zoom / pan control unit 154. The shutter control unit 151 instructs the driver 137 to control the shutter operation. The subject shake correction calculation unit 152 calculates the subject angular velocity as a blur correction amount for the subject. The subject shake correction calculation unit 152 sends the calculated subject angular velocity to the lens control unit 112. Specifically, since the motion vector detection unit 145 detects the motion of the image as motion vectors, the subject shake correction calculation unit 152 reliably detects the vector corresponding to the subject (subject vector) from the detected motion vectors. Then, the subject shake correction calculation unit 152 calculates the subject angular velocity based on the detected subject vector. The camera control unit 141 sends the calculated subject angular velocity to the lens control unit 112. The shutter speed calculation unit 153 calculates a shutter speed suitable for panning. The zoom / pan control unit 154 controls the zoom position so that the subject as seen from the shooting angle is reflected at a certain size in each frame.

[0031] As described above, the imaging apparatus according to this embodiment includes an image blur correction device that corrects image blur by driving a shift lens 104, which serves as a correction component, in a direction orthogonal to the optical axis. Additionally, a camera control unit 141 and a lens control unit 112 act as control components that correct image blur appearing in the subject during panning by driving the shift lens 104 based on the subject angular velocity obtained from the captured image. The camera control unit 141 and the lens control unit 112 drive the shift lens 104 based on the subject angular velocity calculated according to the correction position of the subject blur.

[0032] exist Figure 1 When the camera body 130 is powered on via the operation unit 136, the camera control unit 141 detects the power connection and, under its control, performs power supply and initial settings for each circuit of the camera body 130. Additionally, power is supplied to the interchangeable lens 100, and its initial settings are performed under the control of the lens control unit 112. Then, the lens control unit 112 and the camera control unit 141 begin communication at a predetermined time. Communication data sent from the camera body 130 to the interchangeable lens 100 includes, for example, the camera's status and shooting settings. Communication data sent from the interchangeable lens 100 to the camera body 130 includes, for example, lens focal length information and angular velocity information.

[0033] Figure 2This diagram illustrates an example configuration of camera shake correction control. In the interchangeable lens 100, the angular velocity detection unit 111 detects shake applied to the imaging device due to camera shake, etc., and the camera shake correction control unit 121 performs camera shake correction control. Furthermore, in the case of… Figure 2 In the description, regarding Figure 1 Common configurations will be assigned the same reference numerals, and their descriptions will be omitted.

[0034] The camera shake correction control unit 121 includes an offset removal unit 201 to a pulse width modulation unit 208. The offset removal unit 201 is, for example, a filter calculation unit with a high-pass filter (hereinafter referred to as HPF), and removes the DC component included in the output of the angular velocity detection unit 111. The gain-phase calculation unit 202 amplifies and phase-compensates the angular velocity signal whose offset component has been removed by the offset removal unit 201. Therefore, the gain-phase calculation unit 202 includes: an amplifier that amplifies the angular velocity signal with the offset component removed at a predetermined gain; and a phase compensation filter that performs phase compensation on the angular velocity signal.

[0035] Integrator 203 has the function of changing the characteristics of any frequency band, integrating the output of gain phase calculation unit 202, and calculating the driving amount of shift lens 104. If the angular velocity of angular velocity detection unit 111 is equal to or greater than a certain value for a predetermined time period, integrator 203 determines that panning is in progress and gradually changes the cutoff frequency of HPF of offset removal unit 201 to the high-frequency side. As integrator 203 gradually changes the cutoff frequency to the high-frequency side, the target signal of camera shake correction control gradually decreases, and the correction optical system (shift lens) returns to the optical center position. If the correction optical system corrects an angular velocity high enough to determine that panning is in progress without changing the cutoff frequency to the high-frequency side, the correction optical system will reach the correction limit point, and the photographer will see an unnatural change in the angle of view.

[0036] The image stabilization control determination unit 204 acquires the outputs of integrator 203 and another integrator 225 (described below), and outputs either one to subtractor 205. The image stabilization control determination unit 204 switches the signal used to drive the shift lens 104 according to the output of camera information acquisition unit 226 as follows: For example, when the shooting mode is set to pan assist mode, the image stabilization control determination unit 204 selects the output of integrator 225 calculated by pan control unit 122. On the other hand, when the shooting mode is set to a mode other than pan assist mode, the image stabilization control determination unit 204 selects the output of integrator 203 calculated by camera shake correction control unit 121.

[0037] The position detection unit 105 detects the position of the shift lens 104, and the amplifier circuit 114 amplifies the position detection signal. The analog (A) to digital (D) converter 206 digitizes the position detection signal amplified by the amplifier circuit 114 and outputs the result to the subtractor 205.

[0038] Subtractor 205 performs subtraction with the output of anti-jitter control judgment unit 204 as positive input and the output of A / D converter 206 as negative input, and outputs the deviation data as the subtraction result to controller 207. Controller 207 includes: an amplifier that amplifies the deviation data output by subtractor 205 at a predetermined gain; and a phase compensation filter. The deviation data is processed by the amplifier and phase compensation filter in controller 207 and output to pulse width modulation unit 208. Pulse width modulation unit 208 acquires the output data of controller 207, modulates the waveform (i.e., PWM waveform) data used to change the duty cycle of the pulse waveform, and outputs the modulated data to driver 113 for driving the shift lens.

[0039] The driver 113 is, for example, a voice coil motor that drives the shift lens 104. The driver 113 moves the shift lens 104 in a direction perpendicular to the optical axis of the imaging optical system according to the output of the pulse width modulation unit 208.

[0040] Here, the control and configuration of the panning control unit 122 in panning assist mode will be described. When the user operates the operation unit 136 to set the panning assist mode, the camera control unit 141 switches to panning assist control. Additionally, switching information is sent from the camera control unit 141 to the lens control unit 112, and the lens control unit 112 switches to panning assist control.

[0041] The camera information acquisition unit 226 acquires various types of camera information sent from the camera control unit 141 via the communication control unit 211. This camera information includes settings and release information for panning assist modes, etc. The camera information acquisition unit 226 outputs the information used for judgment and processing to the image stabilization control judgment unit 204.

[0042] Angular velocity output unit 222 acquires the output of offset removal unit 201, i.e., the angular velocity signal after offset component removal. Angular velocity output unit 222 transmits the angular velocity signal to camera control unit 141 via communication control unit 211 and mounting contact unit 115. Subject angular velocity acquisition unit 223 acquires the subject angular velocity calculated by subject shake correction calculation unit 152 inside camera body 130 via mounting contact units 144 and 115 and communication control unit 211. Then, when correcting subject blur based on the acquired subject angular velocity, subject angular velocity acquisition unit 223 outputs the subject angular velocity to be used in calculating the drive amount of shift lens 104.

[0043] Subtractor 224 performs subtraction with the output of offset removal unit 201 as positive input and the output of subject angular velocity acquisition unit 223 as negative input. The deviation is calculated by subtracting the angular velocity indicated by the angular velocity signal with the offset component removed from the subject angular velocity output by subject angular velocity acquisition unit 223. Subtractor 224 outputs the deviation to integrator 225. Integrator 225 integrates the deviation and outputs the result of the integration to anti-shake control judgment unit 204.

[0044] Will use Figure 3 The flowchart describes a method for controlling subject blur correction. Figure 3 This is a flowchart illustrating an example of subject blur correction performed as a panning aid control.

[0045] In step S301, the camera control unit 141 acquires the position information of the shift lens 104 within the replaceable lens 100. The vector value δv detected by the motion vector detection unit 145 is obtained by adding the following values ​​(δv = δg + δo): the value obtained by converting the angular velocity of the imaging device into the displacement δg on the imaging surface, and the value obtained by converting the driving amount of the shift lens 104 into the displacement δo on the imaging surface.

[0046] In step S302, the camera control unit 141 acquires angular velocity information output by the angular velocity detection unit 111 in the interchangeable lens 100 and the angular velocity detection unit 171 in the camera body 130. Then, the camera control unit 141 uses focal length [mm], frame rate [frame / sec] and pixel pitch [um / pixel] to convert the angular velocity [deg / sec] indicated by the angular velocity information into a displacement amount [pixel / frame] on the image plane.

[0047] In step S303, the motion vector detection unit 145 detects motion vectors indicated by the amount of motion in the images across multiple frames. In step S304, the camera control unit 141 calculates a histogram of the motion vectors based on the information obtained in steps S301 to S303. Details of the histogram will be described below.

[0048] In step S305, the camera control unit 141 determines whether a subject vector has been detected based on the histogram calculated in step S304. If a subject vector has been detected, the process proceeds to step S306. On the other hand, if a subject vector has not been detected, the process proceeds to step S301, and the next frame is ready for processing.

[0049] Here, we will use Figure 4 Describes the detection and judgment method of the subject vector. Figure 4 This is a graph used to describe the detection of the subject vector. The histogram calculated in step S304 is used to determine the detection of the subject vector. Figure 4 The histogram shown is in Figure 3 An example of the histogram calculated in step S304. The horizontal axis represents vector values. The vertical axis represents frequencies.

[0050] The motion vector detection unit 145 detects the motion amount of the previous frame in each block of the vector detection frame arranged at a predetermined position as vector values. The camera control unit 141 calculates a histogram of the vector values ​​detected by the motion vector detection unit 145. The motion vector detection unit 145 does not have the function of classifying each detected block as a subject vector or a background vector. Therefore, based on the histogram, the camera control unit 141 determines the motion vectors that start from the angular velocity information 401, which is converted into the displacement δg on the imaging plane, and exist in the background range 402 as background vector group 403. The camera control unit 141 determines the motion vectors that exist outside the background range 402 and have a frequency exceeding a predetermined threshold 404 as subject vectors 405. Furthermore, in the background range 402, the length (the width of the vector values ​​considered as the background range) can be changed according to the focal length or the detection accuracy of the motion factor by the motion vector detection unit 145.

[0051] Furthermore, the camera control unit 141 can determine whether a subject vector has been detected as follows: Instead of the angular velocity signal, the camera control unit 141 can determine, based on the amount of defocus obtained in the focus adjustment process (defocusing), a vector existing in a region with a depth difference within a predetermined range from the defocus amount of the focus frame. Then, the camera control unit 141 can determine vectors existing outside the region with the predetermined depth difference as a background vector group. Alternatively, the camera control unit 141 can use both the defocus amount and the angular velocity signal to determine the subject vector and the background vector group.

[0052] Will describe again Figure 3 .

[0053] In step S306, the camera control unit 141 calculates the subject angular velocity to be used when calculating the drive amount of the shift lens 104 to correct subject blur. Specifically, the camera control unit 141 selects the subject vector to be used for subject blur correction from the subject vectors 405 detected in step S305. This is because the vector value changes according to the position in the subject (the amount of subject blur changes).

[0054] For example, when a photographer pans to capture a train moving in a straight line, the front and rear sides of the train have different vector values. If the camera moves at the same speed as the subject, that is, the camera moves parallel to the subject without changing the shooting direction of the camera, then the front and rear sides of the subject have equal vector values. However, in actual panning, the photographer does not move the camera parallel to the subject, but moves the camera while changing the angle. Therefore, when viewing the subject from the camera, the front and rear sides of the subject have different vectors. As the focal length has a wider angle and the shooting distance becomes shorter, the front and rear sides of the subject have more significantly different vectors. Conversely, if the shooting distance is infinitely long, the shooting situation is equivalent to shooting parallel to the subject, so the entire subject often appears to be stationary. In order to correct the blur at a point on the subject in the correction of subject blur, in step S306, the subject vector to be used in the control is finally selected from the subject vectors 405 and converted into the subject angular velocity.

[0055] In step S307, the camera control unit 141 obtains the difference between the subject angular velocity calculated in step S306 and the subject angular velocity in the previous frame, in order to calculate the subject angular acceleration.

[0056] Here, we will use Figures 5 to 6B Describe the angular velocity and angular acceleration of the subject as seen from the photographer's perspective. Figure 5It is a diagram used to describe the angle between the camera device (the photographer 500) and the subject, depending on the movement of the subject. Figure 6A It is a diagram showing the relationship between the angle between the camera device and the subject and the angular velocity of the subject. Figure 6B It is a diagram showing the relationship between the angle between the camera device and the subject and the angular acceleration of the subject.

[0057] If the speed of the subject is set to v[m / s] and the distance to the subject is set to L[m], then the angle θ[deg] between the subject and the photographer is expressed by the following equation 1.

[0058]

[0059] Here, since the angular velocity ω[deg / sec] is indicated by the first derivative of the angle θ, it is expressed by the following equation 2.

[0060]

[0061] Additionally, due to angular acceleration α [deg / sec] 2 Indicated by the first derivative of the angular velocity ω, it is therefore expressed by the following equation 3.

[0062]

[0063] therefore, Figure 5 Equation 1 shows the angle θ formed by the subject as seen from the camera. Figure 6A and Figure 6B The relationship between the angle θ formed by the subject and the angular velocity ω and angular acceleration α of the subject, as indicated by Equations 2 and 3, is shown. Figure 6A The subject angular velocity 601 shown is the subject angular velocity calculated in step S306. Figure 6B The subject angular acceleration 602 shown is the subject angular acceleration calculated in step S307.

[0064] Will describe again Figure 3 .

[0065] In step S308, the camera control unit 141 instructs the memory 142 to maintain a history of the subject angular velocity and subject angular acceleration calculated for each frame. In step S311, the history of subject angular velocity and subject angular acceleration is used to estimate the subject angular velocity during the exposure time period, as will be described below. The reason for estimating the subject angular velocity during the exposure time period is that no evaluation image to be used for motion vector detection is acquired during the exposure time period, and no motion vector is detected that serves as the basis for calculating the subject angular velocity and subject angular acceleration.

[0066] In step S309, the camera control unit 141 instructs the LCD 140 to display candidates for subject blur correction positions. This will be used... Figure 7 and Figure 8 A detailed description of the candidate display for subject blur correction positions. Figure 7 This is a diagram illustrating an example display of a user interface for subject blur correction. In step S309, boxes (boxes 701 and 702) indicating candidate position information for use as subject blur correction positions are overlaid on the live view image, and as shown... Figure 7 The following is shown. Furthermore, although this embodiment describes an example of using boxes to represent candidate location information for subject blur correction, this disclosure is not limited thereto, and the method of representation is not important, as long as it enables the user to identify the locations listed as candidates for subject blur correction.

[0067] Figure 8 This is a flowchart illustrating the process of displaying candidates for subject blur correction positions.

[0068] In step S801, the camera control unit 141 calculates the center of gravity G of the subject. Here, it will use... Figure 9 Describe the calculation of the centroid G. Figure 9 This is a diagram used to describe the calculation of the centroid. If the coordinates of point A are set to (x1, y1), the coordinates of point B are set to (x2, y2), the coordinates of point C are set to (x3, y3), the mass of point A is set to m1, the mass of point B is set to m2, and the mass of point C is set to m3, then the coordinates of the centroid G (x1, y1) can be expressed by the following equation 4. g y g ).

[0069]

[0070] Here, assuming all points on the image have equal mass, the centroid G on the image can be calculated using the following equation 5. The camera control unit 141 calculates the centroid G based on the center coordinates (x, y) of each vector frame that has been detected as the vector determined to be the subject vector. n y n To calculate the coordinates (x, y) of the centroid G of the subject. g y g ).

[0071]

[0072] Will describe again Figure 8 .

[0073] In step S802, the camera control unit 141 determines the subject blur correction position based on the center of gravity of the subject calculated in step S801. At this time, the camera control unit 141 determines the subject blur correction position based on the composition from the center of gravity of the subject, subject information, etc.

[0074] Will use Figure 10A and Figure 10B This describes a method for determining the blur correction position of a subject based on its center of gravity. Figure 10A and Figure 10B It is a diagram used to describe the position of subject blur correction based on the center of gravity of the subject. Figure 10A This is a diagram showing the subject region. The subject region is the area where the subject vector was detected in step S305. (As used...) Figure 4 As described, the camera control unit 141 separates the subject vector and background vector from the detected vector values, and detects the subject vector. Figure 10A The area indicated by the diagonal line in the image is the subject area.

[0075] Figure 10B This diagram illustrates the subject blur correction position based on the subject's center of gravity. The camera control unit 141 calculates the center of gravity G of the subject region based on the center coordinates of each vector frame that detects a vector determined to be a subject vector (i.e., the center coordinates of each vector frame within the subject region). Then, the camera control unit 141 determines the subject blur correction position based on composition or subject information from the center of gravity G. Subject information used to determine the subject blur correction position includes the subject's movement direction, movement speed, and the angle θ formed with the subject.

[0076] For example, when the subject is moving along the upper right side of the frame, the center of gravity G(x) in the direction of the subject's movement... g y g Starting from the center point G(x), set the upper right vector box as the subject blur correction position. Additionally, in specific compositions, the position can be adjusted from the center point G(x). g y g The vector box at the predetermined position is set as the subject blur correction position. When shooting an approaching motorcycle, if the angle θ formed with the subject, as expressed by Equation 1, has a high value, the motorcycle is shot with a composition viewed from the front. Therefore, the vector box at the position above the center of gravity G in the direction of travel is set as the subject blur correction position, so that the subject blur correction position is close to the front of the motorcycle.

[0077] In step S803, the camera control unit 141 instructs the LCD 140 to display a frame indicating the subject blur correction position determined in step S802, starting from the center of gravity of the subject.

[0078] In step S804, the camera control unit 141 determines whether a subject can be detected. If a subject can be detected, the process proceeds to step S805. Otherwise, if a subject cannot be detected, the process proceeds to step S810. Here, subject detection in this step is performed using any method other than the method used in step S305 to detect the subject using motion vectors. For example, the subject can be detected by comparing it with pre-registered subject model information (pattern) (pattern matching), or by using a detection learning model (machine learning result). Alternatively, a histogram method based on color information (such as the hue or saturation of an image) or brightness information can be used to detect the subject, or by using distance information to the subject. Furthermore, the number of subjects to be detected is not limited to one, and multiple subjects can be detected.

[0079] In step S805, it is determined whether the features of the subject detected in step S804 can be detected. Multiple features of the subject can be detected. If the features can be detected, the process proceeds to step S806. On the other hand, if no features are detected, the process proceeds to step S809.

[0080] For example, in a motorsports scene, the features of the subject include helmets, motorcycle headlights, etc., and if the subject is a person or animal, the features include the face, eyes, etc. Any method can be used to detect the features of the subject. For example, features can be detected by comparing pre-registered feature point model information (patterns) (pattern matching), or features can be detected using a detection learning model (machine learning results).

[0081] In step S806, the camera control unit 141 instructs the LCD 140 to display a frame indicating the subject blur correction position based on the feature parts of the subject determined in step S805.

[0082] In step S809, the camera control unit 141 instructs the LCD 140 to display a frame of the entire subject.

[0083] In step S807, it is determined whether the user has already manually selected a box. If the user has already specified a box, it is assumed that manual selection has been performed, and the process proceeds to step S808. On the other hand, if the user has not specified a box, it is assumed that manual selection has not been performed, and the process proceeds to step S810.

[0084] Here, we will use Figure 7 This describes an example of manual settings for box selection performed by the user. Box 702 is a box indicating the subject blur correction position based on the centroid point of the subject vector displayed in step S803. Box 701 is a box indicating the subject blur correction position based on the features of the subject displayed in step S806. A motorcycle has three main features, such as the vehicle, the rider, and the helmet. In this embodiment, an example of detecting the helmet as a feature of the subject is presented.

[0085] The user manually specifies the frame position from multiple displayed frames where he or she wants to correct the subject blur. The frame can be specified, for example, by touching the frame displayed on the LCD 140, which is a touch panel, or by operating the joystick or button of the operation unit 136.

[0086] In step S808, the camera control unit 141 instructs the LCD 140 to display the frame selected by the user in step S807 more prominently than other frames. Examples of this highlighting include changing the frame's color, making the frame blink, etc. Furthermore, in this embodiment, the frame indicating candidate positions for subject blur correction based on the subject's center of gravity displayed in step S803 is highlighted until the user selects that frame.

[0087] If no subject is detected in step S804, or if no frame selection is manually set in step S807, the process proceeds to step S810. In step S810, the camera control unit 141 instructs the LCD 140 to highlight the subject blur correction position based on the centroid of the region where the subject vector was detected.

[0088] As described above, in this embodiment, candidate locations for subject blur correction are displayed on the image, and the user can specify a location from the candidates. The candidate locations for subject blur correction include correction locations starting from the centroid of the region where the subject vector has been detected (correction locations based on the detection results of the first detection unit) and correction locations based on the features of the subject (correction locations based on the detection results of the second detection unit). Furthermore, although the following examples have been described in this embodiment, this disclosure is not limited thereto: subject blur correction locations based on the centroid of the subject vector, locations where subject features are detected, or the entire location of the subject are displayed as candidate locations for subject blur correction. For example, a user-specified location can be included in the candidates.

[0089] Will describe again Figure 3 .

[0090] In step S310, the camera control unit 141 determines whether the photographer has pressed the release button to give a command to start exposure. If a command to start exposure has been given, the process proceeds to step S311. If no command to start exposure has been given, the process returns to step S301.

[0091] In step S311, the lens control unit 112 calculates (estimates) the subject angular velocity during the exposure time period. Specifically, firstly, the camera control unit 141 sends the subject angular velocity and subject angular acceleration before exposure at the subject blur correction position determined in step S309 to the lens control unit 112 of the interchangeable lens 100. The subject angular velocity and subject angular acceleration before exposure sent to the lens control unit 112 are the history of subject angular velocities and subject angular accelerations calculated in steps S306 and S307 and maintained in memory 142 in step S308. The lens control unit 112 calculates the subject angular velocity during the exposure time period based on the subject angular velocity and subject angular acceleration at the subject blur correction position received from the camera control unit 141. Furthermore, for the subject angular velocity calculated on the interchangeable lens side, the release time delay from pressing the release button to the start of exposure is also taken into account, and the subject angular velocity is calculated by estimating the movement of the subject during the exposure time period.

[0092] In step S312, the lens control unit 112 drives the shift lens 104, which serves as a correction optics system for optically correcting camera shake, based on the subject angular velocity calculated in step S311. This operation corrects subject blur at the user-selected subject blur correction position. Additionally, the lens control unit 112 drives the zoom lens 103 to focus on the user-selected subject blur correction position.

[0093] In step S313, the camera control unit 141 determines whether the predetermined exposure time has elapsed. If the exposure time has elapsed, the process ends. Then, after development is complete, refer to... Figure 3 The described process begins from the next frame. On the other hand, if the exposure time has not yet elapsed, the process returns to step S312 and the shift lens 104 is continuously driven until the exposure time has elapsed.

[0094] Furthermore, although this embodiment is based on the assumption of a mirrorless camera without a mirror, this disclosure can also be applied to single-lens reflex cameras. Needless to say, the difference from a mirrorless camera is that if the video signal input to the metering sensor is instead input to the image sensor 132 instead of the video signal to be input to the motion vector detection unit 145, equivalent control can be performed. Additionally, although an example of correcting subject blur by driving the shift lens 104 has been described in this embodiment, the correction method is not limited to this; for example, subject blur can be optically corrected by driving the image sensor inside the camera body 130. Furthermore, candidate frames indicating the location for subject blur correction and frames indicating the location as the AF target can be displayed on the LCD 140, allowing the user to identify each of the subject blur correction location and the AF target location. In this case, if the display modes (size, form, color, etc.) of each frame are set differently, the user can easily identify each of the candidate frames indicating the subject blur correction location and the frames indicating the AF target location. Furthermore, when candidate frames indicating the subject blur correction position overlap with frames indicating the AF object position, the display mode of these frames can differ from the case where the frames do not overlap.

[0095] According to this embodiment, as described above, multiple candidate locations for subject blur correction are displayed, and the user can select a correction location from the candidates, thus allowing subject blur correction to be performed at the user-indicated correction location. As a result, a camera device can be provided that enables subject blur correction at the location intended by the photographer.

[0096] (Other embodiments)

[0097] The embodiments of the present invention can also be implemented by providing software (programs) that perform the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads out and executes the program.

[0098] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

[0099] This application claims priority to Japanese Patent Application 2020-065610, filed on April 1, 2020, the entire contents of which are incorporated herein by reference.

Claims

1. A camera device, comprising: The first detection unit is configured to detect the subject; The second detection unit is configured to detect the subject in a different manner than the method by which the first detection unit detects the subject. The display unit is configured to display candidates for correcting subject blur based on each of the detection results from the first detection unit and the second detection unit. as well as The control unit is configured to control the correction of subject blur based on the correction position specified from the candidates.

2. The camera device according to claim 1, wherein, The first detection unit detects the subject based on motion vectors.

3. The camera device according to claim 1, wherein, The display unit calculates the center of gravity of the subject detected by the first detection unit, and displays candidate correction positions for correcting subject blur based on the center of gravity.

4. The camera device according to claim 3, wherein, The display unit determines and displays the correction position based on any one of the following: the center of gravity, the direction of movement of the subject, the angle formed by the subject and the camera device, and the composition of the image.

5. The camera device according to any one of claims 1 to 4, wherein, The second detection unit detects the subject based on any one of the following: color information, brightness information, distance information to the subject, pattern matching, and machine learning results.

6. The camera device according to claim 5, wherein, The second detection unit detects the features of the detected subject.

7. The camera device according to claim 6, wherein, The second detection unit detects the features of the subject based on either pattern matching or machine learning results.

8. The camera device according to claim 6, wherein, When the second detection unit detects a feature of the subject, the display unit displays the position corresponding to the feature as a candidate for the correction position to correct the subject blur. When the second detection unit does not detect a feature of the subject, the display unit displays the position corresponding to the subject detected by the second detection unit as a candidate for the correction position to correct the subject blur.

9. The camera device according to claim 1, wherein, When multiple candidates for the correction position are displayed, the display unit highlights the candidate for the correction position specified by the user.

10. The camera device according to claim 1, wherein, When multiple candidates for the correction position are displayed, the display unit highlights the correction position based on the detection result of the first detection unit until the user specifies the correction position.

11. The camera device according to claim 1, wherein, If the user does not specify a correction position, the control unit corrects the subject blur at the correction position based on the detection result of the first detection unit.

12. The camera device according to claim 1, wherein, The control unit corrects subject blur by driving the correction unit based on the subject angular velocity calculated at a specified correction position for subject blur.

13. A control method for a camera device, the control method comprising: The first detection step is used to detect the subject; The second detection step is used to detect the subject in a method different from that of the first detection step; The display step is used to display candidates for the correction position to correct the subject blur based on the detection results of the first detection step and the detection results of the second detection step. as well as A control step for controlling the correction of subject blur based on a correction position specified from the candidates.

Images