Electronic device, control method of electronic device, computer program product, and computer readable storage medium

Abstract

The invention provides an electronic device, a control method of the electronic device, a computer program product, and a computer readable storage medium. The electronic apparatus acquires information relating to a difference between a posture of a grip portion held by a user and a posture of an imaging unit arranged to be able to change relative to the posture of the grip portion, and performs control to display an image captured by the imaging unit, in a case where the posture of the grip portion is changed, the information relating to the difference between the posture of the grip portion held by the user and the posture of the imaging unit. The posture of the imaging unit follows the posture of the grip portion more gently than a change in the posture of the grip portion, and is controlled based on the information such that a range or a position predicted as an imaging range or a position of the imaging range at the time of completion of the following of the imaging unit is recognizable on the image.

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method of the electronic device, a computer program product, and a computer-readable storage medium, and particularly relates to control for preventing shaking of a captured video during shooting. Background Art

[0002] In recent years, Vlog shooting has been widely carried out. In Vlog shooting, an individual records daily life in a video for sending on SNS or the like. A person who performs shooting often performs Vlog shooting while holding a photographing device by hand and walking. For this reason, as Vlog shooting, shooting using an electric gimbal device for preventing shaking of a captured video is widely carried out.

[0003] When shooting using an electric gimbal device, it may be desirable to suppress movement (posture change) of a photographing device (imaging unit) due to camera shake, and to moderate movement such as panning or tilting of the photographing device desired by the person performing shooting. Japanese Unexamined Patent Application Publication No. 2015-180917 discloses a technique of dividing movement of a photographing device into movement of a low-frequency component and movement of a high-frequency component and eliminating each movement in an appropriate form. In the technique disclosed in Japanese Unexamined Patent Application Publication No. 2015-180917, when changing from a non-panning state to a panning state, movement of the low-frequency component is gently suppressed.

[0004] However, when moderating movement (posture change) of a photographing device (imaging unit), the timing at which the movement of the photographing device becomes stable is delayed with respect to the completion timing of a posture change operation (operation of changing the posture of the photographing device) of the person performing shooting. Therefore, there may be a difference between the direction desired by the person performing shooting and the direction in which the photographing device finally becomes stable. In a case where the direction in which the photographing device finally becomes stable is different from the direction desired by the person performing shooting, when adjusting the direction of the photographing device again, an unnatural captured video is obtained. Summary of the Invention

[0005] The present invention provides a technique that allows a person performing shooting to easily direct an imaging unit in a direction desired by the person performing shooting in a case where a posture change of the imaging unit is moderated.

[0006] An electronic device according to the present invention includes: an acquisition unit configured to acquire information related to a difference between a posture of a holding part held by a user and a posture of an imaging unit arranged to be able to change its posture relative to the holding part; and a display control unit configured to control to display an image captured by the imaging unit, wherein when the posture of the holding part changes, the posture of the imaging unit follows the posture of the holding part more gently than the change of the posture of the holding part, and the display control unit performs control based on the information such that a range or position predicted as a shooting range or a position of the shooting range when the follow-up of the imaging unit is completed can be recognized on the image.

[0007] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A and Figure 1B are external views of a digital camera;

[0009] Figure 2 are block diagrams showing the configuration of the digital camera;

[0010] Figure 3A and Figure 3B are schematic views of a setting screen related to predictive display;

[0011] Figure 4 is a flowchart of shooting mode processing;

[0012] Figure 5 is a flowchart of exercise amount acquisition processing;

[0013] Figure 6 is a flowchart of prediction processing;

[0014] Figure 7 is a flowchart of predictive display processing;

[0015] Figure 8 are schematic views showing the transition of displaying a video in the point display mode;

[0016] Figure 9 are schematic views showing the transition of displaying a video in the frame + grid line display mode;

[0017] Figure 10 are schematic views showing the transition of displaying a video in the only frame display mode;

[0018] Figures 11A to 11C are schematic views showing display examples of out-of-range indicators; and

[0019] Figure 12It is a schematic diagram showing the transition of displaying a video in a mask mode. Detailed implementation

[0020] Embodiments of the present invention will be described below with reference to the accompanying drawings. Figure 1A and Figure 1B It is an external view of a digital camera 100 (imaging device), which is an example of a device (electronic device) to which the present invention can be applied. Figure 1A It is a front perspective view of the digital camera 100, and Figure 1B It is a rear perspective view of the digital camera 100.

[0021] The digital camera 100 includes a grip portion 90, an imaging unit 110, and a posture control unit 300. The imaging unit 110 (imaging section) is arranged so that the posture of the imaging unit 110 can be changed relative to the grip portion 90. The posture control unit 300 includes a pan axis drive unit 301, a roll axis drive unit 302, and a pitch axis drive unit 303. The pan axis drive unit 301 is fixed to the grip portion 90, and the pitch axis drive unit 303 is fixed to the imaging unit 110. By driving (rotating) at least one of the pan axis drive unit 301, the roll axis drive unit 302, and the pitch axis drive unit 303, the posture of the imaging unit 110 relative to the grip portion 90 can be maintained or changed. The grip portion 90 and the posture control unit 300 may be separable, or the imaging unit 110 and the posture control unit 300 may be separable. In the case where the imaging unit 110 and the posture control unit 300 are separable, the imaging unit 110 may be a smart phone or the like. The posture control unit 300 is a unit like an electric gimbal device, but a non-electric gimbal device may be used instead of the posture control unit 300.

[0022] The display unit 28 displays images and various information. The touch panel 70a can detect a touch operation on the display surface (touch operation surface) of the display unit 28. The shutter button 61 is an operation member for giving a shooting instruction. The mode selector switch 60 is an operation member for switching between various modes. The power switch 72 is an operation member for switching between turning on (ON) and turning off (OFF) the power of the digital camera 100.

[0023] The controller wheel 73 is a rotatable operation member and is used, for example, to indicate selection items. When a rotation operation of the controller wheel 73 is performed, an electric pulse signal is generated according to the operation amount (rotation amount), and each unit of the digital camera 100 is controlled based on the pulse signal. The angle and rotation speed of the rotation operation of the controller wheel 73 can be determined by the pulse signal. The controller wheel 73 only needs to be an operation member capable of detecting a rotation operation, and can be, for example, a dial operation member in which the controller wheel 73 itself rotates according to the rotation operation to generate a pulse signal. The controller wheel 73 can be an operation member including a touch sensor (so-called touch wheel), or can be an operation member that detects the rotational movement of a user's finger on the controller wheel 73 without rotating the controller wheel 73 itself.

[0024] The four-way key 74 is configured to be able to press each of the upper, lower, left, and right parts, and can perform processing corresponding to the pressed part of the four-way key 74. The SET button 75 is a push button and is mainly used to determine selection items. The movie button 76 is a push button and is used to give an instruction to start or stop movie shooting (recording). The menu button 81 is a push button for performing an instruction operation to display a menu screen, and when the menu button 81 is pressed, a menu screen for various settings can be displayed on the display unit 28. The user can intuitively perform various settings by using the menu screen displayed on the display unit 28, the four-way key 74, and the SET button 75.

[0025] Figure 2 is a block diagram showing the configuration of the digital camera 100.

[0026] The imaging lens 103 is a lens group including a zoom lens and a focusing lens. The shutter 101 is a shutter having an aperture function. The imaging processing unit 22 is an imaging element (image sensor) composed of a CCD or CMOS element or the like that converts an optical image into an electrical signal. The A / D converter 23 converts the analog signal output from the imaging processing unit 22 into a digital signal. The shutter 102 covers the imaging system including the imaging lens 103, the shutter 101, and the imaging processing unit 22 of the digital camera 100, thereby preventing contamination or damage to the imaging system.

[0027] The image processing unit 24 performs predetermined processing (such as pixel interpolation, resizing processing such as reduction, and color conversion processing, etc.) on the data from the A / D converter 23 or the data from the memory control unit 15. In addition, the image processing unit 24 performs predetermined arithmetic processing by using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the calculation results obtained by the image processing unit 24. As a result, through-the-lens (TTL) type autofocus (AF) processing, automatic exposure (AE) processing, and flash pre-emission (EF) processing are performed. In addition, the image processing unit 24 performs predetermined arithmetic processing by using the captured image data, and performs TTL type automatic white balance (AWB) processing based on the obtained calculation results.

[0028] The memory control unit 15 controls the data transmission and reception between the A / D converter 23, the image processing unit 24, and the memory 32. The output data from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15. Alternatively, the output data from the A / D converter 23 is written into the memory 32 via the memory control unit 15 without the intervention of the image processing unit 24. The memory 32 stores the image data obtained by the imaging processing unit 22 and converted into digital data by the A / D converter 23, as well as the image data to be displayed on the display unit 28. The memory 32 has a sufficiently large storage capacity to store a predetermined number of still images, as well as movie images and voices for a predetermined time. In addition, the memory 32 also serves as a memory (video memory) for image display. The D / A converter 13 converts the image data to be displayed and stored in the memory 32 into an analog signal, and supplies the analog signal to the display unit 28. Therefore, the display unit 28 displays the image data to be displayed and written into the memory 32 via the D / A converter 13.

[0029] The display unit 28 is a display such as an LCD or an organic EL, and performs a display corresponding to the analog signal from the D / A converter 13. The digital signal that has been A / D converted by the A / D converter 23 and accumulated in the memory 32 is converted into an analog signal in the D / A converter 13, and the analog signal is sequentially transmitted to the display unit 28 and displayed on the display unit 28, whereby a live view display (LV display) can be performed. Hereinafter, the image displayed in the live view display is referred to as a live view image (LV image).

[0030] The non-volatile memory 56 is a memory that serves as an electrically erasable and recordable recording medium, and is, for example, an EEPROM. In the non-volatile memory 56, constants, programs, etc. for the operation of the system control unit 50 are stored. The program used here is a computer program for executing various flowcharts described later. The system control unit 50 is a control unit including at least one processor and / or at least one circuit, and controls the entire digital camera 100. The system control unit 50 implements the processing described later by executing the program stored in the non-volatile memory 56. The system memory 52 is, for example, a RAM, and the system control unit 50 loads constants, variables, and programs read from the non-volatile memory 56 for the operation of the system control unit 50 into the system memory 52. In addition, the system control unit 50 also performs display control by controlling the memory 32, the D / A converter 13, the display unit 28, etc.

[0031] The system timer 53 is a timer unit that counts the time used for various controls and the time of the built-in clock.

[0032] The communication unit 54 transmits video signals and voice signals to an external device wirelessly or via a wired cable, and receives video signals and voice signals from the external device. The communication unit 54 can also be connected to a wireless local area network (LAN) and the Internet. In addition, the communication unit 54 can also communicate with an external device via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging processing unit 22 and images recorded in the recording medium 200, and can receive image data and various other types of information from an external device.

[0033] The posture detection unit 55 detects the posture of the holding unit 90 relative to the direction of gravity. Based on the posture detected by the posture detection unit 55, it can be determined whether the image captured by the imaging processing unit 22 is an image captured with the holding unit 90 held horizontally or vertically. The system control unit 50 can add direction information corresponding to the posture detected by the posture detection unit 55 to the image file of the image captured by the imaging processing unit 22, or rotate and record the image. Additionally, the system control unit 50 can operate the posture control unit 300 based on information changed according to the posture detected by the posture detection unit 55 to maintain the posture of the imaging unit 110 relative to the direction of gravity. The system control unit 50 can also change the posture of the imaging unit 110 while suppressing sudden changes in the posture of the imaging unit 110. For example, an acceleration sensor or a gyro sensor, etc., can be used for the posture detection unit 55. The movement of the holding unit 90 (such as whether the holding unit 90 is panning, tilting, rolling, lifting, or stationary, etc.) can also be detected by using an acceleration sensor or a gyro sensor as the posture detection unit 55.

[0034] The power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit for switching the blocks to be powered on, etc., and detects whether a battery is installed, the battery type, the remaining battery power, etc. In addition, the power control unit 80 controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the required voltage to the part including the recording medium 200 during the necessary time period. The power supply unit 30 is a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter.

[0035] The recording medium I / F 18 is an interface to a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium for recording captured images, such as a memory card, and is composed of a semiconductor memory, an optical disc, a magnetic disk, etc.

[0036] The operation unit 70 is an input unit that receives operations (user operations) from the user and is used to input various operation instructions to the system control unit 50. By, for example, selecting and operating various function icons displayed on the display unit 28, functions are appropriately assigned to the operation members of the operation unit 70 for each scenario, and the operation members of the operation unit 70 serve as various function buttons. Examples of function buttons include an end button, a return button, an image feed button, a skip button, a zoom-out button, and an attribute change button.

[0037] As Figure 2As shown, the operation unit 70 includes a mode selector switch 60, a shutter button 61, a power switch 72, a touch panel 70a, and other operation members 70b, etc. The other operation members 70b include a controller dial 73, a four-way key 74, a setting button 75, a movie button 76, a menu button 81, etc.

[0038] The mode selector switch 60 switches the operation mode of the system control unit 50 to any one of a still image recording mode, a movie shooting mode, a playback mode, etc. The still image recording mode includes modes such as an automatic shooting mode, an automatic scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode), etc. In addition, the still image recording mode also includes various scene modes and a custom mode, etc., which are shooting settings for different shooting scenes. The user can directly switch the operation mode to any one of these modes with the mode selector switch 60. Alternatively, the user can temporarily switch the screen to a list screen of the shooting mode with the mode selector switch 60, and then selectively switch the mode to any one of the multiple display modes with other operation members. Similarly, the movie shooting mode can include multiple modes.

[0039] The shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on in the middle of the operation of the shutter button 61 in response to a so-called half-press (shooting preparation indication), and generates a first shutter switch signal SW1. The system control unit 50 starts shooting preparation operations such as an automatic focusing (AF) process, an automatic exposure (AE) process, an automatic white balance (AWB) process, or a flash pre-flash (EF) process with the first shutter switch signal SW1. The second shutter switch 64 is turned on when the operation of the shutter button 61 is completed in response to a so-called full-press (shooting indication), and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting process operations from reading the signal from the imaging processing unit 22 until the captured image is written as an image file to the recording medium 200 with the second shutter switch signal SW2.

[0040] The touch panel 70a is a touch sensor that detects various touch operations on the display surface of the display unit 28 (the operation surface of the touch panel 70a). The touch panel 70a and the display unit 28 can be integrally configured. For example, the touch panel 70a is configured such that the light transmittance does not interfere with the display on the display unit 28, and is attached to the upper layer of the display surface of the display unit 28. Then, the input coordinates on the touch panel 70a are associated with the display coordinates on the display surface of the display unit 28. As a result, a graphical user interface (GUI) can be provided that is configured as if the user can directly operate the screen displayed on the display unit 28.

[0041] For the touch panel 70a, the system control unit 50 can detect the following operations or states:

[0042] · The operation of a finger or a pen that has not touched the touch panel 70a newly touching the touch panel 70a, that is, the start of touching (hereinafter referred to as "touching");

[0043] · The state where a finger or a pen is touching the touch panel 70a (hereinafter referred to as "continuous touch");

[0044] · The operation where a finger or a pen is moving while touching the touch panel 70a (hereinafter referred to as "touch movement");

[0045] · The operation where a finger or a pen that has touched the touch panel 70a is released from the touch panel 70a, that is, the end of touching (hereinafter referred to as "touch stop"); and

[0046] · The state where nothing is touching the touch panel 70a (hereinafter referred to as "not touched").

[0047] When touching is detected, continuous touch is detected simultaneously. After touching, continuous touch is usually detected unless touch stop is detected. Even if touch movement is detected, continuous touch is still detected. Even if continuous touch is detected, touch movement is not detected as long as the touch position does not move. After touch stop of all fingers and pens that have touched the touch panel 70a is detected, not touched is detected.

[0048] These operations, states, and the position coordinates of a finger or a pen touching the touch panel 70a are notified to the system control unit 50 via an internal bus. Then, based on the information notified to the system control unit 50, the system control unit 50 determines which operation (touch operation) has been performed on the touch panel 70a. Regarding a touch movement, based on the change in the position coordinates, the movement direction of the finger or the pen moving on the touch panel 70a can be determined for each vertical component and each horizontal component on the touch panel 70a. When a touch movement of a predetermined distance or more is detected, it is determined that a swipe operation has been performed. An operation of quickly moving a finger a specific distance while touching the touch panel 70a and then releasing the finger is called a flick. In other words, a "flick" is an operation of quickly swiping across the touch panel 70a to flick the touch panel 70a with a finger. When a touch movement of a predetermined speed or more and a predetermined distance or more is detected and then touch stop is detected, it can be determined that a flick has been performed (it can be determined that a flick has been performed after a swipe operation). In addition, a touch operation in which multiple places (for example, two points) are touched (multi-touch) and the touch positions are brought closer to each other is called a "pinch", and a touch operation in which the touch positions are moved away from each other is called a "spread". A spread and a pinch are collectively called a pinch operation (or simply referred to as a "pinch"). The touch panel 70a can be any type of touch panel among various types such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared light type, an electromagnetic induction type, an image recognition type, and an optical sensor type. There are types that detect a touch due to contact with the touch panel and types that detect a touch due to a finger or a pen approaching the touch panel, but any of these types can be used.

[0049] In the present embodiment, when the posture of the holding unit 90 changes, the system control unit 50 controls the posture of the imaging unit 110 such that the posture of the imaging unit 110 follows the posture of the holding unit 90 more gently than the change in the posture of the holding unit 90. The posture of the imaging unit 110 is controlled via the posture control unit 300. However, when such control is performed, the user (the person taking the shot) cannot grasp the direction in which the imaging unit 110 finally stabilizes, and there is a possibility of a difference between the direction the user wants and the direction in which the imaging unit 110 finally stabilizes. In the case where the direction in which the imaging unit 110 finally stabilizes is different from the direction the user wants, when the direction of the imaging unit 110 is adjusted again, an unnatural captured video is obtained.

[0050] Therefore, in the present embodiment, the system control unit 50 predicts the imaging range or the position of the imaging range when the following of the imaging unit 110 is completed, and makes the imaging range or the position of the imaging range recognizable on the LV image. As a result, the user can easily grasp the finally stabilized direction of the imaging unit 110 from the predicted range or the predicted position, and can adjust the direction of the holding unit 90 before the imaging unit 110 is stabilized. As a result, the user can easily direct the imaging unit 110 in the direction he / she desires. The predicted range is a range predicted as the imaging range when the following of the imaging unit 110 is completed, and the predicted position is a position predicted as the position of the imaging range when the following of the imaging unit 110 is completed.

[0051] Figure 3A FIG. is a schematic diagram showing a setting screen 310 related to predictive display (display for recognizing a predicted range or a predicted position). The setting screen 310 includes selection items 311 to 314. When the selection item 311 is selected, the system control unit 50 sets (enables) a mode for displaying a frame representing the predicted range, and when the selection item 312 is selected, the system control unit 50 sets a mode for displaying a frame representing the predicted range and grid lines of the frame. When the selection item 313 is selected, the system control unit 50 sets a mode for displaying a point representing the predicted position, and when the selection item 314 is selected, the system control unit 50 sets a mode for not performing predictive display. In this way, the system control unit 50 changes the display form of the predictive display or switches whether to perform predictive display according to an instruction from the user. Note that the predictive display mode is not limited to this, and a mode for masking an area outside the predicted range in the LV image can be set. The mask is, for example, a semi-transparent gray mask. Two or more of a point, a frame, grid lines, and a mask can be combined and displayed appropriately. For example, various modes can be set, such as a mode for displaying all of a point, a frame, grid lines, and a mask, a mode for displaying a point, a frame, and a mask, a mode for displaying a point and a frame, a mode for displaying a point and a mask, and a mode for displaying a frame and a mask.

[0052] In the present embodiment, the system control unit 50 controls not to perform predictive display when the difference between the posture of the imaging unit 110 and the posture of the holding unit 90 is less than a threshold value, and controls to perform predictive display when the difference is equal to or greater than the threshold value. Note that the system control unit 50 can control to perform predictive display regardless of whether the difference between the posture of the imaging unit 110 and the posture of the holding unit 90 is less than the threshold value. The system control unit 50 can control not to perform predictive display when the difference between the posture of the imaging unit 110 and the posture of the holding unit 90 is equal to or less than the threshold value, and can control to perform predictive display when the difference is greater than the threshold value.

[0053] Figure 3B is a schematic diagram showing a setting screen 320 related to the above threshold. The setting screen 320 includes text boxes 321 and 322. When Figure 3A the selection item 311 or the selection item 312 in is selected, the system control unit 50 uses the value input in the text box 321 as the threshold. When Figure 3A the selection item 313 in is selected, the system control unit 50 uses the value input in the text box 322 as the threshold. In this way, the system control unit 50 changes the threshold according to an instruction from the user.

[0054] Figure 4 is a flowchart of the shooting mode process performed by the digital camera 100. Figure 4 The shooting mode process in loads a program stored in the non-volatile memory 56 into the system memory 52 by the system control unit 50 and executes the program. For example, when the shooting mode is set, the system control unit 50 starts Figure 4 the shooting mode process.

[0055] In S401, the system control unit 50 obtains posture information indicating the posture of the holding unit 90 from the posture detection unit 55, and stores the posture information as shake correction center data in the memory 32 via the memory control unit 15.

[0056] In S402, the system control unit 50 displays the LV image on the display unit 28.

[0057] In S403, the system control unit 50 performs a motion amount acquisition process for obtaining a motion amount (a change amount of the posture of the imaging unit 110) to be instructed to the posture control unit 300. The system control unit 50 stores information on the acquired motion amount in the memory 32 via the memory control unit 15. Details of the motion amount acquisition process will be described later with reference to Figure 5 the description.

[0058] In S404, the system control unit 50 determines whether to perform predictive display. If the system control unit 50 determines to perform predictive display, the process proceeds to S405; otherwise, the process proceeds to S407.

[0059] The system control unit 50 acquires the predicted display mode and threshold setting information from the non-volatile memory 56, and acquires the amount-of-motion information of the posture control unit 300 (the information stored in the memory 32 in S403) from the memory 32 via the memory control unit 15. When a mode in which predictive display is not performed is set, the system control unit 50 determines that predictive display is not performed. When a mode for displaying a frame indicating the prediction range, a mode for displaying a frame indicating the prediction range and the grid lines of the frame, or a mode for displaying a point indicating the predicted position is set, the system control unit 50 determines whether the amount of motion of the posture control unit 300 is equal to or greater than the threshold. When the difference between the posture of the imaging unit 110 and the posture of the grip unit 90 is less than the threshold, the amount of motion of the posture control unit 300 is also less than the threshold. Then, when the difference between the posture of the imaging unit 110 and the posture of the grip unit 90 is equal to or greater than the threshold, the amount of motion of the posture control unit 300 is also equal to or greater than the threshold. Therefore, the system control unit 50 determines to perform predictive display when the amount of motion is equal to or greater than the threshold, and determines not to perform predictive display when the amount of motion is less than the threshold.

[0060] In S405, the system control unit 50 performs a prediction process for predicting the imaging range or the position of the imaging range when the following of the imaging unit 110 is completed. Details of the prediction process will be described later with reference to Figure 6 Describe the details of the prediction process.

[0061] In S406, the system control unit 50 performs a predictive display process for displaying the prediction range or the predicted position in an identifiable manner. Details of the predictive display process will be described later with reference to Figure 7 Describe the details of the predictive display process.

[0062] In S407, the system control unit 50 acquires the amount-of-motion information of the posture control unit 300 (the information stored in the memory 32 in S403) from the memory 32 via the memory control unit 15, and instructs the amount of motion to the posture control unit 300. The posture control unit 300 operates with the instructed amount of motion. As a result, the posture of the imaging unit 110 changes by an amount corresponding to the amount of motion of the posture control unit 300.

[0063] In S408, the system control unit 50 determines whether the end of the shooting mode process has been instructed. If the system control unit 50 determines that the end of the shooting mode process has been instructed, the shooting mode process ends; otherwise, the process proceeds to S402. For example, when the mode selector switch 60 has been operated, that is, when switching to another operation mode has been instructed, the system control unit 50 determines that the end of the shooting mode process has been instructed. In addition, when the power switch 72 has been operated, that is, when turning off the power of the digital camera 100 has been instructed, the system control unit 50 determines that the end of the shooting mode process has been instructed.

[0064] Figure 5 Yes Figure 4 Flowchart of the exercise amount acquisition process performed in S403.

[0065] In S501, the system control unit 50 acquires posture information indicating the posture of the grip portion 90 from the posture detection unit 55, and stores the posture information as current posture data in the memory 32 via the memory control unit 15.

[0066] In S502, the system control unit 50 acquires the blur correction center data and the current posture data from the memory 32 via the memory control unit 15. Then, the system control unit 50 stores, via the memory control unit 15, data representing the following posture as new blur correction center data in the memory 32, in which the posture represented by the blur correction center data (target posture) approaches the posture represented by the current posture data (current posture). When the difference between the target posture and the current posture is equal to or greater than a predetermined amount, the system control unit 50 generates data representing a posture that makes the target posture approach the current posture by the predetermined amount as the new blur correction center data. When the difference between the target posture and the current posture is equal to or less than the predetermined amount, the system control unit 50 generates the same data as the current posture data as the new blur correction center data. By making the blur correction center data approach the current posture data in this way, the posture of the imaging unit 110 can follow the posture of the grip portion 90. As the predetermined amount (the amount of change in the target posture) becomes smaller, the posture of the imaging unit 110 follows the posture of the grip portion 90 more smoothly (at a slower speed as the predetermined amount becomes smaller).

[0067] In S503, the system control unit 50 acquires the blur correction center data from the memory 32 via the memory control unit 15. Then, the system control unit 50 calculates the exercise amount of the posture control unit 300 so that the posture of the imaging unit 110 is maintained at the posture of the blur correction center data, and stores the information of the calculated exercise amount in the memory 32 via the memory control unit 15.

[0068] Figure 6 Yes Figure 4 Flowchart of the prediction process performed in S405 of

[0069] In S601, the system control unit 50 obtains the image sensor size information of the imaging processing unit 22 from the non-volatile memory 56, and stores the image sensor size information as sensor size information in the memory 32 via the memory control unit 15.

[0070] In S602, the system control unit 50 obtains the lens focal length information of the imaging lens 103 from the imaging lens 103, and stores the lens focal length information as focal length information in the memory 32 via the memory control unit 15.

[0071] In S603, the system control unit 50 obtains the sensor size information and the focal length information from the memory 32 via the memory control unit 15. Then, the system control unit 50 calculates the imaging angle of view of the imaging unit 110 based on the obtained information, and stores the calculated imaging angle of view information (imaging angle of view information) in the memory 32 via the memory control unit 15. For example, by using Equation 1 below, the imaging angle of view θ [deg] in the horizontal direction can be calculated based on the sensor size x [mm] in the horizontal direction and the focal length f [mm]. Then, the imaging angle of view in the vertical direction can be calculated by using the sensor size in the vertical direction instead of the sensor size in the horizontal direction.

[0072] [Mathematical formula 1]

[0073]

[0074] In S604, the system control unit 50 obtains the current pose data, the blur correction center data, and the imaging angle of view information from the memory 32 via the memory control unit 15. Then, the system control unit 50 calculates the difference between the current pose data and the blur correction center data as the difference between the pose of the grip portion 90 and the pose of the imaging unit 110 (pose difference). Then, the system control unit 50 calculates the imaging range when the follow-up of the imaging unit 110 is completed based on the pose difference and the imaging angle of view. For example, the system control unit 50 calculates the position and inclination of the imaging range when the follow-up of the imaging unit 110 is completed. When the imaging angle of view is 46 degrees and there is a 23-degree pose difference, the end of the LV image is the center of the imaging range when the follow-up is completed. The system control unit 50 stores the calculated position information as prediction position information in the memory 32 via the memory control unit 15, and stores the calculated inclination information as prediction inclination information in the memory 32 via the memory control unit 15.

[0075] Figure 7 Yes Figure 4Flowchart of the prediction display process performed in S406.

[0076] In S701, the system control unit 50 obtains the setting information of the prediction display mode from the non-volatile memory 56 and determines whether the mode for displaying the points indicating the prediction positions (dot display mode) is set. If the system control unit 50 determines that the dot display mode is set, the process proceeds to S702; otherwise, the process proceeds to S704.

[0077] In S702, the system control unit 50 obtains the prediction position information from the memory 32 via the memory control unit 15 and determines whether the prediction position overlaps with the current imaging range (whether the prediction position is within the current imaging range). If the system control unit 50 determines that the prediction position overlaps with the current imaging range, the process proceeds to S703; otherwise, the process proceeds to S708.

[0078] In S703, the system control unit 50 displays the points indicating the prediction positions (prediction points) on the display unit 28. Figure 8 It is a schematic diagram showing the transition of the video display in the dot display mode. Figure 8 It shows the transition of the video display when the user turns the holding unit 90 to the lower right. Before the user changes the posture of the holding unit 90, the prediction points are not displayed, and only the LV image is displayed. When the user changes the posture of the holding unit 90, the prediction points 800 are displayed. The prediction points 800 represent the center position of the imaging range when the follow-up of the imaging unit 110 is completed. Therefore, the prediction points 800 gradually approach the center of the LV image. Note that the prediction position is not limited to the center position of the imaging range when the follow-up of the imaging unit 110 is completed, and for example, it can be the upper left corner of the imaging range when the follow-up of the imaging unit 110 is completed.

[0079] In S704, the system control unit 50 obtains the prediction position information and the prediction inclination information from the memory 32 via the memory control unit 15 and determines the prediction range based on this information. Then, the system control unit 50 determines whether the prediction range overlaps with the current imaging range (whether at least a part of the prediction range is within the current imaging range). If the system control unit 50 determines that the prediction range overlaps with the current imaging range, the process proceeds to S705; otherwise, the process proceeds to S708.

[0080] In S705, the system control unit 50 acquires the setting information of the prediction display mode from the non-volatile memory 56, and determines whether the mode for displaying the frame representing the prediction range and the grid lines of the frame (frame + grid line display mode) is set. If the system control unit 50 determines that the frame + grid line display mode is set, the process proceeds to S706. Otherwise (if the mode for displaying only the frame representing the prediction range (only frame display mode) is set), the process proceeds to S707.

[0081] In S706, the system control unit 50 acquires the prediction position information and the prediction inclination information from the memory 32 via the memory control unit 15, and determines the prediction range based on this information. Then, the system control unit 50 displays the frame (prediction frame) representing the prediction range and the grid lines of the frame on the display unit 28. Figure 9 is a schematic diagram showing the transition of the video displayed in the frame + grid line display mode. Figure 9 Shows the transition of the video displayed when the user turns the holding unit 90 downward to the right. Before the user changes the posture of the holding unit 90, neither the prediction frame nor the grid lines are displayed, and only the LV image is displayed. When the user changes the posture of the holding unit 90, the prediction frame 901 and the grid lines 902 are displayed. The grid lines 902 are the lines that divide the area (prediction range) surrounded by the prediction frame 901 into four areas of two rows and two columns, and represent the center position of the area surrounded by the prediction frame 901. Note that the number of divisions by the grid lines is not particularly limited, and can be a fixed value or can be arbitrarily set (changed) by the user.

[0082] In S707, the system control unit 50 acquires the prediction position information and the prediction inclination information from the memory 32 via the memory control unit 15, and determines the prediction range based on this information. Then, the system control unit 50 displays the frame (prediction frame) representing the prediction range on the display unit 28. Figure 10 is a schematic diagram showing the transition of the video displayed in the only frame display mode. Figure 10 Shows the transition of the video displayed when the user turns the holding unit 90 downward to the right. Before the user changes the posture of the holding unit 90, the prediction frame is not displayed, and only the LV image is displayed. When the user changes the posture of the holding unit 90, the prediction frame 1000 is displayed.

[0083] In S708, the system control unit 50 displays an out-of-range indicator indicating that the prediction range or the prediction position does not overlap with the current imaging range on the display unit 28. The out-of-range indicator indicates, for example, the direction of the prediction range or the prediction position relative to the current imaging range. Figures 11A to 11C is a schematic diagram showing a display example of the out-of-range indicator. In Figure 11AIn [the situation], since the predicted position 1101 appears in the direction to the right from the current imaging range, an out-of-range indicator 1102 indicating the right direction is displayed. In Figure 11B In [the situation], since the predicted position 1101 appears in the direction downward from the current imaging range, an out-of-range indicator 1103 indicating the downward direction is displayed. In Figure 11C In [the situation], since the predicted position 1101 appears in the direction to the lower right from the current imaging range, an out-of-range indicator 1102 indicating the right direction and an out-of-range indicator 1103 indicating the downward direction are displayed. Other indicators indicating the lower right direction may be displayed.

[0084] As described above, as a prediction display mode, a mode (masking mode) for masking an area outside the prediction range in the LV image can be set. Figure 12 is a schematic diagram showing the transition of the displayed video in the masking mode. Figure 12 shows the transition of the displayed video when the user turns the holding unit 90 to the lower right. Before the user changes the posture of the holding unit 90, the mask is not displayed, and only the LV image is displayed. When the user changes the posture of the holding unit 90, a mask 1200 for covering the area outside the prediction range is displayed. When the following of the imaging unit 110 is completed, the mask 1200 is not displayed.

[0085] As described above, according to the present embodiment, the imaging range or the position of the imaging range when the following of the prediction imaging unit 110 is completed is predicted, and the imaging range or the position of the imaging range is made recognizable on the LV image. As a result, the user can easily grasp the direction in which the imaging unit 110 finally stabilizes from the prediction range or the prediction position, and can adjust the direction of the holding unit 90 before the imaging unit 110 stabilizes. As a result, the user can easily direct the imaging unit 110 to the direction he / she wants.

[0086] Note that the above various types of control may be processing performed by one piece of hardware (e.g., a processor or a circuit) or otherwise. The processing may be shared among a plurality of hardware (e.g., a plurality of processors, a plurality of circuits, or a combination of one or more processors and one or more circuits) to control the entire device.

[0087] In addition, the above-mentioned processor is a broad processor and includes a general-purpose processor and a dedicated processor. Examples of the general-purpose processor include a central processing unit (CPU), a microprocessing unit (MPU), a digital signal processor (DSP), and the like. Examples of the dedicated processor include a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and the like. Examples of the PLD include a field programmable gate array (FPGA), a complex programmable logic device (CPLD), and the like.

[0088] The above embodiments (including modification examples) are merely examples. Any configuration obtained by appropriately modifying or changing some configurations of the embodiments within the scope of the subject matter of the present invention is also included in the present invention. The present invention also includes other configurations obtained by appropriately combining various features of the embodiments.

[0089] For example, although the system control unit 50 disposed in the holding unit 90 performs various controls, the control unit that performs the above various controls may be disposed in a part (device) different from the holding unit 90. For example, the control unit may be disposed in the imaging unit 110, or may be disposed in the posture control unit 300. In addition, although the prediction range or the prediction position is made recognizable by the display of items, there is no particular limitation on the method for making the prediction range or the prediction position recognizable. For example, the prediction range or the prediction position may be made recognizable by changing the pixel value of the LV image.

[0090] After the posture of the holding unit 90 is changed, the system control unit 50 may stop the prediction display in response to the elapse of a predetermined time (for example, one second, three seconds, or five seconds) in a state where the holding unit 90 has stopped. This improves the visibility of the LV display. As a result, for example, the user can easily change the posture of the holding unit 90 again at an appropriate timing. The stop of the prediction display can be interpreted as the cancellation of the state where the prediction range or the prediction position becomes recognizable. Even when a predetermined time has elapsed in a state where the holding unit 90 has stopped, the prediction display may continue when the difference between the posture of the imaging unit 110 and the posture of the holding unit 90 is equal to or greater than a threshold value.

[0091] The system control unit 50 may control the image processing unit 24 to detect a specific subject area (for example, a person area larger than a predetermined size) from the LV image. Then, the system control unit 50 may perform control such that the prediction range or the prediction position is not recognizable on the specific subject area in the LV image. This can improve the visibility of the specific subject area.

[0092] The system control unit 50 can make the display form of the prediction display different between the case where the prediction range or prediction position is close to the center of the LV image and the case where the prediction range or prediction position is far from the center. For example, in the case where the prediction range or prediction position is close to the center of the LV image, a prediction box can be displayed, while in the case where the prediction range or prediction position is far from the center of the LV image, a prediction point indicating the center position of the prediction range can be displayed. This makes it possible to suppress the reduction in visibility of the central part of the LV image due to the prediction display. The change in the display form of the prediction display can be a change in the type of the prediction display, or not necessarily a change in the type of the prediction display. The change in the display form of the prediction display can include a change in color, a change in brightness, or a change in the type of the frame (line) or point. In order to suppress the reduction in visibility of the LV image, in the case where the prediction range or prediction position is close to the center of the LV image, the prediction box can be displayed with high saliency, while in the case where the prediction range or prediction position is far from the center of the LV image, the prediction box can be displayed with low saliency. In the case where the prediction range or prediction position is close to the center of the LV image, the prediction point can be displayed with low saliency, while in the case where the prediction range or prediction position is far from the center of the LV image, the prediction point can be displayed with high saliency. Saliency can be considered as the degree of prominence or visibility. A display with high saliency is, for example, a display with a dark color or a low transparency, while a display with low saliency is, for example, a display with a light color or a high transparency.

[0093] According to the present invention, in the case where the posture change of the imaging unit is made gentle, a person who performs shooting can easily direct the imaging unit in the direction that the person who performs shooting desires.

[0094] Other embodiments

[0095] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (program) that executes the functions of the above-described embodiments to a system or device through a network or various storage media, and a method in which a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0096] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An electronic device, comprising: an acquisition unit configured to acquire information related to a difference between a posture of a grip held by a user and a posture of an imaging unit arranged to be able to change the posture relative to the grip; as well as a display control unit configured to control the display of the image captured by the camera unit, Wherein, when the posture of the grip portion changes, The posture of the camera unit follows the posture of the grip more gradually than the posture of the grip changes, and The display control unit performs control based on the information so that a range or a position predicted as an imaging range or a position of the imaging range when following of the imaging unit is completed can be recognized on the image.

2. The electronic device according to claim 1, wherein: The electronic device is a camera device, and the camera device further includes: The grip portion; The camera unit; and A control unit is configured to control the posture of the camera unit so that the posture of the camera unit follows the posture of the grip more gradually than the change of the posture of the grip when the posture of the grip changes.

3. The electronic device according to claim 1 or 2, further comprising: a detection unit configured to detect the posture of the holding portion, Wherein, the acquisition unit acquires the information based on the posture of the holding part.

4. The electronic device according to claim 1 or 2, wherein: The display control unit performs control to display a frame indicating the predicted range.

5. The electronic device according to claim 1 or 2, wherein: The display control unit performs control to display a frame indicating the predicted range and grid lines of the frame.

6. The electronic device according to claim 1 or 2, wherein: The display control unit performs control to display a point indicating the predicted position.

7. The electronic device according to claim 1 or 2, wherein: The display control unit performs control to mask an area outside the predicted range in the image.

8. The electronic device according to claim 1 or 2, wherein: The display control unit changes a display form in which the predicted range or position can be identified according to an instruction from a user.

9. The electronic device according to claim 1 or 2, wherein: The display control unit switches whether the predicted range or position can be identified according to an instruction from a user.

10. The electronic device according to claim 1 or 2, wherein: When the difference between the posture of the imaging unit and the posture of the grip is smaller than a threshold value, the display control unit performs control so that the predicted range or position cannot be recognized.

11. The electronic device according to claim 10, wherein: The display control unit changes the threshold value according to an instruction from a user.

12. The electronic device according to claim 1 or 2, wherein: In a case where the predicted range or position does not overlap with the current imaging range, the display control unit controls to display an indicator indicating that the predicted range or position does not overlap with the current imaging range.

13. The electronic device according to claim 12, wherein: The indicator indicates the direction of the predicted range or position relative to the current camera range.

14. The electronic device according to claim 1 or 2, wherein: The display control unit performs control to cancel a state in which the predicted range or position can be recognized in response to a lapse of a predetermined time in a state in which the grip portion is stopped.

15. The electronic device according to claim 1 or 2, wherein: The display control unit performs control so that the predicted range or position cannot be recognized on a specific object area in the image.

16. The electronic device according to claim 1 or 2, wherein: The display control unit makes a display form in which the predicted range or position can be identified differ between a case where the predicted range or position is close to a center of the image and a case where the predicted range or position is far from the center.

17. A method for controlling an electronic device, comprising: an acquisition step for acquiring information related to a difference between a posture of a grip held by a user and a posture of an imaging unit arranged to be able to change the posture relative to the grip; as well as A display control step is used to control the display of the image captured by the camera unit, Wherein, when the posture of the grip portion changes, The posture of the camera unit follows the posture of the grip more gradually than the posture of the grip changes, and In the display control step, control is performed based on the information so that a range or a position predicted as the imaging range when the following of the imaging unit is completed or the position of the imaging range can be recognized on the image.

18. A computer program product comprising a program for causing a computer to execute each step of the control method according to claim 17.

19. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 17.

Images