Image capture device, image capture device control method, and program

Abstract

To provide an imaging device and a method for supporting a camera user in taking moving pictures with an appropriate camerawork.SOLUTION: A digital camera 100 includes an imaging unit 106, and a live view image 500 captured by the imaging unit 106 is displayed on a display unit 105. When a camerawork to be used for shooting a video is selected from a plurality of cameraworks representing a method for shooting a video, the display unit 105 is controlled to display guides (guide frame 502, message 503, arrow 504) that instruct the user to move in accordance with the selected camerawork, on the basis of the camerawork information of the selected camerawork.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] In recent years, video shooting using cameras such as digital still cameras and digital video cameras has become very popular. In video shooting, shooting with appropriate camerawork to realize the intended video expression requires a lot of experience and skill. Patent Document 1 discloses an imaging device that provides advice to a user on camera settings and the like so that the user can take better still images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-128251 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, a beginner who has just purchased a camera does not understand what techniques are available for video expression, and even if there is a video expression he or she wants to express, he or she does not know how to use the camera to shoot it. Therefore, it is difficult to shoot a video with appropriate camerawork. Patent Document 1 can provide advice on still image shooting, but cannot provide camera users, especially beginners, with appropriate information on camerawork when shooting a video, which requires constant changes in the relationship between the camera and the camera user and the subject.

[0005] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide an imaging apparatus and method that assists a camera user in capturing moving images with appropriate camerawork. [Means for solving the problem]

[0006] The present invention is an imaging device having an imaging means, wherein a live view image captured by the imaging means is displayed on a display unit, and when a camerawork to be used for shooting a video is selected from a plurality of cameraworks representing a method of shooting a video, the imaging device further comprises a control means for controlling the display unit to display a guide instructing a user to move in accordance with the selected camerawork based on attribute information of the selected camerawork. Effect of the Invention

[0007] According to the present invention, it is possible to provide an imaging device and method that assists a camera user in capturing moving images with appropriate camerawork. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating an example of the hardware configuration of a digital camera. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of a digital camera. [Diagram 3] 10 is a diagram showing an example of the configuration of data stored in a camera work information storage unit; [Figure 4] FIG. 4 is a diagram showing an example of the configuration of camera work information. [Diagram 5] FIG. 13 is a diagram showing an example of a recommended camera work notification screen. [Figure 6] FIG. 13 is a diagram showing an example of a recommended camera work details screen. [Figure 7] FIG. 13 is a diagram showing an example of a display screen during video shooting. [Figure 8] 10 is a flowchart showing a main process of video shooting. [Figure 9] 11 is a flowchart showing a process executed in a photography assist mode. [Figure 10] 10 is a flowchart showing details of assistance during shooting processing. [Figure 11] FIG. 13 is a diagram showing an example of a display screen during video shooting. [Figure 12]FIG. 13 is a diagram showing an example of a display screen during video shooting. [Figure 13] FIG. 2 is a diagram illustrating an example of a functional configuration of a digital camera. [Figure 14] 10 is a diagram showing an example of the configuration of data stored in a camera work information storage unit; [Figure 15] 11 is a flowchart showing a process executed in a photography assist mode. [Figure 16] 10 is a flowchart showing details of assistance during shooting processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the accompanying drawings. The following embodiment does not limit the scope of the invention according to the claims. Note that in this specification, the term "image" is a concept that includes not only "still images" but also "moving images."

[0010] <Embodiment 1> FIG. 1 shows the hardware configuration of a digital camera. Digital camera 100 shown in FIG. 1 is an example of an imaging device. Hereinafter, digital camera 100 may be simply referred to as camera. Digital camera 100 includes a CPU 101, a primary storage unit 102, a communication control unit 103, an input unit 104, a display unit 105, an imaging unit 106, a secondary storage unit 107, a camera control unit 108, and a lens control unit 109. CPU 101 is connected to each of these components via a bus 110. Note that digital camera 100 may be configured as a mobile device such as a smartphone or tablet with a camera function.

[0011] A CPU 101 (Central Processing Unit) controls the entire digital camera 100. The CPU 101 reads out a program stored in the secondary storage unit 107, expands it in a work area formed in the primary storage unit 102, and operates according to the expanded program to realize the processing of the flowcharts described below. The primary storage unit 102 forms a work area that temporarily stores various programs executed by the CPU 101 and data related to these programs.

[0012] The communication control unit 103 is connected to an external electronic device (not shown), such as a PC (personal computer), via a communication cable or the like, and controls the transmission and reception of data such as images. Note that the connection with the external electronic device does not necessarily have to be a wired connection via a communication cable or the like, and may be a wireless connection such as infrared or Wi-Fi. The communication control unit 103 may also be connected to a mobile device such as a smartphone. The communication control unit 103 may also be connected to a network via a communication cable or the like, and control the transmission and reception of data with a cloud server on the network.

[0013] The input unit 104 includes a power key, a shutter key, movement keys for instructing to move the cursor position on the display screen of the display unit 105, and a decision key for instructing to set the content selected on the display screen. The input unit 104 also includes a mode changeover switch (not shown) for changing the operation mode of the digital camera 100, such as a normal shooting mode, a shooting assist mode, an end mode, and a playback mode. The input unit 104 outputs an operation signal corresponding to each key operation and switch operation by the user to the CPU 101. The input unit 104 and the display unit 105 may be configured by a touch panel display having an input function and a display function. The keys and switches corresponding to various operations may be displayed on the touch panel display. The input unit 104 may output an operation signal corresponding to a touch operation on the touch panel display to the CPU 101.

[0014] The display unit 105 is configured with a monitor (display screen) such as an LCD (Liquid Crystal Display). The display unit 105 displays a live view image captured by the imaging unit 106, images stored in the secondary storage unit 107, and the like on the display screen in accordance with an instruction of a display signal input from the CPU 101. The imaging unit 106 includes an imaging element that receives subject light incident via an optical lens device, converts the received light into an imaging signal, and outputs the imaging signal, and an A / D conversion unit that converts the imaging signal output from the imaging element into image data and outputs the image data. The imaging unit 106 writes the image data output from the A / D conversion unit directly or indirectly into the secondary storage unit 107. The imaging unit 106 is an example of an imaging means.

[0015] The secondary storage unit 107 stores various programs, camera work information, images captured by the imaging unit 106, etc. The camera work information will be described in detail later. The camera control unit 108 operates the camera body. Specifically, the camera control unit 108 operates a drive mechanism that moves the camera body in at least one of six axial directions, including the x-, y-, and z-axial directions in three-dimensional space and the rotational directions around each of these axes, in accordance with a control signal input from the CPU 101, thereby changing the position and orientation of the camera body. The camera control unit 108 includes a drive device such as a motor, a drive circuit, and the like. The camera control unit 108 also includes an acceleration sensor, an angular velocity sensor, and the like for detecting the position and orientation of the camera body. The lens control unit 109 operates the lens. Specifically, the lens control unit 109 operates the optical lens device of the imaging unit 106 in accordance with a control signal input from the CPU 101, and changes the zoom, focus, aperture, etc. The lens control unit 109 includes a driving device such as a motor, a driving circuit, etc.

[0016] Next, the functional configuration of the digital camera 100 according to this embodiment will be described with reference to Fig. 2. The digital camera 100 functions as each functional unit shown in Fig. 2 by the CPU 101 executing a program stored in the secondary storage unit 107. The functions of the camerawork information storage unit 201, subject recognition unit 202, match determination unit 203, camerawork notification unit 204, and camerawork selection unit 205 of the digital camera 100 will be described.

[0017] The camerawork information storage unit 201 stores one or more pieces of camerawork information in association with each piece of information about a plurality of subjects. Here, an example is shown in which camerawork information suitable for a type of subject is associated with that type of subject. Types of subjects include the whole body of a person, a person's face, a landscape, an animal, etc. Camerawork information suitable for the size, position, and movement information (static or dynamic object, movement speed, movement direction) of a subject may be associated with those.

[0018] Fig. 3 shows the data structure stored in the camerawork information storage unit 201 according to this embodiment. As shown in Fig. 3, the camerawork information storage unit 201 stores a data table showing the association of camerawork with respect to the type of subject. The camerawork includes follow, dolly in, dolly out, circle, fix, pan, tilt, zoom in, zoom out, focus shift (in), focus shift (out), tilt follow, and the like.

[0019] Follow, dolly in, dolly out, and circle are shooting methods in which a user shoots a video while moving with the camera body. Follow is a shooting method in which a user shoots a video while following the movement of a subject. Dolly in and dolly out are shooting methods in which a user shoots a video while moving toward or away from a subject. Circle is a shooting method in which a user shoots a video while moving in a circle around the subject. Fix is ​​a shooting method in which a camera body is fixed and a video is shot. Pan is a shooting method in which a video is shot while moving the camera body in the horizontal direction. Tilt is a shooting method in which a video is shot while moving the camera body in the vertical direction. Zoom in and zoom out are shooting methods in which a video is shot while operating the zoom function of the imaging unit 106. Focus shift (in) and focus shift (out) are shooting methods in which a video is shot while operating the focus function of the imaging unit 106. Tilt follow is a shooting method in which a video is shot while moving the user and the camera body at the same time. The types of camera work are not limited to those described above.

[0020] FIG. 4 shows an example of the configuration of camerawork information for each camerawork. The camerawork information is created in advance according to a template. The camerawork information includes attribute information necessary for shooting a video with that camerawork, such as the way and route of the user's camera body and information on the way the camera itself moves. Specifically, the information is in the form of characters, schematic diagrams and images showing the composition during video shooting, and numerical values ​​(movement speed, movement direction, movement time, movement angle, camera setting value, etc. of the camera body). The various numerical values ​​included in the camerawork information may be changed by the user as appropriate by touch operation or key operation from the input unit 104. The camerawork information may further include examples of videos shot with that camerawork (example videos) and videos (shooting point videos) that record the method of shooting with that camerawork from a third person's perspective. FIG. 4 shows examples of camerawork information for following and camerawork information for panning.

[0021] It should be noted that the information held by the camerawork information holding unit 201 is stored in the secondary storage unit 107, but some of the information may be stored in an external storage device connected via the communication control unit 103. For example, the above example videos and videos regarding shooting methods may be stored in an external storage device or a cloud server.

[0022] The subject recognition unit 202 first acquires a group of images captured by the imaging unit 106 from the current time to a predetermined time ago as video data being captured from the secondary storage unit 107. Then, the subject recognition unit 202 analyzes the acquired video data to recognize the subject appearing in the video data. Specifically, the subject recognition unit 202 acquires the type, size, position, movement information, and the like of the subject using information such as color distribution and exposure distribution in the image obtained from the video data, and information on their time changes. The movement information is information on whether the subject is a static object or a dynamic object, and information on the movement speed and movement direction of the subject if it is a dynamic object. Types of subjects recognized by the subject recognition unit 202 include a person, a whole body of a person, a face of a person, a front of a person, a back of a person, eyes, an animal, a landscape (without people), a building, and the like. Note that when the subject recognition unit 202 recognizes multiple subjects from the video data, it may acquire information on each of the multiple subjects. It may also be possible to identify a main subject from the multiple subjects and acquire only information on the main subject.

[0023] The match determination unit 203 compares the information of the subject acquired by the subject recognition unit 202 with the information of the subject associated with the camerawork information held by the camerawork information holding unit 201 to determine whether there is camerawork information that matches the two. If there is matching camerawork information, the match determination unit 203 determines, as the recommended camerawork, at least one piece of camerawork information associated with the same subject information as the information of the subject acquired by the subject recognition unit 202. If there is no matching camerawork information, the match determination unit 203 determines, as the recommended camerawork, at least one piece of camerawork information that has been determined in advance. In the example of FIG. 3, the camerawork information is associated with the type of subject, so the recommended camerawork is determined using the type of subject, but the recommended camerawork may be determined using the size, position, movement information, etc. of the subject.

[0024] The camerawork notification unit 204 displays a notification based on the recommended camerawork determined by the match determination unit 203 on the display unit 105. Alternatively, instead of displaying on the display unit 105 of the digital camera 100, the camerawork notification unit 204 may communicate with an external electronic device via the communication control unit 103 and display, for example, on a display screen of a PC or a mobile terminal. Furthermore, instead of displaying, the camerawork notification unit 204 may notify using another method, such as using an audio output unit (not shown).

[0025] The operation of the camerawork notification unit 204 when the whole body of a person is recognized by the subject recognition unit 202 will be specifically described with reference to FIG. 3. In this case, the match determination unit 203 determines, as recommended cameraworks, follow, dolly in, circle, tilt, zoom in, zoom out, and tilt follow, which are associated with a person (whole body), from among the multiple cameraworks stored in the camerawork information storage unit 201. The camerawork notification unit 204 then notifies the determined recommended camerawork. Note that, when there are multiple pieces of camerawork information associated with the same type of subject as the type of subject acquired by the subject recognition unit 202, the camerawork notification unit 204 may narrow down the information to a few and notify the user. In this case, the camerawork notification unit 204 may notify only the camerawork information previously set by the user. Alternatively, it may notify only a predetermined number of camerawork information with higher priority.

[0026] FIG. 5 shows an example of a recommended camerawork notification screen. The recommended camerawork notification screen 300 is displayed on the display unit 105. The recommended camerawork notification screen 300 in FIG. 5 displays a recommended camerawork notification 301 for four recommended cameraworks: circle, dolly-in, follow, and pan. The camerawork notification unit 204 acquires each piece of camerawork information for the recommended camerawork from the camerawork information storage unit 201, and displays the recommended camerawork notification 301 using the acquired camerawork information. In the example of FIG. 5, the camerawork notification unit 204 displays the recommended camerawork notification 301 by superimposing an arrow or explanatory text on a thumbnail image of a schematic diagram or example video included in each of the camerawork information for circle, dolly-in, follow, and pan. The arrows shown in FIG. 5 typically represent the movement of the camera user or the camera itself during video shooting.

[0027] Furthermore, a camerawork selection area 302 is displayed on the recommended camerawork notification screen 300. The camerawork selection area 302 is provided with a UI for selecting one recommended camerawork from the notified recommended cameraworks. When a recommended camerawork is selected in the camerawork selection area 302 by the user's operation of the input unit 104, the camerawork notification unit 204 displays a detailed explanation of the selected recommended camerawork on the display unit 105.

[0028] FIG. 6 shows an example of a recommended camerawork detail screen being selected. The recommended camerawork detail screen 400 is a screen that displays a detailed explanation of the recommended camerawork being selected in FIG. 5, and is displayed on the display unit 105. The recommended camerawork detail screen 400 displays, as explanatory information, a sample video 401, a summary explanation 402, shooting points 403, and the like. The information displayed as explanatory information is included in the camerawork information, and is stored in the camerawork information storage unit 201 as described above. In addition, a camerawork selection key 404 is displayed on the recommended camerawork detail screen 400. When the camerawork selection key 404 is selected by the user through the input unit 104, the camerawork selection unit 205 selects the recommended camerawork being selected as the camerawork to be used for video shooting.

[0029] Next, the functions of the camera work setting unit 206, camera instruction unit 207, user instruction unit 208, image shooting unit 209, and image recording unit 210 of the digital camera 100 will be described.

[0030] The camerawork setting unit 206 acquires camerawork information about the camerawork selected by the camerawork selection unit 205 from the camerawork information storage unit 201, and provides it to the camera instruction unit 207 or the user instruction unit 208. Furthermore, the camerawork setting unit 206 instructs the camera instruction unit 207 or the user instruction unit 208 to start operational control of the camerawork selected by the camerawork selection unit 205.

[0031] For example, if the selected camera work is "pan," the camera work setting unit 206 notifies the camera instruction unit 207 of the panning angle and time. Information on the panning angle and time is included in the "pan" camera work information stored in the camera work information storage unit 201. Note that the panning angle and time may be changed appropriately by the user through touch operations or key operations from the input unit 104.

[0032] Furthermore, if the selected camerawork is "follow", the camerawork setting unit 206 notifies the user instruction unit 208 of the subject information acquired by the subject recognition unit 202 and the "follow" camerawork information stored in the camerawork information storage unit 201. The subject information notified to the user instruction unit 208 includes the size, position, movement information, etc. of the subject.

[0033] When the start of camerawork operational control is instructed, the camera instruction unit 207 instructs the camera control unit 108 and the lens control unit 109 to start the operation. In the case of the above-mentioned "pan" example, the camera instruction unit 207 outputs a control signal based on the panning angle, time, etc. notified by the camerawork setting unit 206 to the camera control unit 108. The camera control unit 108 performs a panning operation of the camera in accordance with the control signal from the camera instruction unit 207. Note that the start of the operation of the camera body and the lens and the start of the operational control of the camerawork do not necessarily have to be simultaneous.

[0034] When an instruction to start camerawork operational control is given, the user instruction unit 208 controls the display unit 105 to display a guide instructing the user's movements and camera operations. FIG. 7 shows an example of a display screen on which a guide is displayed when the selected camera work is "follow". The image capturing unit 209, which will be described later, displays images captured by the imaging unit 106 on the display unit 105 in real time. As a result, a live view image is displayed on the display unit 105. The live view image 500 in FIG. 7 shows a scene in which a person is walking along the beach in the evening, moving away from the camera user. Based on information provided by the camera work setting unit 206, the user instruction unit 208 displays a guide superimposed on the live view image 500, instructing the user to move in accordance with the selected camera work.

[0035] Specifically, the user instruction unit 208 displays a guide frame 502 on the live view image 500 to guide the position of a main subject 501 (a person's entire body) in the live view image 500. The guide frame 502 instructs the user where and at what size the main subject should be maintained in the screen during "follow" video shooting. The position and size of the guide frame 502 may be set in advance. Alternatively, the user instruction unit 208 may automatically set the position and size of the guide frame 502 in accordance with the size and position of the subject in the live view image 500 before or at the start of the selected camerawork operation control.

[0036] The message 503 and the arrow 504 instruct the user on how to move with the camera during video shooting, and on details such as points to be careful of when moving. The size and direction of the arrow 504 may be set in advance. Alternatively, the user instruction unit 208 may automatically set the size and direction of the arrow 504 according to the moving speed and moving direction of the subject in the live view image 500 before or at the start of the camerawork operation control. The guide frame 502, the message 503, and the arrow 504 are examples of the guide. By the user moving while following the guide and shooting the subject, it is possible to shoot an appropriate video with the selected camerawork. In addition, the user instruction unit 208 also has a function of updating the guide based on the information of the subject acquired by the subject recognition unit 202 during the camerawork operation control. In addition, the display unit 105 displays a recording start key 505 together with the live view image 500. When the recording start key 505 is selected by the user's operation from the input unit 104, the image recording unit 210 accepts an instruction to start recording.

[0037] The image capturing unit 209 captures an image of a subject using the image capturing unit 106 . When the image recording unit 210 receives an instruction to start recording, the image recording unit 210 records the moving image data captured by the image capturing unit 209 from when the instruction to start recording is received until the end of recording as a moving image file in the secondary storage unit 107 or the like.

[0038] 8 is a flowchart showing the main process of video shooting. The process of this flowchart is realized by CPU 101 loading a program stored in secondary storage unit 107 into primary storage unit 102 and executing it. In the following description, each process (step) is represented by adding an S to the beginning, and the notation of the process (step) is omitted. First, in S601, the CPU 101 analyzes the current operation mode. The current operation mode can be changed by operating a mode changeover switch (not shown) of the input unit 104, or the like.

[0039] In S602, the CPU 101 determines whether the current operation mode is the normal shooting mode, and if it is determined that it is the normal shooting mode (S602; yes), the process proceeds to S603. The normal shooting mode is a shooting mode in which shooting is performed by a process that is predetermined as a standard in the digital camera 100. On the other hand, if it is determined that the current operation mode is not the normal shooting mode (S602; no), the process proceeds to S604. In S603, the CPU 101 executes the process in the normal shooting mode, after which the process returns to S601.

[0040] In S604, the CPU 101 determines whether the current operation mode is the shooting assist mode or not, and if it is determined that the current operation mode is the shooting assist mode (S604; yes), the process proceeds to S605. On the other hand, if it is determined that the current operation mode is not the shooting assist mode (S604; no), the process proceeds to S606. In S605, the CPU 101 executes a process flow of the shooting assist mode. The process flow of the shooting assist mode will be described in detail with reference to Fig. 9. After that, the process returns to S601. In S606, the CPU 101 determines whether the current operation mode is the end mode or not, and if it is determined that the current operation mode is not the end mode (S606; no), the process proceeds to S607. On the other hand, if it is determined that the current operation mode is the end mode (S606; yes), the process of the flowchart in FIG. 8 ends.

[0041] In S607, the CPU 101 executes processing according to the current operation mode, after which the process returns to S601. For example, when the current operation mode is the playback mode, the CPU 101 displays a list of images recorded in the secondary storage unit 107 on the display screen of the display unit 105, and displays an image selected from the list of images.

[0042] Fig. 9 shows a process flow of the capture assist mode according to this embodiment. The capture assist mode process is executed in S605 of Fig. 8. During the capture assist mode process, the image capture unit 209 captures images, and a group of images captured by the image capture unit 209 from the current time until a predetermined time ago are stored in the secondary storage unit 107 as video data being captured. The video data being captured is updated successively. It is also assumed that a live view image is being displayed on the display unit 105. In S701, the object recognition unit 202 analyzes video data being captured, and acquires information such as the type of object appearing in the video data and whether the object is a dynamic object or a static object.

[0043] In S702, the match determination unit 203 compares the subject information acquired in S701 with the subject information associated with the camerawork information stored in the camerawork information storage unit 201, and determines whether there is camerawork information that matches the information acquired in S701. If the match determination unit 203 determines that there is matching camerawork information (S702; yes), it determines the matching camerawork information as the recommended camerawork and proceeds to S703. On the other hand, if it is determined that there is no matching camerawork information (S702; no), the process proceeds to S704.

[0044] In S703, the camerawork notification unit 204 performs notification based on the determined recommended camerawork. The camerawork notification unit 204 displays a recommended camerawork notification screen 300 (FIG. 5) on the display unit 105, for example. In S704, the camerawork notification unit 204 performs notification based on camerawork information of predetermined camerawork (camerawork that corresponds to "other than the above" in FIG. 3). In S705, when the user operates the input unit 104 to specify a preferred camerawork from among the notified cameraworks, the camerawork selection unit 205 selects the specified camerawork. The camerawork selection unit 205 acquires camerawork information about the selected camerawork from the camerawork information storage unit 201 and notifies the camerawork setting unit 206.

[0045] In S706, the camerawork setting unit 206 starts controlling the operation of the camerawork based on the camerawork information about the camerawork selected in S705, thereby starting video shooting according to the selected camerawork. In S707, the camerawork setting unit 206 judges whether the camerawork selected in S705 is a camerawork that requires the user to move or is a camerawork that can be handled by the camera's operational control. In this embodiment, if the camerawork is a camerawork that can be handled by the control of the camera control unit 108 and the lens control unit 109, it is judged to be a camerawork that can be handled by the camera's operational control. Examples of camerawork that can be handled by the camera's operational control include panning, tilting, zooming in, zooming out, and focus shifting. Note that the camerawork operation during the action shooting may be realized by the user's manual operation. In other words, the camerawork that is the target of the assist during shooting is not limited to the camerawork that requires the user to move. In the assist during shooting for such camerawork, a guide that instructs the user to operate the camera (for example, zoom operation) is displayed on the live view image.

[0046] If the camerawork setting unit 206 determines that the camerawork selected in S705 is compatible with camera operation control (S707; camera operation), the process proceeds to S708. On the other hand, if the camerawork setting unit 206 determines that the camerawork requires the user to move (S707; user movement), the process proceeds to S711. In S708, the camera instruction unit 207 controls the camera control unit 108 and the lens control unit 109 in accordance with the camera work information about the camera work selected in S705 to perform operational control of the camera body and the lens, such as panning, tilting, zooming, and focusing. Simultaneously with the start of operational control of the camera work or upon receiving an instruction from the user to start recording, the image recording unit 210 starts recording the video data captured by the image capturing unit 209 as a video file.

[0047] In S709, the camera instruction unit 207 detects from the camera control unit 108 and the lens control unit 109 that the operations of the camera body and the lens have ended. In S710, the image recording unit 210 ends recording of the moving image data. Then, the image shooting unit 209 ends shooting. After that, the process of the shooting assist mode ends. In step S711, the CPU 101 executes the process for assisting during shooting. After that, when the process for assisting during shooting ends, the process for the shooting assist mode ends.

[0048] Fig. 10 shows a process flow of the assist during shooting according to this embodiment. The assist during shooting process is executed in S711 of Fig. 9. Also, at the same time as the assist during shooting process starts, or when an instruction to start recording is received from the user (pressing the recording start key 505), the image recording unit 210 starts recording the video data captured by the image capturing unit 209 as a video file. In S800, the user instruction unit 208 displays a guide for instructing the user's movement superimposed on the live view image based on the camera work information about the camera work selected in S705. The user instruction unit 208 displays, for example, the display screen of FIG. 7 on the display unit 105. In S801, the image capturing unit 209 determines whether or not the camerawork motion control is continuing. If the image capturing unit 209 determines that the camerawork motion control is continuing (S801; yes), the process proceeds to S802, and if the image capturing unit 209 determines that the camerawork motion control has ended (S801; no), the process proceeds to S808. For example, when the subject goes out of the frame from the video data being captured, the camerawork setting unit 206 instructs the user instruction unit 208 to end the camerawork motion control.

[0049] In S802, the subject recognition unit 202 analyzes video data being captured, and calculates the size, position, movement information (movement speed, movement direction), and the like, of the subject captured in the video data. In S803, the user instruction unit 208 judges whether the camerawork operation selected in S705 is performed appropriately based on the information of the subject calculated in S802. If the user instruction unit 208 judges that the camerawork operation is not performed appropriately (S803; no), the process proceeds to S804, and if the user instruction unit 208 judges that the camerawork operation is performed appropriately (S803; yes), the process returns to S800. When returning to S800, the user instruction unit 208 may update the display of the guide based on the camerawork information about the camerawork selected in S705 and the size, position, movement information, etc. of the subject calculated in the previous S802. Specifically, the user instruction unit 208 may change the size or position of the guide frame or the direction of the arrow. This allows the user to be instructed to move so that the camerawork operation is performed appropriately in accordance with the current position and movement of the subject during video shooting.

[0050] For example, suppose that the display screen shown in Fig. 7 above is displayed during the processing of the in-shooting assist. According to the guide instructions shown in Fig. 7, the user needs to shoot while moving so that main subject 501 in live view image 500 fits at an appropriate size within guide frame 502. In other words, the movement of the user following the guide instructions during video shooting results in appropriate camerawork operations. Fig. 11 is an example of a display screen that is displayed some time after the display screen of Fig. 7 is displayed. A main subject 901 in a live view image 900 in Fig. 11 has shifted from a guide frame 902 to the left of the screen, and is also too small relative to the size of the guide frame 902. In this way, the subject may extend beyond the guide frame in the left / right direction of the screen, or may become too small (or too large). This indicates that the user is not moving in accordance with the instructions of the guide.

[0051] In S804, the camera instruction unit 207 calculates the amount of camera correction required to realize appropriate camerawork operation based on the information on the subject calculated in S802 and the camerawork information on the camerawork selected in S705. Specifically, the camera instruction unit 207 calculates the amount of correction for the camera position and orientation and the magnification ratio, taking into account the focal length of the lens, the angle of view, the amount of deviation of the subject from the desired position, and the like.

[0052] 11, with regard to the left-right shift of the subject, the camera instruction unit 207 calculates a correction amount indicating how much (how many degrees) the camera should be tilted left-right to correct the shift, based on the lens angle of view (angle) and the ratio of the shift amount to the screen. Also, with regard to the subject becoming too small, the camera instruction unit 207 can correct the size of the subject by zooming in, and calculates this correction amount from the current size of the subject and the size at the start of the assist during shooting, and the amount of change in the angle of view due to zooming (magnification rate).

[0053] In S805, the camera instruction unit 207 compares the amount of correction calculated in S808 with a preset threshold value to determine whether or not the correction can be handled by operational control of the camera body or lens. If the camera instruction unit 207 determines that the amount of correction is smaller than the threshold value and that the correction can be handled by operational control of the camera body or lens (S805; yes), the process proceeds to S806. On the other hand, if the amount of correction is equal to or greater than the threshold value and that the correction cannot be handled by operational control of the camera body or lens (S805; no), the process proceeds to S807.

[0054] In the above example where the subject becomes too small, this can be corrected by zooming in, but zooming causes not only the size of the subject but also the range of the background reflected. If this change is too large, the video as a whole will not be smooth, so there is an allowable range for correcting the size of the subject by zooming. Also, there is a limit to the magnification ratio. Therefore, a threshold may be set in advance for the amount of correction that can be handled by controlling the operation of the camera body or lens. The threshold may be fixed or may be changeable as appropriate by the user. Also, when the threshold is 0 (when no threshold is set), the camera instruction unit 207 does not control the operation of the camera body or lens.

[0055] In S806, the camera instruction unit 207 performs operation control such as driving the camera body left and right or zooming in / out based on the correction amount calculated in S804, using the camera control unit 108 or the lens control unit 109. Thereafter, the process proceeds to S800.

[0056] In S807, the camera instruction unit 207 controls the camera control unit 108 or the lens control unit 109 to drive the camera body left and right or zoom in and out based on the correction amount calculated in S804. Simultaneously with this control, the user instruction unit 208 displays correction information on the live view image to instruct the user to correct the movement. Note that although both the camera body and the lens are controlled in S806 and S807, a configuration in which only one of the camera body and the lens is controlled may be used.

[0057] FIG. 12 is an example of a display screen on which a guide instructing the user to correct the movement is displayed in response to the display screen of FIG. 11 being displayed. A live view image 1000, a main subject 1001, and a guide frame 1002 in FIG. 12 correspond to the live view image 900, the main subject 901, and the guide frame 902 in FIG. 11. The user instruction unit 208 displays correction text 1003 on the live view image 1000. The correction text 1003 displays a comment instructing the user to correct the movement. For example, in the case of a correction instruction to "follow the subject faster," the user instruction unit 208 displays the arrow 1004 in an emphasized manner by blinking, changing the color, changing the size, or the like. Also, in the case of a correction instruction to "fit the subject within the frame," the user instruction unit 208 displays the guide frame 1002 in an emphasized manner by blinking, changing the color, or the like in the same manner as the arrow 1004. The correction text 1003, the guide frame 1002, and the highlighted arrow 1004 are examples of correction information that instruct the user to correct his or her movement.

[0058] In S808, the image capture unit 209 ends the video capture. After that, the process of assist during capture ends. The processes from S800 to S807 are continuously repeated during video capture. In this way, the camera instruction unit 207 displays a guide to instruct the movement of the moving person to be corrected in response to detecting that the movement of the moving person is not in accordance with the camerawork during video capture. In addition to displaying a guide to instruct the movement of the moving person to be corrected, the camera instruction unit 207 also controls the operation of the camera body and the lens so as to correct deviations from the appropriate operation of the camerawork.

[0059] According to the present embodiment as described above, it is possible to support a user by displaying a guide for instructing the user's movement and camera operation so that the user can shoot a video in accordance with the camera work that he or she intends. It is also possible to present to the user camera work suitable for shooting a video of a subject being shot.

[0060] <Embodiment 2> In the first embodiment, a method for determining a recommended camera work according to information on a subject is described. In the present embodiment, a method for determining a recommended camera work by taking into account the scene being shot in addition to information on a subject is described. In the following, differences from the first embodiment are mainly described, and explanations of similar points are omitted.

[0061] Fig. 13 shows the functional configuration of digital camera 100 according to this embodiment. The functional configuration shown in Fig. 13 differs from the functional configuration in Fig. 2 in that a scene recognition unit 1303 is provided between object recognition unit 1302 (corresponding to object recognition unit 202) and match determination unit 1304 (match determination unit 203). In addition, the data configuration held by camerawork information holding unit 1301 differs from that of embodiment 1.

[0062] FIG. 14 shows the data configuration stored in the camerawork information storage unit 1301 according to this embodiment. As shown in FIG. 14, the camerawork information storage unit 1301 stores a data table showing the association of camerawork with respect to the type of subject for each of a plurality of scenes. For example, if the subject is a "person (whole body)" and the scene is a "blue sky", the match determination unit 1304 determines circle, tilt, zoom out, and tilt follow as recommended cameraworks from among a plurality of cameraworks. Note that the camerawork notification unit 1305 (corresponding to the camerawork notification unit 204) may notify only camerawork information that the user has set in advance to be notified as recommended camerawork information when there are a plurality of recommended cameraworks.

[0063] The scene recognition unit 1303 analyzes the video data being shot and recognizes the scene being shot. Specifically, the scene recognition unit 1303 recognizes the type of scene using information such as color distribution and exposure distribution in the image obtained from the video data, and information on their changes over time. Types of scenes recognized by the scene recognition unit 1303 include bright, dark, blue sky, evening scene, spotlight, etc.

[0064] The match determination unit 1304 compares the subject information acquired by the subject recognition unit 1302 and the type of scene recognized by the scene recognition unit 1303 with information associated with the camerawork information stored in the camerawork information storage unit 201. Then, as a result of the comparison, it is determined whether or not there is camerawork information that matches these.

[0065] FIG. 15 shows a process flow of the shooting assist mode according to this embodiment. S1401 and S1403 to S1412 in the process flow shown in Fig. 15 are similar to S701 to S711 in the process flow in Fig. 9. That is, the process flow of the capture assist mode in this embodiment is the process flow of the capture assist mode in the first embodiment with the addition of a scene recognition step (S1402).

[0066] In S1402, the scene recognition unit 1303 analyzes video data being shot and recognizes the scene. In S1403, the match determination unit 203 determines a recommended camera work based on the subject information and the scene recognition result acquired in S1401 and S1402. The following processing steps are the same as those in the first embodiment, except for the process of assist during shooting (S1412).

[0067] Fig. 16 illustrates a process flow of the assist during shooting according to this embodiment. The process flow of the assist during shooting is a process executed in S1412 of Fig. 15. S1500 to S1501 and S1503 to S1509 of the process flow shown in Fig. 16 are similar to S800 to S808 of the process flow of Fig. 10. In other words, the process flow of the assist during shooting in this embodiment is the process flow of the assist during shooting in the first embodiment with the addition of a camera shake correction step (S1502).

[0068] In S1502, the camera instruction unit 207 receives signals from the acceleration sensor and angular velocity sensor included in the camera control unit 108, and changes the position and orientation of the camera body so as to offset the change in the position and orientation of the camera body, thereby correcting the position and orientation of the camera body. This makes it possible to more actively correct the effects of the user's camera shake and small shakes caused by the user's movement on the captured video when the amount of camera correction is calculated in the subsequent step S1505.

[0069] In the present embodiment as described above, it is possible to recommend camera work appropriate to the scene, and further, it is possible to appropriately correct camera shake during movie shooting.

[0070] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0071] The disclosure of each of the above-described embodiments includes the following configurations, methods, and programs. (Configuration 1) An imaging device having an imaging means, A live view image captured by the imaging means is displayed on a display unit, a control means for controlling, when a camerawork to be used for shooting a video is selected from among a plurality of cameraworks representing a method of shooting a video, to display on the display unit a guide instructing a user on movements in accordance with the selected camerawork based on attribute information of the selected camerawork; An imaging device comprising: (Configuration 2) an acquisition means for acquiring information of a subject from an image captured by the imaging means during shooting of a moving image; and The imaging device described in configuration 1, characterized in that the control means updates the display of the guide based on attribute information of the selected camerawork and information of the subject acquired by the acquisition means. (Configuration 3) an acquisition means for acquiring information of a subject from an image captured by the imaging means during shooting of a moving image; and The imaging device described in configuration 1 or 2, characterized in that the control means controls the display unit to display correction information instructing a user to correct the movement when a deviation from the movement in line with the selected camerawork is detected based on attribute information of the selected camerawork and information about the subject acquired by the acquisition means. (Configuration 4) an acquisition means for acquiring information of a subject from an image captured by the imaging means during shooting of a moving image; and The imaging device described in any one of configurations 1 to 3, characterized in that the control means controls the operation of the device body or the lens when it detects a deviation from the movement in accordance with the selected camerawork based on attribute information of the selected camerawork and information of the subject acquired by the acquisition means. (Configuration 5) a determination means for determining a recommended camerawork from among a plurality of cameraworks representing a method of shooting a video, based on information of a subject recognized from an image captured by the imaging means during shooting of the video; A notification means for notifying the camera work determined by the determination means; 5. The imaging device according to any one of configurations 1 to 4, further comprising: (Configuration 6) The imaging device according to configuration 5, wherein camera work to be used for shooting a video is selected from the camera works notified by the notifying means. (Configuration 7) The imaging device described in configuration 5, characterized in that the determination means determines a recommended camera work from among the multiple camera works based on a scene recognized from an image captured by the imaging means during video shooting. (Configuration 8) 8. The imaging device according to any one of configurations 1 to 7, wherein the attribute information includes a composition and a moving direction and a moving speed of the imaging device during shooting of a moving image. (Configuration 9) an acquisition means for acquiring information of a subject from an image captured by the imaging means during shooting of a moving image; and The imaging device described in configuration 1 is characterized in that when the control means detects a deviation from the movement along the selected camerawork based on attribute information of the selected camerawork and information of the subject acquired by the acquisition means, the control means calculates a correction amount for the device body or lens to shoot a video along the selected camerawork based on attribute information of the selected camerawork and information of the subject acquired by the acquisition means. (Configuration 10) 10. The imaging device according to configuration 9, wherein the control means changes the position and orientation of the device body so as to correct the position of the subject in the live view image based on the correction amount. (Configuration 11) 10. The imaging device according to configuration 9, wherein the control means changes a magnification ratio of a lens so as to correct a size of a subject in the live view image based on the amount of correction. (Configuration 12) 10. The imaging device according to configuration 9, wherein the control means controls the display unit to display correction information instructing a user to correct a movement based on the correction amount. (Configuration 13) The imaging device described in any one of configurations 1 to 12, characterized in that the guide is information for instructing a user on at least one of the position and size of the subject in the live view image, and the direction and speed of movement of the device body. (Configuration 14) 14. The imaging device according to any one of configurations 1 to 13, wherein the camerawork is a shooting method in which a user shoots a moving image while moving around while holding a camera body. (method) A method for controlling an imaging device having an imaging means, comprising: A live view image captured by the imaging means is displayed on a display unit, a control step of controlling the display unit to display a guide instructing a user on movements in accordance with the selected camerawork based on attribute information of the selected camerawork when the camerawork to be used for shooting the video is selected from among a plurality of cameraworks representing methods for shooting the video; 13. A method for controlling an imaging apparatus comprising: (program) A program for operating a computer of an imaging device having an imaging means, A live view image captured by the imaging means is displayed on a display unit, a control means for controlling, when a camerawork to be used for shooting a video is selected from among a plurality of cameraworks representing a method of shooting a video, to display on the display unit a guide instructing a user on movements in accordance with the selected camerawork based on attribute information of the selected camerawork; A program comprising:

Claims

1. An imaging device having an imaging means, The live view image captured by the aforementioned imaging means is displayed on the display unit. A control means controls the display unit to display a guide that instructs the user to move in accordance with the selected camera work, based on the attribute information of the selected camera work, when a camera work to be used for shooting a video is selected from among multiple camera work methods representing video shooting methods. An imaging device characterized by having the following features.

2. An acquisition means for acquiring subject information from images captured by the aforementioned imaging means during video recording. It further possesses, The imaging apparatus according to claim 1, characterized in that the control means updates the display of the guide based on the attribute information of the selected camera work and the subject information acquired by the acquisition means.

3. An acquisition means for acquiring subject information from images captured by the aforementioned imaging means during video recording. It further possesses, The imaging apparatus according to claim 1, characterized in that the control means controls the display unit to display correction information instructing the user to correct the movement when it detects a deviation from the movement along the selected camera work based on the attribute information of the selected camera work and the subject information acquired by the acquisition means.

4. An acquisition means for acquiring subject information from images captured by the aforementioned imaging means during video recording. It further possesses, The imaging apparatus according to claim 1, characterized in that the control means controls the operation of the apparatus body or lens when it detects a deviation from the movement along the selected camera work based on the attribute information of the selected camera work and the subject information acquired by the acquisition means.

5. A decision means that determines a recommended camera work from among multiple camera works representing video recording methods, based on information about the subject recognized from the image captured by the imaging means during video recording, A notification means for notifying the camera work determined by the aforementioned determination means, The imaging apparatus according to claim 1, further comprising the following:

6. The imaging device according to claim 5, characterized in that a camera work to be used for shooting a video is selected from among the camera works notified by the notification means.

7. The imaging device according to claim 5, characterized in that the determination means determines a recommended camera work from among the plurality of camera works based on a scene recognized from an image captured by the imaging means during video recording.

8. The imaging device according to claim 1, characterized in that the attribute information includes the composition, the direction of movement and the speed of movement of the imaging device during video recording.

9. An acquisition means for acquiring subject information from images captured by the aforementioned imaging means during video recording. It further possesses, The imaging apparatus according to claim 1, characterized in that the control means calculates a correction amount for the apparatus body or lens for shooting a video along the selected camera work when it detects a deviation from the movement along the selected camera work based on the attribute information of the selected camera work and the subject information acquired by the acquisition means.

10. The imaging apparatus according to claim 9, characterized in that the control means controls the position and orientation of the apparatus body to correct the position of the subject in the live view image based on the correction amount.

11. The imaging apparatus according to claim 9, characterized in that the control means controls the magnification of the lens to change in order to correct the size of the subject in the live view image based on the correction amount.

12. The imaging apparatus according to claim 9, characterized in that the control means controls the display unit to display correction information that instructs the user to correct the movement based on the correction amount.

13. The imaging device according to claim 1, characterized in that the guide is information for instructing the user on at least one of the following: the position and size of the subject in the live view image, and the direction and speed of movement of the device body.

14. The imaging device according to claim 1, characterized in that the camera work is a shooting method in which the user moves while holding the camera body to shoot video.

15. A control method for an imaging device having an imaging means, The live view image captured by the aforementioned imaging means is displayed on the display unit. A control process that, when a camera work to be used for shooting a video is selected from among multiple camera works representing video shooting methods, controls the display unit to display a guide that instructs the user to move in accordance with the selected camera work, based on the attribute information of the selected camera work. A control method for an imaging device, characterized by having the following features.

16. A program for operating the computer of an imaging device having an imaging means, The live view image captured by the aforementioned imaging means is displayed on the display unit. A control means controls the display unit to display a guide that instructs the user to move in accordance with the selected camera work, based on the attribute information of the selected camera work, when a camera work to be used for shooting a video is selected from among multiple camera work methods representing video shooting methods. A program characterized by having the following features.

Images