Imaging device, imaging method, and program

Abstract

To provide an imaging device, an imaging method, and a program.SOLUTION: In an imaging device, a control unit has a first video mode in which a video file of a video captured by an imaging unit is generated and still images can be extracted from the video file, and when a direction detection unit determines occurrence of a direction change of an imaging device with respect to a gravity direction in the first video mode, the control unit divides the video to generate a first video and a second video. The control unit stores a first video file of the first video and a second video file of the second video in a storage unit. The first video is earlier in time than the second video. The direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with a horizontal direction, a second state in which the imaging device is aligned with the gravity direction, or a third state in which the imaging device is changing between the first state and the second state, and the first video includes a video during a period in which the imaging device is in the third state.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an imaging device, an imaging method, and a program for capturing moving images, and more particularly to an imaging device, an imaging method, and a program capable of extracting frames constituting a moving image as still images. [Background technology]

[0002] There is a known technique for shooting a video in order to extract a still image after shooting. In this technique, a video is shot and recorded in order to capture a momentary event that may occur at any time, and the user selects a desired frame after shooting. However, it is difficult to predict when an event will occur during video shooting. Therefore, there are problems in that it is necessary to continue shooting the video for a certain period of time, which may cause the remaining capacity of the recording medium to be low, or that it may take a great deal of effort for the user to select a frame in which an event occurred from a long video.

[0003] In order to address the above-mentioned issues, tagging or marking of videos can be considered. For example, an event is detected, and a tag representing event information is added to a frame when the event occurs. For example, Patent Document 1 describes that a user can select the timing of extracting still images from a video by marking the video at the timing of acquiring a still image at the instruction of the user during video shooting, and extracting and displaying the marked portion and video frames before and after the marking portion after the video shooting ends. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-32303 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when adding the above-mentioned tags, the tags representing the event information are not standardized and can only be recognized by specific devices including the imaging device used for shooting, and in order to play back the event frames, it is necessary to analyze the tag information of all frames, which causes a problem that it takes time to play back the event frames. In addition, since the technology described in Patent Document 1 only records the marking information, it is necessary to decode the video (still image processing) every time a marked frame is extracted and displayed, which places a high processing load on the imaging device. As such, conventional technologies do not allow users to easily select a video frame that they want to extract as a still image.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide an imaging device, an imaging method, and a program that allow a user to easily select a frame of a video that he or she wishes to extract as a still image. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, an imaging device according to a first aspect of the present invention is an imaging device comprising a direction detection unit that detects a direction of the imaging device relative to the direction of gravity, a shooting unit that shoots a video, and a control unit, wherein the control unit has a first video mode in which a video file of the video shot by the shooting unit is generated and still images can be extracted from the video file, and when, in the first video mode, the direction detection unit determines that a change in the direction of the imaging device relative to the direction of gravity has occurred, the control unit divides the video to generate a first video and a second video, and the control unit saves a first video file of the first video in a first folder in a memory unit, and saves a second video file of the second video in a second folder in the memory unit.

[0008] Usually, in still image shooting, a user holds a camera (imaging device) horizontally (with the bottom of the camera parallel to the ground) or vertically (with the bottom of the camera parallel to the direction of gravity) according to a desired composition. In contrast, in video shooting, the vertical and horizontal orientations are rarely switched during shooting. On the other hand, in video shooting for extracting still images, due to the characteristic that frames are extracted after the fact, it is expected that the way the camera is held will change between vertical and horizontal even during video shooting. This change in vertical and horizontal orientation can be considered as a kind of event caused by the user's action. From this perspective, in the first aspect, when a change occurs in the direction of the imaging device (the attitude of the imaging device) with respect to the direction of gravity, the video is divided to generate a first video and a second video, and the first video file of the first video and the second video file of the second video are stored in separate folders (first folder, second folder). This makes it easy to identify the attitude of the imaging device when each video was shot, and the user can easily select a video including a frame that he or she wants to extract as a still image by using the attitude of the imaging device as a clue. Furthermore, since the frame to be extracted as a still image can be found within the divided moving image whose recording time has been shortened, the desired frame can be quickly found.

[0009] In this way, according to the first aspect, the user can easily select a frame of a video that he / she wishes to extract as a still image. Furthermore, since the first aspect does not require the use of information such as tags or markings, frame selection and still image extraction can be performed by other imaging devices, image playback devices, and image processing devices based on the video recorded in the first aspect. The storage unit may be provided in the imaging device or may be an external device.

[0010] In the imaging device according to the second aspect, in the first aspect, the control unit generates a third moving image that is between the first moving image and the second moving image in time, and the third moving image includes a moving image of a first period in which a direction change occurs. In the second aspect, the period in which a direction change occurs (first period) is made into a moving image (third moving image) separate from the first and second moving images, so that the user can easily distinguish it.

[0011] In the imaging device according to the third aspect, in the first aspect, the first video is earlier in time than the second video, the direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with the horizontal direction, a second state in which the imaging device is aligned with the direction of gravity, or a third state in which the imaging device is changing between the first state and the second state, and the first video includes a video of a period in which the imaging device is in the third state. In the third aspect, the third state can be, for example, an intermediate state between the first and second states, a state in which the orientation of the imaging device is uncertain, or a state in which the attitude of the imaging device is changing between the first state and the second state.

[0012] In the imaging device according to the fourth aspect, in the second aspect, the control unit does not store the third moving image in the storage unit. According to the fourth aspect, by not storing the moving image during the period (first period) in which the direction change occurs, it is possible to save the capacity of the storage unit.

[0013] In the imaging device according to the fifth aspect, in the second aspect, the control unit stores the third moving image in a first folder in the storage unit, that is, the third moving image is stored in the same folder as the first moving image.

[0014] In the imaging device according to the sixth aspect, in the fourth or fifth aspect, the third moving image also includes a moving image of the second period after the direction change is completed. In the sixth aspect, since the shooting conditions may not be stable due to a change in composition even after the direction change is completed, the third moving image is included in the second period even after the direction change is completed.

[0015] In the imaging device according to the seventh aspect, in the first or second aspect, the first moving image also includes a moving image of a second period after the direction change is completed. In the sixth aspect, since the shooting conditions may not be stable due to a change in composition even after the direction change is completed, the first moving image is included in the second period after the direction change is completed.

[0016] The imaging device according to the eighth aspect is the sixth or seventh aspect, in which the second period is a period during which the imaging device adjusts the exposure of the moving image, the white balance of the moving image, or the focus of a subject in the moving image. The eighth aspect specifically specifies the content of the second period.

[0017] In the imaging device according to a ninth aspect, in any one of the first to eighth aspects, the control unit imparts information about the orientation of the imaging device relative to the direction of gravity to the first moving image file and the second moving image file. For example, the information can be imparted to the file name or a part of the file name, a header or a footer of the moving image file, etc.

[0018] The imaging device according to a tenth aspect is any one of the first to ninth aspects, wherein the control unit has a reception unit that receives a user's instruction, and the timing to end the second moving image corresponds to reception by the reception unit of a user's instruction to end the second moving image or an instruction to end moving image shooting. According to the tenth aspect, the user can end the second moving image at a desired timing.

[0019] An imaging device according to an eleventh aspect is any one of the first to tenth aspects, and includes a reception unit for receiving a user's instruction or an event detection unit for detecting a specific event in the control unit, and the control unit does not divide the video if the direction detection unit detects a change in direction while the reception unit is receiving the user's instruction or while the event detection unit is detecting the specific event. According to the eleventh aspect, the video is not divided if there is a change in direction (posture change) of the imaging device during the user's instruction or the occurrence of the specific event, so that a series of scenes related to the user's instruction or the event can be confirmed in a single video.

[0020] The imaging device according to a twelfth aspect is any one of the first to eleventh aspects, in which the control unit has a second moving image mode with shooting conditions different from the first moving image mode, and the control unit does not divide the moving image when the direction detection unit detects a direction change in the second moving image mode. For example, the second moving image mode can be a normal moving image mode in which still images are not extracted from the moving image.

[0021] The imaging device according to the thirteenth aspect is the same as the twelfth aspect, in which at least one of the shutter speed, autofocus speed, autoexposure tracking speed, and white balance tracking speed is set faster in the first moving image mode than in the second moving image mode, and / or the frame rate is set higher than in the second moving image mode. The thirteenth aspect specifies one aspect of the difference in shooting conditions between the first moving image mode and the second moving image mode, and by setting the shooting conditions of the first moving image mode in this way, it is possible to extract high-quality still images from the moving image.

[0022] In an imaging device according to a fourteenth aspect, in any one of the first to thirteenth aspects, the first moving image file and the second moving image file have at least a part in common, excluding the filename extension, which allows a user to easily identify that the first moving image file and the second moving image file are files generated by dividing a single moving image.

[0023] In order to achieve the above-mentioned object, the imaging method according to the 15th aspect of the present invention is an imaging method for an imaging device including a direction detection unit for detecting a direction of the imaging device relative to the direction of gravity, a shooting unit for shooting a video, and a control unit having a first video mode capable of extracting a still image from a file of the video shot by the shooting unit, the imaging method including a determination step in the first video mode in which the direction detection unit determines whether or not a change in the direction of gravity of the imaging device has occurred, a video generation step in which the control unit divides the video to generate a first video and a second video when it is determined that a change in the direction has occurred in the first video mode, and a video storage step in which the first video file of the first video is stored in a first folder of the storage unit and the second video file of the second video is stored in a second folder of the storage unit. According to the imaging method according to the 15th aspect, the user can easily select a desired frame from the video shot in the same manner as in the first aspect. Note that the 15th aspect may further include the same configuration as the second to 14th aspects.

[0024] In order to achieve the above object, a program according to a 16th aspect of the present invention is a program for operating an imaging device including a direction detection unit for detecting a direction of the imaging device relative to the direction of gravity, a shooting unit for shooting a video, and a control unit having a first video mode capable of extracting a still image from a file of the video shot by the shooting unit, the program being configured such that in the first video mode, the direction detection unit determines whether or not a change in the direction of the imaging device relative to the direction of gravity has occurred, and if it is determined that a change in the direction has occurred in the first video mode, the control unit divides the video to generate a first video and a second video, stores a first video file of the first video in a first folder in the storage unit, and stores a second video file of the second video in a second folder in the storage unit. According to the program according to the 16th aspect, as in the first and 15th aspects, a user can easily select a desired frame from a shot video. The 16th aspect may further include the same configuration as the second to 14th aspects. Furthermore, the programs of these aspects may be recorded on a non-transitory recording medium on which computer-readable codes are recorded. Effect of the Invention

[0025] As described above, according to the imaging device, imaging method, and program of the present invention, a user can easily select a desired frame from a captured video. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing the configuration of a camera according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a functional configuration of the image processing device. [Diagram 3] FIG. 3 is a diagram showing the coordinate system of the camera. [Figure 4] FIG. 4 is a diagram showing an example of posture determination and video division. [Diagram 5] FIG. 5 is a flowchart showing the process of the imaging method. [Figure 6] FIG. 6 is a diagram showing changes in camera posture and division and storage of a video. [Figure 7] FIG. 7 is a diagram showing an example of a folder structure. [Figure 8] FIG. 8 is a diagram showing another example of a folder structure. [Figure 9] FIG. 9 is a diagram showing an example of icon display for vertical moving images and horizontal moving images. [Figure 10] FIG. 10 is a diagram showing a state in which thumbnail images are added to vertical and horizontal videos. [Figure 11] FIG. 11 is another flowchart showing the process of the imaging method. [Figure 12] FIG. 12 is a diagram showing another example of posture determination and video division. [Figure 13] FIG. 13 is a diagram showing still another example of posture determination and video division. [Figure 14] FIG. 14 is a diagram showing still another example of posture determination and video division. [Figure 15] FIG. 15 is a diagram showing still another example of posture determination and video division. [Figure 16] FIG. 16 is yet another flowchart showing the process of the imaging method. [Figure 17] FIG. 17 is yet another flowchart showing the process of the imaging method. [Figure 18] FIG. 18 is a flowchart showing the process of recording moving images in the first and second modes. [Figure 19] FIG. 19 is a flowchart (continuation of FIG. 18) showing the moving image recording process in the first and second modes. [Figure 20] FIG. 20 is a diagram showing how a frame to be extracted as a still image is selected. [Figure 21] FIG. 21 is an external view of the smartphone according to the second embodiment. [Figure 22] FIG. 22 is a block diagram showing a configuration of a smartphone according to the second embodiment. [Diagram 23] FIG. 23 is a diagram showing the orientation of the smartphone according to the second embodiment. [Figure 24]FIG. 24 is a diagram showing how the attitude of a smartphone is determined based on an angle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of an imaging device, an imaging method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

[0028] <First embodiment> <Overall configuration of the imaging device> 1 is a diagram showing the configuration of a camera 10 (imaging device) according to the first embodiment. The camera 10 is made up of an interchangeable lens 100 (shooting section, imaging device) and an imaging device body 200 (imaging device), and a subject image (optical image) is formed on an imaging element 210 by a photographing lens including a zoom lens 110 (described later). The interchangeable lens 100 and the imaging device body 200 can be attached and detached via a mount (not shown).

[0029] <Interchangeable lens configuration> The interchangeable lens 100 includes a zoom lens 110, a focus lens 120, an aperture 130, and a lens driver 140. The lens driver 140 drives the zoom lens 110 and the focus lens 120 forward and backward in response to a command from an image processing device 240 (lens driver control unit 240F in FIG. 2) to perform zoom (optical zoom) adjustment and focus adjustment. The zoom adjustment and focus adjustment may be performed in response to a command from the image processing device 240, or in response to a zoom operation or focus operation (rotation of a zoom ring or focus ring, not shown, etc.) performed by a user. The lens driver 140 controls the aperture 130 in response to a command from the image processing device 240 to adjust the exposure. Meanwhile, information such as the positions of the zoom lens 110 and the focus lens 120 and the aperture of the aperture 130 is input to the image processing device 240. The interchangeable lens 100 has an optical axis L.

[0030] <Configuration of the imaging device body> The imaging device body 200 includes an imaging element 210 (imaging section), an AFE 220 (AFE: Analog Front End, imaging section), an A / D converter 230 (A / D: Analog to Digital, imaging section), an image processing device 240, an operation section 250, a storage section 260, a monitor 270, and an attitude sensor 280 (direction detection section). The imaging device body 200 may include a shutter (not shown) for blocking light transmitted through the imaging element 210. The imaging element 210 includes a light receiving surface on which a large number of light receiving elements are arranged in a matrix. Then, subject light transmitted through the zoom lens 110, the focus lens 120, and the aperture 130 is imaged on the light receiving surface of the imaging element 210 and converted into an electrical signal by each light receiving element. R (red), G (green), or B (blue) color filters are provided on the light receiving surface of the imaging element 210, and a color image of the subject can be obtained based on the signals of each color. Various photoelectric conversion elements such as a CMOS (Complementary Metal-Oxide Semiconductor) and a CCD (Charge-Coupled Device) can be used as the image sensor 210. The AFE 220 performs noise removal and amplification of the analog image signal output from the image sensor 210, and the A / D converter 230 converts the captured analog image signal into a digital image signal with a gradation range.

[0031] <Configuration of image processing device> 2 is a diagram showing the functional configuration of the image processing device 240. The image processing device 240 includes an image acquisition unit 240A, a direction detection unit 240B (direction detection unit), a control unit 240C (control unit), an event detection unit 240D (event detection unit), a still image extraction unit 240E (still image extraction unit), and a lens drive control unit 240F (lens drive control unit). The control unit 240C has a reception unit that receives user instructions, and this reception unit can receive user instructions via the operation unit 250. The image processing device 240 performs processes such as shooting a moving image and generating a file, generating a still image file, processing a plurality of frames that constitute a moving image, and extracting a still image, based on a digital image signal input from the A / D converter 230. Details of the processes performed by the image processing device 240 will be described later.

[0032] The functions of the image processing device 240 can be realized by using various processors. The various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions. The above-mentioned various processors also include a GPU (Graphics Processing Unit), which is a processor specialized for image processing, and a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture. Furthermore, the above-mentioned various processors also include dedicated electric circuits, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for executing specific processing.

[0033] When the above-mentioned processor or electric circuit executes software (program), the processor (computer) readable code of the software to be executed is stored in a non-transitory recording medium such as a ROM (Read Only Memory). Then, the processor refers to the software. The software stored in the non-transitory recording medium includes a program for executing the imaging method according to the present invention (a program for operating the imaging device according to the present invention). The code may be recorded in a non-transitory recording medium such as various magneto-optical recording devices and semiconductor memories instead of a ROM. When processing using the software, for example, a RAM (Random Access Memory) is used as a temporary storage area, and data stored in, for example, an EEPROM (Electronically Erasable and Programmable Read Only Memory) not shown can also be referenced.

[0034] In addition to the above-mentioned components, the image processing device 240 includes a ROM 242. The ROM 242 stores computer-readable code of programs required for capturing, recording, displaying, and the like of images (including a program for executing the imaging method according to the present invention).

[0035] <Operation section> The operation unit 250 has a release button, operation buttons (e.g., a cross button, a Quick button, an OK button, etc.), dials, switches, etc. (not shown), and allows the user to perform various operations such as setting the shooting mode, issuing a video shooting instruction, issuing a still image extraction instruction, etc. The monitor 270 (touch panel type) may be used as the operation unit 250.

[0036] <Storage section> The storage unit 260 (storage unit) is composed of various types of magneto-optical recording media, non-transitory recording media such as semiconductor memory, and their control circuits, and stores moving images, still images, still images extracted from moving images, etc. The recording media may be of a type that is detachable from the imaging device main body 200. The captured images (moving images, still images) may be transmitted to an external recording medium (storage unit) by, for example, wireless communication and stored therein.

[0037] <Monitor and viewfinder> Monitor 270 (display device) is configured with a touch panel type liquid crystal display panel, and can display moving images, still images, frames for extracting still images, etc. Monitor 270 can be disposed on the back side, top side, etc. of imaging device main body 200. Camera 10 may also be equipped with a finder.

[0038] <Shooting mode> The camera 10 can set the shooting mode to one of a still image shooting mode, a video shooting mode for extracting still images (first video mode), and a normal video shooting mode (second video mode). The still image shooting mode and the normal video shooting mode are the same modes as those of a normal digital camera. On the other hand, the video shooting mode for extracting still images is a mode in which still images can be extracted from a video file, and shoots a video with different shooting conditions from the normal video shooting mode (a video with shooting conditions that prioritize the extraction of still images over the viewing of the video itself). Specifically, in the video shooting mode for extracting still images, at least one of the shutter speed, autofocus speed, autoexposure tracking speed, and white balance tracking speed is set faster than in the normal video shooting mode, and / or the frame rate is set higher than in the normal video shooting mode. In addition, the resolution and frame rate are set to the highest values ​​that can be set in the camera 10 (for example, 4,000×2,000 pixels, 30 frames / second), and the color tone is also set on the premise of still image extraction. The upper limit of the ISO sensitivity is also set higher than in the normal video shooting mode.

[0039] For example, the shutter speed is set to a value corresponding to the frame rate of the video to be recorded in the normal video shooting mode (1 / 30 seconds when the frame rate is 30 frames / second). On the other hand, in the video mode for extracting still images, it is set to a value faster than the frame interval (for example, less than 1 / 30 seconds). In the normal video shooting mode, the shutter speed is set to a value corresponding to the frame rate of the video so that the video is played back smoothly, but in this case, blurring may occur for a moving subject. For this reason, in the video shooting mode for extracting still images, the shutter speed is set faster than in the normal video shooting mode (faster than the frame interval), which makes it possible to extract high-quality still images with less blurring of the subject. Similarly, the shutter speed can be increased by raising the upper limit of the ISO sensitivity (ISO: International Organization for Standardization), which makes it possible to extract still images with less blurring. In addition, the autofocus speed, the autoexposure tracking speed, the autowhite balance tracking speed, etc. are set faster than in the normal video shooting mode. This makes it possible to obtain many frames in which the subject is in focus, frames with appropriate exposure, etc. Regarding the frame rate, by setting a high rate, the frame interval of the video is shortened, and the number of frames that can be extracted as still images increases.

[0040] According to the above-mentioned still image extraction video shooting mode, frames constituting a video can be extracted as still images after the fact. Therefore, the user can easily take photos of events (natural phenomena, accidents, happenings, etc.) that may occur at any time, photos of subjects whose state changes with the passage of time, and photos of the instantaneous state of a moving subject. At this time, as will be described in detail later, still images can be extracted not only at the timing when recording of a still image is instructed, but also at other timings. Therefore, the user can obtain still images at desired timings. Also, by setting shooting conditions suitable for still image extraction (the above-mentioned shutter speed, resolution, frame rate, etc.), still images of high quality can be extracted. In addition to such after-the-fact extraction of still images, in the still image extraction video shooting mode and normal video shooting mode, still images can be taken (saved) during video shooting.

[0041] <Camera posture> FIG. 3 is a diagram showing an example of a coordinate system defined for the camera 10. In the example shown in FIG. 3, the X axis is the optical axis direction (direction toward the subject), the Z axis is the direction toward the top of the imaging device body 200, and (X axis, Y axis, Z axis) form a right-handed coordinate system. Whether the camera 10 is in a vertical position (the top and bottom surfaces of the camera 10 are parallel to the vertical direction) or a horizontal position (the top and bottom surfaces of the camera 10 are parallel to the ground) can be determined, for example, from the angle around the X axis (0 deg is set when the top and bottom surfaces of the imaging device body 200 are parallel to the ground). FIG. 4 is a diagram showing an example of the relationship between the angle around the X axis and the attitude (vertical or horizontal). The direction detection unit 240B (direction detection unit) can determine the angle as "horizontal" when it is in the range of (-45 deg or more and +45 deg or less) and (+135 deg or more and +225 deg or less), and can determine the angle as "vertical" otherwise. As described above, camera 10 is equipped with attitude sensor 280 (e.g., an acceleration sensor that measures acceleration in three-axis directions; see FIG. 1), so direction detection unit 240B can detect the attitude of camera 10 (direction relative to the direction of gravity) based on the measurement results of attitude sensor 280.

[0042] 4 is an example, and the orientation may be determined within a different range. For example, the direction detection unit 240B may determine the orientation as "horizontal" when the angle is within the ranges of (-30deg or more and +30deg or less) and (+150deg or more and +210deg or less), and may determine the orientation as "vertical" in other cases.

[0043] <Processing in video capture mode for extracting still images> An imaging method in the camera 10 configured as described above will be described. Fig. 5 is a flowchart showing processing in a moving image shooting mode for extracting still images. For example, the camera 10 is set to the moving image shooting mode for extracting still images by operating a mode dial (not shown) of the operation unit 250. Then, the control unit 240C sets the shooting unit (interchangeable lens 100, image sensor 210, AFE 220, A / D converter 230) to the first moving image mode. Then, the processing of Fig. 5 is started, and the control unit 240C initializes the state of the camera 10 (acquiring and displaying a live view image, initializing the attitude, starting attitude determination, etc.) (step S100: initialization step).

[0044] <Video recording start command> The control unit 240C determines whether or not a moving image recording start instruction has been issued (step S110: shooting step, moving image generation step). For example, when a release button (not shown) of the operation unit 250 is pressed down, it can be determined that a "moving image recording start instruction has been issued." If the determination is affirmative, the process proceeds to step S120.

[0045] <Posture determination> The direction detection unit 240B detects the orientation (orientation relative to the direction of gravity) of the camera 10 (imaging device) based on the output of the orientation sensor 280 (step S120: orientation detection step, determination step). As described above with reference to Figs. 3 and 4, the direction detection unit 240B determines whether the orientation is "vertical," "horizontal," or "undefined" based on the angle around the X-axis. An "undefined" orientation can include a state in which the angle around the Y-axis is large and "vertical" or "horizontal" cannot be specified, such as when the user holds the camera 10 facing the ground or the sky. Note that the direction detection unit 240B does not need to detect the "undefined" state of the camera 10.

[0046] <Recording in folders according to posture> If the orientation of the camera 10 is determined to be "horizontal" or "undefined" in step S120, the control unit 240C compresses the captured video to create a new video file. Then, the control unit 240C starts recording (saving) the video file in a horizontal video folder (first folder) in the storage unit 260 (step S130: video saving step). The video can be compressed in MPEG format (MPEG2, MPEG4, etc.). On the other hand, if the orientation of the camera 10 is determined to be "vertical", the control unit 240C starts recording (saving) the video file in a vertical video folder (second folder) separate from the horizontal video folder (step S140: video saving step). Audio may also be recorded and saved using a microphone (not shown) provided in the camera 10. Note that "horizontal video" and "vertical video" are videos when the orientation is horizontal or undefined, and when the orientation is vertical, respectively, and the "horizontal video folder" and "vertical video folder" are folders for saving horizontal videos and vertical videos, respectively.

[0047] <Handling cases where the posture is uncertain> In the flowchart of Fig. 5, "if the orientation changes from horizontal to undefined" and "if it changes from undefined to horizontal", recording continues in the same folder and the same file. And, "if the orientation changes from vertical to undefined" and "if it changes from undefined to vertical", it is determined that "the orientation has changed" and the video is divided and recorded in a different folder. However, if the orientation is undefined, it may be recorded in the vertical video folder, or in a folder different from the horizontal video folder and the vertical video folder (see other aspects described below).

[0048] The direction detection unit 240B continues detecting the attitude of the camera 10 in parallel with shooting the video, and judges whether or not a change in the direction of the camera 10 (image capture device) with respect to the direction of gravity has occurred (step S170: judgment step). Whether or not a change in the direction has occurred can be judged based on the output of the attitude sensor 280 (for example, acceleration in three axial directions). If the direction detection unit 240B judges that a change in the direction has occurred, specifically, that the attitude of the camera 10 has changed between "horizontal or indefinite" and "vertical" (Yes in step S170), the control unit 240C closes the video file being recorded and divides the video (step S180: video generation step). Then, the judgment in step S190 is denied, and the process returns to step S120.

[0049] In step S120, the control unit 240C judges whether the orientation of the camera 10 is "vertical", "horizontal", or "undefined" as in the case of the start of recording the moving image file. Then, the control unit 240C saves the new moving image file related to the division in the horizontal moving image folder or the vertical moving image folder of the storage unit 260 according to the result. For example, if the orientation of the camera 10 changes from "horizontal" at the start of recording to "vertical", the control unit 240C saves the moving image file (the first moving image file of the first moving image) in the horizontal moving image folder at the start of recording, and saves the new moving image file (the second moving image file of the second moving image) generated due to the change in orientation in the vertical moving image folder. Note that in step S170, if the orientation of the camera changes again (Yes in step S170), the moving image does not need to be closed. This is because if the moving image is repeatedly divided every time the orientation of the camera changes, the number of moving image files will increase too much.

[0050] The image acquisition unit 240A, the direction detection unit 240B, and the control unit 240C continue to divide the video and save the video files according to the orientation of the camera 10 until the end of video shooting (while the determination in step S190 is No). It is preferable that the direction detection unit 240B and the control unit 240C add information about the orientation of the camera 10 (direction relative to the direction of gravity) to the first video file and the second video file so that the information can be referenced when displaying the icon of the video file or playing the video. The direction detection unit 240B and the control unit 240C can record information about the orientation of the camera 10 in the header or footer. For example, the information can be set to "01" for portrait, "02" for landscape, and "03" for other (diagonal, indefinite, etc.).

[0051] As shown in the example of FIG. 6, the camera 10 may transition from a horizontal state to a vertical state and end video shooting in the vertical state. In this case, the end timing of the second video (vertical video; video in the vertical state) corresponds to the reception of a user's instruction to end the second video or video shooting by the reception unit in the control unit 240C. This reception unit can receive user instructions via the operation unit 250. Similarly, the camera 10 may transition from a vertical state to a horizontal state again and end video shooting in the horizontal state. In this case, the end timing of the first video (horizontal video; video in the horizontal state) corresponds to the reception of a user's instruction to end the first video or video shooting by the reception unit. These video end timings are similar to those in the aspects described below.

[0052] FIG. 6 is a diagram showing an example of determining a change in direction based on an angle around the X-axis. In the example of FIG. 6, the orientation is determined to be "horizontal" between the elapsed time of zero and t1, and "vertical" when the angle exceeds 45 deg (threshold value) at the elapsed time of t1, and the video is divided at the elapsed time of t1. The control unit 240C saves the video file of the first video (first video file) from the elapsed time of zero to t1 in a horizontal video folder (first folder), and saves the video file of the second video (second video file) after the elapsed time t1 in a vertical video folder (second folder). Since the orientation of the camera 10 may fluctuate minutely, it may be determined that "the orientation has changed" when the changed state of the orientation continues for a predetermined time or more. Although FIG. 6 shows an example in which the orientation of the camera 10 changes from horizontal to vertical, the same process can be performed when the orientation changes from vertical to horizontal (in this case, the vertical video becomes the first video, and the horizontal video becomes the second video. Also, the vertical video folder becomes the first folder, and the horizontal video folder becomes the second folder; the same applies to other aspects).

[0053] Control unit 240C closes the video file (step S180) not only when the attitude of camera 10 has changed (Yes in step S170), but also when it determines in step S150 that "an instruction to end video recording (an instruction to end video shooting) has been received" and when it determines in step S160 that "the remaining capacity of the recording media is insufficient." However, in these cases, video shooting is also ended (Yes in step S190). In step S150, control unit 240C can determine that "an instruction to end video recording has been received" when a release button (not shown) of operation unit 250 is pressed, when an end instruction is given via monitor 270, etc.

[0054] In the first embodiment, when it is determined that "the direction (attitude) of the camera 10 (imaging device) relative to the direction of gravity has changed in the still image extraction moving image shooting mode (first moving image mode)," the moving image is divided to generate a horizontal moving image (first moving image) and a vertical moving image (second moving image). Then, the moving image file of the horizontal moving image (first moving image file) is saved in a horizontal moving image folder (first folder), and the moving image file of the vertical moving image (second moving image file) is saved in a vertical moving image folder (second folder).

[0055] This allows the user to easily identify the orientation of the camera 10 when each video was shot, and allows the user to easily select a video containing a frame that the user wishes to extract as a still image by using the orientation of the camera 10 as a clue. Also, the user can quickly find a desired frame because the user only needs to search for a frame to extract as a still image within the video that has been divided and has a shorter recording time. Thus, according to the first embodiment, the user can easily select a desired frame from the shot video.

[0056] <Example of folder structure> FIG. 7 is a diagram showing an example of a folder structure for saving videos. In the example shown in the figure, a folder is generated for each shooting date, and within each folder for each shooting date, a folder is generated for each shooting number. For example, in the flowchart of FIG. 5, the period from when the start of video recording is instructed in step S110 to when video shooting is ended in step S190 can be treated as one shooting number. For each folder for each shooting number, a folder for horizontal videos and a folder for vertical videos are provided, and horizontal videos and vertical videos are saved, respectively. Such a folder structure can be displayed on the monitor 270 by the control unit 240C.

[0057] 8 shows a horizontal video folder (first folder) and a vertical video folder (second folder) displayed as horizontal and vertical icons, respectively, on monitor 270. According to this manner of folder display, the user can easily visually distinguish between the vertical video folder and the vertical video folder, and can easily select a desired frame.

[0058] <Video file icon and file name> FIG. 9 is a diagram showing an example of icon display of a video file. Part (a) of the figure shows a state in which a video file of a horizontal video is displayed with the characters "horizontal video" added to a horizontal icon. The symbol in the icon indicates that the file can be played by specifying the file. Similarly, part (b) of FIG. 9 shows an example of icon display of a video file of a vertical video, in which the characters "vertical video" are added to a vertical icon. In the example of FIG. 9, the file name is composed of the file creation date (October 23, 2018), shooting number (001), vertical / horizontal identification information (horizontal video is H (Horizontal) and vertical video is V (Vertical)), file number (001, 002,...), file size (10MB, etc.), and extension (here, ".mpg"). The video file of the horizontal video (first video file) and the video file of the vertical video (second video file) have some file names in common except for the extension. The vertical / horizontal identification information is an example of information regarding the direction of the camera 10 (imaging device) relative to the gravity direction.

[0059] The file name may include information indicating the type of file, the file creation date and time, and vertical / horizontal identification information. For example, a horizontal video file created on 2018 / 1 / 1 12:30 may be given the file name "IMG201801011230_H.MOV", and a vertical video file created on 2018 / 1 / 1 12:40 may be given the file name "IMG201801011240_H.MOV". In the file name, "IMG" is an example of information indicating that it is an image file. Even in this case, the file name may include information indicating the file size.

[0060] Fig. 10 shows a thumbnail image displayed on an icon. Part (a) of the figure is an example for horizontal video, and part (b) is an example for vertical video. The image to be displayed may be the first frame of the horizontal or vertical video, or a frame in the middle (for example, a frame where an event occurs).

[0061] By displaying the video files as icons, thumbnails, and file names as exemplified above, the user can easily distinguish between vertical and horizontal videos, and can easily select a video containing a desired frame. Note that the icon display, thumbnail display, and file name assignment of video files can be performed by the control unit 240C.

[0062] <Other aspects of video division and recording> Other aspects of dividing and recording (saving) a moving image will be described below. Note that in the flowcharts described below, the same steps as those in FIG. 5 are given the same step numbers, and detailed descriptions thereof will be omitted.

[0063] (Mode 1: A mode in which the period during posture change is made into an independent video file) In the aspect described in Fig. 5, if the attitude of camera 10 (angle around the X-axis) exceeds a threshold value, it is determined that "a change in orientation has occurred." However, in aspect 1 shown in Fig. 11, a video during the orientation change is saved in a different folder as a video file separate from the horizontal video and vertical video. Fig. 11 is a flowchart showing the process in aspect 1, and the orientation detection unit 240B determines whether the orientation of camera 10 (orientation relative to the direction of gravity) is "horizontal," "vertical," or "changing" (step S122: orientation detection step).

[0064] If the posture is "landscape", the control unit 240C starts recording (saving) the video file (first video file) in the horizontal video folder (first folder) as a horizontal video (first video) (step S130). If the posture is "vertical", the control unit 240C starts recording in the vertical video folder (second folder) as a vertical video (second video) (step S140). If the posture is "changing", the control unit 240C starts recording in the changing video folder (third folder) as a changing video (third video) (step S124). In aspect 1, in addition to the horizontal video folder and vertical video folder, a changing video folder can be created in the lower hierarchical level of the folder for each shooting number.

[0065] In the first aspect, when the direction detection unit 240B judges that "the attitude change of the camera 10 has started or completed" (Yes in step S172 (judgment step)), the video is divided (closing the video file, generating a new video file). At this time, since the attitude of the camera 10 may fluctuate minutely due to camera shake or other factors, the direction detection unit 240B judges that "the attitude change has started" when the angle around the X-axis has changed by more than a predetermined value in a certain period of time. Similarly, it is preferable that the direction detection unit 240B judges that "the attitude change has completed" when the time during which the angle fluctuation falls within a predetermined range continues for more than a certain period of time. For example, in the example shown in FIG. 12, the angle changes significantly between elapsed times t1 and t2, and the angle is stable between elapsed times t3 and t4.

[0066] Therefore, the direction detection unit 240B judges that "the posture is changing" from elapsed time t2 to t4, and judges that "the posture change is complete" at elapsed time t4 (taking into consideration the time lag between the actual posture of the camera 10 and the posture determination result change). In response to this judgment, the control unit 240C generates a changing video (third video) that is between the horizontal video (first video; from elapsed time zero to t2) and the vertical video (second video; after elapsed time t4) in time from elapsed time t2 to t4. As shown in FIG. 12, the changing video (third video) includes video of the first period in which the posture of the camera 10 is changing (a direction change is occurring).

[0067] (Mode 2: Continuous recording in the video file of the previous period during posture changes) In the above-mentioned aspect 1, the period during which the posture is changing (first period) is recorded as an independent video (third video). However, in aspect 2, the period during which the posture is changing is continuously recorded in a video file for the period before the change started. Specifically, as shown in FIG. 13, in aspect 2, the direction detection unit 240B detects whether the camera 10 (imaging device) is in the first state (a state in which the bottom surface of the camera 10 is aligned with the horizontal direction; landscape state), the second state (a state in which the bottom surface of the camera 10 is aligned with the direction of gravity; portrait state), or the third state (a state changing between the first state and the second state). Then, the control unit 240C records the video during the period in which the camera 10 is in the third state (posture is changing) in the first folder (folder for landscape video) by including it in the first video.

[0068] (Mode 3: A mode in which a part of the period after the posture change is completed is continuously recorded in a video file of the period during the change) In a situation where the attitude of the camera 10 has changed, the composition on the screen of the monitor 270 may change significantly. In that case, it may be necessary for the camera 10 (imaging device) to adjust the exposure (AE: Auto Exposure) of the video, the white balance (AWB: Auto White Balance) of the video, or the focus (AF: Auto Focus) of the subject in the video. Therefore, in the third aspect, a determined period (second period) after the end of the attitude change is treated in the same way as when the attitude of the camera 10 is changing, even if the attitude of the camera 10 is stable. Specifically, for example, as shown in FIG. 14, the direction detection unit 240B treats the period from elapsed time t2 to t4 (first period) and also the period from elapsed time t4 to t5 (second period) as "the attitude of the camera 10 is changing."

[0069] The first and second periods are recorded (saved) as a third video. That is, the third video also includes the video of the second period after the direction change is completed. The control unit 240C can divide the video and record the third video as a video file (third video file) separate from the horizontal video (first video) and vertical video (second video) in a horizontal video folder (first folder). However, the third video may be recorded in an independent folder for videos during change (third folder). Images during a period of posture change or when the shooting conditions are unstable may be of low importance, but in aspect 3, such periods are saved as a video separate from the horizontal video and vertical video, so that it is easy to select frames to be extracted as still images and organize video files and folders.

[0070] (Mode 4: A mode in which a part of the period after the posture change is completed is continuously recorded in the original video file) In the above-mentioned aspect 3, the period during which the attitude of the camera 10 is changing (first period) and the period determined after the attitude change ends (second period) are recorded as the "third video." However, in aspect 4, as shown in Fig. 15, the first period (from elapsed time t2 to t4) and the second period (from elapsed time t4 to t5) are recorded as the "first video" in the horizontal video folder (first folder).

[0071] (Mode 5: Do not record video during posture changes) In the above-mentioned aspects 1 to 4, video is recorded during the period during which the posture is changing (and the period thereafter until the posture is stabilized), but a mode in which video is not recorded during the posture change can also be adopted. Specifically, as shown in the flowchart of Fig. 16, the direction detection unit 240B detects whether the posture of the camera 10 is horizontal (first state), vertical (second state), or changing (third state) (step S122). Then, if the posture is changing, recording of a video file is not started (return to step S122).

[0072] Furthermore, if the camera 10 starts changing its orientation (Yes in step S174 (determination step)) after starting to record a video when the orientation of the camera 10 is landscape or portrait (steps S130, S140), the control unit 240C closes the video file in step S180 and returns to step S122. Then, the camera 10 resumes recording a video after the orientation of the camera 10 becomes landscape or portrait (steps S130, S140). The closed video file is saved in the corresponding folder. Images during a change in orientation may be of low importance, but in aspect 5, the video is not saved during such a period, so the remaining capacity of the recording medium can be saved. Furthermore, the user can easily select frames to be extracted as still images and organize video files and folders.

[0073] (Mode 6: Video is not split immediately after an event occurs, even if there is a change in posture) Mode 6 shown in the flowchart of FIG. 17 is another mode for the case where the change in attitude of camera 10 has started or completed (Yes in step S172 (judgment step)). In this mode, if a specific event has occurred between the start or completion of the change in attitude of camera 10 and a determined time (described as "T seconds ago" in step S176), control unit 240C does not close the video file (Yes in step S176 and does not proceed to step S180). Therefore, in this case, the video is not divided. The same applies not only to the case where a specific event has occurred up until a certain time ago, but also to the case where a specific event is being detected (the event is ongoing) and the case where the reception unit is receiving a user's instruction (including the case where a user's instruction or a process based on a user's instruction is ongoing).

[0074] The control unit 240C can set the value of T (for example, 30 seconds, but other values ​​are also acceptable) in response to a user's operation via the operation unit 250 or without a user's operation. The event detection unit 240D (see FIG. 2) may automatically detect the amount of movement, changes in size and shape, changes in brightness, and the like of the subject by known image processing to determine whether a specific event has occurred. In addition, the event detection unit 240D may determine whether a specific event has occurred based on a user's instruction via the operation unit 250 or the reception unit (including the case where the monitor 270 is used as a touch panel). For example, the control unit 240C may determine that a "(specific) event has occurred" when the reception unit receives an instruction to shoot a still image while the user is shooting a video. When the event detection unit 240D and the control unit 240C detect an event, it is preferable that the event detection unit 240D and the control unit 240C add information indicating the event detection timing to the video file (for example, record it in the header or footer). By referring to this information, the user can easily select a frame that he or she wants to extract as a still image (described later).

[0075] As described above, even if there is a change in the attitude of the camera 10, by not dividing the video if an event occurred before the attitude change, the user can check a series of events related to the event in one video and can easily select a desired frame from the captured video. Note that although the flowchart in Fig. 17 is based on Fig. 11 (a mode in which the video during a change in attitude is treated as a different video from the horizontal video and vertical video and is saved in a different folder), it may also be based on the flowcharts in Figs. 5 and 15.

[0076] In addition, in camera 10, the manner in which the moving image is to be divided and recorded (saved) may be determined in accordance with a user's operation via operation unit 250, including manners 1 to 6 described above.

[0077] <Processing when considering the first and second modes> Although the processing in the first moving image mode has been described in the flowcharts shown in Figures 5, 11, 16, and 17, processing in consideration of the first moving image mode and the second moving image mode will be described with reference to Figures 18 and 19. Note that the same processing as in Figures 5, 11, 16, and 17 is given the same step number, and detailed description will be omitted.

[0078] The control unit 240C determines whether the camera 10 is in the first moving image mode (moving image mode for extracting still images) or the second moving image mode (normal moving image mode) (step S112). In the first moving image mode, the control unit 240C sets the shooting conditions suitable for the first moving image mode (step S114). As described above in the "shooting mode" section, the shooting conditions set in step S114 are shooting conditions that prioritize the extraction of still images over the viewing of the moving image itself. The process from step S114 onwards in the first moving image mode is the same as that in the flowchart of FIG. 5, and therefore a detailed description will be omitted.

[0079] On the other hand, if the result of the determination in step S112 is the second movie mode, the control unit 240C sets the shooting conditions suitable for the second movie mode (step S191 in FIG. 19). The direction detection unit 240B determines the attitude of the camera 10 (step S192), and the control unit 240C starts recording (saving) the movie file in the second movie folder of the storage unit 260 (step S193: movie saving step). The processes of steps S194 and S195 are the same as the processes of steps S150 and S160 for the first movie mode. If an instruction to end movie recording is received (Yes in step S194) or if the remaining capacity of the recording medium is not sufficient (No in step S195), the control unit 240C closes the movie file (step S197) and returns to step S190 in FIG. 18 (determination of whether movie shooting is ended or not).

[0080] If there is no instruction to end video recording (No in step S194) and the remaining capacity of the recording medium is sufficient (Yes in step S195), control unit 240C continues recording the video. However, in the second video mode, unlike the first video mode, the video is not divided even if direction detection unit 240B does not detect a change in attitude (change in direction) of camera 10 (No in step S196) or detects a change in attitude (change in direction) of camera 10 (Yes in step S196). The reason that the video is not divided in the second video mode even if the attitude of camera 10 changes is because the second video mode is not a mode for shooting a video with the aim of extracting still images, and it is not necessary to divide the video to make it easier to find a specific scene as in the first video mode.

[0081] <Example of folder structure> When the first and second video modes are taken into consideration, the control unit 240C can generate a folder for the first video mode and a folder for the second mode in a folder for each shooting date in the storage unit 260 (storage unit). In this case, the control unit 240C generates a folder for each shooting number in the folder for the first video mode, and provides a folder for horizontal videos (example of the first folder) and a folder for vertical videos (example of the second folder) in the folder for each shooting number, and stores the horizontal videos (example of the first video) and the vertical videos (example of the second video), respectively. On the other hand, for the folder for the second video mode, the control unit 240C records the files of the videos shot on that shooting date in the folder for each shooting date. Also, the control unit 240C may set the folders for the first and second video modes at a higher level and provide a folder for the shooting date at a lower level, rather than providing folders for the first and second video modes at a lower level than the folder for the shooting date. Such a folder configuration can be displayed on the monitor 270 by the control unit 240C, as in the case of FIG. 7.

[0082] <Extracting still images> As will be described in the following example, the frames constituting a moving image can be extracted as still images by the camera 10. Note that still images may also be extracted by another device such as a personal computer using the moving image file recorded by the camera 10.

[0083] (Example 1) The control unit 240C plays and displays on the monitor 270 a file selected by the user from among the horizontal video, vertical video, or changing video (first video, second video, third video) files. Instead of continuous playback, frame-by-frame playback (displaying one frame at a time in response to a user operation) may be used. The control unit 240C stops playback in response to a user operation, and the still image extraction unit 240E extracts the frame as a still image by confirming the selection of the frame displayed at the time of stopping in response to the user operation.

[0084] (Example 2) When information indicating the detection of an event is added to the video file, the control unit 240C may refer to the information and display a plurality of frames including the frame in which the event was detected on the monitor 270 (see FIG. 1). FIG. 20 shows an example of such a display, in which five frames 1051 to 1055 including frame 1052 are displayed. Furthermore, for frame 1052 in which a still image is recorded (recording of a still image is an example of the occurrence of an event), a camera-shaped icon 1060 is added by the still image extraction unit 240E. This allows the user to know that this is a frame in which a still image is recorded.

[0085] The range of frames for extracting still images to be displayed may include a frame before (frame 1051) and a frame after (frames 1053 to 1055) in time with respect to the "frame corresponding to the recorded still image file" (frame 1052 in the example of FIG. 20). The range of frames for extracting still images to be displayed may be in a form of "including only frames before in time" or "including only frames after in time". The range of frames for extracting still images to be displayed may be set according to a user's specification, or may be set independently of the user's specification. The range of frames to be displayed may be specified by time (for example, one second before and after a frame in which a still image is recorded) or by the number of frames (for example, ten frames before and after a frame in which a still image is recorded). In the initial state, only frames recorded as still images may be displayed in a list, and frames before and / or after a frame selected from among them may be displayed.

[0086] <Select the frame to extract> The user can select a frame from the displayed frames for extracting still images. In the example of FIG. 20, the user can select a frame by checking the check box 1070 for a desired frame via the operation unit 250 and / or the monitor 270, and can also select multiple frames. Note that the frame in which the still image specified by the user is recorded is not necessarily recorded at the best timing. There are cases in which the user wants to select another frame by displaying other related frames together. Therefore, it is preferable to display on the monitor 270 the frame in which the still image is recorded, as well as the frames before and / or after it, so that the user can select from them. FIG. 20 shows a state in which the user has selected a frame 1053 different from the frame in which the still image is recorded (frame 1052 with icon 1060 attached).

[0087] The still image extraction unit 240E extracts the selected frame as a still image (still image extraction step). Since the moving image file is stored in a moving image format such as MPEG format, the still image extraction unit 240E converts the data of the selected frame into a still image format (JPEG format, etc.). The user can extract a still image at a desired timing from the moving images (first moving image, second moving image, third moving image).

[0088] <Second embodiment> In the first embodiment, the camera 10, which is a digital camera, has been described, but the configuration of the imaging device is not limited to this. Other imaging devices of the present invention include, for example, a built-in or external camera for a PC (PC: Personal Computer), or The device may be a mobile terminal device having a photographing function, as described below.

[0089] Examples of a portable terminal device that is an embodiment of the imaging device of the present invention include a mobile phone, a smartphone, a PDA (Personal Digital Assistant), and a portable game machine. A smartphone will be taken as an example and described in detail below with reference to the drawings.

[0090] FIG. 21 is a diagram showing the appearance of a smartphone 1 (imaging device) which is an embodiment of the imaging device of the present invention, where (a) is a front view and (b) is a rear view. The smartphone 1 shown in FIG. 21 has a flat housing 2, and is provided with a display input section 20 in which a display panel 21 (display device) as a display section and an operation panel 22 (operation section) as an input section are integrated on one side of the housing 2. The housing 2 also includes a speaker 31, a microphone 32, an operation section 40 (operation section), camera sections 41 and 42 (imaging device, shooting section, event detection section, control section, still image extraction section, lens drive control section), and a strobe 43. Note that the configuration of the housing 2 is not limited to this, and for example, a configuration in which the display section and the input section are independent may be adopted, or a configuration having a folding structure or a sliding mechanism may be adopted.

[0091] Fig. 22 is a block diagram showing the configuration of the smartphone 1 shown in Fig. 21. As shown in Fig. 22, the smartphone 1 includes, as main components, a wireless communication unit 11, a display input unit 20, a call unit 30, an operation unit 40, camera units 41 and 42, a flash 43, a storage unit 50, an external input / output unit 60, a GPS receiver unit 70 (GPS: Global Positioning System), a motion sensor unit 80, a power supply unit 90, and a main control unit 101 (photographing unit, video file generating unit, still image file generating unit, event detection unit, display control unit, still image extraction unit, lens drive control unit). In addition, the smartphone 1 includes, as a main function, a wireless communication function for performing mobile wireless communication via a base station device and a mobile communication network.

[0092] The wireless communication unit 11 performs wireless communication with a base station device accommodated in a mobile communication network in accordance with instructions from the main control unit 101. Using such wireless communication, various file data such as voice data and image data, e-mail data, and the like are transmitted and received, and Web data, streaming data, and the like are received.

[0093] The display input unit 20 is a so-called touch panel that displays images (still images and / or moving images), text information, etc. under the control of the main control unit 101 to visually convey information to the user and detects user operations on the displayed information, and is equipped with a display panel 21 and an operation panel 22.

[0094] In the display panel 21, an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like is used as a display device. The operation panel 22 is placed so that an image displayed on the display surface of the display panel 21 can be seen, and is a device that detects one or more coordinates operated by a conductor such as a user's finger or pen. When such a device is operated by a conductor such as a user's finger or pen, the operation panel 22 outputs a detection signal generated by the operation to the main control unit 101. Next, the main control unit 101 detects the operation position (coordinates) on the display panel 21 based on the received detection signal.

[0095] 21, the display panel 21 and the operation panel 22 of the smartphone 1 illustrated as an embodiment of the imaging device of the present invention are integrated to form the display input unit 20, but the operation panel 22 is disposed so as to completely cover the display panel 21. When such an arrangement is adopted, the operation panel 22 may also have a function of detecting user operations in an area outside the display panel 21. In other words, the operation panel 22 may have a detection area for the overlapping portion that overlaps the display panel 21 (hereinafter referred to as the display area), and a detection area for the other outer edge portion that does not overlap the display panel 21 (hereinafter referred to as the non-display area).

[0096] The call unit 30 includes a speaker 31 and a microphone 32, and can convert the user's voice input through the microphone 32 into voice data that can be processed by the main control unit 101 and output it to the main control unit 101, and can decode voice data received by the wireless communication unit 11 or the external input / output unit 60 and output it from the speaker 31. Also, as shown in Fig. 21, for example, the speaker 31 can be mounted on the same surface as the surface on which the display input unit 20 is provided, and the microphone 32 can be mounted on the side surface of the housing 2.

[0097] The operation unit 40 is a hardware key using a key switch or the like, and is a device that accepts instructions from a user. For example, as shown in Fig. 21, the operation unit 40 is a push-button switch mounted on the side of the housing 2 of the smartphone 1, which is turned on when pressed with a finger or the like, and turned off by the restoring force of a spring or the like when the finger is released.

[0098] The storage unit 50 stores the control program and control data of the main control unit 101, application software, address data associated with names and telephone numbers of communication partners, data of sent and received e-mails, Web data downloaded by Web browsing, downloaded content data, and also temporarily stores streaming data, etc. The storage unit 50 is composed of an internal storage unit 51 built into the smartphone and an external storage unit 52 having a removable external memory slot. The internal storage unit 51 and the external storage unit 52 constituting the storage unit 50 are realized using known storage media.

[0099] The external input / output unit 60 serves as an interface with all external devices connected to the smartphone 1. The smartphone 1 is directly or indirectly connected to other external devices through the external input / output unit 60 by communication or the like. Examples of communication means include a Universal Serial Bus (USB), IEEE1394, and a network (e.g., the Internet and wireless LAN). Other examples of communication means include Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra WideBand) (registered trademark), and ZigBee (registered trademark).

[0100] Examples of external devices connected to the smartphone 1 include a wired / wireless headset, a wired / wireless external charger, a wired / wireless data port, and a memory card connected via a card socket, and a Subscriber Identity Module Card (SIM) / User Identity Module Card (UIM) card. In addition, external devices such as an external audio and video device connected via an audio and video I / O (Input / Output) terminal, a wirelessly connected external audio and video device, a wired / wireless connected smartphone, a wired / wireless connected PDA, a wired / wireless connected personal computer, and earphones can also be connected. The external input / output unit 60 can transmit data received from such external devices to each component inside the smartphone 1, and can transmit data inside the smartphone 1 to an external device.

[0101] The motion sensor unit 80 includes, for example, a three-axis acceleration sensor and a tilt sensor, and detects the physical movement of the smartphone 1 according to instructions from the main control unit 101. By detecting the physical movement of the smartphone 1, the moving direction, acceleration, and attitude of the smartphone 1 are detected. Such detection results are output to the main control unit 101. The power supply unit 90 supplies power stored in a battery (not shown) to each unit of the smartphone 1 according to instructions from the main control unit 101.

[0102] The main control unit 101 includes a microprocessor, operates according to the control program and control data stored in the storage unit 50, and controls each unit of the smartphone 1 including the camera unit 41. The main control unit 101 also includes a mobile communication control function that controls each unit of the communication system to perform voice communication and data communication via the wireless communication unit 11, and an application processing function.

[0103] The main control unit 101 also has an image processing function for displaying video on the display input unit 20 based on image data (still image or moving image data) such as received data or downloaded streaming data. The image processing function refers to a function in which the main control unit 101 decodes image data, performs image processing on the decoded result, and displays the image on the display input unit 20.

[0104] The camera units 41 and 42 are digital cameras (imaging devices) that take electronic pictures using imaging elements such as CMOS and CCD. In addition, under the control of the main control unit 101, the camera units 41 and 42 can convert image data (moving images, still images) obtained by imaging into compressed image data such as MPEG or JPEG, and can record the image data in the storage unit 50 or output the image data through the external input / output unit 60 or the wireless communication unit 11. In addition, under the control of the main control unit 101, the camera unit 41 can also divide and combine moving images, obtain high-quality still images (RAW images, etc.), replace and process frames, and extract still images from moving images. In the smartphone 1 shown in FIGS. 21 and 22, one of the camera units 41 and 42 can be used to take pictures, or the camera units 41 and 42 can be used simultaneously to take pictures. When the camera unit 42 is used, a strobe 43 can be used.

[0105] The camera units 41 and 42 can be used for various functions of the smartphone 1. For example, the smartphone 1 can display images acquired by the camera units 41 and 42 on the display panel 21. The smartphone 1 can use the images from the camera units 41 and 42 as one of the operation inputs of the operation panel 22. The smartphone 1 can also detect the position by referring to the images from the camera units 41 and 42 when the GPS receiver 70 detects the position. Furthermore, the smartphone 1 can determine the optical axis direction of the camera unit 41 of the smartphone 1 and the current usage environment by referring to the images from the camera units 41 and 42 without using a three-axis acceleration sensor or by using the three-axis acceleration sensor in combination. Of course, the smartphone 1 can also use the images from the camera units 41 and 42 in application software. In addition, the smartphone 1 can add location information acquired by the GPS receiving unit 70, voice information acquired by the microphone 32 (which may be converted to text information by the main control unit or the like), posture information acquired by the motion sensor unit 80, and the like to the image data of a still image or a video, and record the data in the storage unit 50. The smartphone 1 can also output the image data of the still image or the video through the external input / output unit 60 or the wireless communication unit 11.

[0106] <Smartphone posture> Part (a) of FIG. 23 is a diagram showing an example of a coordinate system defined for the smartphone 1, and in this example, the attitude is determined by the angle around the X-axis. Part (a) shows a vertical state with an angle of 0 deg (second state in which the smartphone 1 (imaging device) is aligned with the direction of gravity), and part (b) of the same figure shows a horizontal state with an angle of 90 deg (first state in which the smartphone 1 (imaging device) is aligned with the horizontal direction). Also, FIG. 24 is a diagram showing an example of the relationship between the angle around the X-axis and the attitude (vertical or horizontal) of the smartphone 1. Note that, as in the case of the camera 10 according to the first embodiment (see FIG. 4), the attitude and state (first state, second state, third state) may be defined with the horizontal state (state shown in part (b) of FIG. 23) as the reference (angle 0 deg).

[0107] The smartphone 1 having the above-mentioned configuration can also execute the processes of the imaging method according to the present invention (shooting, dividing, and recording a video, recording a still image, extracting a still image, etc.) in the same manner as the camera 10 according to the first embodiment. Specifically, the processes executed by the image processing device 240 (each unit shown in FIG. 2) in the first embodiment (including the processes of aspects 1 to 6) can be executed by the camera units 41 and 42 and the main control unit 101 in the smartphone 1. In addition, the functions of the operation unit 250, the storage unit 260, and the monitor 270 in the first embodiment can be realized by the operation unit 40, the storage unit 50, and the operation panel 22, and the display panel 21 and the operation panel 22, respectively, in the smartphone 1.

[0108] As a result, the smartphone 1 of the second embodiment can achieve the same effects as the camera 10 of the first embodiment (such as being able to easily select frames to extract as still images and organize video files and folders).

[0109] Although the embodiments and other aspects of the present invention have been described above, the present invention is not limited to the above-described embodiments and aspects, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0110] 1. Smartphone 2. Chassis 10. Camera 11 Wireless Communication Section 20 Display and input section 21 Display Panel 22 Operation Panel 30 Call section 31 Speaker 32 Microphones 40 Control section 41 Camera Division 42 Camera section 43 Strobe 50 Storage section 51 Internal storage 52 External memory unit 60 External input / output section 70 GPS receiver 80 Motion sensor unit 90 Power supply section 100 Interchangeable Lenses 101 Main control unit 110 Zoom Lens 120 Focus Lens 130 Aperture 140 Lens drive unit 200 Imaging device body 210 Image sensor 220 AFE 230 A / D Converter 240 Image Processing Device 240A Image acquisition unit 240B Direction detector 240C Control Unit 240D Event detection unit 240E Still image extraction section 240F Lens drive control unit 242 ROM 250 Operation section 260 Storage section 270 Monitor 280 Attitude Sensor 1051 Frame 1052 Frame 1053 Frame 1054 Frames 1055 Frame 1060 Icon 1070 Checkbox L optical axis S100~S197 Each step of the imaging method ST1 GPS satellite STn GPS satellite

Claims

1. 1. An imaging device, comprising: a direction detection unit that detects a direction of the imaging device with respect to a gravity direction; A filming department that shoots videos; A control unit; Equipped with the control unit has a first video mode capable of generating a video file of the video captured by the imaging unit and extracting a still image from the video file; when the direction detection unit determines that a direction change has occurred in the imaging device with respect to a gravity direction in the first video mode, the control unit divides the video to generate a first video and a second video; The control unit is storing a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; the first video precedes the second video in time; the direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with a horizontal direction, a second state in which the imaging device is aligned with a gravity direction, or a third state in which the imaging device is changing between the first state and the second state; The first moving image includes a moving image captured during a period in which the imaging device is in the third state.

2. 1. An imaging device, comprising: a direction detection unit that detects a direction of the imaging device with respect to a gravity direction; A filming department that shoots videos; A control unit; Equipped with the control unit has a first video mode capable of generating a video file of the video captured by the imaging unit and extracting a still image from the video file; when the direction detection unit determines that a direction change has occurred in the imaging device with respect to a gravity direction in the first video mode, the control unit divides the video to generate a first video and a second video; The control unit is storing a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; the first video precedes the second video in time; The imaging device, wherein the first video also includes a video of a second period after the direction change ends.

3. The imaging device according to claim 2 , wherein the second period is a period during which the imaging device adjusts exposure of a moving image, a white balance of a moving image, or a focus of a subject in the moving image.

4. The imaging device according to claim 1 , wherein the control unit provides the first moving image file and the second moving image file with information about a direction of the imaging device relative to a gravity direction.

5. The control unit has a reception unit that receives an instruction from a user, The imaging device according to claim 1 , wherein the timing to end the second moving image corresponds to the acceptance by the acceptance unit of an instruction to end the second moving image or an instruction to end shooting of a moving image from the user.

6. a reception unit that receives an instruction from a user or an event detection unit that detects a specific event, The imaging device according to claim 5 , wherein the control unit does not divide the video when the direction detection unit detects the direction change while the reception unit is receiving a user's instruction or while the event detection unit is detecting a specific event.

7. the control unit has a second movie mode having shooting conditions different from those of the first movie mode, The imaging device according to claim 1 , wherein the control unit is configured not to divide the moving image when the direction detection unit detects the direction change in the second moving image mode.

8. 8. The imaging device according to claim 7, wherein in the first video mode, at least one of a shutter speed, an autofocus speed, an autoexposure tracking speed, and a white balance tracking speed is set faster than in the second video mode, and / or a frame rate is set higher than in the second video mode.

9. The imaging device according to claim 1 , wherein the first moving image file and the second moving image file have at least a part in common except for an extension of a file name.

10. The imaging device according to claim 1 , wherein the control unit stores the first moving image file in a first folder of the storage unit, and stores the second moving image file in a second folder of the storage unit.

11. 1. An imaging method for an imaging device including: a direction detection unit that detects a direction of the imaging device relative to a gravity direction; an imaging unit that captures a moving image; and a control unit having a first moving image mode that generates a moving image file of the moving image captured by the imaging unit and is capable of extracting a still image from the moving image file, a moving image generating step of dividing a moving image and generating a first moving image and a second moving image when the direction detection unit determines that a direction change has occurred in the first moving image mode with respect to a gravity direction of the imaging device, by the control unit; a moving image storage step in which the control unit stores a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; a detection step in which the direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with a horizontal direction, a second state in which the imaging device is aligned with a gravity direction, or a third state in which the imaging device is changing between the first state and the second state; having the first video precedes the second video in time; The imaging method, wherein the first moving image includes a moving image captured during a period in which the imaging device is in the third state.

12. 1. An imaging method for an imaging device including: a direction detection unit that detects a direction of the imaging device relative to a gravity direction; an imaging unit that captures a moving image; and a control unit having a first moving image mode that generates a moving image file of the moving image captured by the imaging unit and is capable of extracting a still image from the moving image file, a moving image generating step of dividing a moving image and generating a first moving image and a second moving image when the direction detection unit determines that a direction change has occurred in the first moving image mode with respect to a gravity direction of the imaging device, by the control unit; a moving image storage step in which the control unit stores a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; having the first video precedes the second video in time; The imaging method, wherein the first moving image also includes a moving image of a second period after the directional change is completed.

13. A program for operating an imaging device including: a direction detection unit that detects a direction of the imaging device relative to a gravity direction; an imaging unit that captures a moving image; and a control unit having a first moving image mode that generates a moving image file of the moving image captured by the imaging unit and is capable of extracting a still image from the moving image file, the program comprising: cause the control unit to divide a moving image to generate a first moving image and a second moving image when the direction detection unit determines that a direction change has occurred in the imaging device with respect to a gravity direction in the first moving image mode; causing the control unit to store a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; causing the direction detection unit to detect whether the imaging device is in a first state in which the imaging device is aligned with a horizontal direction, a second state in which the imaging device is aligned with a gravity direction, or a third state in which the imaging device is changing between the first state and the second state; the first video precedes the second video in time; The first video is a program including a video during a period when the imaging device is in the third state.

14. A program for operating an imaging device including: a direction detection unit that detects a direction of the imaging device relative to a gravity direction; an imaging unit that captures a moving image; and a control unit having a first moving image mode that generates a moving image file of the moving image captured by the imaging unit and is capable of extracting a still image from the moving image file, the program comprising: cause the control unit to divide a moving image to generate a first moving image and a second moving image when the direction detection unit determines that a direction change has occurred in the imaging device with respect to a gravity direction in the first moving image mode; causing the control unit to store a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; the first video precedes the second video in time; The program, wherein the first video also includes a video of a second period after the direction change is completed.

Images