Imaging device, imaging method, and program

Abstract

To provide an imaging device, an imaging method, and a program.SOLUTION: In an imaging device, a control unit has a first video mode in which a video file of a video captured by an imaging unit is generated and still images can be extracted from the video file, and when a direction detection unit determines occurrence of a direction change of an imaging device with respect to a gravity direction in the first video mode, the control unit divides the video to generate a first video and a second video. The control unit stores a first video file of the first video and a second video file of the second video in a storage unit. The first video is earlier in time than the second video. The direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with a horizontal direction, a second state in which the imaging device is aligned with the gravity direction, or a third state in which the imaging device is changing between the first state and the second state, and the first video includes a video during a period in which the imaging device is in the third state.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an imaging device, an imaging method, and a program for capturing moving images, and more particularly to an imaging device, an imaging method, and a program that are capable of extracting frames that constitute a moving image as still images. [Background technology]

[0002] There is a known technique for capturing video with the aim of extracting a still image after shooting. With this technique, a video is shot and recorded to capture, for example, a momentary event that may occur at any time, and the user selects a desired frame after shooting. However, it is difficult to predict when an event will occur during video shooting. Therefore, there are problems such as the need to continue shooting the video for a certain period of time, which can lead to the remaining capacity of the recording media being exhausted, or the user having to make a great effort to select a frame in which an event occurred from a long video.

[0003] To address the above-mentioned issues, tagging or marking of videos can be considered. For example, an event is detected, and a tag representing event information is added to a frame when the event occurs. Patent Document 1, for example, describes a technique in which a user marks a video at the timing of capturing a still image during video capture, and then extracts and displays the marked location and video frames before and after the video capture is completed, allowing the user to select the timing of extracting a still image from the video. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-32303 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when adding the above-mentioned tags, tags representing event information are not standardized and can only be recognized by specific devices, including the imaging device used for shooting. Furthermore, playing back event frames requires analyzing the tag information of all frames, which results in a time-consuming process. Furthermore, the technology described in Patent Document 1 only records marking information, so video decoding (still image processing) is required each time a marked frame is extracted and displayed, placing a heavy processing load on the imaging device. Thus, conventional technologies have not allowed users to easily select video frames they wish to extract as still images.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide an imaging device, an imaging method, and a program that allow a user to easily select a frame of a video that they want to extract as a still image. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, an imaging device according to a first aspect of the present invention is an imaging device comprising: a direction detection unit that detects the direction of the imaging device relative to the direction of gravity; a shooting unit that shoots video; and a control unit, wherein the control unit generates a video file of the video shot by the shooting unit and has a first video mode in which still images can be extracted from the video file, and when, in the first video mode, the direction detection unit determines that a change in the direction of the imaging device relative to the direction of gravity has occurred, the control unit divides the video to generate a first video and a second video, and the control unit saves the first video file of the first video in a first folder in the memory unit and saves the second video file of the second video in a second folder in the memory unit.

[0008] Typically, when capturing still images, a user holds a camera (image capture device) horizontally (with the bottom of the camera parallel to the ground) or vertically (with the bottom of the camera parallel to the direction of gravity) depending on the desired composition. In contrast, when capturing video, switching between portrait and landscape orientations during capture is rare. On the other hand, when capturing video for extracting still images, due to the nature of frame extraction performed after the fact, it is expected that the camera will be held vertically and horizontally even during video capture. This change between portrait and landscape orientation can be considered a type of event caused by the user's actions. From this perspective, in the first aspect, when a change in the orientation of the image capture device relative to the direction of gravity (the orientation of the image capture device) occurs, the video is divided to generate a first video and a second video, and the first video file of the first video and the second video file of the second video are saved in separate folders (first folder, second folder), respectively. This makes it easy to identify the orientation of the image capture device when each video was captured, and the user can easily select a video containing a frame they want to extract as a still image by using the orientation of the image capture device as a clue. Furthermore, since the frame to be extracted as a still image can be found within the divided video whose recording time has been shortened, the desired frame can be found quickly.

[0009] Thus, according to the first aspect, a user can easily select a frame of a video that he or she wishes to extract as a still image. Furthermore, since the first aspect does not require the use of information such as tags or markings, frame selection and still image extraction can be performed in other imaging devices, image playback devices, and image processing devices based on the video recorded in the first aspect. The storage unit may be included in the imaging device or may be an external device.

[0010] In the imaging device according to the second aspect, the control unit generates a third moving image that is located between the first moving image and the second moving image in time, and the third moving image includes a moving image of a first period in which a direction change occurs. In the second aspect, the period in which a direction change occurs (first period) is made into a moving image (third moving image) separate from the first and second moving images, making it easy for the user to distinguish.

[0011] In the imaging device according to the third aspect, the first video is taken before the second video in time, the direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with the horizontal direction, a second state in which the imaging device is aligned with the direction of gravity, or a third state in which the imaging device is changing between the first and second states, and the first video includes video of a period in which the imaging device is in the third state. In the third aspect, the third state can be, for example, an intermediate state between the first and second states, a state in which the orientation of the imaging device is indefinite, or a state in which the attitude of the imaging device is changing between the first and second states.

[0012] The imaging device according to the fourth aspect is the same as the second aspect, except that the control unit does not store the third moving image in the storage unit. According to the fourth aspect, by not storing the moving image during the period (first period) in which a direction change occurs, it is possible to save capacity in the storage unit.

[0013] An imaging device according to a fifth aspect is the second aspect, wherein the control unit saves the third moving image in a first folder in the storage unit, that is, the third moving image is saved in the same folder as the first moving image.

[0014] In the imaging device according to the sixth aspect, in the fourth or fifth aspect, the third moving image also includes a moving image of the second period after the direction change has ended. In the sixth aspect, since the shooting conditions may not be stable due to a change in composition even after the direction change has ended, the third moving image is included in the second period even after the direction change has ended.

[0015] In the imaging device according to the seventh aspect, in the first or second aspect, the first moving image also includes a moving image of a second period after the direction change has ended. In the sixth aspect, since the shooting conditions may not be stable due to a change in composition even after the direction change has ended, the first moving image includes the moving image of the second period after the direction change has ended.

[0016] An imaging device according to an eighth aspect is the sixth or seventh aspect, wherein the second period is a period during which the imaging device adjusts the exposure of a moving image, the white balance of a moving image, or the focus of a subject in the moving image. The eighth aspect specifically defines the content of the second period.

[0017] In a ninth aspect of the present invention, the image capture device is any one of the first to eighth aspects, and the control unit assigns information about the orientation of the image capture device relative to the direction of gravity to the first moving image file and the second moving image file. For example, the information can be assigned to the file name or part of the file name, or to the header or footer of the moving image file.

[0018] The imaging device according to a tenth aspect is any one of the first to ninth aspects, wherein the control unit has a reception unit that receives user instructions, and the timing to end the second moving image corresponds to reception by the reception unit of an instruction from the user to end the second moving image or an instruction to end moving image shooting. According to the tenth aspect, the user can end the second moving image at a desired timing.

[0019] An imaging device according to an eleventh aspect is any one of the first to tenth aspects, and further includes a reception unit for receiving user instructions or an event detection unit for detecting a specific event within the control unit, and the control unit does not split the video if the direction detection unit detects a change in direction while the reception unit is receiving the user instructions or while the event detection unit is detecting a specific event. According to the eleventh aspect, the video is not split if there is a change in direction (posture) of the imaging device during the user instruction or the occurrence of a specific event, so that a series of scenes related to the user instruction or the event can be viewed in a single video.

[0020] In a twelfth aspect of the present invention, in the imaging device of any one of the first to eleventh aspects, the control unit has a second moving image mode whose shooting conditions are different from those of the first moving image mode, and the control unit does not divide the moving image when the direction detection unit detects a change in direction in the second moving image mode. For example, the second moving image mode can be a normal moving image mode that does not extract still images from the moving image.

[0021] An imaging device according to a thirteenth aspect is the twelfth aspect, wherein in the first moving image mode, at least one of the shutter speed, autofocus speed, autoexposure tracking speed, and white balance tracking speed is set faster than in the second moving image mode, and / or the frame rate is set higher than in the second moving image mode. The thirteenth aspect specifies one aspect of the difference in shooting conditions between the first moving image mode and the second moving image mode, and by setting the shooting conditions of the first moving image mode in this way, it is possible to extract high-quality still images from the moving image.

[0022] In an imaging device according to a fourteenth aspect, in any one of the first to thirteenth aspects, the first moving image file and the second moving image file have at least a portion in common except for the filename extension, which allows a user to easily distinguish that the first moving image file and the second moving image file are files generated by dividing a single moving image.

[0023] To achieve the above-mentioned object, a fifteenth aspect of the present invention provides an imaging method for an imaging device including a direction detection unit that detects the direction of the imaging device relative to the direction of gravity, a shooting unit that shoots a video, and a control unit having a first video mode capable of extracting still images from a file of the video shot by the shooting unit. The imaging method includes a determination step in which, in the first video mode, the direction detection unit determines whether a change in the direction of gravity of the imaging device has occurred; a video generation step in which, if it is determined that a change in direction has occurred in the first video mode, the control unit divides the video to generate a first video and a second video; and a video storage step in which the control unit saves a first video file of the first video in a first folder in the storage unit and a second video file of the second video in a second folder in the storage unit. The fifteenth aspect of the present invention allows a user to easily select a desired frame from a video shot, similar to the first aspect. The fifteenth aspect may further include the same configuration as any of the second to fourteenth aspects.

[0024] To achieve the above-mentioned object, a sixteenth aspect of the present invention provides a program for operating an imaging device including a direction detection unit that detects the direction of the imaging device relative to the direction of gravity, a shooting unit that shoots video, and a control unit having a first video mode capable of extracting still images from a file of the video shot by the shooting unit. In the first video mode, the direction detection unit determines whether a change in the direction of the imaging device relative to the direction of gravity has occurred. If it is determined that a change in direction has occurred in the first video mode, the control unit divides the video to generate a first video and a second video, and stores a first video file of the first video in a first folder in the storage unit and a second video file of the second video in a second folder in the storage unit. The sixteenth aspect of the present invention allows a user to easily select a desired frame from a shot video, similar to the first and fifteenth aspects. The sixteenth aspect may further include the same configuration as the second to fourteenth aspects. Furthermore, the programs of these aspects may be recorded on a non-transitory recording medium recording computer-readable code. [Effects of the Invention]

[0025] As described above, the imaging device, imaging method, and program of the present invention allow a user to easily select a desired frame from a captured video. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing the configuration of a camera according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the functional configuration of the image processing apparatus. [Figure 3] FIG. 3 is a diagram showing the coordinate system of the camera. [Figure 4] FIG. 4 is a diagram illustrating an example of posture determination and video division. [Figure 5] FIG. 5 is a flowchart showing the process of the imaging method. [Figure 6] FIG. 6 is a diagram showing how the camera changes its posture and how the video is divided and saved. [Figure 7] FIG. 7 is a diagram showing an example of a folder structure. [Figure 8] FIG. 8 is another diagram showing an example of a folder structure. [Figure 9] FIG. 9 is a diagram showing an example of icon display for vertical moving images and horizontal moving images. [Figure 10] FIG. 10 is a diagram showing how thumbnail images are added to vertical and horizontal moving images. [Figure 11] FIG. 11 is another flowchart showing the process of the imaging method. [Figure 12] FIG. 12 is a diagram showing another example of posture determination and video division. [Figure 13] FIG. 13 is a diagram showing yet another example of posture determination and video division. [Figure 14] FIG. 14 is a diagram showing yet another example of posture determination and video division. [Figure 15] FIG. 15 is a diagram showing yet another example of posture determination and video division. [Figure 16] FIG. 16 is yet another flowchart showing the process of the imaging method. [Figure 17] FIG. 17 is yet another flowchart showing the process of the imaging method. [Figure 18] FIG. 18 is a flowchart showing the process of recording moving images in the first mode and the second mode. [Figure 19] FIG. 19 is a flowchart (continuation of FIG. 18) showing the moving image recording process in the first mode and the second mode. [Figure 20] FIG. 20 is a diagram showing how a frame to be extracted as a still image is selected. [Figure 21] FIG. 21 is an external view of the smartphone according to the second embodiment. [Figure 22] FIG. 22 is a block diagram showing the configuration of a smartphone according to the second embodiment. [Figure 23] FIG. 23 is a diagram showing the orientation of the smartphone according to the second embodiment. [Figure 24]FIG. 24 is a diagram showing how the attitude of a smartphone is determined based on an angle. DETAILED DESCRIPTION OF THE INVENTION

[0027] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an imaging device, an imaging method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

[0028] First Embodiment <Overall configuration of the imaging device> 1 is a diagram showing the configuration of a camera 10 (imaging device) according to the first embodiment. The camera 10 is made up of an interchangeable lens 100 (photographing section, imaging device) and an imaging device body 200 (imaging device), and a subject image (optical image) is formed on an imaging element 210 by a photographing lens including a zoom lens 110 (described later). The interchangeable lens 100 and the imaging device body 200 can be attached and detached via a mount (not shown).

[0029] <Interchangeable lens configuration> The interchangeable lens 100 includes a zoom lens 110, a focus lens 120, an aperture 130, and a lens driver 140. The lens driver 140 drives the zoom lens 110 and the focus lens 120 forward and backward in response to commands from an image processing device 240 (lens driver control unit 240F in FIG. 2 ) to perform zoom (optical zoom) adjustment and focus adjustment. The zoom adjustment and focus adjustment may be performed in response to commands from the image processing device 240, or in response to zoom and focus operations (such as rotation of a zoom ring or focus ring, not shown) performed by the user. The lens driver 140 also controls the aperture 130 in response to commands from the image processing device 240 to adjust exposure. Information such as the positions of the zoom lens 110 and the focus lens 120 and the aperture of the aperture 130 is input to the image processing device 240. The interchangeable lens 100 has an optical axis L.

[0030] <Configuration of the imaging device body> The imaging device main body 200 includes an imaging element 210 (imaging unit), an AFE 220 (AFE: Analog Front End, imaging unit), an A / D converter 230 (A / D: Analog to Digital, imaging unit), an image processing device 240, an operation unit 250, a storage unit 260, a monitor 270, and an attitude sensor 280 (direction detection unit). The imaging device main body 200 may include a shutter (not shown) for blocking light transmitted through the imaging element 210. The imaging element 210 includes a light receiving surface on which a large number of light receiving elements are arranged in a matrix. Subject light transmitted through the zoom lens 110, the focus lens 120, and the aperture 130 is focused on the light receiving surface of the imaging element 210 and converted into an electrical signal by each light receiving element. R (red), G (green), or B (blue) color filters are provided on the light receiving surface of the imaging element 210, and a color image of the subject can be obtained based on the signals of each color. Note that various photoelectric conversion elements such as a CMOS (Complementary Metal-Oxide Semiconductor) and a CCD (Charge-Coupled Device) can be used as the image sensor 210. The AFE 220 performs noise removal and amplification of the analog image signal output from the image sensor 210, and the A / D converter 230 converts the captured analog image signal into a digital image signal with a wide gradation range.

[0031] <Configuration of image processing device> 2 is a diagram showing the functional configuration of the image processing device 240. The image processing device 240 includes an image acquisition unit 240A, a direction detection unit 240B (direction detection unit), a control unit 240C (control unit), an event detection unit 240D (event detection unit), a still image extraction unit 240E (still image extraction unit), and a lens drive control unit 240F (lens drive control unit). The control unit 240C has a reception unit that receives user instructions, and this reception unit can receive user instructions via the operation unit 250. The image processing device 240 performs processes such as shooting a video and generating a file, generating a still image file, processing multiple frames that constitute the video, and extracting still images based on the digital image signal input from the A / D converter 230. The processes performed by the image processing device 240 will be described in detail below.

[0032] The functions of the image processing device 240 can be realized using various processors. The various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions. The above-mentioned various processors also include a GPU (Graphics Processing Unit), which is a processor specialized for image processing, and a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacturing. Furthermore, the above-mentioned various processors also include dedicated electrical circuits, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for executing specific processing.

[0033] When the above-described processor or electrical circuit executes software (programs), processor-readable code for the software to be executed is stored in a non-transitory recording medium such as a ROM (Read Only Memory). The processor then references the software. The software stored in the non-transitory recording medium includes a program for executing the imaging method of the present invention (a program for operating the imaging device of the present invention). The code may be recorded in a non-transitory recording medium such as various magneto-optical recording devices or semiconductor memory instead of a ROM. When processing using the software, for example, a RAM (Random Access Memory) is used as a temporary storage area, and data stored in, for example, an EEPROM (Electronically Erasable and Programmable Read Only Memory), not shown, may also be referenced.

[0034] In addition to the above-mentioned components, the image processing device 240 also includes a ROM 242. The ROM 242 stores computer-readable code of programs required for capturing, recording, displaying, etc. of images (including a program for executing the imaging method according to the present invention).

[0035] <Operation section> The operation unit 250 has a release button, operation buttons (for example, a cross button, a Quick button, an OK button, etc.), dials, switches, etc. (not shown), and the user can perform various operations such as setting the shooting mode, issuing a command to shoot a video, issuing a command to extract a still image, etc. Note that the monitor 270 (touch panel type) may also be used as the operation unit 250.

[0036] <Storage section> The storage unit 260 (storage unit) is composed of various types of magneto-optical storage media, non-transitory storage media such as semiconductor memory, and their control circuits, and stores moving images, still images, still images extracted from moving images, etc. The storage media may be of a type that is detachable from the imaging device main body 200. The captured images (moving images, still images) may be transmitted to and stored in an external storage medium (storage unit) via, for example, wireless communication.

[0037] <Monitor and viewfinder> Monitor 270 (display device) is configured with a touch panel type liquid crystal display panel and can display moving images, still images, frames for extracting still images, etc. Monitor 270 can be placed on the back side or top side of imaging device main body 200. Camera 10 may also be equipped with a viewfinder.

[0038] <Shooting mode> The camera 10 can be set to one of three shooting modes: still image shooting mode, video shooting mode for extracting still images (first video mode), and normal video shooting mode (second video mode). The still image shooting mode and normal video shooting mode are similar to those of a normal digital camera. On the other hand, the video shooting mode for extracting still images is a mode that allows still images to be extracted from a video file and captures videos under different shooting conditions than the normal video shooting mode (videos shot under shooting conditions that prioritize extracting still images over viewing the video itself). Specifically, in the video shooting mode for extracting still images, at least one of the shutter speed, autofocus speed, autoexposure tracking speed, and white balance tracking speed is set faster than in the normal video shooting mode, and / or the frame rate is set higher than in the normal video shooting mode. In addition, the resolution and frame rate are set to the highest values ​​possible on the camera 10 (e.g., 4,000 x 2,000 pixels, 30 frames per second), and the color tone is also set based on the assumption of still image extraction. The upper limit of the ISO sensitivity is also higher than in the normal video shooting mode.

[0039] For example, in normal video shooting mode, the shutter speed is set to a value corresponding to the frame rate of the video being recorded (1 / 30 seconds if the frame rate is 30 frames per second). On the other hand, in video shooting mode for still image extraction, the shutter speed is set to a value faster than the frame interval (for example, less than 1 / 30 seconds). In normal video shooting mode, the shutter speed is set to a value corresponding to the video frame rate to ensure smooth video playback, but in this case, blurring of moving subjects may occur. For this reason, in video shooting mode for still image extraction, the shutter speed is set faster than in normal video shooting mode (faster than the frame interval), which enables the extraction of high-quality still images with less subject blur. Similarly, by increasing the upper limit of the ISO sensitivity (ISO: International Organization for Standardization), the shutter speed can be increased, thereby enabling the extraction of still images with less blur. In addition, the autofocus speed, autoexposure tracking speed, autowhite balance tracking speed, etc. are set faster than in normal video shooting mode. This allows the capture of more frames with the subject in focus and with appropriate exposure. Regarding the frame rate, setting a high rate shortens the frame interval of the video, and increases the number of frames that can be extracted as still images.

[0040] The above-described still image extraction video capture mode allows frames constituting a video to be extracted as still images after the fact. This allows users to easily capture photos of events (natural phenomena, accidents, unexpected happenings, etc.) that may occur at any time, or photos of the instantaneous state of a moving subject or a subject whose state changes over time. As will be described in detail later, still images can be extracted not only when a still image recording command is issued but also at other times. This allows users to obtain still images at desired times. Furthermore, by setting shooting conditions suitable for still image extraction (such as the shutter speed, resolution, and frame rate described above), high-quality still images can be extracted. In addition to this post-event extraction of still images, still images can be captured (saved) during video capture in the still image extraction video capture mode and normal video capture mode.

[0041] <Camera posture> FIG. 3 is a diagram showing an example of a coordinate system defined for camera 10. In the example shown in FIG. 3, the X axis is the optical axis direction (the direction toward the subject), and the Z axis is the direction toward the top of image capture device body 200, and (X axis, Y axis, Z axis) form a right-handed coordinate system. Whether camera 10 is in a vertical position (the top and bottom surfaces of camera 10 are parallel to the vertical direction) or a horizontal position (the top and bottom surfaces of camera 10 are parallel to the ground) can be determined, for example, from the angle around the X axis (0° is defined as when the top and bottom surfaces of image capture device body 200 are parallel to the ground). FIG. 4 is a diagram showing an example of the relationship between the angle around the X axis and the orientation (vertical or horizontal). Direction detection unit 240B (direction detection unit) determines the angle as "horizontal" when it is within the ranges of (-45° to +45°) and (+135° to +225°), and determines the angle as "vertical" otherwise. As described above, the camera 10 is equipped with an attitude sensor 280 (for example, an acceleration sensor that measures acceleration in three axis directions; see FIG. 1), and therefore the direction detection unit 240B can detect the attitude of the camera 10 (the direction relative to the direction of gravity) based on the measurement results of the attitude sensor 280.

[0042] 4 is an example, and the orientation may be determined within a different range. For example, the direction detection unit 240B may determine the orientation as "landscape" when the angle is within the ranges of (-30 degrees or more and +30 degrees or less) and (+150 degrees or more and +210 degrees or less), and may determine the orientation as "portrait" in other cases.

[0043] <Processing in video shooting mode for extracting still images> An imaging method for the camera 10 configured as described above will now be described. Fig. 5 is a flowchart showing processing in a moving image capturing mode for still image extraction. For example, the camera 10 is set to the moving image capturing mode for still image extraction by operating a mode dial (not shown) on the operation unit 250. Then, the control unit 240C sets the capturing unit (interchangeable lens 100, image sensor 210, AFE 220, A / D converter 230) to the first moving image mode. Then, the processing of Fig. 5 starts, and the control unit 240C initializes the state of the camera 10 (acquiring and displaying a live view image, initializing the attitude, starting attitude determination, etc.) (step S100: initialization step).

[0044] <Video recording start command> The control unit 240C determines whether a command to start video recording has been issued (step S110: shooting step, video generation step). For example, if a release button (not shown) of the operation unit 250 is pressed down, it can be determined that a "command to start video recording has been issued." If the determination is affirmative, the process proceeds to step S120.

[0045] <Posture determination> The direction detection unit 240B detects the orientation (orientation relative to the direction of gravity) of the camera 10 (imaging device) based on the output of the orientation sensor 280 (step S120: orientation detection step, determination step). As described above with reference to FIGS. 3 and 4, the orientation detection unit 240B determines whether the orientation is "portrait," "landscape," or "indeterminate" based on the angle around the X axis. An "indeterminate" orientation can include a state in which the angle around the Y axis is large and it is not possible to identify "portrait" or "landscape," such as when the user holds the camera 10 facing the ground or the sky. Note that the direction detection unit 240B does not have to detect the "indeterminate" state of the camera 10.

[0046] <Recording to folders according to posture> If the orientation of camera 10 is determined to be "landscape" or "undefined" in step S120, control unit 240C compresses the captured video to create a new video file. Then, control unit 240C starts recording (saving) the video file in a horizontal video folder (first folder) in storage unit 260 (step S130: video saving step). Video compression can be performed in MPEG format (MPEG2, MPEG4, etc.), for example. On the other hand, if the orientation of camera 10 is determined to be "portrait," control unit 240C starts recording (saving) the video file in a vertical video folder (second folder) separate from the horizontal video folder (step S140: video saving step). Audio may also be recorded and saved using a microphone (not shown) provided in camera 10. Note that "horizontal video" and "vertical video" refer to videos when the orientation is landscape or undefined, and when the orientation is portrait, respectively. The "horizontal video folder" and "vertical video folder" are folders for saving horizontal video and vertical video, respectively.

[0047] <Handling when posture is uncertain> 5, recording continues in the same folder and the same file when "the orientation changes from landscape to undefined" and when "the orientation changes from undefined to landscape." Furthermore, when "the orientation changes from portrait to undefined" and when "the orientation changes from undefined to portrait," it is determined that "the orientation has changed," and the video is divided and recorded in a separate folder. However, when the orientation is undefined, the video may be recorded in a folder for portrait video, or in a folder separate from the landscape video folder and the vertical video folder (see other aspects described below).

[0048] The direction detection unit 240B continues detecting the orientation of the camera 10 while capturing the video, and determines whether or not a change in the orientation of the camera 10 (image capture device) with respect to the direction of gravity has occurred (step S170: determination step). Whether or not a change in orientation has occurred can be determined based on the output of the orientation sensor 280 (for example, acceleration in three axial directions). If the direction detection unit 240B determines that a change in orientation has occurred, specifically, if it determines that the orientation of the camera 10 has changed between "horizontal or indeterminate" and "portrait" (Yes in step S170), the control unit 240C closes the video file being recorded and divides the video (step S180: video generation step). Then, the determination in step S190 is negative, and the process returns to step S120.

[0049] Upon returning to step S120, the control unit 240C determines whether the orientation of the camera 10 is “portrait,” “landscape,” or “undefined,” just as when recording of the video file began. Then, depending on the result, the control unit 240C saves the new video file resulting from the split in a folder for horizontal videos or a folder for vertical videos in the storage unit 260. For example, if the orientation of the camera 10 changes from “landscape” at the start of recording to “portrait,” the control unit 240C saves the video file (the first video file of the first video) in the folder for horizontal videos at the start of recording, and saves the new video file (the second video file of the second video) generated as a result of the change in orientation in the folder for vertical videos. Note that, in step S170, if the orientation of the camera changes again (Yes in step S170), the video does not need to be closed. This is because repeatedly dividing the video every time the orientation of the camera changes would result in an excessive number of video files.

[0050] The image acquisition unit 240A, the direction detection unit 240B, and the control unit 240C continue to divide the video and save the video files according to the orientation of the camera 10 until the video capture is completed (while the determination in step S190 is No). The direction detection unit 240B and the control unit 240C preferably assign information about the orientation of the camera 10 (orientation relative to the direction of gravity) to the first video file and the second video file so that it can be referenced when displaying the video file icon or playing the video. The direction detection unit 240B and the control unit 240C can record information about the orientation of the camera 10 in the header or footer. For example, the information can be set to "01" for portrait, "02" for landscape, and "03" for other orientations (diagonal, indefinite, etc.).

[0051] As shown in the example of FIG. 6, camera 10 may transition from a landscape orientation to a portrait orientation and end video shooting in the portrait orientation. In this case, the timing to end the second video (portrait video; video in portrait orientation) corresponds to the reception of a user's instruction to end the second video or an instruction to end video shooting by a reception unit within control unit 240C. This reception unit can receive user instructions via operation unit 250. Similarly, camera 10 may transition from a portrait orientation to a landscape orientation again and end video shooting in the landscape orientation. In this case, the timing to end the first video (landscape video; video in landscape orientation) corresponds to the reception of a user's instruction to end the first video or an instruction to end video shooting by the reception unit. These video end timings also apply to aspects described below.

[0052] FIG. 6 is a diagram illustrating an example of determining a change in orientation based on the angle around the X-axis. In the example of FIG. 6, the orientation is determined to be "landscape" between elapsed time zero and t1, and "vertical" at elapsed time t1 when the angle exceeds 45 degrees (threshold value). This results in the video being divided at elapsed time t1. The control unit 240C saves the video file of the first video (first video file) from elapsed time zero to t1 in a horizontal video folder (first folder), and saves the video file of the second video (second video file) from elapsed time t1 onward in a vertical video folder (second folder). Because the orientation of the camera 10 may fluctuate minutely, it may be determined that "the orientation has changed" when the changed orientation continues for a predetermined period of time or longer. While FIG. 6 illustrates an example in which the orientation of the camera 10 changes from landscape to portrait, a change from portrait to landscape can also be processed in a similar manner (in this case, the vertical video becomes the first video, and the horizontal video becomes the second video. The vertical video folder becomes the first folder, and the horizontal video folder becomes the second folder; the same applies to other aspects).

[0053] The control unit 240C closes the video file (step S180) not only when the attitude of the camera 10 has changed (Yes in step S170), but also when it determines in step S150 that an instruction to end video recording (an instruction to end video shooting) has been issued, or when it determines in step S160 that the remaining capacity of the recording media is insufficient. However, in these cases, the control unit 240C also ends video shooting (Yes in step S190). In step S150, the control unit 240C can determine that an instruction to end video recording has been issued when the release button (not shown) of the operation unit 250 is pressed, when an instruction to end video recording is issued via the monitor 270, or the like.

[0054] In the first embodiment, when it is determined that "the orientation (posture) of the camera 10 (imaging device) relative to the direction of gravity has changed in the still image extraction moving image shooting mode (first moving image mode)," the moving image is divided to generate a horizontal moving image (first moving image) and a vertical moving image (second moving image). Then, the moving image file of the horizontal moving image (first moving image file) is saved in a horizontal moving image folder (first folder), and the moving image file of the vertical moving image (second moving image file) is saved in a vertical moving image folder (second folder).

[0055] This allows the user to easily identify the orientation of camera 10 when each video was shot, and to easily select a video containing a frame that the user wants to extract as a still image by using the orientation of camera 10 as a clue. Also, the user can quickly find the desired frame because they only need to search for the frame to extract as a still image within the video that has been divided and has a shorter recording time. Thus, according to the first embodiment, the user can easily select the desired frame from the video that has been shot.

[0056] <Example of folder structure> FIG. 7 shows an example of a folder structure for saving videos. In the example shown in the figure, a folder is created for each shooting date, and within each folder for each shooting date, a folder is created for each shooting number. For example, in the flowchart of FIG. 5, the period from when the start of video recording is instructed in step S110 to when video recording is stopped in step S190 can be treated as one shooting number. For each folder for each shooting number, a folder for horizontal videos and a folder for vertical videos are provided, and horizontal videos and vertical videos are saved, respectively. This folder structure can be displayed on the monitor 270 by the control unit 240C.

[0057] 8 shows a state in which a horizontal video folder (first folder) and a vertical video folder (second folder) are displayed as horizontal and vertical icons, respectively, on monitor 270. This manner of folder display allows the user to easily visually distinguish between the vertical video folder and the vertical video folder, and allows the user to easily select a desired frame.

[0058] <Video file icon and file name> FIG. 9 shows an example of a video file icon display. Part (a) of the figure shows a horizontal icon with the words "horizontal video" attached to it for a horizontal video file. The symbol in the icon indicates that the file can be played by specifying it. Similarly, part (b) of FIG. 9 shows an example of an icon display for a vertical video file, with the words "vertical video" attached to it. In the example of FIG. 9, the file name consists of the file creation date (October 23, 2018), the shooting number (001), vertical / horizontal identification information (H (Horizontal) for horizontal video, V (Vertical) for vertical video), the file number (001, 002, ...), the file size (e.g., 10 MB), and the extension (here, ".mpg"). The horizontal video file (first video file) and the vertical video file (second video file) share some of the same file name except for the extension. The vertical / horizontal identification information is an example of information regarding the orientation of the camera 10 (image capture device) relative to the direction of gravity.

[0059] The file name may include information indicating the file type, the file creation date and time, and portrait / landscape identification information. For example, a landscape video file created on 2018 / 1 / 1 12:30 may be given the file name "IMG201801011230_H.MOV," and a portrait video file created on 2018 / 1 / 1 12:40 may be given the file name "IMG201801011240_H.MOV." In the file name, "IMG" is an example of information indicating that it is an image file. Even in this case, the file name may include information indicating the file size.

[0060] Figure 10 shows how a thumbnail image is displayed on an icon. Part (a) of the figure is an example of a horizontal video, and part (b) is an example of a vertical video. The image to be displayed may be the first frame of a horizontal or vertical video, or an intermediate frame (for example, a frame where an event occurs).

[0061] By displaying the video file as an icon, thumbnail, and giving a file name as exemplified above, the user can easily distinguish between vertical and horizontal videos, and can easily select a video containing a desired frame. Note that the icon display, thumbnail display, and file name giving of the video file can be performed by the control unit 240C.

[0062] <Other aspects of video division and recording> Other aspects of dividing and recording (saving) a moving image will be described below. Note that in the flowcharts described below, the same steps as those in Fig. 5 are given the same step numbers, and detailed descriptions thereof will be omitted.

[0063] (Mode 1: A mode in which the period during posture change is recorded as an independent video file) In the aspect described with reference to Fig. 5, if the attitude of camera 10 (angle around the X axis) exceeds a threshold value, it is determined that "a change in orientation has occurred." However, in aspect 1 shown in Fig. 11, a video during an orientation change is saved in a different folder as a video file separate from the horizontal and vertical video. Fig. 11 is a flowchart showing the processing in aspect 1, in which the orientation detection unit 240B determines whether the orientation of camera 10 (orientation relative to the direction of gravity) is "horizontal," "vertical," or "changing" (step S122: orientation detection step).

[0064] If the orientation is "landscape", the control unit 240C starts recording (saving) the video file (first video file) in the horizontal video folder (first folder) as a horizontal video (first video) (step S130). If the orientation is "portrait", the control unit 240C starts recording in the vertical video folder (second folder) as a vertical video (second video) (step S140). If the orientation is "changing", the control unit 240C starts recording in the changing video folder (third folder) as a changing video (third video) (step S124). In aspect 1, in addition to the horizontal video folder and vertical video folder, a changing video folder can be created in the hierarchy below the folder for each shooting number.

[0065] In aspect 1, if the direction detection unit 240B determines that "a change in the attitude of the camera 10 has started or completed" (Yes in step S172 (determination step)), the video is divided (the video file is closed and a new video file is created). At this time, since the attitude of the camera 10 may fluctuate slightly due to camera shake or other factors, the direction detection unit 240B determines that "a change in attitude has started" if the angle around the X axis has changed by more than a predetermined value in a certain period of time. Similarly, it is preferable that the direction detection unit 240B determines that "a change in attitude has completed" if the angle fluctuation remains within a predetermined range for more than a certain period of time. For example, in the example shown in FIG. 12, the angle changes significantly between elapsed times t1 and t2, and the angle stabilizes between elapsed times t3 and t4.

[0066] Therefore, the direction detection unit 240B determines that "the posture is changing" from elapsed time t2 to t4, and determines that "the posture change is complete" at elapsed time t4 (taking into account the time lag between the actual posture of camera 10 and the posture determination result change). In response to this determination, the control unit 240C generates a moving image during change (third moving image) that is located between the horizontal moving image (first moving image; from elapsed time zero to t2) and the vertical moving image (second moving image; after elapsed time t4) in time from elapsed time t2 to t4. As shown in FIG. 12, the moving image during change (third moving image) includes moving images of the first period when the posture of camera 10 is changing (when a direction change is occurring).

[0067] (Mode 2: During posture changes, recording continues in the video file of the previous period) In the above-described embodiment 1, the period during which the posture is changing (first period) is recorded as an independent video (third video). However, in embodiment 2, the period during which the posture is changing is continuously recorded in a video file for the period before the change began. Specifically, as shown in FIG. 13 , in embodiment 2, the orientation detection unit 240B detects whether the camera 10 (image capture device) is in the first state (a state in which the bottom surface of the camera 10 is aligned with the horizontal direction; landscape state), the second state (a state in which the bottom surface of the camera 10 is aligned with the direction of gravity; portrait state), or the third state (a state changing between the first state and the second state). Then, the control unit 240C records the video during the period in which the camera 10 is in the third state (posture is changing) in the first folder (a folder for landscape video) by including it in the first video.

[0068] (Mode 3: A mode in which a part of the period after the posture change has ended is continuously recorded in a video file of the period during the change) When the attitude of the camera 10 changes, the composition on the screen of the monitor 270 may change significantly. In this case, the camera 10 (image capture device) may need to take a period to adjust the exposure (AE: Auto Exposure) of the video, the white balance (AWB: Auto White Balance) of the video, or the focus (AF: Auto Focus) of the subject in the video. Therefore, in aspect 3, a predetermined period (second period) after the end of the attitude change is treated in the same way as when the attitude of the camera 10 is changing, even if the attitude of the camera 10 is stable. Specifically, as shown in FIG. 14, for example, the direction detection unit 240B treats the period from elapsed time t2 to t4 (first period) and also the period from elapsed time t4 to t5 (second period) as "the attitude of the camera 10 is changing."

[0069] The first and second periods are then recorded (saved) as a third video. That is, the third video also includes the video of the second period after the orientation change has ended. The control unit 240C can divide the video and record the third video as a video file (third video file) separate from the horizontal video (first video) and vertical video (second video) in a folder for horizontal videos (first folder). However, the third video may also be recorded in an independent folder for videos during changes (third folder). Images taken during posture changes or periods when the shooting conditions are unstable may be less important. However, in aspect 3, such periods are saved as a video separate from the horizontal and vertical videos, making it easy to select frames to extract as still images and organize video files and folders.

[0070] (Mode 4: A mode in which part of the period after the posture change is completed is continuously recorded in the original video file) In the above-described aspect 3, the period during which the attitude of the camera 10 is changing (first period) and the period determined after the attitude change has ended (second period) are referred to as the "third video." However, in aspect 4, as shown in Fig. 15, the first period (from elapsed time t2 to t4) and the second period (from elapsed time t4 to t5) are recorded as the "first video" in the horizontal video folder (first folder).

[0071] (Mode 5: Video is not recorded during posture changes) In the above-described modes 1 to 4, video is recorded during the period when the posture is changing (and the period thereafter until the posture stabilizes), but a mode in which video is not recorded during the posture change can also be adopted. Specifically, as shown in the flowchart of Fig. 16, the orientation detection unit 240B detects whether the posture of the camera 10 is horizontal (first state), vertical (second state), or changing (third state) (step S122). If the posture is changing, recording of a video file is not started (return to step S122).

[0072] Furthermore, if the camera 10 starts changing its orientation (Yes in step S174 (determination step)) after starting video recording when the camera 10 is in landscape or portrait orientation (steps S130 and S140), the control unit 240C closes the video file in step S180 and returns to step S122. The camera 10 then resumes video recording after the camera 10 returns to landscape or portrait orientation (steps S130 and S140). The closed video file is saved in the corresponding folder. Images taken during a change in orientation may be of low importance, but in aspect 5, videos are not saved during such periods, thereby conserving the remaining capacity of the recording media. The user can also easily select frames to extract as still images and organize video files and folders.

[0073] (Mode 6: Video is not split immediately after an event occurs, even if there is a posture change) Mode 6 shown in the flowchart of FIG. 17 is another mode for when the change in attitude of camera 10 has started or completed (Yes in step S172 (determination step)). In this mode, if a specific event has occurred between the start or completion of the change in attitude of camera 10 and a predetermined time (described as "T seconds ago" in step S176), control unit 240C does not close the video file (Yes in step S176, and the process does not proceed to step S180). Therefore, in this case, the video is not divided. The same applies not only to cases where a specific event has occurred within a certain time ago, but also to cases where a specific event is being detected (the event is ongoing) and cases where the reception unit is currently receiving a user instruction (including cases where a user instruction or processing based on a user instruction is ongoing).

[0074] The control unit 240C can set the value of T (for example, 30 seconds, but other values ​​are also acceptable) in response to user operation via the operation unit 250 or independently. The event detection unit 240D (see FIG. 2) may automatically detect changes in the amount of movement, size, shape, or brightness of the subject using known image processing to determine whether a specific event has occurred. Alternatively, the event detection unit 240D may determine whether a specific event has occurred based on a user instruction via the operation unit 250 or the reception unit (including when the monitor 270 is used as a touch panel). For example, the control unit 240C may determine that a "(specific) event has occurred" when the reception unit receives an instruction to capture a still image while the user is capturing a video. When an event is detected, the event detection unit 240D and the control unit 240C preferably add information indicating the event detection timing to the video file (for example, record it in the header or footer). By referencing this information, the user can easily select a frame to extract as a still image (as described below).

[0075] As described above, even if there is a change in the attitude of the camera 10, by not dividing the video if an event occurred before the attitude change, the user can check the series of events related to the event in a single video and can easily select a desired frame from the captured video. Note that although the flowchart in Fig. 17 is based on Fig. 11 (in which the video during a change in attitude is treated as a separate video from the horizontal video and vertical video and is saved in a separate folder), it may also be based on the flowcharts in Figs. 5 and 15.

[0076] In camera 10, the manner in which the video is divided and recorded (saved) may be determined in accordance with a user's operation via operation unit 250, including manners 1 to 6 described above.

[0077] <Processing when considering the first and second modes> While the processing in the first moving image mode has been described in the flowcharts shown in Figures 5, 11, 16, and 17, processing in consideration of the first moving image mode and the second moving image mode will now be described with reference to Figures 18 and 19. Note that the same steps as those in Figures 5, 11, 16, and 17 are given the same step numbers, and detailed descriptions thereof will be omitted.

[0078] The control unit 240C determines whether the camera 10 is in the first moving image mode (moving image mode for extracting still images) or the second moving image mode (normal moving image mode) (step S112). If the camera 10 is in the first moving image mode, the control unit 240C sets shooting conditions suitable for the first moving image mode (step S114). As described above in the "Shooting Mode" section, the shooting conditions set in step S114 are shooting conditions that prioritize extracting still images over viewing the moving image itself. The processing from step S114 onwards in the first moving image mode is the same as that in the flowchart of FIG. 5, and therefore a detailed description thereof will be omitted.

[0079] On the other hand, if the result of the determination in step S112 is the second movie mode, control unit 240C sets shooting conditions suitable for the second movie mode (step S191 in FIG. 19). Orientation detection unit 240B determines the attitude of camera 10 (step S192), and control unit 240C starts recording (saving) a movie file in a second movie folder in storage unit 260 (step S193: movie saving step). The processes of steps S194 and S195 are the same as the processes of steps S150 and S160 for the first movie mode. If an instruction to end movie recording has been issued (Yes in step S194) or if the remaining capacity of the recording medium is insufficient (No in step S195), control unit 240C closes the movie file (step S197) and returns to step S190 in FIG. 18 (determining whether movie shooting has ended).

[0080] If there is no instruction to end video recording (No in step S194) and the remaining capacity of the recording medium is sufficient (Yes in step S195), control unit 240C continues recording the video. However, unlike in the first video mode, in the second video mode, the direction detection unit 240B does not divide the video even if it does not detect a change in the attitude (change in direction) of camera 10 (No in step S196) or if it does detect a change in the attitude of camera 10 (Yes in step S196). The reason why the video is not divided in the second video mode even if the attitude of camera 10 changes is because the second video mode is not a mode for shooting video with the aim of extracting still images, and there is no need to divide the video to make it easier to find specific scenes as in the first video mode.

[0081] <Example of folder structure> When considering the first and second video modes, the control unit 240C can create a folder for the first video mode and a folder for the second mode within a folder for each shooting date in the storage unit 260 (storage unit). In this case, the control unit 240C creates a folder for each shooting number within the folder for the first video mode, and within each folder for each shooting number, creates a folder for horizontal video (example of the first folder) and a folder for vertical video (example of the second folder), and saves the horizontal video (example of the first video) and the vertical video (example of the second video), respectively. On the other hand, for the folder for the second video mode, the control unit 240C records video files shot on that shooting date in a folder for each shooting date. Furthermore, instead of creating folders for the first and second video modes below a folder for the shooting date, the control unit 240C may place the folders for the first and second video modes at a higher level and create a folder for the shooting date below them. This folder configuration can be displayed on the monitor 270 by the control unit 240C, as in the case of FIG. 7.

[0082] <Extracting still images> As will be explained in the following example, the camera 10 can extract frames constituting a moving image as still images. Note that still images may also be extracted using another device such as a personal computer using a moving image file recorded by the camera 10.

[0083] (Example 1) The control unit 240C plays and displays on the monitor 270 a file selected by the user from among the moving image files of horizontal moving images, vertical moving images, or changing moving images (first moving image, second moving image, third moving image). Instead of continuous playback, frame-by-frame playback (displaying one frame at a time in response to a user operation) may be used. The control unit 240C stops playback in response to a user operation, and by confirming the selection of the frame displayed at the time of stopping in response to the user operation, the still image extraction unit 240E extracts that frame as a still image.

[0084] (Example 2) If information indicating the detection of an event is added to the video file, control unit 240C may refer to that information and display multiple frames, including the frame in which the event was detected, on monitor 270 (see FIG. 1). FIG. 20 shows an example of such a display, displaying five frames 1051 to 1055, including frame 1052. Furthermore, for frame 1052 in which a still image has been recorded (recording of a still image is an example of the occurrence of an event), still image extraction unit 240E has added a camera-shaped icon 1060. This allows the user to understand that this is a frame in which a still image has been recorded.

[0085] The range of frames for extracting still images to be displayed may include frames that are temporally earlier (frame 1051) and later (frames 1053-1055) than the "frame corresponding to the recorded still image file" (frame 1052 in the example of FIG. 20). The range of frames for extracting still images to be displayed may include only frames that are temporally earlier or only frames that are temporally later. The range of frames for extracting still images to be displayed may be set according to user designation, or may be set independently. The range of frames to be displayed may be specified by time (e.g., one second before and one second after the frame in which the still image was recorded) or by the number of frames (e.g., ten frames before and ten frames after the frame in which the still image was recorded). Alternatively, only frames recorded as still images may be displayed in a list in the initial state, and frames temporally earlier and / or later than a selected frame may be displayed.

[0086] <Select the frame to extract> The user can select a frame from the displayed frames for extracting still images. In the example of FIG. 20 , the user can select a frame by checking the check box 1070 for the desired frame via the operation unit 250 and / or monitor 270. Multiple frames can also be selected. Note that the frame in which the still image specified by the user was recorded may not necessarily have been recorded at the best timing. Displaying other related frames together may encourage the user to select another frame. Therefore, it is preferable to display the frame in which the still image was recorded on the monitor 270 so that the user can select from frames before and / or after the frame. FIG. 20 shows a state in which the user has selected frame 1053, which is different from the frame in which the still image was recorded (frame 1052 marked with icon 1060).

[0087] The still image extraction unit 240E extracts the selected frame as a still image (still image extraction step). Since the moving image file is stored in a moving image format such as MPEG format, the still image extraction unit 240E converts the data of the selected frame into a still image format (JPEG format, etc.). The user can extract a still image at a desired timing from the moving images (first moving image, second moving image, third moving image).

[0088] <Second embodiment> In the first embodiment, the camera 10 is a digital camera, but the configuration of the imaging device is not limited to this. Other imaging devices of the present invention include, for example, a built-in or external PC camera (PC: Personal Computer), or The device may be a mobile terminal device with a photographing function, as described below.

[0089] Examples of portable terminal devices that are embodiments of the imaging device of the present invention include mobile phones, smartphones, PDAs (Personal Digital Assistants), and portable game consoles. A smartphone will be used as an example and described in detail below with reference to the drawings.

[0090] FIG. 21 illustrates the appearance of a smartphone 1 (imaging device) according to an embodiment of the present invention, with (a) being a front view and (b) being a rear view. The smartphone 1 illustrated in FIG. 21 has a flat housing 2, and on one side of the housing 2, a display panel 21 (display device) as a display unit and a display / input unit 20 integrated with an operation panel 22 (operation unit) as an input unit. The housing 2 also includes a speaker 31, a microphone 32, an operation unit 40 (operation unit), camera units 41 and 42 (imaging device, shooting unit, event detection unit, control unit, still image extraction unit, lens drive control unit), and a flash 43. The configuration of the housing 2 is not limited to this; for example, the display unit and the input unit may be independent of each other, or the housing 2 may have a foldable structure or a sliding mechanism.

[0091] Fig. 22 is a block diagram showing the configuration of the smartphone 1 shown in Fig. 21. As shown in Fig. 22, the smartphone 1 includes, as its main components, a wireless communication unit 11, a display / input unit 20, a call unit 30, an operation unit 40, camera units 41 and 42, a flash unit 43, a storage unit 50, an external input / output unit 60, a GPS receiver unit 70 (GPS: Global Positioning System), a motion sensor unit 80, a power supply unit 90, and a main control unit 101 (a photographing unit, a video file generating unit, a still image file generating unit, an event detecting unit, a display control unit, a still image extracting unit, and a lens driving control unit). The smartphone 1 also includes, as its main function, a wireless communication function for performing mobile wireless communication via a base station device and a mobile communication network.

[0092] The wireless communication unit 11 performs wireless communication with a base station device accommodated in the mobile communication network in accordance with instructions from the main control unit 101. Using such wireless communication, various file data such as audio data and image data, e-mail data, etc. are sent and received, and web data, streaming data, etc. are received.

[0093] The display input unit 20 is a so-called touch panel that, under the control of the main control unit 101, displays images (still images and / or moving images), text information, etc. to visually convey information to the user and detects user operations on the displayed information, and is equipped with a display panel 21 and an operation panel 22.

[0094] The display panel 21 uses an LCD (Liquid Crystal Display), an OLED (Organic Electro-Luminescence Display), or the like as a display device. The operation panel 22 is placed so that an image displayed on the display surface of the display panel 21 can be seen, and is a device that detects one or more coordinates operated by a conductor such as a user's finger or pen. When such a device is operated by a conductor such as a user's finger or pen, the operation panel 22 outputs a detection signal generated by the operation to the main control unit 101. Next, the main control unit 101 detects the operation position (coordinates) on the display panel 21 based on the received detection signal.

[0095] 21 , the display panel 21 and operation panel 22 of the smartphone 1 exemplified as an embodiment of the imaging device of the present invention are integrated to form the display input unit 20, with the operation panel 22 being arranged to completely cover the display panel 21. When such an arrangement is adopted, the operation panel 22 may also have a function to detect user operations in areas outside the display panel 21. In other words, the operation panel 22 may have a detection area for the overlapping portion that overlaps the display panel 21 (hereinafter referred to as the display area), and a detection area for the remaining outer edge portion that does not overlap the display panel 21 (hereinafter referred to as the non-display area).

[0096] The call unit 30 includes a speaker 31 and a microphone 32, and can convert the user's voice input through the microphone 32 into voice data that can be processed by the main control unit 101 and output it to the main control unit 101, and can decode voice data received by the wireless communication unit 11 or the external input / output unit 60 and output it from the speaker 31. Also, as shown in Fig. 21, for example, the speaker 31 can be mounted on the same surface as the display input unit 20, and the microphone 32 can be mounted on the side of the housing 2.

[0097] The operation unit 40 is a hardware key using a key switch or the like, and is a device that accepts instructions from a user. For example, as shown in Fig. 21, the operation unit 40 is a push-button switch that is mounted on the side of the housing 2 of the smartphone 1 and turns on when pressed with a finger or the like, and turns off when the finger is released due to the restoring force of a spring or the like.

[0098] The storage unit 50 stores the control program and control data of the main control unit 101, application software, address data associated with names and telephone numbers of communication partners, data of emails sent and received, web data downloaded via web browsing, downloaded content data, and also temporarily stores streaming data, etc. The storage unit 50 is composed of an internal storage unit 51 built into the smartphone and an external storage unit 52 with a removable external memory slot. The internal storage unit 51 and the external storage unit 52 constituting the storage unit 50 are realized using known storage media.

[0099] The external input / output unit 60 serves as an interface with all external devices connected to the smartphone 1. The smartphone 1 is directly or indirectly connected to other external devices via the external input / output unit 60 through communication or the like. Examples of communication or the like include a universal serial bus (USB), IEEE 1394, and a network (e.g., the Internet, a wireless LAN). Other examples of communication or the like include Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra WideBand) (registered trademark), and ZigBee (registered trademark).

[0100] Examples of external devices connected to the smartphone 1 include a wired / wireless headset, a wired / wireless external charger, a memory card connected via a wired / wireless data port or a card socket, and a Subscriber Identity Module (SIM) / User Identity Module (UIM) card. Other external devices that can be connected include external audio and video devices connected via an audio and video I / O (Input / Output) terminal, wirelessly connected external audio and video devices, a wired / wirelessly connected smartphone, a wired / wireless connected PDA, a wired / wireless connected personal computer, and earphones. The external input / output unit 60 can transfer data received from such external devices to various components within the smartphone 1 and can transfer data within the smartphone 1 to external devices.

[0101] The motion sensor unit 80 includes, for example, a three-axis acceleration sensor and an inclination sensor, and detects the physical movement of the smartphone 1 according to instructions from the main control unit 101. By detecting the physical movement of the smartphone 1, the direction of movement, acceleration, and attitude of the smartphone 1 are detected. These detection results are output to the main control unit 101. The power supply unit 90 supplies power stored in a battery (not shown) to each unit of the smartphone 1 according to instructions from the main control unit 101.

[0102] The main control unit 101 includes a microprocessor, operates according to control programs and control data stored in the storage unit 50, and controls all the components of the smartphone 1, including the camera unit 41. The main control unit 101 also includes a mobile communication control function that controls all the components of the communication system to perform voice communication and data communication via the wireless communication unit 11, and an application processing function.

[0103] The main control unit 101 also has an image processing function for displaying video on the display input unit 20 based on image data (still image or moving image data) such as received data or downloaded streaming data. The image processing function refers to a function in which the main control unit 101 decodes image data, performs image processing on the decoded result, and displays the image on the display input unit 20.

[0104] The camera units 41 and 42 are digital cameras (imaging devices) that capture images electronically using imaging elements such as CMOS or CCD. Under the control of the main control unit 101, the camera units 41 and 42 can convert captured image data (video and still images) into compressed image data, such as MPEG or JPEG, and store the image data in the storage unit 50 or output the image data via the external input / output unit 60 or wireless communication unit 11. Under the control of the main control unit 101, the camera unit 41 can also split and combine video, acquire high-quality still images (RAW images, etc.), swap and process frames, and extract still images from video. In the smartphone 1 shown in FIGS. 21 and 22, one of the camera units 41 and 42 can be used to capture images, or the camera units 41 and 42 can be used simultaneously. When using the camera unit 42, a flash 43 can be used.

[0105] The camera units 41 and 42 can be used for various functions of the smartphone 1. For example, the smartphone 1 can display images acquired by the camera units 41 and 42 on the display panel 21. The smartphone 1 can also use images from the camera units 41 and 42 as one of the operation inputs to the operation panel 22. The smartphone 1 can also detect its location by referring to images from the camera units 41 and 42 when the GPS receiver 70 detects its location. Furthermore, the smartphone 1 can determine the optical axis direction of the camera unit 41 of the smartphone 1 or the current usage environment by referring to images from the camera units 41 and 42, without using a triaxial acceleration sensor or by using the triaxial acceleration sensor in combination. Of course, the smartphone 1 can also use images from the camera units 41 and 42 in application software. Additionally, the smartphone 1 can add location information acquired by the GPS receiving unit 70, voice information acquired by the microphone 32 (which may be converted to text information by the main control unit or the like), posture information acquired by the motion sensor unit 80, and the like to image data of a still image or video, and record the added data in the storage unit 50. The smartphone 1 can also output the image data of the still image or video through the external input / output unit 60 or the wireless communication unit 11.

[0106] <Smartphone posture> Part (a) of FIG. 23 is a diagram showing an example of a coordinate system defined for the smartphone 1, and in this example, the attitude is determined by the angle around the X-axis. Part (a) shows a portrait state at an angle of 0° (second state in which the smartphone 1 (imaging device) is aligned with the direction of gravity), and part (b) of the same figure shows a landscape state at an angle of 90° (first state in which the smartphone 1 (imaging device) is aligned with the horizontal direction). FIG. 24 is a diagram showing an example of the relationship between the angle around the X-axis and the attitude (portrait or landscape) of the smartphone 1. Note that, as in the case of the camera 10 according to the first embodiment (see FIG. 4), the attitude and state (first state, second state, third state) may be defined using the landscape state (the state shown in part (b) of FIG. 23) as the reference (angle 0°).

[0107] The smartphone 1 having the above-described configuration can also execute the processes of the imaging method according to the present invention (shooting, dividing, and recording video, recording still images, extracting still images, etc.) in the same way as the camera 10 according to the first embodiment. Specifically, the processes executed by the image processing device 240 (each unit shown in FIG. 2 ) in the first embodiment (including the processes of aspects 1 to 6) can be executed by the camera units 41 and 42 and the main control unit 101 in the smartphone 1. In addition, the functions of the operation unit 250, the storage unit 260, and the monitor 270 in the first embodiment can be realized by the operation unit 40, the storage unit 50, and the operation panel 22, and the display panel 21 and the operation panel 22, respectively, in the smartphone 1.

[0108] As a result, the smartphone 1 according to the second embodiment can also achieve the same effects as the camera 10 according to the first embodiment (such as being able to easily select frames to extract as still images and organize video files and folders).

[0109] Although the embodiments and other aspects of the present invention have been described above, the present invention is not limited to the above-described embodiments and aspects, and various modifications are possible within the scope of the spirit of the present invention. [Explanation of symbols]

[0110] 1. Smartphone 2. Case 10 Camera 11. Wireless Communication Section 20 Display and input section 21 Display panel 22 Operation panel 30 Telephone section 31 Speaker 32 microphones 40 Control section 41 Camera Department 42 Camera Department 43 Strobe 50 Storage section 51 Internal storage 52 External memory unit 60 External input / output section 70 GPS receiver 80 Motion sensor unit 90 Power supply section 100 interchangeable lenses 101 Main control unit 110 Zoom Lens 120 focus lens 130 aperture 140 Lens drive unit 200 Imaging device body 210 Image sensor 220 AFE 230 A / D converter 240 Image Processing Device 240A Image acquisition unit 240B Direction detection unit 240C Control Unit 240D Event detection unit 240E Still image extraction section 240F Lens drive control unit 242 ROM 250 Operation section 260 Storage section 270 monitors 280 Attitude Sensor 1051 frames 1052 frames 1053 frames 1054 frames 1055 frames 1060 Icons 1070 Checkbox L optical axis S100~S197 Each step of the imaging method ST1 GPS satellite STn GPS satellite

Claims

1. An imaging device, a direction detection unit that detects a direction of the imaging device relative to a direction of gravity; A filming department that shoots videos, A control unit; Equipped with the control unit has a first video mode capable of generating a video file of the video captured by the imaging unit and extracting a still image from the video file; the control unit, in the first video mode, when the direction detection unit determines that a change in direction of the imaging device with respect to a direction of gravity has occurred, divides the video to generate a first video and a second video; The control unit storing a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; the first video is earlier in time than the second video; the direction detection unit detects whether the imaging device is in a first state in which it is aligned with a horizontal direction, a second state in which it is aligned with a direction of gravity, or a third state in which the imaging device is changing between the first state and the second state; the first moving image includes a moving image captured during a period in which the imaging device is in the third state; The direction detection unit determining that the imaging device is in a horizontal position as the first state when an angle of the imaging device around the X axis is in a range of −45 degrees or more and +45 degrees or less, or in a range of +135 degrees or more and +225 degrees or less, where the optical axis direction of the imaging device is an X axis of the imaging device; determining that the imaging device is in the vertical position as the second state when the angle of the imaging device around the X axis is not within a range of −45 degrees or more and +45 degrees or less, or not within a range of +135 degrees or more and +225 degrees or less; the angle of the imaging device around the X-axis is 0° when the top and bottom surfaces of the body of the imaging device are parallel to the ground and the top surface is vertically above the bottom surface; Imaging device.

2. The direction detection unit determining that the imaging device is in the horizontal position when the angle of the imaging device about the X axis is in the range of −30 degrees or more and +30 degrees or less, and in the range of +150 degrees or more and +210 degrees or less; 2. The imaging device according to claim 1, wherein the imaging device is determined to be in the portrait position when the angle of the imaging device about the X axis is not in the range of −30 degrees or more and +30 degrees or less, or not in the range of +150 degrees or more and +210 degrees or less.

3. An imaging device, a direction detection unit that detects a direction of the imaging device relative to a direction of gravity; A filming department that shoots videos, A control unit; Equipped with the control unit has a first video mode capable of generating a video file of the video captured by the imaging unit and extracting a still image from the video file; the control unit, in the first video mode, when the direction detection unit determines that a change in direction of the imaging device with respect to a direction of gravity has occurred, divides the video to generate a first video and a second video; The control unit storing a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; the first video is earlier in time than the second video; The imaging device, wherein the first video also includes a video of a second period after the direction change has ended.

4. The imaging device described in claim 3, wherein the second period is a period during which the imaging device adjusts the exposure of the video, the white balance of the video, or the focus of a subject in the video.

5. An imaging device as described in any one of claims 1 to 4, wherein the control unit provides the first video file and the second video file with information regarding the orientation of the imaging device relative to the direction of gravity.

6. The control unit has a reception unit that receives instructions from a user, The imaging device according to claim 1 , wherein the timing to end the second moving image corresponds to the acceptance by the accepting unit of an instruction to end the second moving image or an instruction to end moving image shooting from the user.

7. The control unit includes a reception unit that receives user instructions or an event detection unit that detects a specific event, The imaging device according to claim 6, wherein the control unit does not divide the video if the direction detection unit detects the change in direction while the reception unit is receiving a user instruction or while the event detection unit is detecting a specific event.

8. The control unit has a second video mode having different shooting conditions from the first video mode, The imaging device according to claim 1 , wherein the control unit does not divide the moving image when the direction detection unit detects the change in direction in the second moving image mode.

9. An imaging device as described in Claim 8, wherein the first video mode has at least one of the shutter speed, autofocus speed, autoexposure tracking speed, and white balance tracking speed set to a higher speed than the second video mode, and / or the frame rate set to a higher speed than the second video mode.

10. An imaging device described in any one of claims 1 to 9, wherein the first video file and the second video file have at least some parts in common except for the file name extension.

11. An imaging device described in any one of claims 1 to 10, wherein the control unit stores the first video file in a first folder of the memory unit and stores the second video file in a second folder of the memory unit.

12. A direction detection unit that detects the direction of the imaging device relative to the direction of gravity, and a shooting unit that shoots a video, a control unit having a first video mode that is capable of generating a video file of a video captured by the shooting unit and extracting a still image from the video file, a moving image generating step in which the control unit divides the moving image to generate a first moving image and a second moving image when the direction detection unit determines that a change in direction of the imaging device with respect to a gravity direction has occurred in the first moving image mode; a moving image saving step in which the control unit saves a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; a detection step in which the direction detection unit detects whether the imaging device is in a first state in which the imaging device is aligned with a horizontal direction, a second state in which the imaging device is aligned with a direction of gravity, or a third state in which the imaging device is changing between the first state and the second state; and the first video is earlier in time than the second video; the first moving image includes a moving image captured during a period in which the imaging device is in the third state; In the detecting step, the direction detecting unit determining that the imaging device is in a horizontal position as the first state when an angle of the imaging device around the X axis is in a range of −45 degrees or more and +45 degrees or less, or in a range of +135 degrees or more and +225 degrees or less, where the optical axis direction of the imaging device is an X axis of the imaging device; determining that the imaging device is in the vertical position as the second state when the angle of the imaging device around the X axis is not within a range of −45 degrees or more and +45 degrees or less, or not within a range of +135 degrees or more and +225 degrees or less; the angle of the imaging device around the X-axis is 0° when the top and bottom surfaces of the body of the imaging device are parallel to the ground and the top surface is vertically above the bottom surface; Imaging method.

13. The direction detection unit determining that the imaging device is in the horizontal position when the angle of the imaging device about the X axis is in the range of −30 degrees or more and +30 degrees or less, and in the range of +150 degrees or more and +210 degrees or less; 13. The imaging method according to claim 12, wherein the imaging device is determined to be in the portrait position when the angle around the X axis of the imaging device is not in the range of −30 degrees or more and +30 degrees or less, or not in the range of +150 degrees or more and +210 degrees or less.

14. A direction detection unit that detects the direction of the imaging device relative to the direction of gravity, and a shooting unit that shoots a video, a control unit having a first video mode that is capable of generating a video file of a video captured by the shooting unit and extracting a still image from the video file, a moving image generating step in which the control unit divides the moving image to generate a first moving image and a second moving image when the direction detection unit determines that a change in direction of the imaging device with respect to a gravity direction has occurred in the first moving image mode; a moving image saving step in which the control unit saves a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; and the first video is earlier in time than the second video; An imaging method in which the first video also includes a video of a second period after the direction change has ended.

15. A program for operating an imaging device comprising: a direction detection unit that detects the direction of the imaging device relative to the direction of gravity; a shooting unit that shoots video; and a control unit having a first video mode that generates a video file of the video shot by the shooting unit and is capable of extracting still images from the video file, causing the control unit to divide a moving image to generate a first moving image and a second moving image when the direction detection unit determines that a change in direction of the imaging device with respect to a gravity direction has occurred in the first moving image mode; causing the control unit to store a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; causing the direction detection unit to detect whether the imaging device is in a first state in which it is aligned with a horizontal direction, a second state in which it is aligned with a direction of gravity, or a third state in which the imaging device is changing between the first state and the second state; the first video is earlier in time than the second video; the first moving image includes a moving image captured during a period in which the imaging device is in the third state; The direction detection unit determining that the imaging device is in a horizontal position as the first state when an angle of the imaging device around the X axis is in a range of −45 degrees or more and +45 degrees or less, and in a range of +135 degrees or more and +225 degrees or less, where the optical axis direction of the imaging device is an X axis of the imaging device; determining that the imaging device is in a vertical position as the second state when the angle of the imaging device around the X axis is not within a range of −45 degrees or more and +45 degrees or less, or not within a range of +135 degrees or more and +225 degrees or less; the angle of the imaging device around the X-axis is 0° when the top and bottom surfaces of the body of the imaging device are parallel to the ground and the top surface is vertically above the bottom surface; program.

16. The direction detection unit, determining that the imaging device is in the horizontal position when the angle of the imaging device around the X axis is in the range of −30 degrees or more and +30 degrees or less, and in the range of +150 degrees or more and +210 degrees or less; 16. The program according to claim 15, wherein the image capturing device is determined to be in the portrait position when the angle of the image capturing device about the X axis is not in the range of −30 degrees or more and +30 degrees or less, or not in the range of +150 degrees or more and +210 degrees or less.

17. A direction detection unit that detects the direction of the imaging device relative to the direction of gravity, and a shooting unit that shoots a video, a control unit having a first video mode that is capable of generating a video file of a video captured by the imaging unit and extracting a still image from the video file, causing the control unit to divide a moving image to generate a first moving image and a second moving image when the direction detection unit determines that a change in direction of the imaging device with respect to a gravity direction has occurred in the first moving image mode; causing the control unit to store a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit; the first video is earlier in time than the second video; The first video also includes a video of a second period after the direction change is completed.

Images