Electronic device, method for controlling electronic device, and program

Abstract

It is not possible to efficiently generate a single video file containing the video intended by the user. [Solution] The electronic device of the present invention comprises: an imaging means for capturing video; a generation means for generating a single video file from a plurality of videos captured by the imaging means; a detection means for detecting features of the video captured by the imaging means; a means for generating cumulative information indicating the cumulative number of a plurality of items related to the video, wherein when a feature corresponding to the plurality of items is detected by the detection means, the generation means adds the cumulative number of the item; and a notification means for notifying information about the item based on a surrounding environment image captured by the imaging means while waiting to capture, if the cumulative number of the item in the cumulative information generated by the generation means is less than a predetermined threshold.

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method for an electronic device, and a program.

Background Art

[0002] In recent years, there has been an increasing need to shoot videos with a digital camera and not only enjoy the captured videos but also have fun by editing them according to the user's preferences. One example of such editing is to combine multiple videos to generate a single video file. When generating a single video file, it is required that the video file contains the videos intended by the user.

[0003] Patent Document 1 discloses that when a photo is taken, in the captured normal shooting image data, the number of shots and elements of shooting tendencies (such as full body, smiling face, up, etc.) for each person are aggregated. Also, Patent Document 1 discloses that the aggregated number of shots for each person is displayed on a confirmation screen separate from the screen during shooting. Further, Cited Document 1 discloses assisting the next shooting by displaying a message indicating that the shooting of a person with fewer shots than other people is less, for that person.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, even if a message indicating that the shooting of a person with fewer shots than other people is less is displayed as in Patent Document 1, it is not always the case that the person can be shot.

[0006] Therefore, the present invention aims to efficiently generate a single video file containing the video intended by the user. [Means for solving the problem]

[0007] To achieve the above objective, the electronic device of the present invention is characterized by comprising: an imaging means for capturing video; a generation means for generating a single video file from a plurality of videos captured by the imaging means; a detection means for detecting features of the video captured by the imaging means; a means for generating cumulative information indicating the cumulative number of a plurality of items related to the video, wherein when a feature corresponding to the plurality of items is detected by the detection means, the generation means adds the cumulative number of said item; and a notification means for notifying information about the said item based on a surrounding environment image captured by the imaging means while waiting to capture, if the cumulative number of said item in the cumulative information generated by the generation means is less than a predetermined threshold. [Effects of the Invention]

[0008] According to the present invention, a single video file containing the video intended by the user is efficiently generated. [Brief explanation of the drawing]

[0009] [Figure 1] This is an external view of a digital camera 100 (imaging device) as an example of a device (electronic device) to which the present invention can be applied. [Figure 2] This block diagram shows the configuration of a digital camera 100, which is an example of a device (electronic device) to which the present invention can be applied. [Figure 3] This is an example of cumulative information 300 according to this embodiment. [Figure 4] This is an example of a shooting standby screen 400 displayed on the display unit 28 according to this embodiment. [Figure 5] This is an example of notifying a user of items that are available for capture when they are attempting to take a picture, but whose cumulative count is insufficient. [Figure 6]This is an example of a screen 600 displayed on the display unit 28 during the acquisition of ambient environment data according to this embodiment. [Figure 7] This flowchart explains the process of notifying the user of items that are available for capture when they are attempting to take a picture, but whose cumulative count is insufficient. [Modes for carrying out the invention]

[0010] Preferred embodiments of the present invention will be described below with reference to the attached drawings.

[0011] [External view of Digital Camera 100] Figures 1(a) and 1(b) are external views of a digital camera 100 (imaging device) as an example of an apparatus (electronic device) to which the present invention can be applied. Figure 1(a) is a front perspective view of the digital camera 100, and Figure 1(b) is a rear perspective view of the digital camera 100.

[0012] The digital camera 100 includes a gripping unit 90, an imaging unit 110, and a posture control unit 300. The imaging unit 110 is configured to change its orientation relative to the gripping unit 90. The user can hold the digital camera 100 towards a subject by gripping the gripping unit 90 with either their left or right hand. The posture control unit 300 includes a pan axis drive unit 301, a roll axis drive unit 302, and a tilt axis drive unit 303. The pan axis drive unit 301 is fixed to the gripping unit 90, and the tilt axis drive unit 303 is fixed to the imaging unit 110. By driving (rotating) at least one of the pan axis drive unit 301, the roll axis drive unit 302, and the tilt axis drive unit 303, the orientation of the imaging unit 110 relative to the gripping unit 90 can be maintained or changed. The gripping unit 90 and the posture control unit 300 may be separable, and the imaging unit 110 and the posture control unit 300 may be separable. If the imaging unit 110 and the attitude control unit 300 are separable, the imaging unit 110 may be a smartphone or the like. The attitude control unit 300 is such as an electric gimbal device, but a non-electric gimbal device may be used instead of the attitude control unit 300.

[0013] The display unit 33 displays images and various information. The touch panel 70a can detect touch operations on the display surface (touch operation surface) of the display unit 33. The shutter button 61 is an operating member for issuing a shooting command. The mode switching switch 60 is an operating member for switching between various modes. The power switch 72 is an operating member for switching the power of the digital camera 100 ON and OFF.

[0014] The controller wheel 73 is a rotatable operating member, used, for example, to indicate a selection item. When the controller wheel 73 is rotated, an electrical pulse signal is generated according to the amount of operation (amount of rotation), and various parts of the digital camera 100 are controlled based on this pulse signal. The angle and number of rotations of the rotation operation on the controller wheel 73 can be determined from this pulse signal. The controller wheel 73 can be any operating member that can detect rotation, for example, a dial operating member in which the controller wheel 73 itself rotates in response to rotation and generates a pulse signal. The controller wheel 73 can also be an operating member that includes a touch sensor (a so-called touch wheel), or the controller wheel 73 itself may not rotate, but may be an operating member that detects the rotational movement of the user's finger on the controller wheel 73.

[0015] The four-way key 74 is configured so that the up, down, left, and right sections can each be pressed, allowing for processing corresponding to the pressed section. The SET button 75 is a push button and is mainly used to confirm selection items. The video button 76 is a push button and is used to instruct the start and stop of video recording. The menu button 81 is a push button used to instruct the display of the menu screen, and when the menu button 81 is pressed, a menu screen with various settings is displayed on the display unit 33. The user can intuitively make various settings using the menu screen displayed on the display unit 33, the four-way key 74, and the SET button 75.

[0016] [Block Diagram of Digital Camera 100] FIG. 2 is a block diagram showing the configuration of digital camera 100.

[0017] The photographing lens 103 is a lens group including a zoom lens and a focus lens. The shutter 101 is a shutter having an aperture function. The imaging processing unit 22 is an image sensor composed of a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) element that converts an optical image into an electrical signal.

[0018] The A / D converter 23 converts the analog signal output from the imaging processing unit 22 into a digital signal. The barrier 102 covers the imaging system including the photographing lens 103, the shutter 101, and the imaging processing unit 22 of the digital camera 100 to prevent dirt and damage to the imaging system.

[0019] The image processing unit 24 performs predetermined processing (such as resizing processing like pixel interpolation and reduction, and color conversion processing) on the data from the A / D converter 23 or the data from the memory control unit 15. Further, the image processing unit 24 performs predetermined arithmetic processing using the captured image data (video), and the system control unit 50 performs exposure control and distance measurement control based on the arithmetic result obtained by the image processing unit 24. Thereby, TTL (Through-the-Lens) type AF (Auto Focus) processing, AE (Automatic Exposure) processing, and EF (Flash Pre-Firing) processing are performed. The image processing unit 24 further performs predetermined arithmetic processing using the captured image data (video) and performs TTL type AWB (Auto White Balance) processing based on the obtained arithmetic result.

[0020] The feature detection unit 25 detects features (categories) such as subject, shooting distance, and camera work from the image data (video) generated by the image processing unit 24. For example, the subject is detected by comparing data learned based on deep learning with image data processed by the image processing unit 24. Examples of subjects to be detected include people, animals (dogs, cats, birds, horses, etc.), vehicles (cars, motorcycles, trains, etc.), food / small objects, landscapes, and buildings. Furthermore, subjects may be detected using training data that allows for more detailed identification of individuals or entities within the categories of people or animals. Note that subject detection based on machine learning is just one example, and subjects may be detected by other methods.

[0021] Furthermore, the shooting distance refers to the distance between the subject and the digital camera 100. For example, the shooting distance is detected based on distance measurement or deep learning. If the distance between the subject and the digital camera 100 is small, it is assumed that the shooting distance is short, the subject is large, and the subject is a close-up of the face. If the distance between the subject and the digital camera 100 is medium, it is assumed that the shooting distance is medium, the subject is of medium size, and the subject is a bust-up shot. And if the distance between the subject and the digital camera 100 is large, it is assumed that the shooting distance is long, the subject is small, and the subject is a full-body shot.

[0022] Furthermore, camera movement is detected based on information regarding the subject's movement detected by the feature detection unit 25 and information regarding the gripping unit 90's movement detected by the attitude detection unit 55, which will be described later. Camera movement includes Fix (fixed), Pan (horizontal), Tilt-up (vertical), Tilt-down (vertical), etc. Other camera movement options include Follow (forward), Lead (backward), Dolly (translational), Push-in (forward), Pull-out (backward), and Circle (rotation).

[0023] The generation unit 26 generates cumulative information showing the cumulative count of multiple items related to the image data (video). First, the image processing unit 24 classifies the image data (video) generated based on the detection results by the feature detection unit 25. Figure 3 shows that each category (camera work, shooting distance (size), subject) has one or more items. When the feature detection unit 25 detects features corresponding to each item in the captured video, it adds the cumulative count of each item and generates cumulative information 300 showing the cumulative count of each item. Then, the cumulative information 300 showing the cumulative count of each item generated for each item in each category is stored in the memory 32. Each time a video is shot, the generation unit 26 sets each of the multiple features detected in the video by the feature detection unit 25 to 1 and adds it to the cumulative count of each item in the cumulative information 300. In this way, the items are classified into each category and stored as cumulative information 300. Based on this cumulative information 300, the generation unit 26 determines which items have insufficient cumulative counts. Furthermore, the generation unit 26 associates the image data (video) with items and stores them in the memory 32. Categories and items will be described later using Figure 3.

[0024] The video compression unit 27 compresses and encodes multiple image data (videos) generated in the image processing unit 24 according to the MPEG compression encoding standard, and generates a video file in MPEG format.

[0025] The memory control unit 15 controls the transmission and reception of data between the A / D converter 23, the image processing unit 24, and the memory 32. Output data from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15. Alternatively, output data from the A / D converter 23 is written to the memory 32 via the memory control unit 15 without going through the image processing unit 24. The memory 32 stores image data obtained by the imaging processing unit 22 and converted into digital data by the A / D converter 23, as well as image data for display on the display unit 33. The memory 32 has sufficient storage capacity to store a predetermined number of still images, a predetermined amount of video footage, and audio. The memory 32 also serves as a memory for image display (video memory). The D / A converter 13 converts the display image data stored in the memory 32 into analog signals and supplies them to the display unit 33. In this way, the display image data written to the memory 32 is displayed by the display unit 33 via the D / A converter 13.

[0026] The display unit 33 is a display such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence), and displays information according to the analog signal from the D / A converter 13. The digital signal, which has been converted from digital to digital by the A / D converter 23 and stored in the memory 32, is converted back to analog by the D / A converter 13 and sequentially transferred to the display unit 33 for display, thereby enabling live view display (LV display). Hereinafter, the image displayed in live view display will be referred to as the live view image (LV image).

[0027] The non-volatile memory 56 is a memory recording medium that can be electrically erased and recorded, such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). The non-volatile memory 56 stores constants and programs for the operation of the system control unit 50. The program referred to here is a computer program for executing various flowcharts, which will be described later.

[0028] The system control unit 50 is a control unit consisting of at least one processor and / or at least one circuit, and controls the entire digital camera 100. The system control unit 50 implements the processes described later by executing programs stored in the non-volatile memory 56. The system memory 52 is, for example, RAM, and the system control unit 50 loads constants, variables, and programs read from the non-volatile memory 56 into the system memory 52 for the operation of the system control unit 50. The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 13, the display unit 33, etc.

[0029] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of the built-in clock.

[0030] The communication unit 54 transmits and receives video and audio signals to and from external devices connected wirelessly or via wired cables. The communication unit 54 can also connect to a wireless LAN (Local Area Network) or the internet. Furthermore, the communication unit 54 can communicate with external devices using Bluetooth® or Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging processing unit 22 and images recorded on the recording medium 200, and can receive image data and other various information from external devices.

[0031] The attitude detection unit 55 detects the orientation of the gripping unit 90 relative to the direction of gravity. Based on the orientation detected by the attitude detection unit 55, the imaging processing unit 22 can determine whether the image captured was taken with the gripping unit 90 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the attitude detection unit 55 to the image file of the image captured by the imaging processing unit 22, or rotate the image before recording. Furthermore, the system control unit 50 can maintain the orientation of the imaging unit 110 relative to the direction of gravity by operating the attitude control unit 300 based on the orientation change information detected by the attitude detection unit 55. The system control unit 50 can also change the orientation of the imaging unit 110 while suppressing rapid changes in the orientation of the imaging unit 110. An acceleration sensor or a gyro sensor can be used as the attitude detection unit 55. It is also possible to detect the movement of the gripping unit 90 (pan, tilt, roll, lift, whether it is stationary or not, etc.) using the acceleration sensor or gyro sensor in the attitude detection unit 55.

[0032] The power control unit 80 consists of a battery detection circuit, a DC-DC converter, a switch circuit for switching which blocks are energized, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, supplying the necessary voltage to each part, including the recording medium 200, for the required period of time. The power supply unit 31 consists of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries and Li batteries, and an AC adapter.

[0033] The recording medium I / F18 is an interface to the recording medium 200, such as a memory card or hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of semiconductor memory, optical disks, magnetic disks, etc.

[0034] The operation unit 70 is an input unit that receives user input (user operation) and is used to input various operation instructions to the system control unit 50. Each operation component of the operation unit 70 is assigned a function as appropriate for each situation by selecting various function icons displayed on the display unit 33, and acts as various function buttons. Examples of function buttons include an exit button, a back button, an image advance button, a jump button, a filter button, and an attribute change button.

[0035] As shown in Figure 2, the operation unit 70 includes a mode switching switch 60, a shutter button 61, a power switch 72, a video button 76, a touch panel 70a, and other operating components 70b. The other operating components 70b include a controller wheel 73, a four-way key 74, a SET button 75, a menu button 81, and the like.

[0036] The mode switch 60 switches the operating mode of the system control unit 50 to one of the following: still image recording mode, video recording mode, playback mode, etc. Modes included in the still image recording mode include auto shooting mode, auto scene detection mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide shooting settings for different shooting scenes. The user can switch directly to any of these modes using the mode switch 60. Alternatively, the user can switch to a list screen of shooting modes using the mode switch 60, and then selectively switch to one of the displayed modes using another operating element. Similarly, the video recording mode may also include multiple modes.

[0037] The shutter button 61 is equipped with a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 turns ON during the operation of the shutter button 61, so-called half-press (instruction to prepare for shooting), and generates a first shutter switch signal SW1. The system control unit 50 starts shooting preparation operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing in response to the first shutter switch signal SW1. The second shutter switch 64 turns ON when the operation of the shutter button 61 is completed, so-called full-press (instruction to shoot), and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting operations from reading the signal from the image processing unit 22 to writing the captured image as an image file to the recording medium 200 in response to the second shutter switch signal SW2.

[0038] The power switch 72 is an operating component that switches the power of the digital camera 100 ON and OFF.

[0039] The video button 76 is a push button used to start and stop video recording.

[0040] The touch panel 70a is a touch sensor that detects various touch operations on the display surface of the display unit 33 (the operating surface of the touch panel 70a). The touch panel 70a and the display unit 33 can be configured as an integrated unit. For example, the touch panel 70a is configured such that its light transmittance does not interfere with the display of the display unit 33, and is mounted on the upper layer of the display surface of the display unit 33. Then, the input coordinates on the touch panel 70a are associated with the display coordinates on the display surface of the display unit 33. This makes it possible to provide a GUI (Graphical User Interface) that makes it seem as if the user can directly operate the screen displayed on the display unit 33.

[0041] The system control unit 50 can detect the following operations or states on the touch panel 70a. - A finger or pen that was not previously touching the touch panel 70a now touches the touch panel 70a, i.e., the start of a touch (hereinafter referred to as Touch-Down). • The state in which the touch panel 70a is being touched with a finger or pen (hereinafter referred to as Touch-On). • The finger or pen is moving while touching the touch panel 70a (hereinafter referred to as Touch-Move). - The finger or pen that was touching the touch panel 70a has been removed (released), i.e., the touch has ended (hereinafter referred to as Touch-Up). • The state in which nothing is being touched on the touch panel 70a (hereinafter referred to as Touch-Off)

[0042] When a touchdown is detected, a touch-on is also detected simultaneously. After a touchdown, touch-ons are usually detected continuously unless a touch-up is detected. Touch-ons are also detected if a touch-move is detected. A touch-move is not detected if the touch position has not moved, even if a touch-on has been detected. A touch-off occurs after all fingers or pens that were touching the screen have been detected as having touched up.

[0043] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 70a, are notified to the system control unit 50 via the internal bus. The system control unit 50 then determines what kind of operation (touch operation) was performed on the touch panel 70a based on the notified information. For touch moves, the direction of movement of the finger or pen moving on the touch panel 70a can also be determined for each vertical and horizontal component on the touch panel 70a based on the change in position coordinates. If it is detected that a touch move has occurred beyond a predetermined distance, it is determined that a slide operation has been performed.

[0044] A flick is an operation in which a finger is touched on the touch panel 70a, moved quickly a certain distance while still in contact with it, and then released. In other words, a flick is an operation in which the finger is quickly swiped across the touch panel 70a. If a touch move of a predetermined distance or more at a predetermined speed or faster is detected, and a touch-up is detected immediately afterward, it can be determined that a flick has occurred (it can be determined that a flick occurred following a slide operation). Furthermore, a touch operation in which multiple locations (for example, two points) are touched together (multitouch) and the touch positions are brought closer together is called a pinch-in, and a touch operation in which the touch positions are moved further apart is called a pinch-out. Pinch-out and pinch-in are collectively referred to as a pinch operation (or simply a pinch).

[0045] The touch panel 70a may be of any type from among various types of touch panels, such as resistive, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, and optical sensor types. It may also be a method that detects a touch when there is contact with the touch panel, or when a finger or pen approaches the touch panel; either method is acceptable.

[0046] Figure 3 shows an example of cumulative information 300 according to this embodiment, representing the cumulative number of items in each category (camera work, shooting distance (size), subject). The items are further subdivisions of each category.

[0047] For example, if a single image data (video) detects "person" as the subject and "Pan (horizontal)" as the camera work, 1 is added to the "person" item in the "subject" category. Furthermore, 1 is added to the "Pan (horizontal)" item in the "camera work" category. In this way, the feature detection unit 25 calculates the cumulative number of each item in the cumulative information 300 by adding 1 to each of the multiple features (categories) detected in a single image data (video). For example, in Figure 3, the cumulative number of the "person" item in the "subject" category is 6, the cumulative number of the "landscape" item is 5, the cumulative number of the "animal" item is 0, the cumulative number of the "vehicle" item is 1, and the cumulative number of the "food / small objects" item is 0.

[0048] Using the cumulative information 300 in Figure 3, the generation unit 26 determines that an item with a cumulative count of 0 has an insufficient cumulative count. In the "Subject" category, the items "Animals" and "Food / Small Items" have a cumulative count of 0 and are determined to be items with an insufficient cumulative count. All other items do not have a cumulative count of 0 and are therefore determined not to have an insufficient cumulative count.

[0049] In the "Camera Work" category, the items "Follow (forward)", "Dolly (parallel movement)", and "Pull-out (backward)" have a cumulative count of 0, and are therefore determined to be items with insufficient cumulative counts. All other items do not have a cumulative count of 0, and are therefore determined not to have insufficient cumulative counts.

[0050] In the "Shooting Distance (Size)" category, there are no items with a cumulative count of 0. Therefore, all items included in this category are determined not to have insufficient cumulative counts.

[0051] In this embodiment, items with a cumulative count of 0 were determined to have insufficient cumulative counts. However, the user may pre-set a threshold, and items below that threshold may be determined to have insufficient cumulative counts. Alternatively, the user may pre-set target values ​​for the cumulative count of each item, or target values ​​for the percentage of each item's cumulative count in the total number of items, and items falling below these target values ​​may be determined to have insufficient cumulative counts. Furthermore, the display order of each item may be determined based on the magnitude of the deviation from the target value.

[0052] However, even if the items with insufficient cumulative counts are identified, there may be cases where the specific subject corresponding to the item with insufficient cumulative count does not exist when the user is trying to take a picture.

[0053] Therefore, in this embodiment, the orientation of the imaging unit 110 relative to the gripping unit 90 is changed, and the imaging unit 110 is moved up, down, left, and right to determine the items that can be photographed when the user is trying to photograph, based on the captured surrounding environment image (surrounding environment data). Then, it is determined whether that item matches the item for which the cumulative count is insufficient. Specifically, the type of subject that can be photographed when the user is taking a picture is determined, and it is determined whether that type of subject matches the type of subject for which the cumulative count is insufficient. If the type of subject that can be photographed matches the type of subject for which the cumulative count is insufficient, information regarding that type of subject is displayed on the display unit 28, making it possible to photograph the subject for which the cumulative count is insufficient.

[0054] Figure 4(a) shows an example of the shooting standby screen 400 displayed on the display unit 28. When the type of subject detected by the feature detection unit 25 in the captured surrounding environment image is food 401, "Food / Small Objects" is determined to be a type of subject that can be photographed. In other words, the "Food / Small Objects" item is an item that can be photographed when the user is about to take a picture. Referring to the cumulative information 300 in Figure 3, the cumulative count of the "Food / Small Objects" item in the "Subject" category is 0, and since this is an item with an insufficient cumulative count, this item matches an item that can be photographed. In other words, the item that can be photographed when the user is about to take a picture and has an insufficient cumulative count is "Food / Small Objects". Therefore, as shown in Figure 4(a), information about the item that can be photographed and is insufficient is displayed on the display unit 28. Specifically, the shooting standby screen 400 in Figure 4(a) displays "Insufficient 'Food / Small Objects'! Shooting recommended".

[0055] In addition, the shooting standby screen 400 in Figure 4(a) may display "Insufficient 'food / small objects'! Shooting recommended," as well as an image of the surrounding environment including subjects that can be photographed. If there are no subjects that can be photographed directly in front of the user, the image data (video) captured with the imaging unit 22 facing a subject that can be photographed may be displayed in live view. Furthermore, the direction in which subjects that can be photographed are located relative to the user's front may be displayed on the display unit 28.

[0056] On the other hand, if the types of subjects that can be photographed do not match the types of subjects for which the cumulative count is insufficient, the user will not be able to photograph the subject for which the cumulative count is insufficient at the time they are trying to photograph it. Therefore, the category that the user focuses on when taking a photograph is changed from "subject" to another category. For example, by focusing on "camera work" or "shooting distance (size)" as another category, if there is an item that the user can photograph and an item for which the cumulative count is insufficient match, information about that item will be displayed on the display unit 28.

[0057] Figure 4(b) shows an example of the shooting standby screen 400 displayed on the display unit 28. When the type of subject detected by the feature detection unit 25 in the captured surrounding environment image (surrounding environment data) is a person 402 and a landscape 403, "person" and "landscape" are determined to be the types of subjects that can be photographed. However, referring to the cumulative information 300 in Figure 3, the items "person" and "landscape" in the "subject" category do not have a cumulative count of 0, so they are determined not to be types of subjects with insufficient cumulative counts. Therefore, the types of subjects that can be photographed and the types of subjects with insufficient cumulative counts do not match. Accordingly, the items to be used for determining the match with the items that can be photographed are items with insufficient cumulative counts in other categories. Specifically, "Follow (forward)", "Dolly (translation)", and "Pull-out (backward)" in the "camera work" category have a cumulative count of 0, so they are candidates for items with insufficient cumulative counts to be used for determining the match with the items that can be photographed. Furthermore, since none of the items in the "Shooting Distance (Size)" category have a cumulative count of 0, they are not candidates for use in determining whether an item is Shootable.

[0058] In the shooting standby screen 400 of Figure 4(b), the type of subject detected by the feature detection unit 25 is a person 402, and the orientation of the person 402 is sideways. Therefore, "Dolly (parallel movement)", a camera work highly compatible with the sideways orientation of person 402, is set as a photonable item. The photonable item "Dolly (parallel movement)" matches "Dolly (parallel movement)", which is one of the candidates for items with insufficient cumulative counts. As a result, when the user is about to take a picture, the item that is photonable and has insufficient cumulative counts is set to "Dolly (parallel movement)". Therefore, as shown in Figure 4(b), information about the photonable and insufficient item is displayed on the display unit 28. Specifically, in the shooting standby screen 400 of Figure 4(b), the message "Insufficient "Dolly (parallel movement)"! Shooting recommended" is displayed.

[0059] As mentioned above, the items that can be photographed are those camera work types that are highly compatible with the orientation and movement of the subject. Furthermore, if there are multiple items with insufficient cumulative counts, and the cumulative counts are the same, the item used to determine whether an item is photographable may be narrowed down to one based on the priority set by the user. For example, if the subject is facing sideways, "Dolly (translation)" may be used to determine whether an item is photographable, even among the items with a cumulative count of 0.

[0060] Next, using Figure 5, we will explain an example of notifying the user of items that are available for photography but have insufficient cumulative counts when the user is attempting to take a picture. After the photography waiting screen 400 in Figure 4 is displayed, if a predetermined time has elapsed without any user operation, screen 500 is displayed on the display unit 28 to prioritize prompting the user to take pictures of items with insufficient cumulative counts. This screen 500 includes not only information about items that are available for photography but have insufficient cumulative counts, but also information about items that are not available for photography but have insufficient cumulative counts. Furthermore, screen 500 first displays the category containing items that are available for photography when the user is attempting to take a picture, from among the items with insufficient cumulative counts. In addition, screen 500 displays the items included in that category in ascending order of cumulative count.

[0061] Figure 5(a) corresponds to the shooting standby screen 400 in Figure 4(a), where the "Food / Small Items" item in the "Subject" category is an item that can be photographed when the user is trying to take a picture, but has an insufficient cumulative count. "Food / Small Items" 501 is displayed on the left of the first row. Items included in the "Subject" category are displayed in ascending order of cumulative count, from left to right in the first row, then left to right in the second row, and so on. Specifically, "Food / Small Items" 501 is on the left of the first row, "Animals" 502 with a cumulative count of 0 is on the right of the first row, "Vehicles" with a cumulative count of 1 is on the left of the second row, "Landscapes" with a cumulative count of 5 is on the right of the second row, and "People" with a cumulative count of 6 is on the left of the third row. Note that the "Food / Small Items" 501, which can be photographed when the user is trying to take a picture, but has an insufficient cumulative count, is marked with a "!!" icon. The "Animals" 502, which has the same cumulative count as 0 but cannot be photographed when the user is taking a picture, is marked with an "!" icon. By displaying these icons in a distinct way, users can see whether or not it is possible to take a photo while they are taking one.

[0062] Figure 5(b) corresponds to the shooting standby screen 400 in Figure 4(b), and the type of subject detected from the surrounding environment image is not an item with an insufficient cumulative count. Therefore, instead of "Subject," in the "Camera Work" and "Shooting Distance" categories, "Dolly (Parallel Movement)" 503, which is an item that can be shot when the user is about to take a picture and has an insufficient cumulative count, is displayed on the left of the first row. Items included in the "Camera Work" category are displayed in ascending order of cumulative count, from left to right in the first row, then from left to right in the second row, and so on. Specifically, "Dolly (Parallel Movement)" 503 is on the left of the first row, "Follow (Forward)" 504 with an cumulative count of 0 is in the center of the first row, and "Pull-out (Reverse)" with an cumulative count of 0 is on the right of the first row. The second row continues with "Tilt-down (vertical)" with a cumulative count of 1 on the left, "Push-in (forward)" with a cumulative count of 1 in the center of the second row, and "Circle (rotation)" with a cumulative count of 1 on the right of the second row. The third row continues with "Fix (fixed)" with a cumulative count of 2 on the left, "Tilt-up (vertical)" with a cumulative count of 2 in the center of the third row, "Lead (backward)" with a cumulative count of 2 on the right of the third row, and "Pan (horizontal)" with a cumulative count of 3 on the left of the fourth row. Additionally, the "Dolly (parallel movement)" 503, which is possible to capture when the user is attempting to take a picture but has insufficient cumulative counts, will display a "!!" icon. Furthermore, the "Follow (forward)" 504 and "Pull-out (backward)" 505, which have the same cumulative count as 0 but are not possible to capture when photographing a person facing sideways because the camera work is incompatible with the subject's orientation, will display a "!" icon. By displaying these icons in a distinct way, users can see whether or not it is possible to take a photo while they are taking one.

[0063] If the cumulative number is the same, the items may be displayed in order based on the priority set in advance by the user, or in an order suitable for photographing the detected subject.

[0064] Furthermore, in the screens 500 shown in Figures 5(a) and (b), the cumulative shooting percentage of the subject and the cumulative shooting percentage of the camera work are displayed using bar graphs. Since the user only needs to recognize the shooting percentage, other display formats such as pie charts are also acceptable instead of bar graphs, or the percentages and cumulative numbers are displayed directly as numbers.

[0065] While the example uses an exclamation mark icon to draw attention, other attention-grabbing indicators, such as changing the color, would also work.

[0066] Next, the process of notifying the system of items that are available for shooting but have insufficient cumulative counts while waiting for shooting will be explained using the flowchart in Figure 7. This process is achieved by the system control unit 50 loading the program stored in the non-volatile memory 56 into the system memory 52 and executing it.

[0067] In S701, the system control unit 50 determines whether the power to the imaging device 100 has been turned ON by operating the power switch 72 by the user. If it determines that the power has been turned ON, the process proceeds to S702; otherwise, it proceeds to S701. Once the power to the imaging device 100 is turned ON, the system control unit 50 displays the image data (video) processed by the image processing unit 24 on the display unit 28 in live view mode and sets the device to standby mode for shooting. The imaging unit 22 captures images at a specific frame rate for recording, and the display unit 28 displays the images at a specific frame rate for display.

[0068] In S702, the system control unit 50 determines whether the user has given an operation instruction to display information on the display unit 28 regarding items that can be photographed but are missing. If it determines that the user has given an operation instruction to display information on items that can be photographed but are missing, the process proceeds to S703; otherwise, the process proceeds to S716.

[0069] In S703, the system control unit 50 changes the orientation of the imaging unit 110 relative to the gripping unit 90 and moves the imaging unit 110 up, down, left, and right to acquire ambient environmental data around the location to be photographed. At that time, as shown in Figure 6, a screen 600 indicating that ambient environmental data is being acquired is displayed on the display unit 28. On the screen 600, guidance indicating that ambient environmental data is being acquired is superimposed on the live view display.

[0070] In S704, the feature detection unit 25 detects a subject from the surrounding environment image acquired in S703. The system control unit 50 then refers to the cumulative information 300 stored in the memory 32, and if the subject classified by the generation unit 26 matches the subject detected from the surrounding environment image, it designates the subject detected from the surrounding environment image as a subject that can be photographed.

[0071] In S705, the system control unit 50 determines that items in the "subject" category of the cumulative information 300 stored in memory 32 have a cumulative count of 0 and that these items have an insufficient cumulative count.

[0072] In S706, the system control unit 50 determines whether the photographable subject detected in S704 matches the item whose cumulative count is insufficient, as determined in S705. If it determines that the photographable subject matches the item whose cumulative count is insufficient, the system proceeds to S707; otherwise, it proceeds to S708.

[0073] In S707, the system control unit 50 displays information about items that can be photographed but have insufficient cumulative counts. As shown in Figure 4, the information displayed about the items is the name of the item.

[0074] In S708, the system control unit 50 determines the items that can be photographed while the user is taking a picture, based on categories other than the subject, such as "camera work" and "shooting distance (size)". For example, the items that can be photographed are determined based on the orientation and movement of the subject, and the types of camera work that are highly compatible with those orientations and movements are selected.

[0075] In S709, the system control unit 50 determines that items in the cumulative information 300 stored in the memory 32 that have a cumulative count of 0, such as "camera work" and "shooting distance (size)," which are categories other than the subject, are items with insufficient cumulative counts.

[0076] In S710, the system control unit 50 determines whether the items that can be photographed, as determined in S708, match the items with insufficient cumulative counts, as determined in S709. If it determines that the items that can be photographed and the items with insufficient cumulative counts match, the process proceeds to S707; otherwise, the process ends.

[0077] In S711, the system control unit 50 determines whether a predetermined time has elapsed since the display unit 28 displayed the shooting standby screen 400. The system timer 53 detects whether the predetermined time has elapsed, and if information regarding an item that is ready for shooting and has an insufficient cumulative count is displayed in S707, the system timer 53 is started. If it is determined that the predetermined time has elapsed, the process proceeds to S712; otherwise, the process proceeds to S711.

[0078] In S712, the system control unit 50 displays all information on the display unit 28, including not only information about items that can be photographed but have insufficient cumulative counts, but also information about items that cannot be photographed but have insufficient cumulative counts. As shown in the screen 500 of Figure 5(a), items in a specific category are sorted and displayed in ascending order of cumulative count. The system also calculates the ratio of the number of each item to the total number of already recorded image data (videos) and displays the result on the display unit 28.

[0079] In S713, the system control unit 50 determines whether it has received an instruction to start displaying information about items in other categories based on an operation by the user on the operation unit 70. If it determines that it has received an instruction to display information about items in other categories, it proceeds to S714; otherwise, it proceeds to S715.

[0080] In S714, the system control unit 50 displays all information about the items in the category for which display has been instructed. Similar to S712, as shown in screen 500 of Figure 5(b), the items in a specific category are sorted and displayed in ascending order of cumulative count. The system also calculates the ratio of the number of each item to the total number of already recorded image data (videos) and displays the result on the display unit 28.

[0081] In S715, the system control unit 50 determines whether it has received an instruction from the user to terminate the display of all information related to the item, based on an operation on the operation unit 70. If it determines that it has received an instruction to terminate the display of all information related to the item, it proceeds to S716; otherwise, it proceeds to S713.

[0082] In S716, the system control unit 50 displays the image data (video) processed by the image processing unit 24 on the display unit 28 in live view.

[0083] In S717, the system control unit 50 determines whether or not a command to start video recording has been issued due to the user pressing the video button 76. If it determines that a command to start video recording has been issued, the process proceeds to S718; otherwise, it proceeds to S702.

[0084] In S718, the system control unit 50 starts recording video. Specifically, the system control unit 50 compresses the video data obtained by the image processing unit 24 in the video compression unit 27 and sends it to the recording medium interface 18. The recording medium interface 18 adds the video data to be recorded this time to the video file already recorded on the recording medium 200 and records them together as a single video file.

[0085] In S719, when the system control unit 50 receives a video recording stop instruction due to the user pressing the video button 76, it stops video recording.

[0086] In S720, the system control unit 50 adds information about the recorded video to the cumulative information 300 and terminates this process. Specifically, the generation unit 26 classifies the multiple features detected by the feature detection unit 25 for the recorded video into items in each category. Based on the classification results, the system control unit 50 adds 1 to the cumulative count for the corresponding items in each category.

[0087] In this way, by checking which items are available for recording and have insufficient cumulative counts while waiting for recording to begin, it is possible to efficiently generate a single video file containing the video intended by the user.

[0088] <Other Embodiments> The various controls described above, which are performed by the system control unit 50, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0089] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0090] The present invention can also be realized by performing the following process: supplying software (programs) that realize the functions of the embodiments described above to a system or device via a network or various storage media, and having the computer (or CPU, MPU, etc.) of that system or device read and execute the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0091] Furthermore, the disclosure of this embodiment includes the following configurations and methods.

[0092] (Composition 1) An imaging means for capturing video, A generation means that generates a single video file from multiple videos captured by the aforementioned imaging means, A detection means for detecting the characteristics of the video captured by the aforementioned imaging means, A means for generating cumulative information showing the cumulative number of multiple items related to the aforementioned video, wherein when a feature corresponding to the multiple items is detected by the detection means, the generation means adds the cumulative number of the item. If the cumulative number of items in the cumulative information generated by the generation means is less than a predetermined threshold, a notification means provides information about the items based on the surrounding environment image captured by the imaging means during the waiting period for shooting. An electronic device characterized by having the following features.

[0093] (Configuration 2) The electronic device according to Configuration 1, wherein the notification means notifies information about an item when the cumulative number of the item in the cumulative information generated by the generation means is less than the predetermined threshold and the item that can be captured in the surrounding environment image captured by the imaging means during the shooting standby period matches.

[0094] (Composition 3) The electronic device according to configuration 1 or 2, wherein the notification means, when the cumulative number of items in the cumulative information generated by the generation means is less than the predetermined threshold, does not match the items that can be captured in the surrounding environment image captured by the imaging means during the shooting standby period, and the cumulative number of items included in a feature different from the feature corresponding to the item is less than the predetermined threshold, notifies information regarding the items included in the different feature based on the surrounding environment image captured by the imaging means during the shooting standby period.

[0095] (Composition 4) The electronic device according to any one of configurations 1 to 3, characterized in that the information relating to the aforementioned item is information relating to a subject, information relating to the camera work of the electronic device, or information relating to the shooting distance.

[0096] (Composition 5) A generation step that generates a single video file from multiple videos captured by an imaging means for capturing video, A detection step of detecting the characteristics of the video captured by the aforementioned imaging means, A means for generating cumulative information showing the cumulative number of multiple items related to the aforementioned video, comprising a generation step of adding the cumulative number of an item when a feature corresponding to the multiple items is detected by the detection means, If the cumulative number of items in the cumulative information generated by the generation step is less than a predetermined threshold, a notification step is made to notify information about the items based on the surrounding environment image captured by the imaging means during the waiting period for shooting. A method for controlling electronic equipment, characterized by having the following features.

[0097] (Composition 6) A program to cause a computer to perform the control method of the electronic device described in Configuration 5.

[0098] (Composition 7) A computer-readable recording medium containing a program that causes the computer to perform the control method of the electronic device described in Configuration 5.

Claims

1. An imaging means for capturing video, A generation means that generates a single video file from multiple videos captured by the aforementioned imaging means, A detection means for detecting the characteristics of the video captured by the aforementioned imaging means, A means for generating cumulative information showing the cumulative number of multiple items related to the aforementioned video, wherein when a feature corresponding to the multiple items is detected by the detection means, the generation means adds the cumulative number of the item. If the cumulative number of items in the cumulative information generated by the generation means is less than a predetermined threshold, a notification means provides information about the items based on the surrounding environment image captured by the imaging means during the waiting period for shooting. An electronic device characterized by having the following features.

2. The electronic device according to claim 1, wherein the notification means notifies information about an item when the cumulative number of the item in the cumulative information generated by the generation means is less than the predetermined threshold and the item that can be captured in the surrounding environment image captured by the imaging means during the shooting standby period matches.

3. The electronic device according to claim 1, wherein the notification means, when the cumulative number of items in the cumulative information generated by the generation means is less than the predetermined threshold of an item, does not match an item that can be captured in the surrounding environment image captured by the imaging means during the shooting standby period, and the cumulative number of items included in a feature different from the feature corresponding to the item is less than the predetermined threshold of the surrounding environment image captured by the imaging means during the shooting standby period, notifies information regarding the item included in the different feature based on the surrounding environment image captured by the imaging means during the shooting standby period.

4. The electronic device according to claim 1, characterized in that the information relating to the aforementioned items is information relating to the subject, information relating to the camera work of the electronic device, or information relating to the shooting distance.

5. A generation step of generating a single video file from multiple videos captured by an imaging means for capturing video, A detection step of detecting the characteristics of the video captured by the aforementioned imaging means, A means for generating cumulative information showing the cumulative number of multiple items related to the aforementioned video, comprising a generation step of adding the cumulative number of an item when a feature corresponding to the multiple items is detected by the detection means, If the cumulative number of items in the cumulative information generated by the generation step is less than a predetermined threshold, a notification step is made to notify information about the items based on the surrounding environment image captured by the imaging means during the waiting period for shooting. A method for controlling electronic equipment, characterized by having the following features.

6. A program for causing a computer to execute the control method for an electronic device described in claim 5.

7. A computer-readable recording medium containing a program for causing a computer to execute the control method of the electronic device described in claim 5.

Images