Electronic apparatus, method for controlling electronic apparatus, and program

Abstract

This electronic apparatus controls display of a specific screen displayed on a display means, and acquires a plurality of moving images captured in each of a plurality of consecutive time slots and a plurality of still images captured during capture of the plurality of moving images. The electronic apparatus (1) displays the plurality of moving images in a first region in the specific screen in chronological order, and (2) displays, for each time slot in which one or more still images were captured among the plurality of time slots, the moving image captured in the time slot and the one or more still images captured in the time slot in the first region in association with each other.

Description

Electronic device, electronic device control method, and program

[0001] The present invention relates to an electronic device, a control method for an electronic device, and a program.

[0002] Digital cameras are known that can capture still images while recording video without stopping the video. In news reporting and other situations, a single cameraman often captures both video and still images (hereafter, video and still images are collectively referred to as "photographic data") to reduce manpower. Therefore, it is important that a single camera can capture still images while recording video.

[0003] Furthermore, when a news article or the like is created using photographic data, the photographer transmits the photographic data to an editor via the Internet, and the editor selects and edits the photographic data.

[0004] However, since videos and still images are saved as separate files, it can be difficult to understand the relationship between them. For this reason, Patent Document 1 makes it easier to check the relationship between videos and still images by displaying still images in association with each other on a timeline during video playback.

[0005] JP 2010-147861 A

[0006] In Patent Document 1, it is certainly possible to easily check the relationship between a selected video and multiple still images. However, it is not possible to check the relationship between multiple videos and multiple still images simultaneously. In particular, when generating a news article, a selection task is performed from multiple short videos and multiple still images, making the selection task difficult even with the technology of the first embodiment.

[0007] Therefore, an object of the present invention is to provide a technique for reducing the workload of selecting images from a plurality of moving images and a plurality of still images.

[0008] One aspect of the present invention is an electronic device that controls the display of a specific screen displayed on a display means, and is characterized by having: an acquisition means that acquires multiple videos shot during multiple consecutive time periods and multiple still images taken while shooting the multiple videos; and a control means that 1) displays the multiple videos in chronological order in a first area on the specific screen, and 2) for each time period in which one or more still images of the multiple time periods were shot, links the video shot during that time period and the one or more still images shot during that time period to each other and displays them in the first area.

[0009] One aspect of the present invention is a control method for an electronic device that controls the display of a specific screen displayed on a display means, comprising: an acquisition step of acquiring a plurality of videos taken during each of a plurality of consecutive time periods and a plurality of still images taken while taking the plurality of videos; and a control step of 1) displaying the plurality of videos in chronological order in a first area on the specific screen, and 2) for each time period in which one or more still images of the plurality of time periods were taken, linking the video taken during that time period and the one or more still images taken during that time period to each other and displaying them in the first area.

[0010] According to the present invention, it is possible to reduce the workload of selecting images from a plurality of moving images and a plurality of still images.

[0011] FIG. 1 is a diagram showing the overall configuration of a shooting and editing system according to the first embodiment. FIGS. 2A and 2B are external views of a camera according to the first embodiment. FIG. 3 is a diagram showing the configuration of a camera according to the first embodiment. FIGS. 4A and 4B are diagrams showing the configuration of a server and a PC according to the first embodiment. FIG. 5 is a diagram showing a screen of an editing application according to the first embodiment. FIG. 6 is a diagram showing a screen of an editing application according to the second embodiment. FIG. 7 is a flowchart showing the operation of a camera according to the first embodiment. FIG. 8 is a flowchart showing the operation of a server according to the first embodiment. FIG. 9 is a flowchart showing the operation of a PC according to the first embodiment. FIG. 10 is a flowchart showing the operation of a camera according to the second embodiment. FIG. 11 is a flowchart showing the operation of a server according to the second embodiment. FIG. 12 is a flowchart showing the operation of a PC according to the second embodiment. FIG. 13 is a flowchart showing processing according to an imaging signal, etc. according to the second embodiment. FIGS. 14A to 14C are diagrams for explaining editing processing according to the third embodiment. FIG. 15 is a flowchart showing the operation of a PC according to the third embodiment.

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] <Embodiment 1> Fig. 1 is a schematic diagram showing an example of the overall configuration of a shooting and editing system according to Embodiment 1. The shooting and editing system according to Embodiment 1 includes a camera 100, a server 300, and a personal computer (PC) 400. The server 300 and the PC 400 are information processing devices (electronic devices) that handle images captured by an imaging device such as the camera 100. Fig. 1 shows a configuration in which the camera 100 and the PC 400 are connected to each other wirelessly or via a wire so as to be able to communicate with each other.

[0014] 2A and 2B are external views showing an example of the appearance of an imaging device, camera 100. Fig. 2A is a perspective view of camera 100 as seen from the front side, and Fig. 2B is a perspective view of camera 100 as seen from the back side.

[0015] The camera 100 has, on its top surface, a shutter button 101, a power switch 102, a mode selector switch 103, a main electronic dial 104, a sub electronic dial 105, a movie button 106, and an extra-viewfinder display 107. The shutter button 101 is an operating member for issuing instructions to prepare for shooting or to issue instructions to shoot. The power switch 102 is an operating member for switching the power of the camera 100 on and off. The mode selector switch 103 is an operating member for switching between various modes. The main electronic dial 104 is a rotary operating member for changing settings such as shutter speed and aperture. The sub electronic dial 105 is a rotary operating member for moving the selection frame (cursor) and scrolling through images. The movie button 106 is an operating member for issuing instructions to start and stop movie shooting (recording). The extra-viewfinder display 107 displays various settings such as shutter speed and aperture.

[0016] The camera 100 has a display unit 108, a touch panel 109, directional keys 110, a SET button 111, an AE lock button 112, a magnification button 113, a playback button 114, a menu button 115, an eyepiece unit 116, an eyepiece detection unit 118, and a touch bar 119 on its rear surface. The display unit 108 displays images and various information. The touch panel 109 is an operation member that detects touch operations on the display surface (touch operation surface) of the display unit 108. The directional keys 110 are an operation member that includes keys (four-way keys) that can be pressed up, down, left, and right. The camera 100 can execute processing corresponding to the position of the directional keys 110 that is pressed. The SET button 111 is an operation member that is mainly pressed to confirm a selection item. The AE lock button 112 is an operation member that is pressed to fix the exposure state in a shooting standby state. The enlargement button 113 is an operating member for switching the enlargement mode on and off in the live view display (LV display) in the shooting mode. When the enlargement mode is on, the live view image (LV image) is enlarged or reduced by operating the main electronic dial 104. The enlargement button 113 is also used to enlarge the playback image or increase the magnification in the playback mode. The playback button 114 is an operating member for switching between the shooting mode and the playback mode. Pressing the playback button 114 in the shooting mode switches to the playback mode, and the latest image recorded on the recording medium 227 (described later) can be displayed on the display unit 108.

[0017] The menu button 115 is an operating member that is pressed to display a menu screen on the display unit 108 that allows various settings to be made. The user can intuitively make various settings using the menu screen displayed on the display unit 108, the directional keys 110, and the SET button 111. The eyepiece unit 116 is a portion through which the user places their eye in front of an eyepiece finder (a peer-type finder) 117. Through the eyepiece unit 116, the user can view an image displayed on an EVF 217 (Electronic Viewfinder) (described below) inside the camera 100. The eyepiece detection unit 118 is a sensor that detects whether the user places their eye in front of the eyepiece unit 116 (eyepiece finder 117).

[0018] The touch bar 119 is a line-shaped touch operation member (line touch sensor) capable of receiving touch operations. The touch bar 119 is positioned so that it can be touched (touched) by the thumb of the right hand when the grip unit 120 is held in the right hand (held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. In other words, the touch bar 119 can be operated while the user places their eye on the eyepiece viewfinder 117, looks through the eyepiece unit 116, and is in a position (shooting posture) in which the user is ready to press the shutter button 101 at any time. The touch bar 119 can receive tap operations (touching and then releasing the touch position without moving it within a predetermined period of time) and left / right slide operations (touching and then moving the touch position while keeping the touch) on the touch bar 119. The touch bar 119 is an operation member different from the touch panel 109 and does not have a display function. The touch bar 119 functions as a multi-function bar (M-Fn bar) to which various functions can be assigned, for example.

[0019] The camera 100 also includes a grip section 120, a thumb rest section 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like. The grip section 120 is a holding section shaped to be easily gripped with the user's right hand when holding the camera 100. When the user holds the camera 100 by gripping the grip section 120 with the little finger, ring finger, and middle finger of the right hand, the shutter button 101 and the main electronic dial 104 are positioned so that they can be operated with the index finger of the right hand. Similarly, the sub electronic dial 105 and the touch bar 119 are positioned so that they can be operated with the thumb of the right hand. The thumb rest section 121 (thumb standby position) is located on the rear side of the camera 100, in a position where it is easy to place the thumb of the right hand holding the grip section 120 when none of the operating components are being operated. The thumb rest section 121 is made of a rubber material or the like to enhance holding strength (grip feel). Terminal cover 122 protects connectors such as connection cables that connect camera 100 to external devices. Lid 123 protects recording medium 227 and the slot by closing the slot for storing recording medium 227, which will be described later. Communication terminal 124 is a terminal for communicating with a lens unit (such as lens unit 200, which will be described later) that is detachable from camera 100.

[0020] Fig. 3 is a block diagram showing an example of the configuration of the camera 100. Note that the same components as those in Fig. 2 are assigned the same reference numerals as in Fig. 2, and descriptions of those components will be omitted as appropriate. In Fig. 3, a lens unit 200 is attached to the camera 100.

[0021] First, the lens unit 200 will be described. The lens unit 200 is a type of interchangeable lens that can be attached to and detached from the camera 100. The lens unit 200 is a single lens and is an example of a normal lens. The lens unit 200 includes an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0022] The aperture 201 is configured to have an adjustable aperture diameter. The lens 202 is composed of multiple lenses. The aperture drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lens 202 to adjust the focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, and the like based on instructions from a system control unit 50 (described later). The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203 and adjusts the focus by changing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera 100. Specifically, communication is performed via a communication terminal 206 of the lens unit 200 and a communication terminal 124 of the camera 100. The communication terminal 206 is a terminal through which the lens unit 200 communicates with the camera 100.

[0023] Next, a description will be given of the camera 100. The camera 100 has a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0024] The shutter 210 is a focal plane shutter that can freely control the exposure time of the image capture unit 211 based on instructions from the system control unit 50. The image capture unit 211 is an image sensor composed of a CCD, CMOS, or other element that converts an optical image into an electrical signal. The image capture unit 211 may also have an image plane phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts the analog signal output from the image capture unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on data from the A / D converter 212 or data from the memory control unit 213. The image processing unit 214 also performs predetermined arithmetic processing using the captured image data, and the system control unit 50 controls exposure and distance measurement based on the obtained arithmetic results. This processing allows for TTL (through-the-lens) AF processing, AE (automatic exposure) processing, EF (pre-flash) processing, etc. Furthermore, the image processing unit 214 performs predetermined calculation processing using the captured image data, and the system control unit 50 performs TTL type AWB (auto white balance) processing based on the obtained calculation results.

[0025] The image data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written to the memory 215 via the memory control unit 213 without going through the image processing unit 214. The memory 215 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212. The memory 215 stores image data to be displayed on the display unit 108 and the EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 215 also serves as a memory for displaying images (video memory).

[0026] The D / A converter 216 converts the display image data stored in the memory 215 into an analog signal and supplies it to the display unit 108 and the EVF 217. Therefore, the display image data written to the memory 215 is displayed on the display unit 108 and the EVF 217 via the D / A converter 216. The display unit 108 and the EVF 217 perform display in accordance with the analog signal from the D / A converter 216. The display unit 108 and the EVF 217 are displays such as LCDs and organic EL displays. The digital signals that have been A / D converted by the A / D converter 212 and stored in the memory 215 are then converted into analog signals by the D / A converter 216. The converted analog signals are sequentially transferred to the display unit 108 and the EVF 217, and images based on the analog signals are displayed on the display unit 108 and the EVF 217. This results in a live view display.

[0027] The system control unit 50 is a control unit comprising at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 executes programs recorded in the nonvolatile memory 219 to realize each process in the flowcharts described below. The system control unit 50 also performs display control by controlling the memory 215, D / A converter 216, display unit 108, EVF 217, etc.

[0028] The camera 100 also includes a system memory 218 , a non-volatile memory 219 , a system timer 220 , a communication unit 221 , an attitude detection unit 222 , and an eye proximity detection unit 118 .

[0029] A RAM, for example, is used as the system memory 218. Constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 219, and the like are loaded into the system memory 218. The nonvolatile memory 219 is an electrically erasable and recordable memory. An EEPROM, for example, is used as the nonvolatile memory 219. Constants and programs for the operation of the system control unit 50 are recorded in the nonvolatile memory 219. The programs here refer to programs for executing the flowcharts described below. The system timer 220 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0030] The communication unit 221 transmits and receives video and audio signals to and from external devices connected wirelessly or via a wired cable. The communication unit 221 can also connect to a wireless LAN (Local Area Network) and the Internet. The communication unit 221 can also communicate with external devices via Bluetooth (registered trademark) and Bluetooth Low Energy. The communication unit 221 can transmit images (including live images) captured by the imaging unit 211, shooting information signals indicating shooting information such as the image file name and shooting date and time, and images recorded on the recording medium 227. The communication unit 221 can receive images and various other information from external devices.

[0031] The orientation detection unit 222 detects the orientation of the camera 100 relative to the direction of gravity. Based on the detected orientation, the orientation detection unit 222 can determine whether an image captured by the imaging unit 211 was captured with the camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the orientation detection unit 222 to the image file of the image captured by the imaging unit 211, or rotate the image according to the detected orientation. The orientation detection unit 222 can be, for example, an acceleration sensor or a gyro sensor. The orientation detection unit 222 can also be used to detect movement of the camera 100 (such as panning, tilting, lifting, or whether the camera 100 is stationary).

[0032] The eyepiece detection unit 118 can detect the approach of an object to the eyepiece unit 116 (eyepiece finder 117). For example, an infrared proximity sensor can be used as the eyepiece detection unit 118. When an object approaches the eyepiece unit 116, infrared light is emitted from the light-emitting unit of the eyepiece detection unit 118, reflected by the object, and received by the light-receiving unit of the infrared proximity sensor. The distance from the eyepiece unit 116 to the object can be determined based on the amount of received infrared light. In this way, the eyepiece detection unit 118 performs eyepiece detection, which detects the proximity of an object to the eyepiece unit 116.

[0033] The eye-contact detection unit 118 is an eye-contact detection sensor that detects the approach (eye-contact) and departure (eye-away) of an eye (object) from the eyepiece unit 116. The eye-contact detection unit 118 detects an eye-contact state when it detects an object approaching the eyepiece unit 116 within a predetermined distance from a non-eye-contact state (non-approach state). On the other hand, the eye-contact detection unit 118 detects an eye-away state when the object that it detected as approaching moves away from the eyepiece state (approach state) by a predetermined distance or more. The threshold for detecting eye-contact and the threshold for detecting eye-away may be different, for example, by providing hysteresis. Furthermore, after detecting eye-contact, the eye-contact state is maintained until eye-away is detected. After detecting eye-away, the eye-away state is maintained until eye-away is detected.

[0034] The system control unit 50 switches between display (display state) and non-display (non-display state) of the display unit 108 and the EVF 217 depending on the state detected by the eye proximity detection unit 118. Specifically, when at least in a shooting standby state and the display destination switching setting is automatic switching, the display destination is set to the display unit 108 and the display is turned on while the EVF 217 is not switched on when the eye is not placed in position. Furthermore, when the eye is placed in position, the display destination is set to the EVF 217 and the display is turned on while the display unit 108 is switched off. Note that the eye proximity detection unit 118 is not limited to an infrared proximity sensor, and other sensors may be used as the eye proximity detection unit 118 as long as they can detect a state that can be considered as eye proximity.

[0035] The camera 100 also has an outside-viewfinder display unit 107, an outside-viewfinder display drive circuit 223, a power supply control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and the like.

[0036] The extra-finder display unit 107 is driven by an extra-finder display drive circuit 223 and displays various settings of the camera 100, such as shutter speed and aperture. The power supply control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switch circuit (a circuit that switches between blocks that receive current), etc. The power supply control unit 224 detects whether a battery is installed, the type of battery, and the remaining battery power. The power supply control unit 224 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the required voltage to each unit, including the recording medium 227, for the required period of time.

[0037] The power supply unit 225 is a primary battery (such as an alkaline battery or a lithium battery), a secondary battery (such as a NiCd battery, a NiMH battery, or a Li battery), an AC adapter, or the like. The recording medium I / F 226 is an interface with a recording medium 227 (such as a memory card or a hard disk). The recording medium 227 is a memory card or the like for recording captured images. The recording medium 227 is composed of a semiconductor memory, a magnetic disk, or the like. The recording medium 227 may be detachable from the camera 100, or may be built into the camera 100.

[0038] The operation unit 228 is an input unit that accepts operations from the user (user operations). The operation unit 228 is used to input various instructions to the system control unit 50. The operation unit 228 includes the shutter button 101, the power switch 102, the mode selector switch 103, the touch panel 109, and other operation units 229. The other operation units 229 include the main electronic dial 104, the sub electronic dial 105, the movie button 106, the direction keys 110, the SET button 111, the AE lock button 112, the enlargement button 113, the playback button 114, the menu button 115, the touch bar 119, and the like.

[0039] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. When the first shutter switch 230 is turned on during operation of the shutter button 101, that is, when the shutter button 101 is pressed halfway (a shooting preparation instruction), it outputs a first shutter switch signal SW1. The system control unit 50 starts shooting preparation processing (AF processing, AE processing, AWB processing, EF processing, etc.) in response to the output of the first shutter switch signal SW1. When the second shutter switch 231 is turned on after operation of the shutter button 101 is completed, that is, when the shutter button 101 is pressed all the way (a shooting instruction), it outputs a second shutter switch signal SW2. In response to the output of the second shutter switch signal SW2, the system control unit 50 starts a series of shooting processes (processing from reading out a signal from the imaging unit 211 to generating an image file including the captured image and writing it to the recording medium 227).

[0040] The mode selector switch 103 switches the operating mode of the system control unit 50 to one of a still image capture mode, a video capture mode, a playback mode, etc. Modes included in the still image capture mode include an auto capture mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). Modes included in the still image capture mode also include various scene modes and custom modes that provide capture settings for specific capture scenes. The user can directly switch to one of the above-mentioned capture modes using the mode selector switch 103. Alternatively, the user can first switch to a list screen of capture modes using the mode selector switch 103, and then selectively switch to one of the displayed modes using the operation unit 228. Similarly, the video capture mode may also include multiple modes.

[0041] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be configured as an integrated unit. For example, the touch panel 109 is attached to the upper layer of the display surface of the display unit 108 so that its light transmittance does not interfere with the display of the display unit 108. By associating input coordinates on the touch panel 109 with display coordinates on the display surface of the display unit 108, a GUI (graphical user interface) can be configured that makes it appear as if the user is directly operating the screen displayed on the display unit 108. The touch panel 109 can use any of a variety of methods, such as a resistive film method, a capacitive method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, or an optical sensor method. Depending on the method, there are methods that detect a touch by contact with the touch panel 109, and methods that detect a touch by the approach of a finger or a pen to the touch panel 109, but either method is acceptable.

[0042] The system control unit 50 can detect the following operations or states on the touch panel 109: A finger or pen that has not been touching the touch panel 109 now touches the touch panel 109, i.e., the start of touching (hereinafter referred to as Touch-Down). A state in which the touch panel 109 is being touched by a finger or pen (hereinafter referred to as Touch-On). A finger or pen moving across the touch panel 109 while still touching it (hereinafter referred to as Touch-Move). A finger or pen that has been touching the touch panel 109 has been released from the touch panel 109, i.e., the end of touching (hereinafter referred to as Touch-Up). A state in which nothing is touching the touch panel 109 (hereinafter referred to as Touch-Off).

[0043] When a touch down is detected, a touch on is also detected at the same time. After a touch down, a touch on is usually continued to be detected unless a touch up is detected. If a touch move is detected, a touch on is also detected at the same time. Even if a touch on is detected, a touch move is not detected unless the touch position moves. Once it is detected that all fingers or pens that were touching have touched up, a touch off occurs.

[0044] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 109, are notified to the system control unit 50 via the internal bus. Based on the notified information, the system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 109. Regarding touch-move, the movement direction of the finger or pen moving on the touch panel 109 can also be determined for each vertical component and horizontal component on the touch panel 109 based on changes in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed.

[0045] A flick is an operation in which a finger is touched to the touch panel 109, quickly moved a certain distance, and then released. In other words, a flick is an operation in which a finger is quickly traced across the touch panel 109 as if flicking. When a touch-move is detected over a predetermined distance or more at a predetermined speed or more, and a touch-up is then detected, it is determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation). Furthermore, a touch operation in which multiple points (for example, two points) are touched together (multi-touch) and the touch positions are brought closer together is called a pinch-in, and a touch operation in which the touch positions are moved farther apart is called a pinch-out. Pinch-out and pinch-in are collectively called a pinch operation (or simply pinch).

[0046] 4A is a block diagram showing an example of the configuration of server 300, which is an external device to PC 400. Server 300 has control unit 301, ROM 302, RAM 303, recording medium 304, communication unit 305, and determination unit 306. These components can communicate with each other via system bus 307. Note that each component of server 300 may be included in camera 100.

[0047] The control unit 301 is, for example, a central processing unit (CPU) and controls the entire server 300.

[0048] The ROM 302 is a read-only memory that non-temporarily stores programs and parameters.

[0049] The RAM 303 is a random access memory that temporarily stores programs and data supplied from external devices and the like.

[0050] The recording medium 304 is a medium (a hard disk or a flash memory) that is fixedly installed in the server 300. Alternatively, the recording medium 304 is a medium (an optical disk, a magnetic card, an optical card, an IC card, a memory card, or the like) that is detachable from the server 300.

[0051] The communication unit 305 communicates with external devices (such as the camera 100 and the PC 400). The communication unit 305 receives video and still images from the camera 100. The received video and still images are stored in the recording medium 304. In response to a request from the PC 400, the communication unit 305 transmits to the PC 400 the images (video and still images) stored in the recording medium 304 and information on the determination result of the state of the still image (whether it is in focus, whether the eye is blinking, etc.). At this time, the communication unit 305 may transmit to the PC 400 a determination signal indicating the information on the determination result and information on the file name of the still image, separately from the still image. The communication unit 305 also receives an imaging information signal indicating imaging information of the still image generated by the camera 100, and transmits the imaging information signal to the PC 400.

[0052] The determination unit 306 determines the state of the still image (whether it is in focus, whether the eye is blinking, etc.) stored in the recording medium 304. The determination unit 306 also stores information on the determination result of the state of the still image in the recording medium 304 together with the still image.

[0053] Furthermore, the system bus 307 connects the components of the server 300 so that they can communicate with each other.

[0054] 4B is a block diagram showing an example of the configuration of an electronic device, a PC 400. The PC 400 has a control unit 401, a ROM 402, a RAM 403, a recording medium 404, an operation unit 405, a display unit 406, and a communication unit 407. These components can communicate with each other via a system bus 408.

[0055] The control unit 401 is, for example, a central processing unit (CPU). The control unit 401 controls the entire PC 400. The control unit 401 also communicates with the server 300 via the communication unit 407 to check whether a new video or still image has been transmitted from the camera 100 to the server 300. When a new video or still image has been transmitted to the server 300, the control unit 401 requests the server 300 to transmit the video or still image via the communication unit 407. The control unit 401 then stores the new video or still image in the recording medium 404. The control unit 401 also automatically selects a video corresponding to a still image selected in response to an operation on the operation unit 405 (a video shot in the same time period as the still image).

[0056] The ROM 402 is a read-only memory that non-temporarily stores programs and parameters.

[0057] The RAM 403 is a random access memory that temporarily stores programs and data supplied from external devices and the like.

[0058] The recording medium 404 is a medium (such as a hard disk or flash memory) that is fixedly installed in the PC 400. Alternatively, the recording medium 404 is a medium (such as an optical disk, a magnetic card, an optical card, an IC card, or a memory card) that is detachable from the PC 400.

[0059] The operation unit 405 accepts operations by a user (editor) on the PC 400. For example, one of one or more still images that satisfy a predetermined condition is selected in response to a user's operation on the operation unit 405. Here, the operation unit 405 may be an operation member (such as a button or a touch panel) provided on the PC 400, or may be an operation member (such as a keyboard or a mouse) that is detachable from the PC 400.

[0060] The display unit 406 displays data stored in the PC 400, data supplied from an external device, and the like. The display unit 406 also displays a list of videos and still images recorded on the recording medium 404 on the same time axis. The display unit 406 can also display only still images that satisfy predetermined conditions. The display unit 406 can also display videos captured during the same time period as a still image selected by the operation unit 405, each arranged in a predetermined area of ​​a specific screen. Furthermore, if reception of a still image is delayed, the display unit 406 displays the reception status of the still image and the determination result in accordance with the shooting information signal and the determination signal. The display unit 406 may be included in the PC 400, or may be a display device separate from the PC 400.

[0061] A communication unit 407 communicates with external devices such as the camera 100 and the server 300. A system bus 408 connects the components of the PC 400 so that they can communicate with each other.

[0062] The basic processing from shooting to editing in the shooting and editing system will be described below. The shooting and editing system includes a camera 100, a server 300, and a PC 400.

[0063] 5 shows an example of a screen (a screen displayed on the display unit 406) of an editing application running on the PC 400. In the editing application, the PC 400 displays videos and still images received from the camera 100 and selects still images and videos (mutually related still images and videos) to be used in a news article. In addition, the editing application enables the PC 400 to edit the text (wording) of the news article and publish a news article (published article) that includes the text of the news article together with the selected still images and videos.

[0064] The screen of the editing application includes a status area 600, a Good area 601, a timeline area 602, and an article area 603. The status area 600 displays the shooting status (status indicating whether the camera 100 is shooting a video or not, and whether a still image has been shot or not).

[0065] The Good area 601 displays one or more still images that satisfy predetermined conditions in the center left side of the editing application screen, arranged in chronological order of their capture. The user can select one of the one or more still images displayed in the Good area 601.

[0066] The timeline area 602 displays videos and still images for each shooting time period on a timeline (in chronological order). The multiple videos displayed in the timeline area 602 are multiple videos shot over consecutive time periods. A time period is a range of a certain length (e.g., one minute). If a still image satisfies certain conditions (such as composition, focus of the main subject, or blinking), a "Good" icon (a display item indicating that the still image was properly shot) is superimposed on a portion of the still image. If a still image does not satisfy the certain conditions, a "Bad" icon (a display item indicating that the still image was not properly shot) may be superimposed on a portion of the still image. Furthermore, an icon corresponding to the state of the still image may be superimposed (displayed) on the still image. For example, an icon indicating that a blinking eye is being displayed on a still image in which a specific subject is blinking, and an icon indicating that the still image was shot with a specific composition may be superimposed on a still image in which a specific subject is blinking.

[0067] The article area 603 is an area for editing a news article. In the article area 603, the user can edit the title and text of the news article. The article area 603 includes a still image area 603A for inserting (placing) a still image and a video area 603B for inserting (placing) a video. When a still image displayed in the Good area 601 is selected by a user operation such as a mouse click, the selected still image is inserted into the still image area 603A of the article area 603. Furthermore, a video captured during the same time period as the still image is inserted into the video area 603B of the article area 603. This makes it possible to realize a preview of the news article in a state close to completion.

[0068] If the user determines from the preview that the news article can be made public, the user presses the Publish button 604. A final confirmation message is then displayed asking whether or not it is OK to make the news article public. If the user's response to the final confirmation message indicates that the user has instructed the news article to be made public, the news article is distributed (made public on the Internet).

[0069] The operation of the camera 100 according to the first embodiment will be described with reference to the flowchart in Fig. 7. The flowchart in Fig. 7 starts when the camera 100 transitions to a shooting preparation state after the power of the camera 100 is turned on (started up).

[0070] In step S701, the system control unit 50 starts image capture and also starts live view display (display of a captured live view image).

[0071] In step S702, the system control unit 50 determines whether an instruction to start video recording has been given (such as an operation of the video button 106 by the photographer). If it is determined that an instruction to start video recording has been given, the process proceeds to step S703. If it is determined that an instruction to start video recording has not been given, the process of step S702 is repeated.

[0072] In step S703, the system control unit 50 starts capturing a moving image using the imaging unit 211 and starts recording the captured moving image on the recording medium 227.

[0073] In step S704, the system control unit 50 determines whether a predetermined time has passed since the previous processing of step S705 was performed. If it is determined that the predetermined time has passed since the previous processing of step S705 was performed, the process proceeds to step S705. If it is determined that the predetermined time has not passed since the previous processing of step S705 was performed, the process proceeds to step S706. The predetermined time is, for example, a relatively short time (any time between 15 seconds and 3 minutes) that is set in advance by the photographer, taking into consideration the performance of the camera 100, the communication environment when transmitting the video, and the like.

[0074] In step S705, the system control unit 50 transmits a predetermined amount of video from the previous time when the process of step S705 was performed to the server 300. The process of transmitting a predetermined amount of video to the server 300 continues until an instruction to end video shooting is given.

[0075] In step S706, the system control unit 50 determines whether or not a still image capture instruction has been issued. If it is determined that a still image capture instruction has been issued, the process proceeds to step S707. If it is determined that a still image capture instruction has not been issued, the process proceeds to step S709.

[0076] In step S 707 , the system control unit 50 captures one still image and records the still image on the recording medium 227 .

[0077] In step S708, the system control unit 50 transmits the still image captured in step S707 to the server 300. Note that when an instruction to capture a still image is given in step S706, the system control unit 50 may continuously capture multiple still images in steps S707 and S708 until an instruction to end still image capture is given, and transmit the captured multiple still images to the server 300.

[0078] In step S709, the system control unit 50 determines whether an instruction to end video shooting has been issued. If it is determined that an instruction to end video shooting has been issued, the process proceeds to step S710. If it is determined that an instruction to end video shooting has not been issued, the process proceeds to step S704.

[0079] In step S710, the system control unit 50 ends shooting of the moving image. Then, the system control unit 50 transmits to the server 300 the remaining portion of the moving image that has not yet been transmitted to the server 300 (the moving image that has not been transmitted in step S705).

[0080] The operation of the server 300 according to the first embodiment will be described with reference to the flowchart of Fig. 8. The processing of the flowchart of Fig. 8 may be executed repeatedly and continuously, or may be executed every time some information is transmitted from the camera 100 to the server 300.

[0081] In step S801, the control unit 301 determines whether or not a video has been received from the camera 100. If it is determined that a video has been received from the camera 100, the process proceeds to step S802. If it is determined that a video has not been received from the camera 100, the process proceeds to step S805.

[0082] In step S802, the control unit 301 stores the video received in step S801 in a database on the recording medium 304.

[0083] In step S803, the control unit 301 determines whether or not a request for video transmission has been made by the PC 400 (whether or not an instruction has been issued to the server 300 to transmit a video to the PC 400). If it is determined that a request for video transmission has been made, the process proceeds to step S804. If it is determined that a request for video transmission has not been made, the process of step S803 is repeated.

[0084] In step S804, control unit 301 transmits to PC 400 any videos that have not yet been transmitted to PC 400. Here, control unit 301 associates information indicating that a video has been transmitted to PC 400 with the video and stores the information in the database. This allows control unit 301 to prevent the same video from being transmitted to PC 400 multiple times.

[0085] In step S805, the control unit 301 determines whether or not a still image has been received from the camera 100. If it is determined that a still image has been received from the camera 100, the process proceeds to step S806. If it is determined that a still image has not been received from the camera 100, the process proceeds to step S801.

[0086] In step S806, the control unit 301 controls the determination unit 306 to determine the state of the still image (such as the composition of the still image, the focus state of the main subject, or whether the eyes are blinking). The control unit 301 can use a predetermined algorithm or AI (Artificial Intelligence) to determine the state of the still image.

[0087] In step S807, the control unit 301 associates the determination result (hereinafter referred to as the "state determination result") of the state of the still image (such as composition, focus state of the main subject, or whether the eyes are blinking) with the still image and stores it in a database on the recording medium 304.

[0088] In step S808, the control unit 301 determines whether or not a request for transmission of a still image has been made by the PC 400 (whether or not an instruction has been issued to the server 300 to transmit a still image to the PC 400). If it is determined that a request for transmission of a still image has been made by the PC 400, the process proceeds to step S809. If it is determined that a request for transmission of a still image has not been made by the PC 400, the process of step S808 is repeated.

[0089] In step S809, the control unit 301 transmits to the PC 400 any still images that have not yet been transmitted to the PC 400, together with the status determination results (status determination result information) for the still images. Here, for the still images that have been transmitted to the PC 400, the control unit 301 associates information indicating that the still images have been transmitted with the still images and stores the information in the database. This allows the control unit 301 to prevent the same still image from being transmitted to the PC 400 multiple times.

[0090] The operation of the PC 400 according to the first embodiment will be described with reference to the flowchart of Fig. 9. The process of the flowchart of Fig. 9 starts when the PC 400 is started.

[0091] In step S901, the control unit 401 starts monitoring the database of the server 300 (the database of the recording medium 304).

[0092] In step S902, the control unit 401 determines whether or not a new video (a video that has not been transmitted to the PC 400) exists in the database of the server 300. If it is determined that a new video (hereinafter referred to as a "new video") exists in the database, the process proceeds to step S903. If it is determined that a new video does not exist in the database, the process proceeds to step S910.

[0093] In step S903, the control unit 401 requests the server 300 to transmit a new video.

[0094] In step S904 , the control unit 401 receives a new video from the server 300 .

[0095] In step S905, the control unit 401 acquires a thumbnail of the new video based on the received new video, and then displays (adds) the thumbnail of the new video in an area corresponding to the time period during which the new video was shot in the timeline area 602 shown in FIG.

[0096] In step S910, the control unit 401 determines whether or not a new still image (a still image that has not been transmitted to the PC 400) exists in the database of the server 300. If it is determined that a new still image (hereinafter referred to as a "new still image") exists in the database, the process proceeds to step S911. If it is determined that a new still image does not exist in the database, the process proceeds to step S901.

[0097] In step S911, the control unit 401 requests the server 300 to transmit a new still image.

[0098] In step S912 , the control unit 401 receives the new still image and the state determination result of the new still image from the server 300 .

[0099] In step S913, the control unit 401 determines whether the new still image satisfies a predetermined condition based on the received state determination result. Here, the predetermined condition is a condition for determining whether an appropriate still image has been captured. Specifically, the predetermined condition may be a condition that the image is captured with a specific composition, a condition that a specific subject does not have their eyes closed, or a condition that a specific subject is in focus. Alternatively, the predetermined condition may be a condition that the image is not captured in backlight, a condition that a person in the still image is looking toward the camera 100, or a condition that multiple people are captured in the still image. If it is determined that the new still image satisfies the predetermined condition, the process proceeds to step S914. If it is determined that the new still image does not satisfy the predetermined condition, the process proceeds to step S915.

[0100] In step S914, the control unit 401 displays (adds) a new still image in the Good area 601 shown in FIG.

[0101] In step S915, the control unit 401 displays (adds) the new still image in the area corresponding to the time period in which the new still image was captured in the timeline area 602 shown in FIG.

[0102] In step S916, the control unit 401 displays (adds) the new still image together with a Good icon in the area corresponding to the time period in which the new still image was captured in the timeline area 602 shown in FIG.

[0103] Through the processing of steps S915 and S916, the new video and new still images captured in the same time period are linked and displayed in the timeline area 602.

[0104] In step S920, the control unit 401 determines whether or not a still image has been selected in the Good area 601 by a user operation. Note that the control unit 401 may also determine whether or not a still image has been selected from either the Good area 601 or the timeline area 602. If it is determined that a still image has been selected, the process proceeds to step S921. If it is determined that a still image has not been selected, the process proceeds to step S901.

[0105] In step S921, the control unit 401 selects a moving image (hereinafter referred to as a "selected moving image") that was shot in the same time period as the still image selected in step S920 (hereinafter referred to as a "selected still image").

[0106] In step S922, the control unit 401 inserts (places) the selected still image in the still image area 603A shown in FIG.

[0107] In step S923, the control unit 401 inserts (places) the selected video in the video area 603B. At this time, the user (editor) can preview (check) the entire news article (news article content), as shown in the article area 603 in FIG.

[0108] In step S924, the control unit 401 determines whether an instruction to end content generation for the news article has been issued by pressing the publish button 604 shown in FIG. 5 . If it is determined that an instruction to end content generation has been issued, the process proceeds to step S925. If it is determined that an instruction to end content generation has not been issued, the process proceeds to step S920. Note that, until the publish button 604 is pressed, the form (position, size, etc.) of a selected still image in a news article or the form of a selected video may be adjustable by a user operation. Furthermore, the text of the article may be changeable by a user operation.

[0109] In step S925, the control unit 401 generates a news article (news article content) including the selected still image, the selected video, and the text of the article.

[0110] In step S926, the control unit 401 determines whether an instruction to upload the news article (news article content) has been issued after the user has confirmed the final confirmation message before publication. If it is determined that an instruction to upload has been issued, the process proceeds to step S927. If it is determined that an instruction to upload has not been issued, the process proceeds to step S924.

[0111] In step S927, the control unit 401 transmits the news article to the distribution server, which then makes the news article available on the Internet.

[0112] According to the first embodiment, by viewing the timeline area, a user can grasp which videos and still images were shot in the same time period. Furthermore, a Good icon is attached to still images that meet certain conditions, allowing the user to select a more suitable still image. Furthermore, when a still image is selected, videos shot in the same time period as the selected still image are selected. Therefore, the user can more easily select still images and videos shot in the same time period. Furthermore, when a still image is selected, the still image and video shot in the same time period are inserted into the news article content, making it easier for the user to create a news article.

[0113] In a second embodiment, the PC 400 displays the reception status of a still image and the reception status of a status determination result in response to a signal received from the server 300. Note that, in the following, descriptions of the configurations and processes that are the same between the first embodiment and the second embodiment will be omitted as appropriate.

[0114] 6 shows an example of a screen display of an editing application running on the PC 400 according to the second embodiment. As in the first embodiment, the editing application displays videos and still images received from the server 300 (camera 100) and allows the user to select still images and videos to use in a news article. By using the editing application, the user can edit the text of the news article and publish the news article including the selected still images and videos.

[0115] In the second embodiment, a still image reception status icon 605 is displayed in the timeline area 602 (an area that displays on a timeline the moving images and still images received from the server 300) when a still image has not yet been received from the server 300. The still image reception status icon 605 is a display item that indicates that a still image will be transmitted in the future. At this time, the PC 400 also displays a Good icon if the still image to be transmitted in the future satisfies predetermined conditions, as in the first embodiment. Thereafter, when the PC 400 receives a still image, it replaces the still image reception status icon 605 with the received still image.

[0116] Furthermore, when the status determination result has not been received from the server 300, the PC 400 displays a status determination result reception status icon 606 together with a still image (or a still image reception status icon 605). The reception status icon 606 is a display item indicating that the status determination result has not been received (it is unclear whether the still image satisfies a predetermined condition). When the PC 400 receives the status determination result, it hides the status determination result reception status icon 606.

[0117] The operation of the camera 100 according to the second embodiment will be described with reference to the flowchart in Fig. 10. In the second embodiment, the process of step S1008 is performed in addition to the process of the flowchart in Fig. 7. As shown in Fig. 10, step S1008 is performed between step S707 and step S708.

[0118] In step S1008, the system control unit 50 transmits a shooting information signal indicating information such as the file name and shooting date and time of the still image (the still image captured in step S707) to the server 300. In other words, the shooting information signal is transmitted to the server 300 before the still image is transmitted.

[0119] The operation of the server 300 according to the second embodiment will be described with reference to the flowchart of Fig. 11. The processing of the flowchart of Fig. 11 starts when the server 300 receives a shooting information signal or an image (moving image or still image) from the camera 100.

[0120] In step S1101, the control unit 301 determines whether or not a shooting information signal has been received from the camera 100. If it is determined that a shooting information signal has been received, the process proceeds to step S1102. If it is determined that a shooting information signal has not been received, the process proceeds to step S1103.

[0121] In step S1102 , the control unit 301 transmits the received shooting information signal to the PC 400 .

[0122] Steps S1103 to S1108 are similar to steps S801 to S806.

[0123] In step S1109 , the control unit 301 generates a determination signal indicating the file name of the still image and the result of the state determination, and transmits the determination signal to the PC 400 .

[0124] In step S1110, the control unit 301 stores the received still image in a database on the recording medium 304.

[0125] In step S1111, the control unit 301 determines whether or not a request to transmit a still image has been made by the PC 400. If it is determined that a request to transmit a still image has been made, the process proceeds to step S1112. If it is determined that a request to transmit a still image has not been made, the process of step S1111 is repeated.

[0126] In step S1112, the control unit 301 transmits to the PC 400 any still images that have not yet been transmitted to the PC 400.

[0127] The operation of the PC 400 according to the second embodiment will be described with reference to the flowchart in Fig. 12. In the second embodiment, in addition to the flowchart in Fig. 9, the process of step S1200 (steps S1328 to S1332 in Fig. 13) is executed. Note that in step S912, the determination signal has already been received, so the control unit 401 receives only the new still image. In step S913, the control unit 401 determines whether the new still image satisfies a predetermined condition based on the determination signal.

[0128] In step S1200, the control unit 401 executes processing in response to reception of the image capturing signal and the determination signal. Details of the processing in step S1200 will be described with reference to the flowchart of FIG.

[0129] In step S1328, the control unit 401 determines whether or not an image capturing information signal has been received from the server 300. If it is determined that an image capturing information signal has been received, the process proceeds to step S1329. If it is determined that an image capturing information signal has not been received, the process proceeds to step S1330.

[0130] In step S1329, the control unit 401 displays a still image reception status icon 605 in the timeline area 602 shown in FIG. 6 . At this time, the control unit 401 references the shooting time periods of the already received videos to determine which video shooting time period includes the shooting date and time recorded in the shooting information signal. Then, the control unit 401 displays the still image reception status icon 605 at a position where a still image in the determined shooting time period should be placed. Here, the control unit 401 calculates the video shooting time period based on the shooting start time and shooting end time (or the video shooting start time and playback time). Note that when reception of the still image is complete, the control unit 401 replaces the still image reception status icon 605 with the still image.

[0131] In step S1330, the control unit 401 determines whether or not a determination signal has been received from the server 300. If it is determined that a determination signal has been received, the process proceeds to step S1331. If it is determined that a determination signal has not been received, the process proceeds to step S1332.

[0132] In step S1331, the control unit 401 displays an icon corresponding to the status determination result of the still image based on the file name recorded in the determination signal. At this time, the control unit 401 superimposes the icon corresponding to the status determination result of the still image on the thumbnail of the still image or the still image reception status icon 605. Here, the icon corresponding to the status determination result is a Good icon if the still image corresponding to the status determination result satisfies a predetermined condition.

[0133] In step S1332, the control unit 401 displays the reception status icon 606 of the status determination result superimposed on the still image thumbnail or the still image reception status icon 605. When the control unit 401 receives a determination signal, it hides the reception status icon 606 of the status determination result or replaces the reception status icon 606 of the status determination result with a Good icon or the like according to the status determination result.

[0134] If the still image reception status icon 605 is displayed in step S1329, the still image reception status icon 605 is then replaced with a thumbnail of the still image in steps S915 and S916. If the status determination result reception status icon 606 is displayed in step S1332, the status determination result reception status icon 606 is then replaced with an icon according to the status determination result in step S916.

[0135] According to the second embodiment, the PC 400 displays the reception status of the still image and the reception status of the status determination result in response to the received shooting information signal and determination signal. This allows the user to know that a still image that meets a predetermined condition will be transmitted to the PC 400 in the future, even if there is a delay in the transmission of the still image or in the processing by the server 300, for example.

[0136] Third Embodiment In a third embodiment, an example will be described in which editing processing of still images and moving images is added to the PC 400 as in the following (1) to (9) in comparison with the first embodiment. Note that in the third embodiment, descriptions of the same configurations and processing as in the first embodiment will be omitted as appropriate.

[0137] (1) The PC 400 can acquire incidental information recorded in images (still images and videos). Specifically, the PC 400 can acquire incidental information from images (still images and videos) recorded on the recording medium 404. The incidental information includes shooting information (information such as the shutter speed or aperture at the time of shooting), attitude information of the camera 100 at the time of shooting, or autofocus information (information such as focus coordinates), shooting date and time information, etc. The incidental information for videos also includes information for each frame. Note that, although an example has been shown in which incidental information is recorded in images (still images and videos), the incidental information may also be recorded in a separate file associated with the image.

[0138] (2) The PC 400 can specify the range of extraction of a still image. Specifically, the PC 400 can specify the range of extraction of a still image in response to a user operation on the operation unit 405. In the third embodiment, the user specifies a rectangular area including a subject area as the range of extraction of a still image. However, when the user specifies the subject (the position of the subject), the PC 400 may specify a rectangle including the specified subject area. Furthermore, the aspect ratio of the rectangle of the extraction range is not particularly limited.

[0139] (3) The PC 400 can detect an area including a subject in a still image. Specifically, the PC 400 detects the subject from the still image and detects an area including the subject (subject area). The method for detecting the subject is not particularly limited, and a known method may be used.

[0140] (4) The PC 400 can determine the cropping range of a still image (the area in the still image to be cropped). Specifically, the PC 400 determines the cropping range based on the accompanying information, the information on the specified cropping range, the information on the subject area, or the like.

[0141] (5) The PC 400 can cut out a partial area of ​​a still image. Specifically, as shown in Fig. 14A, the PC 400 cuts out a partial area from a still image to generate a new still image.

[0142] (6) The PC 400 can determine a frame of a video corresponding to a still image (hereinafter referred to as a "corresponding frame"). Specifically, the PC 400 determines a frame of a video generated during a time period corresponding to the still image, which frame depicts the same scene as the still image, as the corresponding frame. In the third embodiment, the PC 400 acquires the shooting dates and times of the still image and video from the accompanying information of the still image and video, and determines, as the corresponding frame, the frame of all frames of the video whose shooting date and time is closest to that of the still image. The corresponding frame may be determined by any method, such as "a method of comparing the degree of similarity between two images" or "a method of determining corresponding frames based on the correspondence between identifiers (GUIDs) recorded in the accompanying information of the video and still image at the time of shooting."

[0143] (7) The PC 400 can acquire information used to determine the cutout range of a still image (referred to as "range determination information"). The PC 400 acquires the range determination information and also acquires the reason why the cutout range was determined.

[0144] (8) The PC 400 can extract a portion of a playback section from a video. Specifically, the PC 400 extracts a range of the portion of the playback section from the video to generate a new video. The extracted playback section is determined based on range determination information so as to include a corresponding frame (a frame corresponding to a still image). In embodiment 3, the PC 400 determines, for example, as shown in FIG. 14B , the playback section to be extracted is the playback section that includes a subject that was focused on when the still image region was extracted. The PC 400 may also change the length of the extracted playback section depending on the posting destination of the video (e.g., social media or a news article). If the extracted playback section does not reach a specific time in a single video, the PC 400 may extract portions of the playback sections from multiple videos with consecutive shooting times, including the selected video, and combine the extracted playback sections.

[0145] (9) The PC 400 can cut out a partial range of the image of each frame of a video. Specifically, after determining a "video cutout range," the PC 400 cuts out the cutout range from the image of each frame and treats the cutout range of each frame as the image of each frame of a new video. In this way, the PC 400 generates a new video. In embodiment 3, the PC 400 determines the cutout range for each frame in the same manner as for a still image, as shown in FIG. 14C. The PC 400 then calculates a range that includes all of the cutout ranges determined for each frame (the range surrounded by the thickest frame in FIG. 14C) and sets this range as the "video cutout range" (the range to be cut out from each frame). In other words, the range in which a specific subject appears in the image of all frames (each frame) is determined.

[0146] The operation of the PC 400 in the third embodiment will be described with reference to the flowchart in Fig. 15. The processing in the flowchart in Fig. 15 is executed between steps S921 and S922 in the flowchart in Fig. 9 according to the first embodiment. The other processing in the flowchart in Fig. 9 (the processing in steps S901 to S927) is executed in the same manner as in the first embodiment.

[0147] In step S912, the control unit 401 receives not only the still image and the state determination result, but also information on the target subject area at the time of determining the state of the still image as "information used in determining the state of the still image." The target subject area is, for example, the area of ​​the subject that is the target of determination of whether the subject is blinking or whether the subject is in focus. For example, the target subject area is the area of ​​a specific person. In the third embodiment, "information on the target subject area" is used as an example of "information used in determining the state of the still image." However, the "information used in determining the state of the still image" may be the coordinates used in determining the state of the still image, or face information used in determining the state of the still image, or the like.

[0148] In step S1501, the control unit 401 acquires the still image selected in step S920 (selected still image).

[0149] In step S1502, the control unit 401 determines whether the selected still image satisfies a predetermined condition. If it is determined that the selected still image satisfies the predetermined condition, the process proceeds to step S1503. If it is determined that the selected still image does not satisfy the predetermined condition, the process proceeds to step S1505.

[0150] In step S1503, the control unit 401 acquires information on the target subject area at the time of determining the state of the selected still image.

[0151] In step S1504, the control unit 401 identifies a range in the selected still image that includes the target object based on the target object region information acquired in step S1503. The control unit 401 determines the identified range as the "cutout range" of the selected still image.

[0152] In step S1505, the control unit 401 determines whether or not the user has specified a cutout range for the selected still image. If it is determined that the user has specified a cutout range for the selected still image, the process proceeds to step S1506. If it is determined that the user has not specified a cutout range for the selected still image, the process proceeds to step S1507.

[0153] In step S1506, the control unit 401 determines the range specified by the user (the range in the selected still image) as the "cut-out range" of the selected still image.

[0154] In step S1507, the control unit 401 acquires the supplementary information of the selected still image.

[0155] In step S1508, the control unit 401 detects one or more subjects in the selected still image, and then detects the subject closest to the focus coordinates included in the supplementary information as the subject of interest.

[0156] In step S1509, the control unit 401 specifies a range including the target subject based on the target subject detected in step S1508. The control unit 401 determines the specified range as the "cutout range" of the selected still image.

[0157] In step S1510, the control unit 401 cuts out the determined cut-out range from the selected still image to generate a new still image. As a result, the control unit 401 can delete the unnecessary area and obtain a new still image (hereinafter referred to as a "cut-out still image") suitable for the news article.

[0158] In step S1511, the control unit 401 acquires the moving image selected in step S921 (selected moving image).

[0159] In step S1512, the control unit 401 cuts out a moving image of a partial playback section of the selected moving image based on the information used to determine the cut-out range of the selected still image, thereby generating a new moving image.

[0160] In step S1513, the control unit 401 determines a "video clip cutout range" for the video generated in step S1512. The control unit 401 generates a new video (hereinafter referred to as a "cut-out video") by cutting out the "video clip cutout range" from the images of each frame of the video generated in step S1512. Here, the cut-out video is generated so that the subject of interest is captured in the image of each frame of the cut-out video. This makes it possible to obtain a new video that is suitable for a news article, with unnecessary playback sections and unnecessary areas deleted. Note that the "video clip cutout range" can also be determined based on information used to determine the cut-out range of the selected still image (information on the range specified by the user, information on the subject of interest, or additional information on the selected still image).

[0161] In step S1514, the control unit 401 replaces the selected still image and selected moving image selected in steps S920 and S921 with the extracted still image and extracted moving image newly generated in steps S1510 and S1513. In other words, the control unit 401 performs control so that the extracted still image and extracted moving image are used in the processing of steps S922 and S923.

[0162] According to the third embodiment, unnecessary information can be deleted from the selected still image and the selected video, so that still images and videos more suitable for the news article can be easily generated. In other words, more suitable news articles can be generated while saving the user effort.

[0163] Furthermore, in the above, "If A is equal to or greater than B, proceed to step S1; if A is smaller (lower) than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1; if A is equal to or less than B, proceed to step S2." Conversely, "If A is greater (higher) than B, proceed to step S1; if A is equal to or less than B, proceed to step S2" may be read as "If A is equal to or greater than B, proceed to step S1; if A is smaller (lower) than B, proceed to step S2." Therefore, unless a contradiction arises, "equal to or greater than A" may be read as "greater (higher; longer; more) than A," and "equal to or less than A" may be read as "smaller (lower; shorter; fewer) than A." Furthermore, "greater (higher; longer; more) than A" may be read as "greater than A," and "smaller (lower; shorter; fewer) than A" may be read as "equal to or less than A."

[0164] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0165] The processor is a processor in the broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, a central processing unit (CPU), a micro processing unit (MPU), and a digital signal processor (DSP). Dedicated processors include, for example, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a programmable logic device (PLD). Examples of programmable logic devices include a field programmable gate array (FPGA) and a complex programmable logic device (CPLD).

[0166] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0167] The present invention can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit that realizes one or more functions.

[0168] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.

[0169] This application claims priority based on Japanese Patent Application No. 2023-162921, filed on September 26, 2023, the entire contents of which are incorporated herein by reference.

[0170] 400: PC, 401: control unit, 406: display unit, 407: communication unit

Claims

1. An electronic device that controls the display of a specific screen displayed on a display means, comprising: an acquisition means for acquiring a plurality of videos shot during each of a plurality of consecutive time periods and a plurality of still images taken while the plurality of videos are being shot; and a control means for 1) displaying the plurality of videos in chronological order in a first area of ​​the specific screen, and 2) for each time period in which one or more still images were shot among the plurality of time periods, linking the video shot during that time period and the one or more still images shot during that time period to each other and displaying them in the first area.

2. The electronic device according to claim 1, further comprising a selection means for selecting, when one still image is selected from the plurality of still images, a first video that was shot during the time period in which the selected first still image was shot.

3. The electronic device according to claim 2, characterized in that the specific screen has a second area in which mutually related videos and still images are arranged, and when the first still image is selected, the control means arranges the first still image and the first video selected by the selection means in the second area.

4. The electronic device according to claim 3, characterized in that, when the first still image is selected, the control means places a first range, which is a cut-out portion of the first still image, in the second area.

5. The electronic device according to claim 4, characterized in that the control means determines the first range based on a user's specification, information about a specific subject in the first still image, or supplementary information of the first still image.

6. The electronic device described in any one of claims 3 to 5, characterized in that, when the first still image is selected, the control means places in the second area a second video that is cut out of at least either a partial range of the playback section of the first video or a partial range of an image of each frame of the first video.

7. The electronic device according to claim 6, characterized in that the control means determines the range of the playback section of a portion of the first video based on a user's specification, information about a specific subject in the first still image, or supplementary information about the first still image.

8. The electronic device according to claim 6 or 7, characterized in that the control means cuts out a portion of the image of each frame of the first video so that a specific subject is captured in the image of each frame of the second video.

9. The electronic device according to claim 5 or 7, characterized in that the additional information includes at least one of the following information at the time of shooting: shutter speed information, aperture information, information on the attitude of the imaging device, autofocus information, and information on the date and time of shooting.

10. An electronic device as described in any one of claims 3 to 9, characterized in that the second area contains content including mutually related videos and still images, and the content includes text, and the electronic device further has an output means for outputting the content.

11. The electronic device described in any one of claims 1 to 10, characterized in that the acquisition means acquires the multiple still images transmitted from an external device, and the control means, when the acquisition means has not acquired any of the multiple still images, displays in the first area a first display item indicating that the still image is to be transmitted, in place of the still image not acquired by the acquisition means.

12. The electronic device according to claim 11, characterized in that when the acquisition means acquires a still image corresponding to the first display item displayed in the first area, the control means replaces the first display item displayed in the first area with the still image.

13. The electronic device according to any one of claims 1 to 12, characterized in that, when a still image satisfying a predetermined condition is displayed in the first area, the control means further displays a second display item indicating that the still image satisfies the predetermined condition.

14. The electronic device according to claim 13, characterized in that the specific screen has a third area different from the first area, and the control means arranges one or more still images among the plurality of still images that satisfy the specified condition and displays them in the third area.

15. The electronic device described in claim 13 or 14, characterized in that the acquisition means acquires information on a judgment result indicating whether or not each of the multiple still images satisfies the specified condition, and the control means, when the information on the judgment result of any of the multiple still images has not been acquired by the acquisition means, displays a third display item indicating that the judgment result has not been acquired, at a position in the first area corresponding to the still image for which the judgment result has not been acquired.

16. The electronic device according to claim 15, characterized in that when the acquisition means acquires information on the judgment result corresponding to the third display item displayed in the first area, the control means switches the third display item displayed in the first area to a display corresponding to the judgment result.

17. The electronic device according to any one of claims 13 to 16, characterized in that the specified condition is a condition that the photograph is of a specific composition, a condition that a specific subject is not blinking, or a condition that the specific subject is in focus.

18. A control method for an electronic device that controls the display of a specific screen displayed on a display means, comprising: an acquisition step of acquiring a plurality of videos shot during each of a plurality of consecutive time periods and a plurality of still images shot during the shooting of the plurality of videos; and a control step of 1) displaying the plurality of videos in chronological order in a first area on the specific screen, and 2) for each time period in which one or more still images were shot among the plurality of time periods, linking the video shot during that time period and the one or more still images shot during that time period to each other and displaying them in the first area.

19. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 17.

Images