Electronic device, method for controlling electronic device, and program

Abstract

The electronic device controls display of a specific screen displayed on a display section, and acquires a plurality of moving images captured in each of a plurality of consecutive time blocks and a plurality of still images captured during capturing of the plurality of moving images. The electronic device 1) displays a plurality of moving images in a first region of a specific screen in a chronological order, and 2) displays a plurality of moving images in a second region of the specific screen for each time block in which one or more still images are captured among a plurality of time blocks, the moving image captured in the time block and the one or more still images captured in the time block are displayed in the first area in association with each other.

Description

Technical Field

[0001] This invention relates to electronic devices, methods and procedures for controlling electronic devices. Background Technology

[0002] Digital cameras are known capable of capturing still images while simultaneously capturing moving images without stopping the moving image capture. In news reporting and other applications, to save manpower, a single photographer often captures both moving and still images (hereinafter, moving and still images are collectively referred to as "capture data"). Therefore, it is important to be able to capture still images using a single camera while capturing moving images.

[0003] When using footage to generate news articles, photographers transmit the footage to editors via the internet, and the editors then categorize and edit the footage.

[0004] However, because moving and still images are stored as separate files, it can be difficult to identify the relationship between them. Therefore, in PTL 1, still images are displayed in association with the timeline during the playback of moving images to allow for easy identification of the relationship between moving and still images.

[0005] [List of Citations]

[0006] [Patent Literature]

[0007] [Patent Document 1] Japanese Patent Publication No. 2010-147861 Summary of the Invention

[0008] [Technical Issues]

[0009] In PTL 1, it is easy to identify the relationship between a selected moving image and multiple still images. However, it is not possible to simultaneously identify the relationship between multiple moving images and multiple still images. In particular, when generating news articles, etc., classification operations are performed among multiple moving images and multiple still images in a short period of time. Therefore, even using the technique according to the first embodiment of PTL 1, it is difficult to perform classification operations.

[0010] Therefore, the object of the present invention is to provide a technique for reducing the workload of classifying images from multiple moving images and multiple still images.

[0011] [Solution to the problem]

[0012] One aspect of the present invention is an electronic device for controlling the display of a specific screen on a display unit, the electronic device comprising: an acquisition unit that acquires a plurality of moving images captured during a plurality of consecutive time blocks and a plurality of still images captured during the capture of the plurality of moving images; and a control unit that 1) displays the plurality of moving images in chronological order in a first area of ​​the specific screen, and 2) for each time block in which one or more still images are captured, displays the moving images captured during that time block and the one or more still images captured during that time block in association with each other in the first area.

[0013] One aspect of the present invention is a method for controlling an electronic device for controlling the display of a specific screen on a display component, the method comprising: acquiring a plurality of moving images captured during a plurality of consecutive time blocks and a plurality of still images captured during the capture of the plurality of moving images; and performing a control process, the control process 1) displaying the plurality of moving images in chronological order in a first area of ​​the specific screen, and 2) for each time block in which one or more still images are captured, displaying the moving images captured during that time block and the one or more still images captured during that time block in association with each other in the first area.

[0014] [Beneficial effects of the present invention]

[0015] According to the present invention, the workload of classifying images from multiple moving images and multiple still images can be reduced. Attached Figure Description

[0016] [ Figure 1 ] Figure 1 This is an overall configuration diagram of the shooting / editing system according to the first embodiment.

[0017] [ Figure 2 ] Figure 2 (A) and Figure 2 (B) is an external view of the camera according to the first embodiment.

[0018] [ Figure 3 ] Figure 3 This is a configuration diagram of the camera according to the first embodiment.

[0019] [ Figure 4 ] Figure 4 (A) and Figure 4 (B) is a configuration diagram of the server and PC according to the first embodiment.

[0020] [ Figure 5 ] Figure 5 This is a diagram illustrating a screen of an editing application according to the first embodiment.

[0021] [ Figure 6 ] Figure 6 This is a diagram illustrating a screen of an editing application according to the second embodiment.

[0022] [ Figure 7 ] Figure 7 This is a flowchart of the operation of the camera according to the first embodiment.

[0023] [ Figure 8 ] Figure 8 This is a flowchart of the operation of the server according to the first embodiment.

[0024] [ Figure 9 ] Figure 9 This is a flowchart of the operation of the PC according to the first embodiment.

[0025] [ Figure 10 ] Figure 10 This is a flowchart of the operation of the camera according to the second embodiment.

[0026] [ Figure 11 ] Figure 11 This is a flowchart of the operation of the server according to the second embodiment.

[0027] [ Figure 12 ] Figure 12 This is a flowchart of the operation of the PC according to the second embodiment.

[0028] [ Figure 13 ] Figure 13 This is a flowchart of the processing in response to a shooting signal, etc., according to the second embodiment.

[0029] [ Figure 14 ] Figure 14 (A) to Figure 14 (C) is a diagram illustrating the editing process according to the third embodiment.

[0030] [ Figure 15 ] Figure 15 This is a flowchart of the operation of the PC according to the third embodiment. Detailed Implementation

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0032] First Embodiment

[0033] Figure 1This is a schematic diagram illustrating an example of the overall configuration of a shooting / editing system according to a first embodiment. The shooting / editing system according to the first embodiment includes a camera 100, a server 300, and a personal computer (PC) 400. The server 300 and PC 400 are information processing devices (electronic devices) that process images captured by an image capturing device such as the camera 100. Figure 1 The illustration shows a configuration in which the camera 100 and PC 400 communicate with each other wirelessly or via a wired connection.

[0034] Figure 2 (A) and Figure 2 (B) is an external view illustrating an example of the appearance of a camera 100 used as an image capturing device. Figure 2 (A) is a perspective view of camera 100 when viewed from the front, and Figure 2 (B) is a perspective view from the rear view of camera 100.

[0035] The camera 100 includes a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a secondary electronic dial 105, a pan image button 106, and an external viewfinder display unit 107 on its upper surface. The shutter button 101 is an operating component used to issue shooting preparation or shooting commands. The power switch 102 is an operating component used to switch the power of the camera 100 between an on and off state. The mode switch 103 is an operating component used to switch between various modes. The main electronic dial 104 is a rotary operating component used to change the settings of shutter speed, aperture, etc. The secondary electronic dial 105 is a rotary operating component used to perform actions such as moving the selection frame (cursor) and image feeding. The pan image button 106 is an operating component used to issue commands to start and stop pan image shooting (recording). The external viewfinder display unit 107 displays various settings such as shutter speed and aperture.

[0036] The camera 100 includes a display unit 108, a touch panel 109, directional keys 110, a SET button 111, an AE lock button 112, a zoom button 113, a playback button 114, a menu button 115, an eyepiece 116, an eye proximity detection unit 118, and a touch bar 119 on its rear surface. The display unit 108 displays images and various types of information. The touch panel 109 is an operating component used to detect touch operations on the display surface (touch operation surface) of the display unit 108. The directional keys 110 are an operating unit configured to include keys that can be pressed up, down, left, and right (four-way keys). The camera 100 can perform processing based on the pressed position of the directional keys 110. The SET button 111 is an operating component that is normally pressed when confirming the selected item. The AE lock button 112 is an operating component that is pressed when fixing the exposure state in the shooting standby state. The magnification button 113 is an operating component used to switch the magnification mode between an on and off state in the live view display (LV display) of shooting mode. When the magnification mode is on, the main electronic dial 104 is operated to magnify or reduce the live view image (LV image). Additionally, the magnification button 113 is used to magnify the playback image or increase the magnification in playback mode. The playback button 114 is an operating component used to switch between shooting mode and playback mode. In shooting mode, pressing the playback button 114 switches the shooting mode to playback mode, and the latest image from the images recorded on the recording medium 227, described later, can be displayed on the display unit 108.

[0037] The menu button 115 is an operating component that is pressed to display a menu screen on the display unit 108 that allows for various settings. Users can intuitively perform various settings using the menu screen displayed on the display unit 108, the directional keys 110, and the SET button 111. The eyepiece 116 is the part where the user brings their eye closer to the eyepiece viewfinder (inward-looking viewfinder) 117 and looks inside the eyepiece viewfinder 117. Users can visually recognize the video displayed on the EVF 217 (electronic viewfinder), which will be described later, inside the camera 100, via the eyepiece 116. The eye proximity detection unit 118 is a sensor used to detect whether the user brings their eye closer to the eyepiece 116 (eyepiece viewfinder 117).

[0038] The touch bar 119 is a linear touch operation component (linear touch sensor) capable of receiving touch operations. The touch bar 119 is positioned such that while the user holds the grip portion 120 with their right hand (while holding the grip portion 120 with their right pinky, ring finger, and middle finger), they can perform a touch operation (touch) on the touch bar 119 with their right thumb to press the shutter button 101 with their right index finger. In other words, the touch bar 119 is operable when the user brings their eye closer to the viewfinder 117 to look into the viewfinder portion 116, and is always ready (in the shooting posture) to press the shutter button 101. The touch bar 119 can receive tap operations (operations where the touch bar 119 is touched and released within a predetermined time period after a touch without moving the touched position), lateral sliding operations (operations where the touch bar 119 is touched and then the touched position is moved while the touch is maintained), etc. The touch bar 119 is an operating component different from the touch panel 109 and does not have a display function. The touch bar 119 is used as a multi-function bar (M-Fn bar) that can be assigned various functions, for example.

[0039] The camera 100 also includes a grip 120, a thumb rest 121, a terminal cover 122, a cap 123, a communication terminal 124, etc. The grip 120 is shaped to allow the user to easily hold the camera 100 with his or her right hand. The shutter button 101 and the main electronic dial 104 are positioned so that the user can operate the shutter button 101 and the main electronic dial 104 with his or her right index finger while holding the camera 100 with his or her right little finger, right ring finger, and right middle finger. Simultaneously, in a similar configuration, the secondary electronic dial 105 and the touch bar 119 are positioned so that the user can operate the secondary electronic dial 105 and the touch bar 119 with his or her right thumb. The thumb rest 121 (thumb spare position) is a grip provided on the rear part of the camera 100, where the user can easily place his or her right thumb to hold the grip 120 without operating any operating components. The thumb rest 121 is made of a rubber component or the like to enhance holding force (grip). The terminal cover 122 protects connectors such as connecting cables used to connect the camera 100 to external devices (external equipment). The cover 123 protects the recording medium 227, described later, and the slot by closing it. The communication terminal 124 is a terminal for communicating with a lens unit (such as the lens unit 200 described later) detachably from the camera 100.

[0040] Figure 3 This is a block diagram illustrating a configuration example of camera 100. Note that, compared to... Figure 2 The same component in the component is made by the same component as the component in the component. Figure 2 The same reference numerals are used to denote the parts, and descriptions of the parts are appropriately omitted. Figure 3 In the middle, the lens unit 200 is attached to the camera 100.

[0041] First, a description of the lens unit 200 will be given. The lens unit 200 is a replaceable lens type that is detachable from the camera 100. The lens unit 200 is a single lens unit and is an example of a typical lens. The lens unit 200 includes an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, etc.

[0042] Aperture 201 is configured to have an adjustable aperture. Lens 202 is configured to include multiple lenses. Aperture drive circuit 203 adjusts the amount of light by controlling the aperture 201. AF drive circuit 204 drives lens 202 for focusing. Lens system control circuit 205 controls aperture drive circuit 203, AF drive circuit 204, etc., based on instructions from system control unit 50 described later. Lens system control circuit 205 controls aperture 201 via aperture drive circuit 203 and changes the position of lens 202 for focusing via AF drive circuit 204. Lens system control circuit 205 is capable of communicating with camera 100. Specifically, communication is performed via communication terminal 206 of lens unit 200 and communication terminal 124 of camera 100. Communication terminal 206 is a terminal for communication between lens unit 200 and camera 100.

[0043] The following is a description of the camera 100. The camera 100 includes a shutter 210, an image capture unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0044] Shutter 210 is a focal plane shutter capable of freely controlling the exposure time of image capture unit 211 based on instructions from system control unit 50. Image capture unit 211 is an image capture element (image sensor) configured to include CCD, CMOS, or other elements that convert optical images into electrical signals. Image capture unit 211 may also have an imaging plane phase difference sensor that outputs defocus information to system control unit 50. A / D converter 212 converts the analog signal output from image capture unit 211 into a digital signal. Image processing unit 214 performs predetermined processing (such as pixel interpolation, size adjustment processing such as reduction, or color conversion processing) on ​​data from A / D converter 212 or data from memory control unit 213. In addition, image processing unit 214 performs predetermined arithmetic processing using captured image data, and system control unit 50 performs exposure control and distance control based on the obtained arithmetic results. Through this processing, TTL (through-lens) type AF processing, AE (auto exposure) processing, EF (flash pre-emission) processing, etc., are performed. Furthermore, the image processing unit 214 performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs TTL-type AWB (automatic white balance) processing based on the obtained arithmetic results.

[0045] Image data from A / D converter 212 is written to memory 215 via image processing unit 214 and memory control unit 213. Alternatively, image data from A / D converter 212 is written to memory 215 via memory control unit 213 without going through image processing unit 214. Memory 215 stores image data acquired by image capture unit 211 and converted into digital data by A / D converter 212. Memory 215 stores image data to be displayed on display unit 108 or EVF 217. Memory 215 has sufficient storage capacity to store a predetermined number of still images or moving images and audio over a predetermined period. Memory 215 also serves as memory for image display (video memory).

[0046] D / A converter 216 converts the image data stored in memory 215 for display into an analog signal and supplies the analog signal to display unit 108 and EVF 217. Therefore, the image data written to memory 215 for display is displayed on display unit 108 or EVF 217 via D / A converter 216. Display unit 108 and EVF 217 perform display based on the analog signal from D / A converter 216. For example, each of display unit 108 and EVF 217 is a display such as an LCD or OLED. Digital signals that have undergone A / D conversion by A / D converter 212 and are stored in memory 215 are converted into analog signals by D / A converter 216. The resulting analog signals are sequentially transmitted to display unit 108 and EVF 217, and an image based on these analog signals is displayed on display unit 108 and EVF 217. Thus, live view display is performed.

[0047] The system control unit 50 is a control unit that includes at least one processor and / or at least one circuit. In other words, the system control unit 50 may also be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 executes programs recorded in the non-volatile memory 219 to implement each process in the flowchart described later. In addition, the system control unit 50 also performs display control by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, etc.

[0048] Meanwhile, the camera 100 has a system memory 218, a non-volatile memory 219, a system timer 220, a communication unit 221, a posture detection unit 222, and an eye proximity detection unit 118.

[0049] The system memory 218 is, for example, RAM. Constants, variables, and programs read from non-volatile memory 219, used for the operation of the system control unit 50, are loaded into the system memory 218. The non-volatile memory 219 is an electrically erasable / recordable memory. For example, an EEPROM is used as the non-volatile memory 219. Constants, programs, and other information for the operation of the system control unit 50 are recorded in the non-volatile memory 219. The program mentioned herein refers to the program used to execute the flowcharts described later. The system timer 220 is a timing unit that measures the time to be used for various controls and the time of the internal clock.

[0050] Communication unit 221 performs the transmission and reception of video and audio signals with external devices connected wirelessly or via wired cable. Communication unit 221 can also connect to a wireless LAN (local area network) or the Internet. Furthermore, communication unit 221 can communicate with external devices via Bluetooth (registered trademark) or Bluetooth Low Energy. Communication unit 221 can transmit images captured by image capture unit 211 (including live images), image filenames, shooting information signals indicating shooting information such as shooting date and time, and images recorded on recording medium 227. Communication unit 221 can receive images and various other information from external devices.

[0051] The posture detection unit 222 detects the posture of the camera 100 in the direction of gravity. Based on the detected posture, the posture detection unit 222 can determine whether the image captured by the image capture unit 211 was captured when the camera 100 was held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the posture detected by the posture detection unit 222 to the image file of the image captured by the image capture unit 211, or it can rotate the image according to the detected posture. For example, an accelerometer, a gyroscope, etc., can be used as the posture detection unit 222. Movements of the camera 100 (such as panning, tilting, raising, or remaining stationary) can also be detected using the posture detection unit 222.

[0052] The eye proximity detection unit 118 is capable of detecting any object approaching the eyepiece section 116 (eyepiece viewfinder 117). For example, an infrared proximity sensor can be used as the eye proximity detection unit 118. When an object approaches the eyepiece section 116, infrared light projected from the light projection portion of the eye proximity detection unit 118 is reflected by the object and received by the light receiving portion of the infrared proximity sensor. Based on the amount of infrared light received, the distance from the eyepiece section 116 to the object can be determined. Therefore, the eye proximity detection unit 118 performs eye proximity detection to detect the approach distance of the object relative to the eyepiece section 116.

[0053] The eye proximity detection unit 118 is an eye proximity detection sensor used to detect the approach (eye approach) and separation (eye separation) of the eye (object) relative to the eyepiece 116. When an object within a predetermined distance is detected approaching the eyepiece 116 from a non-eye proximity state (non-proximity state), the eye proximity detection unit 118 detects eye proximity. Simultaneously, when the detected approaching object is at a predetermined distance or greater than the eye proximity state (proximity state), the eye proximity detection unit 118 detects eye separation. The threshold for detecting eye proximity and the threshold for detecting eye separation can be different from each other, for example, by setting a hysteresis. It is assumed that after detecting eye proximity, the eye proximity state is maintained until eye separation is detected. It is also assumed that after detecting eye separation, the non-eye proximity state is maintained until eye proximity is detected.

[0054] The system control unit 50 switches between display (display state) and non-display (non-display state) states of the display unit 108 and the EVF 217 based on the state detected by the eye proximity detection unit 118. Specifically, when the camera 100 is at least in shooting standby mode and the display destination switching setting is set to automatic switching, during non-eye proximity periods, the system control unit 50 turns on the display by switching the display destination to the display unit 108, while simultaneously putting the EVF 217 into a non-display state. Conversely, during eye proximity periods, the system control unit 50 turns on the display by switching the display destination to the EVF 217, while simultaneously putting the display unit 108 into a non-display state. It should be noted that the eye proximity detection unit 118 is not limited to an infrared proximity sensor. Any other sensor capable of detecting states considered to be eye proximity states can also be used as the eye proximity detection unit 118.

[0055] In addition, the camera 100 includes an external viewfinder display unit 107, an external viewfinder display drive circuit 223, a power control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, etc.

[0056] The external viewfinder display unit 107 is driven by the external viewfinder display drive circuit 223 to display various camera settings such as shutter speed and aperture. The power control unit 224 is configured to include a battery detection circuit, a DC-DC converter, a switching circuit (for switching which blocks are to be powered on), etc. The power control unit 224 performs detections such as whether a battery is attached, the type of battery, and the battery level. Furthermore, based on the detection results and instructions from the system control unit 50, the power control unit 224 controls the DC-DC converter and supplies the required voltage to each of the units, including the recording medium 227, for the required period of time.

[0057] The power supply unit 225 can be a primary battery (such as an alkaline battery or a lithium battery), a secondary battery (such as a NiCd battery, NiMH battery, or Li battery), an AC adapter, etc. The recording medium I / F 226 is an interface (such as a memory card or hard disk) to the recording medium 227. The recording medium 227 is a memory card used to record captured images. The recording medium 227 is composed of semiconductor memory, a magnetic disk, etc. The recording medium 227 can be detached from the camera 100 or embedded within the camera 100.

[0058] The operation unit 228 is an input unit that receives operations (user operations) from the user. The operation unit 228 is used to input various commands to the system control unit 50. The operation unit 228 includes a shutter button 101, a power switch 102, a mode switch 103, a touch panel 109, and other operation units 229. Other operation units 229 include a main electronic dial 104, a secondary electronic dial 105, a pan image button 106, directional keys 110, a SET button 111, an AE lock button 112, a zoom button 113, a playback button 114, a menu button 115, and a touch bar 119.

[0059] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. When the shutter button 101 is operated—that is, when the first shutter switch 230 is activated in a so-called half-pressed state (shooting preparation command)—the first shutter switch 230 outputs a first shutter switch signal SW1. The system control unit 50, in response to the output of the first shutter switch signal SW1, begins shooting preparation processing (such as AF processing, AE processing, AWB processing, or EF processing). When the shutter button 101 is operated—that is, when the second shutter switch 231 is activated in a so-called fully pressed state (shooting command)—the second shutter button 231 outputs a second shutter switch signal SW2. In response to the output of the second shutter switch signal SW2, the system control unit 50 begins a series of shooting processes (from reading signals from the image capture unit 211 to generating an image file including the captured image and writing the image file to the recording medium 227).

[0060] The mode switching switch 103 switches the operating mode of the system control unit 50 to any one of the following: still image shooting mode, moving image shooting mode, and playback mode. Still image shooting modes include automatic shooting mode, automatic scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). Additionally, still image shooting modes include various scene modes and custom modes that provide shooting settings corresponding to different shooting scenarios. By using the mode switching switch 103, the user can directly switch the operating mode to any of the above shooting modes. Alternatively, after temporarily switching to the shooting mode list screen using the mode switching switch 103, the user can selectively switch to any of the various display modes using the operation unit 228. Similarly, moving image shooting modes may also include multiple modes.

[0061] Touch panel 109 is a touch sensor that detects various touch operations on the display surface (operation surface of touch panel 109) of display unit 108. Touch panel 109 and display unit 108 can be configured integrally. For example, touch panel 109 is attached to the upper layer of the display surface of display unit 108 such that light transmittance does not obstruct the display of display unit 108. By associating input coordinates on touch panel 109 with display coordinates on the display surface of display unit 108, a GUI (Graphical User Interface) can be configured, giving the user the impression that they can directly operate the screen displayed on display unit 108. For touch panel 109, any of various methods such as resistive film method, capacitive method, surface elastic wave method, infrared method, electromagnetic induction method, image recognition method, and optical sensor method can be used. Depending on the method, there are methods that detect touch based on contact with touch panel 109 and methods that detect touch based on a finger or pen approaching touch panel 109, but either method can be used.

[0062] The system control unit 50 can detect the following operations or states on the touch panel 109.

[0063] • A new touch on the touch panel 109 by a finger or pen that has not yet touched the touch panel 109, that is, the start of a touch (hereinafter referred to as touch-down).

[0064] • The touch panel 109 is being touched by a finger or pen (hereinafter referred to as "Touch-On").

[0065] • The state in which a finger or pen moves while touching the touch panel 109 (hereinafter referred to as Touch-Move).

[0066] • Remove (release) a finger or pen from the touch panel 109, i.e., the touch ends (hereinafter referred to as "touch-up").

[0067] • Nothing is touching the state of touch panel 109 (hereinafter referred to as "Touch-Off").

[0068] Touch down is detected simultaneously with touch on. Touch on is typically detected continuously after a touch down, unless a touch release is detected. Additionally, touch on is detected simultaneously with touch movement. Even when touch on is detected, no touch movement is detected unless the touch location is moving. Touch off is detected after all touched fingers or styluses have been released from their positions.

[0069] These operations / status and the position coordinates of the finger or pen touching the touch panel 109 are reported to the system control unit 50 via an internal bus. The system control unit 50 determines what operation (touch operation) has been performed on the touch panel 109 based on the reported information. Regarding touch movement, the direction of movement of the finger or pen moving on the touch panel 109 can also be determined based on changes in position coordinates, for each vertical and horizontal component on the touch panel 109. When a touch movement is detected to have reached a predetermined distance or longer, a swipe operation is determined to have been performed.

[0070] The operation of quickly moving a finger a certain distance while keeping it touching the touch panel 109 and then releasing the finger is called a flick. In other words, a flick is an operation of quickly tracking the touch panel 109 with a finger in a way that is like flipping through the touch panel 109. A flick is determined to have been performed when a touch movement is detected to reach a predetermined distance or longer at a predetermined speed or higher and then the touch is detected to be lifted (it can be determined that a flick occurred after a swipe operation). In addition, a touch operation that simultaneously touches (multi-touch) multiple points (e.g., two points) and moves the corresponding touched positions closer to each other is called a "pinch-in", while a touch operation that moves the corresponding touched positions away from each other is called a "pinch-out". Such pinch-in and release are collectively referred to as pinch-out operations (or simply pinch-out).

[0071] Figure 4(A) is a block diagram illustrating a configuration example of a server 300 as an external device to a PC 400. The server 300 includes a control unit 301, a ROM 302, a RAM 303, a recording medium 304, a communication unit 305, and a determination unit 306. These components can be connected to each other via a system bus 307. Note that each component of the server 300 can also be included in the camera 100.

[0072] Control unit 301 is, for example, a central processing unit (CPU). Control unit 301 controls the entire server 300.

[0073] ROM 302 is a read-only memory that stores programs and parameters, not temporary ones.

[0074] RAM 303 is a random access memory used for temporary storage of programs and data supplied from external devices, etc.

[0075] Recording medium 304 is a medium (hard disk or flash memory) permanently disposed in server 300. Alternatively, recording medium 304 is a medium (such as optical disc, magnetic card, optical card, IC card or memory card) removable from server 300.

[0076] The communication unit 305 communicates with external devices (such as camera 100 and PC 400). The communication unit 305 receives moving and still images from camera 100. The received moving and still images are stored on recording medium 304. In response to a request from PC 400, the communication unit 305 transmits information about the images (moving and still images) stored on recording medium 304, as well as the results of determining the state of the still images (such as whether the still image is in focus or whether the eyes are closed), to PC 400. At this time, the communication unit 305 can also transmit a determination signal indicating information about the determination result and information about the file name of the still image separately from the still image to PC 400. Additionally, the communication unit 305 receives a shooting information signal indicating shooting information of the still image generated by camera 100 and transmits the shooting information signal to PC 400.

[0077] The determining unit 306 determines the state of the still image stored on the recording medium 304 (such as whether the still image is in focus or whether the eyes are closed). The determining unit 306 also stores information about the result of determining the state of the still image on the recording medium 304 together with the still image.

[0078] The system bus 307 connects the various components of the server 300 to each other in a communicative manner.

[0079] Figure 4(B) is a block diagram illustrating a configuration example of a PC 400 as an electronic device. The PC 400 includes a control unit 401, a ROM 402, a RAM 403, a recording medium 404, an operation unit 405, a display unit 406, and a communication unit 407. These components communicate with each other via a system bus 408.

[0080] The control unit 401 is, for example, a central processing unit (CPU). The control unit 401 controls the entire PC 400. Additionally, the control unit 401 communicates with the server 300 via the communication unit 407 to confirm whether a new moving image or still image has been transmitted from the camera 100 to the server 300. When a new moving image or still image has been transmitted to the server 300, the control unit 401 requests the server 300 to transmit the moving image or still image via the communication unit 407. The control unit 401 then stores the new moving image or still image on the recording medium 404. The control unit 401 automatically selects the moving image (the moving image captured during the same time block as the still image) corresponding to the still image selected in response to the operation of the operation unit 405.

[0081] ROM 402 is a read-only memory that stores programs and parameters, not temporary ones.

[0082] RAM 403 is a random access memory used for temporary storage of programs and data supplied from external devices, etc.

[0083] Recording medium 404 is a medium (such as a hard disk or flash memory) permanently disposed in PC 400. Alternatively, recording medium 404 is a medium (such as an optical disc, magnetic card, optical card, IC card, or memory card) removable from PC 400.

[0084] The operation unit 405 receives operations from the user (editor) on the PC 400. For example, in response to the user's operation on the operation unit 405, it selects one of one or more still images that meet predetermined conditions. Here, the operation unit 405 may be an operation component (such as a button or touch panel) disposed in the PC 400, or it may be an operation component (such as a keyboard or mouse) detachable from the PC 400.

[0085] Display unit 406 displays data held by PC 400, data supplied from external devices, etc. Additionally, display unit 406 displays a list of moving images and still images recorded on recording medium 404 on the same timeline. Display unit 406 may also display only still images that meet predetermined conditions. Display unit 406 may also display moving images captured during the same time block as the still image selected by operation unit 405 in a corresponding predetermined area of ​​a specific frame. Furthermore, when the reception of a still image has been delayed, display unit 406 displays the reception status and determination result of the still image based on the capture information signal and determination signal. Note that display unit 406 may be included in PC 400, or it may be a display device separate from PC 400.

[0086] The communication unit 407 communicates with external devices such as the camera 100 and the server 300. The system bus 408 communicatively connects the various components of the PC 400.

[0087] The following is a description of the basic processes from shooting to editing in the shooting and editing system. The photographic editing system includes a camera 100, a server 300, and a PC 400.

[0088] Figure 5 The illustration shows an example of a screen displaying an editing application operating on PC 400 (displayed on display unit 406). In the editing application, PC 400 displays moving and still images received from camera 100 and selects (related to each other) still and moving images to be used in a news article. Furthermore, depending on the editing application, PC 400 can edit the text (wording) in the news article and publish the news article (public article) including the text, along with the selected still and moving images.

[0089] The editing application screen includes a status area 600, a good area 601, a timeline area 602, and a text area 603. In the status area 600, the shooting status is displayed (indicating whether the camera 100 is shooting moving images and whether still image shooting has been performed).

[0090] The qualified area 601 displays one or more still images that meet predetermined conditions, arranged in chronological order of capture time, in the middle left of the editing application screen. Additionally, the user can select any one of the one or more still images displayed in the qualified area 601.

[0091] In timeline region 602, moving images and still images (in chronological order) from each of the shooting time blocks are displayed on the timeline. The multiple moving images displayed in timeline region 602 are multiple moving images taken in consecutive time blocks. Each time block is a time range of a given length (e.g., 1 minute). Here, when a still image meets predetermined conditions (such as composition, focus status of the main subject, or the presence or absence of closed eyes), a pass icon (a display item indicating an appropriately shot still image) is displayed, such that the pass icon is superimposed on a portion of the still image. Note that when a still image does not meet predetermined conditions, a fail icon (a display item indicating that no appropriately shot still image was taken) can also be displayed, such that the fail icon is superimposed on a portion of the still image. Icons corresponding to the state of the still image can also be superimposed (displayed) on the still image. For example, it might be possible to superimpose an icon indicating closed eyes on a still image of a particular subject closing his or her eyes, and in a still image taken with a specific composition, to superimpose an icon indicating a shot taken with a specified composition.

[0092] Article area 603 is used for editing news articles. In article area 603, users can edit the title and text of the news article. Article area 603 includes a still image area 603A for inserting (placing) still images and a moving image area 603B for inserting (placing) moving images. When a still image displayed in qualified area 601 is selected by a user action such as mouse click, the selected still image is inserted into still image area 603A of article area 603. Additionally, moving images taken during the same time block as the still image are inserted into moving image area 603B of article area 603. As a result, a preview close to the complete state of the news article can be achieved.

[0093] As a preview result, when it is determined that publication is possible, the user presses the publish button (604). A final confirmation message then appears, asking if the news article can be published. Once the user's response to the final confirmation message confirms that the user has given the instruction to publish the news article, the news article is distributed (published on the internet).

[0094] refer to Figure 7 The flowchart will provide a description of the operation of the camera 100 according to the first embodiment. When the camera 100 is powered on (activated), it transitions to the shooting preparation state. Figure 7 The flowchart begins.

[0095] In step S701, the system control unit 50 starts capturing images and simultaneously starts real-time view display (display of the captured real-time view image).

[0096] In step S702, the system control unit 50 determines whether a command to start capturing a moving image has been issued (such as the photographer's operation of the moving image button 106). If it is determined that a command to start capturing a moving image has been issued, the process proceeds to step S703. If it is determined that a command to start capturing a moving image has not been issued, the process in S702 is repeated.

[0097] In step S703, the system control unit 50 begins to capture moving images using the image capture unit 211 and simultaneously begins to record the captured moving images on the recording medium 227.

[0098] In step S704, the system control unit 50 determines whether a predetermined time has elapsed since the processing in step S705 was previously executed. If it is determined that a predetermined time has elapsed since the processing in step S705 was previously executed, the process proceeds to step S705. If it is determined that no predetermined time has elapsed since the processing in step S705 was previously executed, the process proceeds to step S706. The predetermined time is a relatively short time (any time between 15 seconds and 3 minutes, etc.) preset by the photographer taking into account factors such as the performance of the camera 100 and the communication environment during the transmission of moving images.

[0099] In step S705, the system control unit 50 transmits a moving image corresponding to a predetermined time period since the processing in step S705 was previously executed to the server 300. The process of transmitting the moving image corresponding to the predetermined time period to the server 300 continues until a command to end video image capture is issued.

[0100] In step S706, the system control unit 50 determines whether a command to capture a still image has been issued. If it is determined that a command to capture a still image has been issued, the process proceeds to step S707. If it is determined that a command to capture a moving image has not been issued, the process proceeds to step S709.

[0101] In step S707, the system control unit 50 captures a still image and records the still image on the recording medium 227.

[0102] In step S708, the system control unit 50 transmits the still image captured in step S707 to the server 300. Note that when the command to capture a still image is issued in step S706, the system control unit 50 can also continuously capture multiple still images in steps S707 and S708 until a command to end still image capture is issued, and then transmit the captured multiple still images to the server 300.

[0103] In step S709, the system control unit 50 determines whether a command to end the capture of moving images has been issued. If it is determined that a command to end the capture of moving images has been issued, the process proceeds to step S710. If it is determined that a command to end the capture of moving images has not been issued, the process proceeds to step S704.

[0104] In step S710, the system control unit 50 ends the capture of the moving image. Then, the system control unit 50 transmits the moving images corresponding to the remaining time that have not yet been transmitted to the server 300 (the moving images that have not been transmitted in step S705) to the server 300.

[0105] refer to Figure 8 The flowchart will provide a description of the operation of server 300 according to the first embodiment. Figure 8 The processing in the flowchart can be performed continuously and repeatedly, or it can be performed each time any information is transmitted from camera 100 to server 300.

[0106] In step S801, the control unit 301 determines whether a moving image has been received from the camera 100. If it is determined that a moving image has been received from the camera 100, the process proceeds to step S802. If it is determined that a moving image has not yet been received from the camera 100, the process proceeds to step S805.

[0107] In step S802, the control unit 301 stores the moving image received in step S801 in the database of the recording medium 304.

[0108] In step S803, the control unit 301 determines whether there is a request to transfer moving images from the PC 400 (an instruction has been issued requesting the server 300 to transfer moving images to the PC 400). If it is determined that a request to transfer moving images has been made, the process proceeds to step S804. If it is determined that no request to transfer moving images has been made, the process in step S803 is repeated.

[0109] In step S804, the control unit 301 transmits motion images that have not yet been transmitted to the PC 400. Here, relative to motion images that have already been transmitted to the PC 400, the control unit 301 stores the motion images in a database in association with information indicating the status of the transmission. This prevents the control unit 301 from transmitting the same motion image to the PC 400 multiple times.

[0110] In step S805, the control unit 301 determines whether a still image has been received from the camera 100. If it is determined that a still image has been received from the camera 100, the process proceeds to step S806. If it is determined that a still image has not been received from the camera 100, the process proceeds to step S801.

[0111] In step S806, the control unit 301 controls the determining unit 306 to determine the state of the still image (such as the composition in the still image, the focus state of the main subject, or whether the closed eyes are present or absent). The control unit 301 may use a predetermined algorithm, predetermined AI (artificial intelligence), etc. to determine the state of the still image.

[0112] In step S807, the control unit 301 associates the result of determining the state of the still image (such as composition, focus state of the main subject, or whether the closed eye state exists or does not exist) (hereinafter referred to as the "state determination result") with the still image, and stores the state determination result and the still image in the database of the recording medium 304.

[0113] In step S808, the control unit 301 determines whether there is a request to transfer a still image from the PC 400 (an instruction has been issued requesting the server 300 to transfer a still image to the PC 400). If it is determined that a request to transfer a still image has been issued from the PC 400, the process proceeds to step S809. If it is determined that no request to transfer a still image has been issued from the PC 400, the process in step S808 is repeated.

[0114] In step S809, the control unit 301 transmits the still images that have not yet been transmitted to the PC 400 along with the still image status determination results (information about the status determination results) to the PC 400. Here, relative to the still images that have already been transmitted to the PC 400, the control unit 301 stores information indicating the status of the transmitted images in association with the still images in the database. This prevents the control unit 301 from transmitting the same still image to the PC 400 multiple times.

[0115] refer to Figure 9 The flowchart will provide a description of the operation of PC 400 according to the first embodiment. When PC 400 is activated, Figure 9 The process begins in the flowchart.

[0116] In step S901, the control unit 401 begins monitoring the database of the server 300 (the database of the recording medium 304).

[0117] In step S902, the control unit 401 determines whether a new moving image (a moving image that has not yet been transmitted to the PC 400) exists in the database of the server 300. When it is determined that a new moving image (hereinafter referred to as "new moving image") exists in the database, the process proceeds to step S903. When it is determined that no new moving image exists in the database, the process proceeds to step S910.

[0118] In step S903, the control unit 401 requests the server 300 to transmit new moving images.

[0119] In step S904, the control unit 401 receives a new moving image from the server 300.

[0120] In step S905, the control unit 401 acquires a thumbnail of the new moving image based on the received new moving image. Then, the control unit 401 displays (adds) the thumbnail of the new moving image to (or to) the image. Figure 5 The timeline region 602 shown corresponds to the time block in which the new moving image was captured.

[0121] In step S910, the control unit 401 determines whether a new still image (a still image that has not yet been transmitted to the PC 400) exists in the database of the server 300. When it is determined that a new still image (hereinafter referred to as "new still image") exists in the database, the process proceeds to step S911. When it is determined that no new still image exists in the database, the process proceeds to step S901.

[0122] In step S911, the control unit 401 requests the server 300 to transmit a new still image.

[0123] In step S912, the control unit 401 receives the new still image and the state determination result of the new still image from the server 300.

[0124] In step S913, the control unit 401 determines whether the new still image meets predetermined conditions based on the received state determination result. Here, predetermined conditions are conditions used to determine whether an appropriate still image has been captured. Specifically, predetermined conditions include conditions such as shooting with a specific composition, a specific subject not closing his or her eyes, and a specific subject being in focus. Alternatively, predetermined conditions may also include conditions such as shooting without backlighting, a person's gaze not pointing towards the camera 100 in the still image, and multiple people in the still image. When it is determined that the new still image meets the predetermined conditions, the process proceeds to step S914. When it is determined that the new still image does not meet the predetermined conditions, the process proceeds to step S915.

[0125] In step S914, the control unit 401 displays (adds) the new still image to (or adds to) the image. Figure 5 The qualified area 601 shown in the figure.

[0126] In step S915, the control unit 401 displays (adds) the new still image to (or adds it to). Figure 5 The timeline region 602 shown corresponds to the time block in which the new still image was captured.

[0127] In step S916, the control unit 401 displays (adds) the new still image along with the qualified icon in (to). Figure 5 The timeline region 602 shown corresponds to the time zone in which the new still image was captured.

[0128] Through the processing in steps S915 and S916, new moving images and new still images captured during the same time block are correlated with each other and displayed in timeline area 602.

[0129] In step S920, the control unit 401 determines whether a still image in the qualified region 601 has been selected by the user. Note that the control unit 401 can also determine whether a still image has been selected from either the qualified region 601 or the timeline region 602. When it is determined that a still image has been selected, the process proceeds to step S921. When it is determined that no still image has been selected, the process proceeds to step S901.

[0130] In step S921, the control unit 401 selects a moving image (hereinafter referred to as "the selected moving image") that was captured during the same time block as the still image (hereinafter referred to as "the selected still image") selected in step S920.

[0131] In step S922, the control unit 401 inserts (places) the selected still image into... Figure 5 In the still image region 603A shown in the figure.

[0132] In step S923, the control unit 401 inserts (places) the selected moving image into the moving image area 603B. At this time, as... Figure 5 As shown in article area 603, users (editors) can preview (confirm) the entire news article (news article content).

[0133] In step S924, the control unit 401 determines whether the button has been pressed. Figure 5 The publish button 604 shown is used to issue a command to end the content generation of the news article. When it is determined that a command to end content generation has been issued, the process proceeds to step S925. When it is determined that a command to end content generation has not been issued, the process proceeds to step S920. Note that until the publish button 604 is pressed, the form (such as position or size) of the selected still image or the form of the selected moving image in the news article can also be adjusted by user operation. Alternatively, the text in the article can also be changed by user operation.

[0134] In step S925, the control unit 401 generates a news article (news article content) that includes the selected still image, the selected moving image, and the text in the article.

[0135] In step S926, after receiving the final confirmation message prior to user confirmation of publication, the control unit 401 determines whether an instruction to upload the news article (news article content) has been issued. If it is determined that an upload instruction has been issued, the process proceeds to step S927. If it is determined that no upload instruction has been issued, the process proceeds to step S924.

[0136] In step S927, the control unit 401 transmits the news article to the distribution server. As a result, the distribution server publishes the news article on the Internet.

[0137] According to the first embodiment, by viewing the timeline area, users can identify moving and still images taken within the same time block. Furthermore, since a qualified icon is attached to still images that meet predetermined conditions, users can select a more preferred still image. Additionally, when a still image is selected, moving images taken within the same time block as the still image are also selected. Therefore, users can more easily select still and moving images taken within the same time block. When a still image is selected, still and moving images taken within the same time block are inserted into the news article content, which facilitates the creation of news articles for the user.

[0138] Second Embodiment

[0139] In the second embodiment, PC 400 displays the reception status of the still image and the reception status determination result based on the signal received from server 300. Note that, hereinafter, descriptions of the same components and processes as in the second and first embodiments will be appropriately omitted.

[0140] Figure 6 The illustration shows a display example of an editing application operating on a PC 400 according to a second embodiment. In the same manner as in the first embodiment, the editing application displays moving and still images received from the server 300 (camera 100), and allows selection of still and moving images to be used in a news article. By using the editing application, users can edit the text in a news article and publish a news article containing the selected still and moving images.

[0141] In the second embodiment, in timeline area 602 (the area for displaying moving and still images received from server 300 on the timeline), a still image reception status icon 605 is displayed at points in time when a still image has not yet been successfully received from server 300. The still image reception status icon 605 is a display item indicating that a still image will be transmitted in the future. At this time, in the same manner as in the first embodiment, PC 400 also displays a pass icon whenever a still image to be transmitted in the future meets predetermined conditions. Then, when a still image is received, PC 400 replaces the still image reception status icon 605 with the received still image.

[0142] At a point in time when a status determination result has not yet been received from server 300, PC 400 displays the status determination result reception status icon 606 along with the still image (or still image reception status icon 605). Reception status icon 606 indicates that a status determination result has not yet been successfully received (it is unknown whether the still image meets the predetermined conditions). When a status determination result is received, PC 400 stops displaying the status determination result reception status icon 606.

[0143] refer to Figure 10 The flowchart will provide a description of the operation of the camera 100 according to the second embodiment. In the second embodiment, besides Figure 7 In addition to the flowchart, the processing in step S1008 is also performed. For example... Figure 10 As shown, step S1008 is performed between step S707 and step S708.

[0144] In step S1008, the system control unit 50 transmits a shooting information signal to the server 300, indicating information such as the filename of the still image (the still image captured in step S707) and the date and time of capture. In other words, the shooting information signal is transmitted to the server 300 before the still image is transmitted.

[0145] refer to Figure 11 The flowchart below will provide a description of the operation of server 300 according to the second embodiment. When server 300 receives a shooting information signal or image (moving image or still image) from camera 100, it begins... Figure 11 The processing in the flowchart.

[0146] In step S1101, the control unit 301 determines whether a shooting information signal has been received from the camera 100. If it is determined that a shooting information signal has been received, the process proceeds to step S1102. If it is determined that a shooting information signal has not been received, the process proceeds to step S1103.

[0147] In step S1102, the control unit 301 transmits the received shooting information signal to the PC 400.

[0148] Steps S1103 to S1108 are the same as steps S801 to S806.

[0149] In step S1109, the control unit 301 generates a determination signal indicating the file name and status determination result of the still image, and transmits the determination signal to the PC 400.

[0150] In step S1110, the control unit 301 stores the received still image in the database of the recording medium 304.

[0151] In step S1111, the control unit 301 determines whether there is a request to transmit a still image from the PC 400. If it is determined that a request to transmit a still image has been issued, the process proceeds to step S1112. If it is determined that no request to transmit a still image has been issued, the process in step S1111 is repeated.

[0152] In step S1112, the control unit 301 transmits still images that have not yet been transmitted to the PC 400.

[0153] refer to Figure 12 The flowchart will provide a description of the operation of PC 400 according to the second embodiment. Here, in the second embodiment, besides Figure 9 In addition to the flowchart, the processing in step S1200 is also performed. Figure 13 (Steps S1328 to S1232 in the above steps). Note that in step S912, a determination signal has already been received, so the control unit 401 only receives the new still image. In step S913, the control unit 401 determines whether the new still image meets the predetermined conditions based on the determination signal.

[0154] In step S1200, the control unit 401 performs processing based on the reception of the capture signal and the determination signal. (Reference) Figure 13 The flowchart will describe the processing details in step S1200.

[0155] In step S1328, the control unit 401 determines whether a shooting information signal has been received from the server 300. If it is determined that a shooting information signal has been received, the process proceeds to step S1329. If it is determined that a shooting information signal has not been received, the process proceeds to step S1330.

[0156] In step S1329, the control unit 401 in Figure 6The still image reception status icon 605 is displayed in the timeline area 602 shown. At this time, the control unit 401 refers to the shooting time blocks of the already received moving images to determine the shooting time block of the moving image, including the shooting date and time recorded in the shooting information signal. Then, the control unit 401 displays the still image reception status icon 605 at the position where the still image should be placed within the determined shooting time block. Here, the control unit 401 calculates the shooting time block of the moving image based on the shooting start time and shooting end time (or the shooting start time and playback time of the moving image). Note that when the still image reception is complete, the control unit 401 replaces the still image reception status icon 605 with the still image.

[0157] In step S1330, the control unit 401 determines whether a confirmation signal has been received from the server 300. If it is determined that a confirmation signal has been received, the process proceeds to step S1331. If it is determined that a confirmation signal has not been received, the process proceeds to step S1332.

[0158] In step S1331, the control unit 401 displays an icon corresponding to the still image's status determination result based on the filename recorded in the determination signal. At this time, the control unit 401 overlays the icon corresponding to the still image's status determination result onto the still image's thumbnail or the still image reception status icon 605. When the still image corresponding to the status determination result meets predetermined conditions, the icon corresponding to the status determination result is a qualified icon.

[0159] In step S1332, the control unit 401 displays the status determination result reception status icon 606, such that the status determination result reception status icon 606 is superimposed on the thumbnail of the still image or the still image reception status icon 605. Note that when a confirmation signal is received, the control unit 401 stops displaying the status determination result reception status icon 606, or replaces the status determination result reception status icon 606 with a pass icon or similar icon corresponding to the status determination result.

[0160] Note that when the still image reception status icon 605 is displayed in step S1329, the still image reception status icon 605 is then replaced with a thumbnail of the still image in steps S915 and S916. When the status determination result reception status icon 606 is displayed in step S1332, the status determination result reception status icon 606 is then replaced with an icon corresponding to the status determination result in step S916.

[0161] According to the second embodiment, the PC 400 displays the still image reception status and the status determination result reception status based on the received shooting information signal and determination signal. As a result, even if there is a delay, for example, in the transmission of the still image or in the processing in the server 300, the user can know that a still image meeting predetermined conditions will be transmitted to the PC 400 in the future.

[0162] Third Embodiment

[0163] The third embodiment will describe an example of adding still and moving image editing processes to the PC 400, as described below in (1) to (9), relative to the first embodiment. Note that in the third embodiment, descriptions of the same components and processes as in the first embodiment are appropriately omitted.

[0164] (1) PC 400 is capable of acquiring ancillary information recorded on images (still images and moving images). Specifically, PC 400 can acquire ancillary information from images (still images and moving images) recorded on recording medium 404. Ancillary information includes shooting information (such as shutter speed or aperture during shooting), camera 100's pose information during shooting, autofocus information (such as focus coordinates), information such as shooting date and time, etc. As ancillary information for moving images, information for each frame is also recorded. Note that although an example of recording ancillary information on images (still images and moving images) is shown, ancillary information can also be recorded in another file associated with the image.

[0165] (2) PC 400 can specify the cropping range of the still image. Specifically, PC 400 can specify the cropping range of the still image in response to a user operation on operation unit 405. In the third embodiment, the user specifies a rectangular region including the subject area as the cropping range of the still image. However, it is also possible that when the user specifies the subject (subject position), PC 400 specifies a rectangle including the specified subject area. The aspect ratio of the rectangle within the cropping range is not particularly limited.

[0166] (3) PC 400 can detect regions in a still image that include a subject. Specifically, PC 400 detects a subject from a still image and detects the region that includes the subject (subject region). The method for detecting the subject is not particularly limited and known methods can also be used.

[0167] (4) PC 400 is able to determine the cropping range of a still image (the area in the still image to be cropped). Specifically, PC 400 determines the cropping range based on accompanying information, information about the specified cropping range, information about the subject area, etc.

[0168] (5) PC 400 can crop a portion of a still image. Specifically, such as... Figure 14 As shown in (A), PC 400 generates a new still image by cropping a portion of the still image.

[0169] (6) PC 400 is able to determine the frame of the moving image corresponding to the still image (hereinafter referred to as the "corresponding frame"). Specifically, PC 400 determines the frame of the moving image generated during a time block corresponding to the time block of the still image, where the scene is the same as the scene of the still image. In the third embodiment, PC 400 obtains the shooting date and time of the still image and the moving image from the accompanying information of the still image and the moving image, and determines the frame whose shooting date and time are closest to the shooting date and time of the still image among all the frames of the moving image as the corresponding frame. As a method for determining the corresponding frame, any method such as "a method of comparing the matching degree between two images" or "a method of determining the corresponding frame based on the correspondence between identifiers (GUIDs) recorded in the accompanying information of the moving image and the still image during the shooting period" can be used.

[0170] (7) PC 400 is able to acquire information used to determine the cropping range of a still image (referred to as "range determination information"). PC 400 acquires the range determination information and also acquires the reason for determining the cropping range.

[0171] (8) PC 400 can cut a portion of a playback segment of a moving image. Specifically, PC 400 generates a new moving image by cutting a range of that portion of the playback segment from the moving image. The playback segment to be cut is determined based on range determination information so as to include the corresponding frame (the frame corresponding to the still image). In the third embodiment, as for example... Figure 14 As shown in (B), the PC400 determines the playback segment to be cut when the subject of interest appears in the still image area during cropping. Additionally, the PC400 can change the length of the playback segment to be cut based on the destination of the moving image (such as an SNS or news article). When the playback segment to be cut does not reach a specific time in the case of a moving image, the PC400 can also cut a portion of the playback segment from multiple moving images that include the selected moving image and have a continuous shooting time, and combine the cut playback segments together.

[0172] (9) PC 400 can cut a portion of the image in each frame of a moving image. Specifically, after determining the "cut range of the moving image", PC 400 cuts the cut range from the image in each frame and processes the cut range of each frame as the image in each frame of a new moving image. This allows PC 400 to generate new moving images. In the third embodiment, as Figure 14 As shown in (C), PC 400 determines the clipping extent in each frame in the same manner as for still images. PC 400 then calculates the range including all clipping extents determined for each frame. Figure 14 The range enclosed by the thickest wireframe in (C) is used to determine the "moving image cropping range" (the range to be cropped from each frame). In other words, the range in which a particular subject is seen in the image in every single frame across all frames is determined.

[0173] refer to Figure 15 The flowchart will describe the operation of PC 400 according to the third embodiment. Figure 15 The processing in the flowchart according to the first embodiment Figure 9 The process is performed between steps S921 and S922 in the flowchart. Figure 9 The processes in the flowchart (the processes in steps S901 to S927) are performed in other respects in the same manner as in the first embodiment.

[0174] Note that in step S912, the control unit 401 receives not only the still image and the state determination result, but also information about the subject region of interest when the state of the still image is determined, as "information for determining the state of the still image". For example, the subject region of interest is the area of ​​the subject that determines whether the subject has closed its eyes, whether the subject is in focus, etc. For example, the subject region of interest is the area of ​​a specific person. In the third embodiment, the example of "information about the subject region of interest" as "information for determining the state of the still image" will be given. However, "information for determining the state of the still image" can also be coordinates for determining the state of the still image, facial information for determining the state of the still image, etc.

[0175] In step S1501, the control unit 401 acquires the still image selected in step S920 (the selected still image).

[0176] In step S1502, the control unit 401 determines whether the selected still image meets predetermined conditions. When it is determined that the selected still image meets the predetermined conditions, the process proceeds to step S1503. When it is determined that the selected still image does not meet the predetermined conditions, the process proceeds to step S1505.

[0177] In step S1503, the control unit 401 acquires information about the subject area of ​​interest when the state of the selected still image is determined.

[0178] In step S1504, the control unit 401, based on the information about the subject region of interest obtained in step S1503, specifies the range of the selected still image that includes the subject of interest. The control unit 401 determines the specified range as the "crop range" of the selected still image.

[0179] In step S1505, the control unit 401 determines whether the user has specified the cropping range of the selected still image. If it is determined that the user has specified the cropping range of the selected still image, the process proceeds to step S1506. If it is determined that the user has not specified the cropping range of the selected still image, the process proceeds to step S1507.

[0180] In step S1506, the control unit 401 determines the range specified by the user (the range in the selected still image) as the "cropping range" of the selected still image.

[0181] In step S1507, the control unit 401 acquires additional information about the selected still image.

[0182] In step S1508, the control unit 401 detects one or more subjects in the selected still image. Then, the control unit 401 detects the subject closest to the focal coordinates included in the accompanying information as the subject of interest.

[0183] In step S1509, the control unit 401 specifies a range including the subject of interest based on the subject of interest detected in step S1508. The control unit 401 determines the specified range as the "crop range" of the selected still image.

[0184] In step S1510, the control unit 401 cuts a determined cut range from the selected still image to generate a new still image. This allows the control unit 401 to obtain a new still image (hereinafter referred to as the "cut still image") with extra areas removed and suitable for a news article.

[0185] In step S1511, the control unit 401 acquires the moving image selected in step S921 (the selected moving image).

[0186] In step S1512, the control unit 401, based on information used to determine the cropping range of the selected still image, crops a portion of the playback segment of the selected moving image. Therefore, the control unit 401 generates a new moving image.

[0187] In step S1513, the control unit 401 determines the "moving image cropping range" of the moving image generated in step S1512. The control unit 401 generates a new moving image (hereinafter referred to as the "cropped moving image") by cropping the "moving image cropping range" from the image in each frame of the moving image generated in step S1512. Here, the cropped moving image is generated such that the subject of interest appears in the image in each frame of the cropped moving image. Therefore, a new moving image that has removed extra playback segments and extra areas and is suitable for a news article can be obtained. Note that the "moving image cropping range" can also be determined based on information used to determine the cropping range of the selected still image (information about the user-specified range, information about the subject of interest, or additional information about the selected still image).

[0188] In step S1514, the control unit 401 replaces the selected still image and selected moving image selected in steps S920 and S921 with the newly generated cropped still image and cropped moving image in steps S1510 and S1513. In other words, the control unit 401 performs control so that the cropped still image and cropped moving image are used in the processing in steps S922 and S923.

[0189] According to the third embodiment, unwanted information can be removed from selected still images and selected moving images, thus easily generating still images and moving images suitable for news articles. In other words, better news articles can be generated while reducing the user's workload.

[0190] Furthermore, in the above description, "when A is not less than B, the process proceeds to step S1, and when A is less than (lower than) B, the process proceeds to step S2" can also be understood as "when A is greater than (higher than) B, the process proceeds to step S1, and when A is not greater than B, the process proceeds to step S2." Conversely, "when A is greater than (higher than) B, the process proceeds to step S1, and when A is not greater than B, the process proceeds to step S2" can also be understood as "when A is not less than B, the process proceeds to step S1, and when A is less than (lower than) B, the process proceeds to S2." Therefore, unless there is a contradiction, "not less than A" can also be understood as "greater than A (higher, longer, or more than A)," and "not greater than A" can also be understood as "less than A (lower, shorter, or less than A)." Additionally, "greater than A (higher, longer, or more than A)" can also be understood as "not less than A," and "less than A (lower, shorter, or less than A)" can also be understood as "not greater than A."

[0191] It should be noted that the various types of control described above can be implemented by a piece of hardware (e.g., a processor or circuit) or other processing. Processing can be shared among multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits), thereby implementing control of the entire device.

[0192] Furthermore, the processors mentioned above are processors in a broad sense, encompassing both general-purpose and special-purpose processors. Examples of general-purpose processors include central processing units (CPUs), microprocessor units (MPUs), and digital signal processors (DSPs). Examples of special-purpose processors include graphics processing units (GPUs), application-specific integrated circuits (ASICs), and programmable logic devices (PLDs). Examples of PLDs include field-programmable gate arrays (FPGAs) and complex programmable logic devices (CPLDs).

[0193] The above embodiments (including variant examples) are merely examples. Any configurations obtained by suitably modifying or changing some configurations of the embodiments within the scope of this disclosure are also included in this disclosure. This disclosure also includes other configurations obtained by suitably combining various features of the embodiments.

[0194] (Other embodiments)

[0195] The present invention can also be implemented by the following steps: supplying a program for implementing one or more of the functions of the above embodiments to a system or device via a network or storage medium, and causing one or more processors in the computer of the system or device to read and execute the program. Furthermore, the present invention can also be implemented by circuitry for implementing one or more functions.

[0196] [List of Reference Symbols]

[0197] 400: Head-mounted display; 401: Control unit; 406: Display unit; 407: Communication unit

Claims

1. An electronic device for controlling the display of a specific image on a display component, the electronic device comprising: An acquisition component acquires multiple moving images captured during consecutive time blocks and multiple still images captured during the acquisition of the multiple moving images; as well as The control unit 1) displays the plurality of moving images in chronological order in a first area of ​​the specific screen, and 2) for each time block in which one or more still images are captured, displays the moving images captured during that time block and the one or more still images captured during that time block in association with each other in the first area.

2. The electronic device according to claim 1, further comprising: The selection component, when a still image is selected from the plurality of still images, selects a first moving image captured during the time block in which the first still image corresponding to the selected still image was captured.

3. The electronic device according to claim 2, in, The specific frame has a second area, in which related moving and still images are arranged, and When the first still image is selected, the control unit arranges the first still image and the first moving image selected by the selection unit in the second area.

4. The electronic device according to claim 3, in, When the first still image is selected, the control unit arranges a first range obtained by cropping a portion of the first still image in the second region.

5. The electronic device according to claim 4, in, The control unit determines the first range based on user specifications, information about a specific object in the first still image, or additional information about the first still image.

6. The electronic device according to any one of claims 3 to 5, in, When the first still image is selected, the control unit arranges a second moving image in the second region, obtained by cutting at least one of the range of a partial playback segment of the first moving image and the range of a portion of each frame of the first moving image.

7. The electronic device according to claim 6, in, The control unit determines the range of the partial playback segment of the first moving image based on user specifications, information about a specific object in the first still image, or accompanying information about the first still image.

8. The electronic device according to claim 6 or 7, in, The control component cuts out a portion of the image of each frame of the first moving image, such that a specific object appears in each frame of the second moving image.

9. The electronic device according to claim 5 or 7, in, The accompanying information includes at least one of the following as information during the shooting period: shutter speed information, aperture information, image capture device posture information, autofocus information, and shooting date and time information.

10. The electronic device according to any one of claims 3 to 9, in, Content including moving and still images related to each other, and including text, is arranged in the second area. The electronic device also includes: An output component that outputs the content.

11. The electronic device according to any one of claims 1 to 10, in, The acquisition component acquires the plurality of still images transmitted from the external device, and At a point in time when the acquisition component has not yet acquired any of the plurality of still images, instead of the still image that has not yet been acquired by the acquisition component, the control component displays a first display item in the first area indicating that the still image is to be transmitted.

12. The electronic device according to claim 11, in, When a still image corresponding to the first display item displayed in the first area is acquired, the control unit replaces the first display item displayed in the first area with the still image.

13. The electronic device according to any one of claims 1 to 12, in, When a still image that meets predetermined conditions is displayed in the first area, the control unit also displays a second display item indicating that the still image meets the predetermined conditions.

14. The electronic device according to claim 13, in, The specific image has a third region that is different from the first region, and The control unit arranges one or more still images that meet the predetermined conditions from among the plurality of still images, and displays the arranged still images in the third region.

15. The electronic device according to claim 13 or 14, in, The acquisition component acquires information regarding the determination result indicating whether each of the plurality of still images meets the predetermined condition, and At a point in time when the information regarding the determination result for any of the plurality of still images has not yet been acquired by the acquisition component, the control component displays a third display item indicating that the determination result has not yet been acquired at the position in the first region corresponding to the still image for which the determination result has not yet been acquired.

16. The electronic device according to claim 15, in, When the acquisition unit acquires information about the determination result corresponding to the third display item displayed in the first area, the control unit switches the third display item displayed in the first area to the display corresponding to the determination result.

17. The electronic device according to any one of claims 13 to 16, in, The predetermined conditions are the conditions under which a specific composition is provided, the conditions under which the specific object does not close its eyes, or the conditions under which the specific object is in focus.

18. A method for controlling an electronic device, the electronic device being used to control the display of a specific image displayed on a display component, the method comprising: The steps of acquiring multiple moving images captured during consecutive time blocks and multiple still images captured during the acquisition of the multiple moving images; as well as The control process includes the steps of: 1) displaying the plurality of moving images in chronological order in a first area of ​​the specific screen; and 2) for each time block in which one or more still images are captured, displaying the moving images captured during that time block and the one or more still images captured during that time block in association with each other in the first area.

19. A program that enables a computer to be used as a component of a control device according to any one of claims 1 to 17.

Images