Electronic device, moving image management method, and computer program product

Abstract

The present disclosure provides an electronic device, a moving image management method, and a computer program product capable of easily editing a moving image in a scene including a plurality of segments. An electronic device (200) for managing a moving image in a scene including a plurality of segments is provided with: a display unit (240) for displaying information; an input unit (230) for inputting an operation by a user; and a control unit (210) that controls the display unit in accordance with the operation input by the input unit. The control unit causes the display unit to display an evaluation screen that prompts evaluation of the moving image associated with each of the plurality of segments, acquires evaluation information indicating the evaluation of the moving image by the user from the input unit, and generates management data on the basis of the acquired evaluation information. The management data manages a priority order in which moving image information indicating each moving image is arranged in an edit screen in which a plurality of moving images associated with a plurality of segments are edited.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a method for managing a dynamic image in a scene including a plurality of cuts. Background Art

[0002] Patent Document 1 discloses a video program production support system that enables the consistent use of scenes in electronic file format from the planning and composition stage to the editing stage. The video program production support system includes a camera that creates take metadata within the structure table metadata for each take, and associates the clip within the program metadata with the take's content data and take metadata. At the end of filming of a selected clip, if the clip is associated with only one take, the camera automatically sets that take as OK (accepted). On the other hand, if the clip is associated with multiple takes, one take is set as OK, and the others are set as NG (not accepted), based on the cameraman's OK / NG selection.

[0003] Patent document 2 discloses an electronic device that appropriately displays images representing content to improve convenience. The electronic device displays a selection image representing one content included in the corresponding group in each area on a timeline arranged along the time axis and displaying a plurality of areas corresponding to a plurality of groups. Furthermore, the electronic device associates the above-mentioned areas and displays non-selection images representing content that is not the selected image among the plurality of contents included in the group corresponding to the area. Patent document 2 discloses a plurality of thumbnail images of the same group, such as the selected image and the non-selection image, arranged in the order in which the dynamic image was recorded. Thus, the user can understand the number of times the dynamic image was taken simply by confirming the number of times the thumbnail image is arranged.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2004-187275

[0007] Patent Document 2: Japanese Patent Application Laid-Open No. 2020-102821 Summary of the Invention

[0008] Problems to be solved by the invention

[0009] The present disclosure provides an electronic device and a dynamic image management method capable of easily editing a dynamic image in a scene including a plurality of segments.

[0010] Means used to solve problems

[0011] In the present disclosure, an electronic device manages dynamic images in a scene including multiple segments. The electronic device includes: a display unit for displaying information; an input unit for inputting user operations; and a control unit for controlling the display unit based on the operations inputted into the input unit. The control unit causes the display unit to display an evaluation screen that prompts evaluation of a dynamic image associated with each of the multiple segments, obtains evaluation information representing the user's evaluation of the dynamic image from the input unit, and generates management data based on the obtained evaluation information. The management data manages and configures the priority order of dynamic image information representing each dynamic image in an editing screen for editing multiple dynamic images associated with the multiple segments.

[0012] In the present disclosure, a dynamic image management method is a method for managing dynamic images in a scene including multiple segments. In this method, a control unit of an electronic device causes a display unit to display an evaluation screen prompting evaluation of a dynamic image associated with each of the multiple segments, obtains evaluation information representing a user's evaluation of the dynamic image from an input unit, and generates management data based on the obtained evaluation information. This management data manages and arranges the priority order of dynamic image information representing each dynamic image in an editing screen for editing multiple dynamic images associated with the multiple segments.

[0013] Effects of the Invention

[0014] According to the electronic device and the dynamic image management method of the present disclosure, a dynamic image in a scene including a plurality of segments can be easily edited. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a diagram illustrating the configuration of an imaging system in Embodiment 1 of the present disclosure.

[0016] Figure 2 This is a diagram showing the structure of a digital camera in an imaging system.

[0017] Figure 3 This is a diagram showing the structure of an information support terminal in an imaging system.

[0018] Figure 4 It is a diagram showing a display example of a function selection screen in the information assistance terminal.

[0019] Figure 5 It is a diagram showing a display example of a scenario input screen in the information assistance terminal.

[0020] Figure 6 This is a diagram illustrating the data structure of the fragment cutout data in the information assistance terminal.

[0021] Figure 7It is a diagram showing a display example of a segment selection screen on the information assistance terminal.

[0022] Figure 8 This is a flowchart illustrating the operation of the clip shooting function in the camera system.

[0023] Figure 9 This is a diagram showing a display example of a rating screen on an information assistance terminal.

[0024] Figure 10 This is a flowchart illustrating the segment list generation process in the camera system.

[0025] Figure 11 This is a diagram for explaining filtering of a fragment list.

[0026] Figure 12 This is a diagram illustrating an example of a fragment list when there are additional fragments.

[0027] Figure 13 This is a flowchart illustrating the processing in the recording mode in the camera system.

[0028] Figure 14 It is a diagram showing a display example in the recording mode of the information assistance terminal.

[0029] Figure 15 This is a diagram illustrating the data structure of moving image metadata in an information support terminal.

[0030] Figure 16 This is a flowchart illustrating the processing in the reproduction mode in the imaging system.

[0031] Figure 17 It is a diagram showing a display example in the reproduction mode of the information assistance terminal.

[0032] Figure 18 This is a diagram showing a display example of a moving image editing screen on a moving image editing PC of the imaging system.

[0033] Figure 19 This is a flowchart illustrating the operation of the export function in the imaging system.

[0034] Figure 20 It is a diagram showing a display example of the export function of the information assistance terminal.

[0035] Figure 21 This is a diagram illustrating an example of a moving image folder in the moving image editing screen.

[0036] Figure 22 This is a diagram illustrating an example of a moving image timeline on a moving image editing screen.

[0037] Figure 23 This is a flowchart illustrating folder formation processing in the camera system.

[0038] Figure 24 This is a diagram illustrating the data structure of management data based on folder configuration processing.

[0039] Figure 25 This is a flowchart illustrating the timeline setting process in the imaging system.

[0040] Figure 26 This is a diagram illustrating the data structure of management data based on the timeline setting process.

[0041] Figure 27 A diagram for explaining a modified example of a digital camera.

[0042] Explanation of symbols

[0043] 10 Camera System

[0044] 100 digital cameras

[0045] 200 Information Auxiliary Terminal

[0046] 210 Control Department

[0047] 220 Storage Department

[0048] 230 Operation Department

[0049] 240 Display unit

[0050] 250 Department of Communications. DETAILED DESCRIPTION

[0051] The following description of the embodiments is described in detail with reference to the accompanying drawings as appropriate. However, detailed descriptions of known matters and repeated descriptions of substantially identical structures may be omitted. In addition, the drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure and are not intended to limit the subject matter described in the claims.

[0052] (Implementation Method 1)

[0053] In the first embodiment of the present disclosure, a system using an electronic device independent of an imaging device that performs moving image capturing will be described.

[0054] 1. Structure

[0055] use Figure 1 The imaging system in the first embodiment of the present disclosure will be described.

[0056] For example, Figure 1As shown, the present system 10 includes a digital camera 100, an information support terminal 200, and a moving image editing personal computer (PC) 300. In the present system 10, the digital camera 100 and the information support terminal 200 are connected for data communication, for example, by wired communication or wireless communication.

[0057] The system 10 can be used, for example, in applications where a user creates a desired moving image work by shooting multiple moving images with a digital camera 100 and then editing them. For example, the system 10 provides information support useful for a series of workflows, including creating a scene representing the concept of the moving image work, repeatedly shooting moving images corresponding to the multiple segments of the scene, and editing the multiple captured moving images.

[0058] In the present system 10, the information support terminal 200 can, for example, manage scenes in a moving image work or control the digital camera 100 to manage the moving image capture for each segment. For example, the live view image from the digital camera 100 can be viewed on the information support terminal 200. The moving image data captured by the digital camera 100 is edited on the moving image editing PC 300. To facilitate such moving image editing on the moving image editing PC 300, the present system 10 uses the data managed on the information support terminal 200.

[0059] In the present system 10, the video editing PC 300 may or may not be communicatively connected to one or both of the digital camera 100 and the information auxiliary terminal 200. For example, data from the digital camera 100 and / or the information auxiliary terminal 200 may be input to the video editing PC 300 via a removable recording medium such as a memory card. The present system 10 may also not include the video editing PC 300.

[0060] 1.1. Structure of a digital camera

[0061] use Figure 2 The structure of the digital camera 100 in this embodiment will be described.

[0062] Figure 2This figure illustrates the configuration of a digital camera 100 in the present system 10. The digital camera 100 is an example of an imaging device in this embodiment. The digital camera 100 in this embodiment includes an image sensor 115, an image processing engine 120, a display monitor 130, and a control unit 135. Furthermore, the digital camera 100 includes a buffer memory 125, a card slot 140, a flash memory 145, an operation unit 150, a communication module 155, a microphone 160, and a speaker 170. Furthermore, the digital camera 100 includes, for example, an optical system 110 and a lens drive unit 112.

[0063] The optical system 110 includes a focus lens, a zoom lens, an optical image stabilization lens (OIS), an aperture, a shutter, and other components. The focus lens is used to change the focus of the subject image formed on the image sensor 115. The zoom lens is used to change the magnification of the subject image formed by the optical system. The focus lens and other components are each composed of one or more lenses.

[0064] The lens driver 112 drives the focus lens and other components of the optical system 110. The lens driver 112 includes a motor and moves the focus lens along the optical axis of the optical system 110 under the control of the control unit 135. The structure that drives the focus lens in the lens driver 112 is implemented by a DC motor, a stepping motor, a servo motor, or an ultrasonic motor.

[0065] The image sensor 115 captures the subject image formed via the optical system 110 and generates image data. This image data represents the image captured by the image sensor 115. The image sensor 115 generates image data for each new frame at a predetermined frame rate (e.g., 30 frames per second). The timing of image data generation and the operation of the electronic shutter in the image sensor 115 are controlled by the control unit 135. The image sensor 115 can use various image sensors, such as a CMOS image sensor, a CCD image sensor, or an NMOS image sensor.

[0066] The image sensor 115 performs operations such as still image capture and through-image capture. Through-images are primarily moving images and are displayed on the display monitor 130 so that the user can determine the composition for capturing still images. Through-images and still images are each an example of captured images in this embodiment. The image sensor 115 is an example of an imaging unit in this embodiment.

[0067] The image processing engine 120 performs various processing on the image data output from the image sensor 115 to generate image data, or performs various processing on the image data to generate an image for display on the display monitor 130. Examples of these various processing include, but are not limited to, white balance correction, gamma correction, YC conversion, electronic zoom, compression, and stretching. The image processing engine 120 may be comprised of hard-wired electronic circuits or a microcomputer or processor running a program.

[0068] The display monitor 130 is an example of a display unit that displays various information. For example, the display monitor 130 displays an image (through image) represented by image data captured by the image sensor 115 and processed by the image processing engine 120. The display monitor 130 also displays menu screens, etc., for the user to configure various settings for the digital camera 100. The display monitor 130 can be configured, for example, with a liquid crystal display device or an organic EL device.

[0069] The operation unit 150 is a general term for hardware keys such as operation buttons and a joystick provided on the exterior of the digital camera 100. It receives user operations. Examples of the operation unit 150 include a release button, a mode dial, and a touch panel. Upon receiving a user operation, the operation unit 150 transmits an operation signal corresponding to the user operation to the control unit 135.

[0070] The control unit 135 comprehensively controls the overall operation of the digital camera 100. The control unit 135 includes a CPU, etc., and the CPU executes programs (software) to implement predetermined functions. Alternatively, the control unit 135 may include a processor comprised of dedicated electronic circuits designed to implement predetermined functions, instead of a CPU. Specifically, the control unit 135 can be implemented using various processors, such as a CPU, MPU, GPU, DSP, FPGA, and ASIC. The control unit 135 may also be comprised of one or more processors. Furthermore, the control unit 135 may be implemented on a single semiconductor chip along with the image processing engine 120 and the like.

[0071] The buffer memory 125 is a recording medium that functions as working memory for the image processing engine 120 and the control unit 135. The buffer memory 125 is implemented using DRAM (Dynamic Random Access Memory) or other means. The flash memory 145 is a non-volatile recording medium. Although not shown, the control unit 135 may include various internal memories, such as a built-in ROM. The ROM stores various programs executed by the control unit 135. Alternatively, the control unit 135 may include a built-in RAM that functions as a work area for the CPU.

[0072] Card slot 140 is a unit for inserting a removable memory card 142. Card slot 140 can electrically and mechanically connect memory card 142. Memory card 142 is an external memory device having a recording element such as a flash memory. Memory card 142 can store data such as image data generated by image processing engine 120.

[0073] The communication module 155 is a module (circuit) that connects to external devices using a given communication standard, either wired or wireless. Examples of given communication standards include USB, HDMI (registered trademark), IEEE 802.11, Wi-Fi, and Bluetooth. The digital camera 100 can communicate with other devices via the communication module 155.

[0074] The microphone 160 includes, for example, one or more microphone elements built into the digital camera 100. The microphone 160 outputs a sound signal representing the sound it picks up to the control unit 135. An external microphone may also be used in the digital camera 100. The digital camera 100 may include a connection unit such as a terminal for connecting to an external microphone in place of the built-in microphone 160, or may include a connection unit such as a terminal for connecting to an external microphone in addition to the microphone 160.

[0075] The speaker 170 includes, for example, one or more speaker elements built into the digital camera 100, and outputs sound to the outside of the digital camera 100 under the control of the control unit 135. An external speaker or headphones may also be used with the digital camera 100. The digital camera 100 may include a connection unit for connecting to an external speaker, etc., instead of the built-in speaker 170, or may include a connection unit for connecting to an external speaker, etc., in addition to the speaker 170.

[0076] 1.2. Structure of the information auxiliary terminal

[0077] use Figure 3 The structure of the information assistance terminal 200 in this embodiment will be described.

[0078] Figure 3 2 is a diagram illustrating a configuration of the information supporting terminal 200. The information supporting terminal 200 is an example of an electronic device such as a smartphone, a tablet terminal, or a PC. Figure 3 The information assistance terminal 200 illustrated in FIG includes a control unit 210 , a storage unit 220 , an operation unit 230 , a display unit 240 , a communication unit 250 , a microphone 260 , and a speaker 270 .

[0079] The control unit 210 includes, for example, a CPU or MPU that cooperates with software to implement given functions. The control unit 210 controls the overall operation of the information auxiliary terminal 200. The control unit 210 reads data and programs stored in the storage unit 220 and performs various calculations to implement various functions.

[0080] The control unit 210, for example, executes a program including a set of commands for implementing the aforementioned functions. The program may be provided via a communication network such as the Internet or stored on a removable recording medium. Alternatively, the control unit 210 may be a hardware circuit such as a dedicated electronic circuit or a reconfigurable electronic circuit designed to implement the aforementioned functions. The control unit 210 may also be comprised of various semiconductor integrated circuits, such as a CPU, MPU, GPU, GPGPU, TPU, microcomputer, DSP, FPGA, and ASIC.

[0081] The storage unit 220 is a storage medium that stores programs and data required to realize the functions of the information assistance terminal 200. Figure 3 As shown, the storage unit 220 includes a storage unit 221 and a temporary storage unit 222 .

[0082] The storage unit 221 stores parameters, data, and control programs for realizing predetermined functions. The storage unit 221 is composed of, for example, an HDD or an SSD. For example, the storage unit 221 stores the aforementioned programs and various image data.

[0083] Temporary storage unit 222 is comprised of RAM, such as DRAM or SRAM, and temporarily stores (i.e., retains) data. For example, temporary storage unit 222 retains image data that is in the process of being edited. Temporary storage unit 222 can function as a work area for control unit 210 or be comprised of a storage area within the internal memory of control unit 210.

[0084] The operating unit 230 is a general term for operating components operated by the user. The operating unit 230 is, for example, a touch panel that is overlapped with the display unit 240 to input various touch operations, and is an example of an input unit of the information auxiliary terminal 200. The input unit can also be a connection software unit that communicates with various external input devices to receive operation signals. The operating unit 230 can be a physical button or switch provided in the information auxiliary terminal 200, or a keyboard, mouse, or touchpad can be used. The operating unit 230 can also be various GUIs such as virtual buttons, icons, cursors, software keyboards, and targets displayed on the display unit 240.

[0085] The display unit 240 is composed of, for example, a liquid crystal display or an organic EL display, and can also display various information such as various GUIs for operating the operation unit 230 and information input from the operation unit 230 .

[0086] The communication unit 250 is a module (circuit) that connects to external devices using a given communication standard, whether wired or wireless. Examples of given communication standards include USB, HDMI, IEEE 802.11, Wi-Fi, and Bluetooth. The communication unit 250 can also connect the information auxiliary terminal 200 to a communication network such as the Internet. The communication unit 250 is an example of an acquisition unit that receives various information from external devices or a communication network.

[0087] Microphone 260 includes, for example, one or more microphone elements built into information auxiliary terminal 200. Microphone 260 outputs a sound signal representing the picked-up sound to control unit 210. Information auxiliary terminal 200 may include a connection portion such as a terminal for connecting to an external microphone instead of the built-in microphone 260, or may include a connection portion such as a terminal for connecting to an external microphone in addition to microphone 260.

[0088] The speaker 270 includes, for example, one or more speaker elements built into the digital camera 100, and outputs sound to the outside of the information auxiliary terminal 200 under the control of the control unit 210. The information auxiliary terminal 200 may include a connection unit for connecting to an external speaker or headphones instead of the built-in speaker 270, or may include a connection unit for connecting to an external speaker or headphones in addition to the speaker 270.

[0089] The above-described configuration of the information auxiliary terminal 200 is merely an example, and the configuration of the information auxiliary terminal 200 is not limited thereto. For example, various display devices such as a projector and a head-mounted display may be used as the display unit 240 of the information auxiliary terminal 200. Furthermore, when using an external display device, the display unit 240 of the information auxiliary terminal 200 may also be an output interface circuit for video signals, for example, in accordance with the HDMI standard.

[0090] 2. Action

[0091] The operation of the system 10 configured as described above will be described below.

[0092] In the present system 10, the information assistance terminal 200 has various functions for sequentially providing information assistance to the user in the workflow of motion picture production. An example of a display screen for selecting various functions of the information assistance terminal 200 is shown in FIG. Figure 4 .

[0093] The display unit 240 of the information assistance terminal 200 is Figure 4 The function selection screen illustrated in FIG displays a scene creation button 11, a capture button 12, and an export button 13. Hereinafter, the longitudinal direction of the screen of the display unit 240 is referred to as the X direction, and the width direction is referred to as the Y direction.

[0094] The scene creation button 11 is a virtual button that accepts user operation to execute a function (i.e., a scene creation function) within the system 10 that provides information support for the user's scene creation process prior to movie shooting. The information support terminal 200 of the system 10 manages information for each segment, known as a shooting interval, that divides the scene thus created. For example, a segment comprises a period used for multiple movie shootings of a scene.

[0095] The capture button 12 is, for example, a virtual button for executing a function (i.e., a clip capture function) that assists in capturing moving images for each segment of a scene created by the scene creation function. The number of takes for a moving image per segment is not limited to one, but may be multiple. In this embodiment, the information support terminal 200 controls the moving image capture performed by the digital camera 100 during the clip capture function and manages the capture results for each segment.

[0096] The export button 13 is a virtual button for executing a function (i.e., an export function) for pre-processing the results of video capture management using the clip capture function for external output and then outputting them. The pre-processing using the export function provides information support that facilitates the process of editing multiple video capture results along a scene-by-scene basis on the video editing PC 300.

[0097] The information assistance terminal 200 of the present system 10 can provide continuous information assistance from scene creation to pre-processing of motion picture editing by, for example, the user sequentially utilizing the functions of the scene creation button 11 , the capture button 12 , and the export button 13 .

[0098] In the present system 10, the function selection screen of the information assistance terminal 200 may further include a delete button for deleting various data in the information assistance. For example, the information assistance terminal 200 may delete all moving image files of the same scene upon user operation of the delete button.

[0099] 2.1. About the scene creation function

[0100] use Figures 5 and 6 The scene creation function in the information assistance terminal 200 of the present system 10 is described.

[0101] Figure 5The following is an example of a scene input screen display in the information assistance terminal 200. For example, when the input of the scene input screen is input from the operation unit 230, Figure 4 When the user clicks the scene creation button 11 in the function selection screen, Figure 5 As illustrated, the control unit 210 of the information assistance terminal 200 causes the display unit 240 to display a scenario input screen.

[0102] The scene input screen is a screen for the user to input the scene into the information assistance terminal 200 in the scene creation function of the present system 10. For example, Figure 5 As shown, the scene input screen includes a storyboard input field 20 for each segment, a segment edit button 14, and a return button 15. The control unit 210 of the information assistance terminal 200 receives various user operations related to the scene input screen displayed on the display unit 240 through the operation unit 230.

[0103] In the information support terminal 200, the storyboard input field 20 receives user input of storyboard information such as an outline of the scene concept for each segment constituting the scene. Figure 5 As shown, the storyboard input field 20 of each clip includes a composition field 21 , a script field 22 , a shooting time field 23 , a shooting location field 24 and a note field 25 .

[0104] The composition field 21 accepts input of image information, such as the composition of the video clip being shot. This image information can be input by drawing via user operation or specifying image data. The script field 22 accepts text input, such as a script for each clip in the scene.

[0105] The Shooting Time field 23 accepts a numerical input indicating the target duration of time for shooting the moving image for that segment. The Shooting Location field 24 accepts input indicating the location where the moving image for that segment was shot. The shooting location can be entered as text or, as appropriate, through a data search. The Notes field 25 accepts input of various user-desired information regarding the shooting of the moving image for that segment, such as the shooting equipment, through text entry.

[0106] exist Figure 5 In the example shown in FIG. 2 , the display unit 240 displays storyboard input fields 20 for two segments. The control unit 210 acquires storyboard information for each segment in response to user input into the various fields 21 to 25 of the storyboard input field 20 for each segment in the scene. On the scene input screen of the information assistance terminal 200, the storyboard input fields 20 for each segment displayed on the display unit 240 can be changed, for example, by a scrolling operation in the X direction for scrolling the storyboard input fields 20 for each segment.

[0107] The clip edit button 14 switches between on and off, allowing various user operations such as adding, deleting, and reordering clips included in a scene. For example, by touching the clip edit button 14 while it is on, the user can arrange a desired number of clips in the temporal sequence of the scene in the storyboard input field 20.

[0108] The return button 15 accepts a user operation to return the screen transition in the information assistance terminal 200 to one screen. For example, the control unit 210 inputs the screen ( Figure 5 ) by the user's operation of the return button 15 in the display unit 240 to transition to the function selection screen ( Figure 4 ). The control unit 210 of this embodiment generates segment cutting data including storyboard information of each segment and stores it in the storage unit 220 as the output of such scene creation function. The segment cutting data at the end of such scene creation function is shown in FIG. Figure 6 .

[0109] For example, Figure 6 As shown, the segment cutting data D1 manages the "script," "composition," "shooting time," "shooting location," "shooting completion flag," and "movie metadata list" by associating them for each "segment number." This segment cutting data D1 is an example of management information in this embodiment.

[0110] For example, the control unit 210 of the information assistance terminal 200 assigns, in ascending order, segment numbers representing the identification information of the segments in the segment cut data D1 to the storyboard input field 20 for each segment, which is configured on the scene input screen. Furthermore, if the order of the segments is changed, the control unit 210 reallocates the segment numbers according to the changed order. For each segment, the control unit 210 records the information entered in the script field 22, composition field 21, shooting time field 23, shooting location field 24, and note field 25 of the storyboard input field 20 in the "Script," "Composition," "Shooting Time," and "Shooting Location" fields of the segment cut data D1, respectively.

[0111] In the segment cut data D1, the "Filming Completion Flag" is set to "Complete" (ON) or "Incomplete" (OFF) to manage whether the segment is filmed. At the end of the scene creation function, the "Filming Completion Flag" for all segments is initially set to "Incomplete."

[0112] The "moving image metadata list" is a list for storing metadata of the moving images shot in association with the clip. When the scene creation function ends, the moving image metadata list is set to an empty value as an initial setting.

[0113] As described above, according to the scene creation function in the information assistance terminal 200 of this system 10, by generating segment cutting data D1 based on user input in the scene input screen, information assistance can be provided to the user in the process of creating the scene of the desired dynamic image work for each segment.

[0114] The scene creation function of the information assistance terminal 200 is not particularly limited to the above. For example, the information assistance terminal 200 may receive a user instruction to output the storyboard information of the scene entered on the scene input screen using a data format that can be shared with other devices (e.g., PDF format) and perform such data output.

[0115] 2.2. About the Clip Shooting Function

[0116] Reference Figure 7 An overview of the operation of the clip shooting function in the information assistance terminal 200 of the present system 10 will be described.

[0117] Figure 7 This figure shows an example of a clip selection screen displayed on the information assistance terminal 200. The clip selection screen is used to select a user-desired clip from among clips set using, for example, the scene creation function, in the clip shooting function of the present system 10. The clip selection screen is an example of a selection screen in the information assistance terminal 200 of this embodiment.

[0118] For example, Figure 7 As shown, the clip selection screen includes a clip list 30, a storyboard display bar 31, a filter button 32, an add clip button 33, a record mode button 34, a playback mode button 35, and a return button 15. The clip list 30 lists various clips that the user can select. The storyboard display bar 31 displays storyboard information for the selected clip. Details of the clip selection screen will be described later.

[0119] In the clip shooting function of the present system 10, the information auxiliary terminal 200 provides a user-friendly Figure 7 The clip selection screen shown here allows users to comprehensively advance the video shooting of each clip while checking various clips. Users may shoot a video in an order different from the order of the clips in the scene, or shoot multiple takes for a single clip.

[0120] To this end, the information support terminal 200 of the present system 10 receives user feedback on the selected clip during the shooting of each take of the moving image, manages whether the shooting of the clip is complete, and displays the progress of the moving image shooting of each clip to the user in the clip list 30. The following describes the details of the operation of the present system 10.

[0121] 2.2.1. Overall operation of the clip shooting function

[0122] use Figures 7 to 9 The overall operation of the clip shooting function in the present system 10 will be described.

[0123] Figure 8 This is a flowchart illustrating the operation of the clip shooting function in the present system 10 . Figure 8 Each process shown in the flowchart is executed by, for example, the control unit 210 of the information auxiliary terminal 200. For example, when the segment cutting data D1 based on the scene creation function is stored in the storage unit 220 and the communication connection with the digital camera 100 is established in the communication unit 250, the function selection screen ( Figure 4 ) is operated, the processing of this flow starts.

[0124] First, the control unit 210 of the information auxiliary terminal 200 generates a segment selection screen ( Figure 7 ) (S1). In this system 10, the clip list generation process (S1) is repeatedly executed, for example, during the execution of the clip shooting function, based on the progress of the movie shooting and various user operations, to sequentially update the clip list 30. Details of the process in step S1 will be described later.

[0125] Next, the control unit 210 performs the following operations based on the generated segment list 30 and the segment cutting data D1, for example, Figure 7 As shown, the display unit 240 displays a segment selection screen ( S2 ).

[0126] For example, Figure 7 As shown, the segment list 30 in the segment selection screen includes a plurality of segment icons 3. Each segment icon 3 represents a separate segment as an option. For example, the selected segment icon 3 is initially set to segment number "1".

[0127] For example, the control unit 210 controls the display unit 240 so that the segment icon 3 representing the currently selected segment is highlighted (S2). For example, the highlighted segment icon 3 may be displayed larger than the other segment icons 3 and surrounded by a frame in an accent color. Furthermore, the control unit 210 refers to the segment segmentation data D1 and causes the storyboard display field 31 to display storyboard information for the segment represented by the currently selected segment icon 3 (S2).

[0128] exist Figure 7 In the example shown, clips numbered "1" and "4" are in the "Unfinished" state, while clips numbered "2" and "3" are in the "Complete" state. In the clip list 30 of this embodiment, the clip icon 3 has a display attribute that distinguishes between the "Complete" and "Unfinished" states. This display attribute is set so that, for example, the display mode for the "Complete" state emphasizes the display mode for the "Unfinished" state.

[0129] For example, Figure 7 As shown, while the display unit 240 displays the clip selection screen, the control unit 210 receives various user operations via the operation unit 230, such as a touch panel (S3). The user operations targeted by step S3 include (I) a clip selection operation, (II) an operation to switch to recording mode, (III) an operation to switch to playback mode, (IV) a filtering operation, (V) an operation to add a clip, and (VI) an end operation.

[0130] The segment selection operation (S3 (I)) is a user operation for changing the selected segment. For example, it is an operation of clicking a segment icon 3 other than the selected segment icon 3 in the segment list 30 displayed on the segment selection screen. The segment selection operation is not limited to this. For example, a sliding operation in the storyboard display bar 31 can also be input as a segment selection operation for changing from the selected segment to an adjacent segment.

[0131] When a clip selection operation is input (step (I) of step S3), the control unit 210 changes the selected clip icon 3 in accordance with the input clip selection operation (step S4), and repeats the process from step S2 onward. As a result, the selected clip icon 3 is changed on the clip selection screen, and the storyboard display bar 31 is displayed for the newly selected clip (step S2).

[0132] The transition operation to recording mode (step (II) of S3) is a user operation for transitioning to recording mode, an action mode for capturing moving images of the selected clip. For example, this operation involves clicking recording mode button 34. Additionally or alternatively, this transition operation may involve sliding the clip selection screen in a predetermined direction within the ±X directions. Furthermore, recording mode button 34 may be omitted.

[0133] When the operation to switch to the recording mode is input (S3 (II)), the control unit 210 executes various processes for recording a single shot of a moving image in association with the selected clip as the recording mode (S5). An example of the display in step S5 is shown in FIG. Figure 9 .

[0134] Figure 9 The following shows an example of a rating screen display in the information support terminal 200. The rating screen is a screen for prompting the user to determine a rating for the moving image of the taken take. The rating screen is an example of an evaluation screen in the information support terminal 200 of this embodiment.

[0135] For example, Figure 9 As shown, the rating screen includes an information display bar 40 for the captured moving image, an OK button 41, a KEEP button 42, and an NG button 43 as rating options. Information display bar 40 displays information related to the captured moving image, such as a thumbnail image of the moving image, the clip number associated with the take, and the take number.

[0136] For example, the OK button 41 indicates an evaluation of "OK," indicating that the user has determined that the take is suitable for the corresponding clip. The Keep button 42 indicates an evaluation of "Keep," indicating that the user is unsure whether to adopt the take. The NG button 43 indicates an evaluation of "NG (No Good)," indicating that the user has clearly determined that the take is not suitable. In this embodiment, the evaluation of "NG" is an example of the first evaluation, and the evaluations of "OK" and "Keep" are each examples of the second evaluation.

[0137] The video recording mode processing (S5) in this embodiment is performed every time a moving image is captured, for example, Figure 9 The control unit 210 obtains the rating information indicating the user's evaluation of the shot based on the rating result of the processing (S5) of the recording mode. Figure 8 As shown, the segment list generation process (S1) is performed again to update the segment list 30. The details of the process of step S5 will be described later.

[0138] The transition to playback mode (step (III) of S3) is a user operation for transitioning to playback mode, an operating mode in which the video captured for the selected clip is played back and displayed. This operation, for example, involves pressing playback mode button 35. Additionally or alternatively, the transition to playback mode may be a sliding operation in the ±X direction of the clip selection screen in the direction opposite to the transition to recording mode. Furthermore, playback mode button 35 may be omitted.

[0139] When a transition operation to playback mode is input (step (III) of step S3), the control unit 210 executes a process (step S6) for reproducing the moving images of various takes related to the selected clip as playback mode. In this embodiment, the playback mode process (step S6) can include re-rating, which changes the evaluation of the moving images for each take. Based on the re-rating results of this playback mode process (step S6), the control unit 210 performs the clip list generation process (step S1) again to update the clip list 30. Details of the process in step S6 will be described later.

[0140] The filter operation (S3(IV)) is a user operation for narrowing down the segments to be displayed in the segment list 30, and is, for example, an operation of the filter button 32. When the filter operation (S3(IV)) is input, the control unit 210 obtains conditions for filtering the segments to be displayed based on the user's selection (S7).

[0141] The information assistance terminal 200 of this embodiment uses the filming location in the storyboard information for each segment as a filter condition for the segment list 30. The control unit 210 performs the segment list generation process again based on the filming location acquired as the filter condition ( S1 ). This updates the segment list 30 to include only the segment icons 3 (described in detail later) corresponding to the filming location in the filter condition.

[0142] The segment addition operation (S3(V)) is a user operation for adding a new segment in addition to the existing segments in the segment list 30, and is, for example, an operation of the segment addition button 33. When the segment addition operation (S3(V)) is input, the control unit 210 sets various information related to the added segment (S8) and performs the segment list generation process (S1) again (described in detail later).

[0143] The ending operation ((VI) of S3) is a user operation for ending the clip shooting function, for example, the clip selection screen ( Figure 7 ) in the return button 15. When the end operation is input (S3 (VI)), the control unit 210, for example, switches the display unit 240 from the segment selection screen to the function selection screen ( Figure 4 ) transition, ending the processing shown in this flow.

[0144] According to the above process, the user of the present system 10 can select the clip selection screen ( Figure 7 ) and confirms various segments (S4), while performing dynamic image shooting of a desired segment (S5), or performing playback display (S6). In this way, the user can easily manage the dynamic image shooting of multiple segments in the scene.

[0145] In the clip selection screen of this embodiment, each clip icon 3 is displayed to distinguish whether it has been filmed or not, thereby preventing the user from forgetting to film a clip. The display of whether each clip has been filmed or not reflects the user's evaluation of the moving image for each take, making it easier to ensure moving image quality that meets the user's expectations. This rating is performed for each take (S5), and can also be re-rated in playback mode (S6). This facilitates quality management of moving image filming that meets the user's expectations.

[0146] In addition, the segment selection screen ( Figure 7 ) is not limited to updating the clip list 30 in accordance with the above-described clip rating / re-rating (S5, S6, S1), but can also be performed to filter or add clips to be displayed (S7, S8, S1). This allows the user to efficiently use desired clips on the clip selection screen during live video shooting, for example, and facilitates the use of the clip shooting function of this system 10.

[0147] In the clip shooting function of this embodiment, the communication connection with the digital camera 100 can be managed. For example, a button for managing the communication connection can be provided on the clip selection screen. The control unit 210 can also disable the operation to switch to the recording mode (S3 (II)) if the communication connection with the digital camera 100 has not yet been established.

[0148] 2.2.2. Fragment list generation process

[0149] use Figures 10 to 12 Come to Figure 8 The details of the fragment list generation process in step S1 are described below.

[0150] Figure 10 This is a flowchart illustrating the segment list generation process ( S1 ) in the present system 10 . Figure 10 The processing shown in the flow is, for example, based on the function selection screen ( Figure 4 Alternatively, the processing of this flow is based on the clip selection screen ( Figure 7 ) in a given user action ( Figure 8S3 (II) to (V)) is started after executing steps S5 to S8.

[0151] First, the control unit 210 of the information auxiliary terminal 200 cuts the data D1 ( Figure 6 ), one segment is selected from all the segments included in the segment list 30 or the like as a check object (S10). The selection in step S10 is performed, for example, in ascending order of the segment numbers in the segment cutout data D1.

[0152] Next, the control unit 210 determines whether the shooting location of the inspection object meets the filtering condition based on the shooting location of the inspection object segment in the segment cut data D1 (S11). Figure 11 Such filtering will be described.

[0153] Figure 11 Show Figure 8 Example of display of the selection dialog box 32a in step S7. For example, when the segment selection screen ( Figure 7 ) When the user clicks the filter button 32 (S3 (IV)), in step S7, the display unit 240 displays the selection dialog box 32a. The selection dialog box 32a includes options for filter conditions such as "All Locations", "Photography Location 1", and "Photography Location 2".

[0154] For example, the filter condition is set to "All locations" as the initial state (see Figure 7 Filter button 32), in this case, the judgment of step S11 is "yes". In addition, in the user's filtering operation ( Figure 8 When a specific shooting location is selected as a filtering condition in step (IV) of step S3 ( S7 ), the control unit 210 determines whether the shooting location of the inspection target segment matches the specific shooting location ( S11 ).

[0155] exist Figure 8 In step S7, the control unit 210 causes the selection dialog box 32a ( Figure 11 ) is displayed, and a user operation of selecting an option in the selection dialog box 32a is accepted. For example, when the user clicks on an option in the selection dialog box 32a, the control unit 210 obtains the shooting location of the option as a filtering condition (S7), and performs the determination in step S11 of the clip list generation process (S1) based on the filtering condition.

[0156] Return to Figure 10If the shooting location of the inspection target does not meet the filtering conditions (No in S11), the control unit 210 proceeds to step S16 without performing steps S12 to S15, such as generating a segment icon 3. Thus, segments with shooting locations different from the filtering conditions are excluded from display in the segment list 30.

[0157] If the imaging location of the inspection object meets the filtering condition (Yes in S11 ), the segment is included in the display target of the segment list 30. In this case, the control unit 210 generates a segment icon 3 corresponding to the segment (S12).

[0158] Next, the control unit 210 determines, for example, based on the segment cut data D1, whether the capture completion flag for the segment for which the segment icon 3 was generated is set to "complete" (S13). For example, the capture completion flag is set to "complete" if the dynamic images associated with the inspection target segment include a dynamic image with an evaluation of "OK" or "hold," and is set to "incomplete" if the dynamic image does not include a dynamic image with an evaluation of "OK" or "hold." The determination in step S13 can also be performed by referring to the evaluations of each dynamic image associated with the inspection target segment, rather than using the capture completion flag.

[0159] When the shooting completion flag of the segment indicates completion (YES in S13 ), the control unit 210 sets a display attribute of the shooting completion state to the corresponding segment icon 3 ( S14 ).

[0160] On the other hand, if the shooting completion flag of the segment indicates not completed but incomplete (No in S13 ), the control unit 210 sets a display attribute of the shooting incomplete state to the corresponding segment icon 3 ( S15 ).

[0161] The control unit 210 determines, for example, based on the segment cutout data D1, whether all segments assigned segment numbers have been checked (S16). This determination in step S16 is performed, for example, within the range of segments that have been included in the segment cutout data D1 since scene creation, unlike additional segments (hereinafter referred to as "normal segments").

[0162] If all normal segments have not been checked (No in S16), the control unit 210 performs the process from step S11 onwards again for the unchecked normal segments. In this way, all normal segments are sequentially checked for display targets in the segment list 30 (Yes in S16).

[0163] When all normal segments are checked (Yes in S16), the control unit 210 determines whether there is an additional segment (S17). Figure 8In step S8, the additional segment is set to, for example, an additional segment number (see Figure 12 ).

[0164] If no additional segments have been added (No in S17), the control unit 210 does not perform the processing of step S18, but instead generates a segment list 30 based on the normal segment inspection results (S11 to S16) (S19). For example, in the segment list 30, the segment icons 3 to be displayed are arranged in ascending order of segment numbers.

[0165] On the other hand, if there are additional segments (YES in S17), the control unit 210 performs various processes related to checking the display targets of the segment list 30 on the additional segments, similar to normal segments (S18). The process of step S18 is performed similarly to steps S11 to S16, with the additional segments being the inspection targets instead of the normal segments. In this case, the control unit 210 generates the segment list 30 based on the inspection results of the normal segments and the additional segments (S11 to S16, S18) (S19).

[0166] When the fragment list 30 is generated in this manner ( S19 ), the control unit 210 ends, for example Figure 8 After completing the processing of step S1, proceed to step S2.

[0167] According to the above segment list generation process (S1), the information assistance terminal 200 of the present system 10 generates a segment list 30 that lists the segments included in the scene based on the segment cutting data D1, so that it can be identified and displayed whether the shooting is completed (S14, S15). When the evaluation of each shot in each segment changes ( Figure 8 The identification display of the segment list 30 is dynamically updated according to the changed evaluation. This makes it easy for the user to confirm the progress of the moving image shooting of multiple segments.

[0168] In addition, in the present system 10, the filming location is used as a filter condition to filter the clip list 30 (see Figure 8 S7 and Figure 10 S11). Thus, the user can select the on-site shooting location as a filter condition, for example, and use the clip selection screen to select clips that should be shot on-site. In this way, the system 10 can easily prevent users from forgetting to shoot clips on-site.

[0169] In addition, in the present system 10, the segment list 30 can also be updated to include additional segments (see Figure 8 S8 and Figure 10S17 to S18). The fragment list 30 in the case of additional fragments is shown in FIG. Figure 12 .

[0170] For example, when the user performs a fragment append operation ( Figure 8 In step (V) of S3 of FIG, the control unit 210 sets the additional segment (S8), and then proceeds to “Yes” in step S17 of the segment list generation process (S1), and checks the additional segment (S18). Figure 12 As shown, the segment list 30 is updated to include the segment icon 3 a of the additional segment.

[0171] According to the additional clip of this system 10, the user can immediately add a clip on the spot such as a shooting location without re-editing the scene. Figure 12 As shown, the segment icon 3a of the additional segment is arranged at the end of the segment list 30 in step S1 after step S8, for example. When there are a plurality of additional segments, they are arranged in ascending order of the additional segment numbers, for example.

[0172] In addition, Figure 8 In step S8, the control unit 210 automatically sets the shooting location in the storyboard information of the additional segment, for example, according to the filtering conditions. Alternatively, the shooting location of the additional segment can also be set by user input. By setting the shooting location of the additional segment in this way, the additional segment can also be set as the object of filtering (S7) in the same way as the normal segment. The storyboard information of the additional segment is set to a null value except for the shooting location, for example. As the composition of the storyboard information, a given image indicating that it is an additional segment can also be used. In addition, the additional segment can also be deleted by a given user operation, for example, the given user operation can be a long press operation of the segment icon 3a of the additional segment.

[0173] 2.2.3. About Video Recording Mode

[0174] use Figures 13 to 15 Come to Figure 8 The details of the processing in the recording mode in step S5 will be described.

[0175] Figure 13 This is a flowchart illustrating the processing ( S5 ) in the recording mode in the present system 10 . Figure 13 For example, the processing shown in the flow is performed on the clip selection screen ( Figure 7 ) is inputted in order to switch to the recording mode ((II) of S3 ).

[0176] First, the control unit 210 of the information assistance terminal 200 switches to the recording mode and switches the display unit 240 to a screen for waiting for video recording (S30). An example of the display of the information assistance terminal 200 in step S30 is shown in FIG. Figure 14 (A).

[0177] For example, Figure 14 As shown in (A) of FIG. 3 , the recording standby screen in step S30 includes a timer button 44, a live view image 45, and a record button 46. Timer button 44 accepts user operation for, for example, setting a timer period for timing at the start of recording. Record button 46 accepts user operation for starting recording of a moving image.

[0178] In the present system 10, when the information auxiliary terminal 200 switches to recording mode, for example, the control unit 210 requests the digital camera 100 to transmit the live view image 45 via the communication unit 250 (S30). For example, in recording mode, the control unit 210 sequentially receives image data of the live view image 45 from the digital camera 100 via the communication unit 250 and displays the live view image 45. Furthermore, the control unit 210 receives audio data, for example, collected by the microphone 160, from the digital camera 100 at appropriate times.

[0179] The control unit 210 sets the timer period according to the user's operation of the timer button 44 (S31). The display example of the information assistance terminal 200 in step S31 is shown in FIG. Figure 14 (B) The timer period can be set for each take or for each clip.

[0180] Figure 14 (B) is right Figure 14 The timer selection bar 44a displayed in response to the operation of the timer button 44 in the shooting standby screen of (A) is exemplified. Figure 14 In (B), an example is shown in which the timer period is set to "5 seconds" in the timer selection bar 44a. The timer selection bar 44a includes options such as "Not Enabled" corresponding to a timer period of 0 seconds and "5 seconds" indicating the numerical value of the timer period. When the user operation of the timer selection bar 44a is received in the operation unit 230, in step S31, the control unit 210 sets the timer period according to the user operation and changes the display unit 240 from Figure 14 (B) Return to Figure 14 (A).

[0181] The control unit 210 performs various controls in response to the user's operation of the record button 46 so as to start recording the video of one take associated with the selected clip (S32). For example, in step S32, the control unit 210 instructs the digital camera 100 via the communication unit 250 to start recording the video. An example of the display of the information support terminal 200 in step S32 is shown in FIG. Figure 14 (C), (D).

[0182] Figure 14 (C) is right in such Figure 14 The following exemplifies a captured screen when the recording button 46 is operated while the timer period is set as in the example of (B). Figure 14 (D) from Figure 14 The image capture screen after the timer period (C) is illustrated. The control unit 210 counts down the time elapsed for the set timer period from the time the record button 46 is operated (S32), and displays it superimposed on the live view image, for example. In this example, the dynamic image recording is performed including such a timer period. In addition, when the timer period is set to "not enabled", the control unit 210 does not specifically perform the countdown in step S32. Figure 14 (C) Such a countdown display, for example, makes the display unit 240 change from Figure 14 (A) transition to Figure 14 (D) This is the shooting picture.

[0183] For example, Figure 14 As shown in (C) and (D) of FIG32 , the shooting screen in step S32 includes a live view image 45, a time display bar 47, a recording stop button 46a, and a mark button 48. The live view image 45 in the shooting screen is highlighted, for example, by a frame indicating that the video is being recorded. The time display bar 47 compares the recording time of the selected clip in the clip cutting data D1 with the elapsed time since the start of recording of the moving image of that take.

[0184] In step S32, the control unit 210 controls the display unit 240 to display the shooting waiting screen ( Figure 14 (A)) Switch to the shooting screen ( Figure 14 (C), (D)). Furthermore, the control unit 210 records a moving image file representing the live view image 45 sequentially received from the digital camera 100 after the record button 46 is pressed in the storage unit 220 of the information support terminal 200 (S32). The moving image file includes, for example, sound data picked up by the microphone 160 in synchronization with the moving image shooting of the digital camera 100.

[0185] Furthermore, the control unit 210 determines a file name for the moving image file based on, for example, the segment cutting data D1 and the number of takes at which shooting for the selected segment has ended. The control unit 210 may also include the determined file name in an instruction to the digital camera 100. The control unit 135 of the digital camera 100 starts moving image shooting in accordance with an instruction, for example, from the information assistance terminal 200 received via the communication module 155.

[0186] At this time, the control unit 135 repeatedly performs the image capture operation of the image sensor 115, and records the resulting moving image data on the memory card 142 via the card slot 140, for example. The moving image data includes, for example, sound data obtained by collecting sound with the microphone 160. The control unit 135 may also start collecting sound in synchronization with the moving image capture in the digital camera 100 in response to such a capture instruction.

[0187] For example, in Figure 14 In the shooting screen (D), the recording stop button 46a accepts a user operation to stop the recording of the moving image. The mark button 48 accepts a user operation to mark a desired moment during the shooting of the moving image. For example, the user can use the mark button 48 to select a moment they want to reference during post-processing of the moving image.

[0188] The control unit 210 then performs various controls to stop recording the moving image in response to the user's operation of the recording stop button 46a (S33). For example, in step S33, the control unit 210 instructs the digital camera 100 to stop recording the moving image via the communication unit 250. Furthermore, the control unit 210 stops recording the moving image of the live view image 45 within the information support terminal 200 (S33). The control unit 135 of the digital camera 100 terminates the moving image recording in accordance with the instruction from the information support terminal 200.

[0189] In addition, the control unit 210 prompts the user to rate the moving images of the shots taken as described above, for example, Figure 9 As shown, the rating screen is displayed on the display unit 240 ( S34 ).

[0190] The control unit 210 receives, for example, Figure 9 User operation of the various buttons 41-43 on the rating screen results in the user's evaluation of the captured take as a result of the rating of the moving image (S35). In this embodiment, the user can select a desired rating from the three aforementioned ratings: "OK," "Reserve," and "NG," each time they capture a moving image, particularly for a moving image captured without interfering with the evaluation of other takes.

[0191] The control unit 210 determines whether the acquired user evaluation is, for example, “NG” ( S36 ). For example, if the user evaluation is “OK” or “Reserved”, the determination in step S36 is “No”.

[0192] If the user's evaluation received is not "NG" (No in S36), the control unit 210 sets the shooting completion flag of the clip associated with the take (i.e., the currently selected clip) in the clip cutting data D1 to "Complete" (S37). For example, the shooting completion flag is switched from "Not Completed" to "Complete" by executing step S37 if the take count of the video is "1" or if the video of the current take is evaluated as "NG" for a take count of "2" or higher.

[0193] On the other hand, if the user's evaluation received is "NG" (YES in S36), the control unit 210 does not specifically update the shooting completion flag and proceeds to step S38. Thus, if the shooting completion flag for the clip corresponding to the moment a moving image rated "NG" is captured is incomplete, that incomplete state is maintained. Alternatively, if, for example, a moving image from a previously captured take has a "hold" or "OK" rating, causing the shooting completion flag to be in the completed state, that completed state is maintained.

[0194] The control unit 210 generates metadata of the moving image of the shot captured as described above, for example, segment cut data D1 recorded in the storage unit 220 (S38). Such moving image metadata D2 is exemplified in Figure 15 .

[0195] For example, Figure 15 As shown, the moving image metadata D2 includes "moving image file name," "rating information," "timer duration," and "marker information." The control unit 210 includes in the moving image metadata D2 the moving image file name determined to reflect the number of takes for the moving image captured in steps S32 and S33, the user rating acquired in step S35, and the timer duration set in step S31. Furthermore, if the user operates the mark button 48, the control unit 210 identifies the time of the user operation within the time the moving image was captured and includes this information as mark information in the moving image metadata D2.

[0196] The control unit 210 cuts the data D1 into segments ( Figure 6) and stores the generated dynamic image metadata D2 (S38). Dynamic image metadata D2 is not particularly limited to the above and may include, for example, the number of takes of the dynamic image in addition to or instead of the dynamic image file name.

[0197] The control unit 210 ends the recording mode processing (S5) based on the storage of the moving image metadata D2 (S38), and proceeds to Figure 8 Step S1.

[0198] Through the above-described recording mode processing (S5), the system 10 can capture and record a moving image of one take of the selected clip, prompting the user to rate it (S32-S35). Furthermore, based on the acquired rating information, the system 10 manages the shooting completion flag for the clip (S36, S37). This allows the user's rating information for each take to be appropriately reflected in the management of whether the clip is shooting completed. Furthermore, through the recording mode processing (S5) of this embodiment, the information support terminal 200 can control the recording and recording of moving images by the digital camera 100, thereby achieving management of moving image shooting.

[0199] In the rating of the moving image for each take ( S34 - S35 ), multiple takes associated with the same segment may have multiple takes with the same rating. For example, multiple takes of the same segment may have the moving image rated "OK."

[0200] In addition, the rating screen displayed in step S34 may be displayed as a dialog box. For example, the control unit 210 may control the display unit 240 so that the display screens before and after step S33 are displayed overlapping the dialog box of the rating screen.

[0201] Recording mode recording waiting screen ( Figure 14 (A)) may also include a return button 15 for returning the screen to the clip selection screen, for example. Alternatively, such a return operation may be a swipe in one of the ±X directions of the dynamic image management screen. Furthermore, the information auxiliary terminal 200 may also switch to playback mode by swiping in the opposite direction.

[0202] 2.2.4. About playback mode

[0203] use Figures 16 and 17 right Figure 8 The processing of the reproduction mode in step S6 is described below.

[0204] Figure 16This is a flowchart illustrating the processing ( S6 ) of the reproduction mode in the present system 10 . Figure 16 For example, the processing shown in the flow is performed on the clip selection screen ( Figure 7 ) is inputted in the operation to shift to the playback mode ((III) of S3 ) starts.

[0205] First, the control unit 210 of the information assistance terminal 200 causes the display unit 240 to transition to a screen for managing the dynamic image of the selected segment based on the dynamic image file of the shot associated with the selected segment and the segment cutting data D1 (S51). An example of the display of the information assistance terminal 200 in step S51 is shown in FIG. Figure 17 (A).

[0206] For example, Figure 17 As shown in (A) of FIG5 , the video management screen in step S51 includes a segment identification column 51, a video list 50, a re-rating button 52, and a return button 15. The segment identification column 51 displays identification information for the selected segment. The video list 50 includes a video icon 5 representing each take of the selected segment. For example, the video icon 5 is composed of a thumbnail image of the video in that take superimposed with rating information.

[0207] In step S51, the control unit 210 generates individual moving image icons 5, for example, by referring to the moving image metadata list for the clip in the clip segment data D1. The control unit 210 arranges the moving image icons 5, for example, in ascending order of take number, to generate a moving image list 50 (S51).

[0208] For example, in Figure 17 While the display unit 240 is displaying the video management screen as shown in (A), the control unit 210 receives various user operations via the operation unit 230 ( S52 ). The user operations targeted in step S52 include (I) a playback selection operation, (II) a re-rating operation, and (III) a return operation.

[0209] The playback selection operation ((I) of S52 ) is a user operation for selecting a video file to be played back and displayed, and is, for example, an operation of clicking a desired video icon 5 in the video list 50 displayed on the video management screen.

[0210] The re-rating operation ((II) in S52 ) is a user operation for re-rating the movie files, and is, for example, an operation of clicking the re-rating button 52 and then clicking a desired movie icon 5 .

[0211] The return operation (step (III) of step S52) is a user operation to return from playback mode to the function selection screen. For example, this can be performed by pressing the return button 15 on the video management screen. The return operation in step (III) of step S52 can also be performed by sliding the video management screen in one of the ±X directions. The information support terminal 200 can also switch to recording mode by sliding in the opposite direction.

[0212] When the playback selection operation is input (S52 (I)), the control unit 210 causes the display unit 240 to transition to a screen for playback display of the selected moving image file (S53). An example of the display of the information assistance terminal 200 in step S53 is shown in FIG. Figure 17 (B).

[0213] For example, Figure 17 As shown in (B), the playback screen in step S53 includes a playback image 53, a playback control bar 54, a mark button 55, and a return button 15. The control unit 210 accepts various user operations related to this playback screen via the operation unit 230. For example, the user can switch between playing and temporarily pausing a moving image by clicking on the playback image 53, or change the playback position by clicking on the playback control bar 54.

[0214] The playback control bar 54 indicates the playback time within the entire duration of the moving image, and a mark 56 is placed at a position indicating a specific time. For example, the control unit 210 places the mark 56 on the playback control bar 54 by referring to the mark information in the moving image metadata D2.

[0215] exist Figure 17 On the playback screen (B), the user can newly place a marker 56 by operating the marker button 55 or change the placement of the marker 56 by dragging the marker 56. Upon receiving any of these various marker operations (YES in S54), the control unit 210 updates the marker information in the moving image metadata D2 in accordance with the marker operation (S55).

[0216] Furthermore, when a user instruction to end the playback of the moving image is input by operating the return button 15 (Yes in S56), the control unit 210 changes the display unit 240 from the playback screen ( Figure 17 (B) Transition to the dynamic image management screen ( Figure 17 (A)), return to step S52.

[0217] On the dynamic image management screen ( Figure 17In (A) of step S52, when a re-rating operation is input ((II) of step S52), the control unit 210 causes the display unit 240 to display a screen for prompting the user to re-rating (S57). An example of the display in step S57 is shown in FIG. Figure 17 (C).

[0218] Figure 17 (C) shows an example of a re-rating dialog box 52a displayed on the display unit 240 and superimposed on the video management screen. Figure 9 ) Similarly, various buttons 41 to 43 are included as options for user evaluation. The control unit 210 receives user operation of the various buttons 41 to 43 in the re-rating dialog box 52a and acquires evaluation information indicating the re-rating result of the take of the moving image icon 5 selected by the user in step (II) of step S52 (S58).

[0219] Next, the control unit 210 updates, for example, the segment cutting data D1 based on the evaluation information from the re-rating results (S59). For example, the control unit 210 rewrites the evaluation information for the take in the selected segment's moving image metadata list, or manages the shooting completion flag for the selected segment in consideration of the re-rating results. For example, if the re-rating changes the evaluation information for all takes associated with the selected segment from "NG" to "Hold" or "OK" for any one take, the shooting completion flag switches from "Not Completed" to "Completed."

[0220] Thereafter, the control unit 210 returns to step S51 and updates the moving image list 50 in the moving image management screen so that the re-ranking result is reflected in the moving image icon 5 of the take.

[0221] For example, when in the dynamic image management screen ( Figure 17 When a return operation is input in step (A) of step (III) of step S52), the control unit 210 ends the processing of the reproduction mode (S6) and returns to step (A) of step (III) of step S52. Figure 8 In this way, when re-rating is performed in the playback mode ( S58 ), the segment list 30 is updated in the subsequent segment list generation process ( S1 ) so as to reflect the new evaluation information.

[0222] Through the above playback mode processing (S6), the user can confirm the various takes of the selected clip through playback display (S53-S56) and re-rating based on the confirmation results (S57-S58). Furthermore, while confirming the video, the user can place a marker 56 at the desired moment, making subsequent editing of the video easier.

[0223] 2.3. About the export function

[0224] Reference Figure 18 An overview of the export function in the information assistance terminal 200 of the present system 10 will be described.

[0225] Figure 18 FIG. 3 shows a display example of a moving image editing screen in the moving image editing PC 300 of the present system 10. Moving image editing PC 300 ( Figure 1 ) For example, the management data output by the export function of the information auxiliary terminal 200 and the dynamic image data based on the shooting result of the digital camera 100 are read into a given editing software, such as Figure 18 As shown, a dynamic image editing screen is displayed. The editing software can be various non-linear editing (NLE) software, such as DaVinci Resolve, Adobe Premiere Pro, Final Cut Pro or Vegas Pro.

[0226] Figure 18 The moving image editing screen shown in the figure includes a material display bar 61, a timeline editing bar 62, a metadata display bar 63, and a preview image 64. The moving image editing screen is a screen for the user to perform various moving image editing operations and is an example of an editing screen using the moving image editing PC 300 as an example of an external device.

[0227] The material display column 61 displays a list of the moving image data (i.e., moving image materials) read as materials for moving image editing in the moving image editing PC 300. In the present system 10, for example, the material display column 61 displays a moving image folder 70 (described in detail later), which is a folder for managing moving image materials for each clip.

[0228] The timeline editing bar 62 displays a movie timeline 80 including multiple movie assets arranged on a timeline 81, and accepts user operations for editing a movie work composed of these assets in the movie timeline 80. Movie timeline 80 has tracks arranged along timeline 81, one row for each movie asset. In movie timeline 80, multiple tracks are arranged in a direction V intersecting timeline 81, allowing different movie assets to be arranged at the same position on timeline 81. Hereinafter, the +V side of this arrangement in direction V will sometimes be referred to as the upper side, and the opposite -V side will sometimes be referred to as the lower side.

[0229] Metadata display field 63 displays metadata of the moving image material displayed in material display field 61 or timeline edit field 62. Preview image 64 displays an image of the moving image material at a time corresponding to the position of playback head 82 on time axis 81 in moving image timeline 80.

[0230] For example, in the dynamic image editing screen ( Figure 18 ), the user can adjust the placement of the moving image material in the timeline editing bar 62 while checking the preview image 64, or place the new moving image material in the material display bar 61 on the moving image timeline 80. Through such user operations, the editing operation of the moving image work is performed in the present system 10.

[0231] For example, to facilitate the user's ability to edit the aforementioned moving image work after executing the clip capture function, the information support terminal 200 of this embodiment uses the export function to systematically generate management data for moving image materials, reflecting the clip cutting and user evaluation. This facilitates the user's ability to edit moving images in this system 10, further improving the efficiency of this process on the moving image editing PC 300. The export function of this system 10 is described in detail below.

[0232] 2.3.1. Overall operation of the export function

[0233] use Figures 19 to 22 The overall operation of the export function in the present system 10 will be described.

[0234] Figure 19 This is a flowchart illustrating the operation of the export function in the present system 10 . Figure 19 Each process shown in the flowchart is executed, for example, by the control unit 210 of the information auxiliary terminal 200. For example, when the segment cutting data D1 including the moving image metadata D2 based on the segment shooting function is stored in the storage unit 220, the function selection screen ( Figure 4 ) is operated, the processing of this flow starts.

[0235] First, the control unit 210 of the information auxiliary terminal 200 sets the number of tracks in the video timeline 80, i.e., the number of tracks, etc., according to the user operation in the operation unit 230 (S61). The display example in step S61 is shown in FIG. Figure 20 .

[0236] Figure 20 For example, the display unit 240 is displayed superimposed on the function selection screen ( Figure 4). For example, the export settings dialog box 16 includes an input field 16a for the number of tracks in the video timeline 80 and a selection field 16b for the rating. The control unit 210 receives user operations on the export settings dialog box 16, for example, using the operation unit 230, and acquires the user settings for the video timeline 80 (S61).

[0237] In the export settings dialog box 16, the track number input field 16a accepts user input of the number of tracks desired to be displayed on the movie timeline 80 within a given numerical range (e.g., 1 to 20). The rating selection field 16b accepts user input of, for example, "OK, Keep Both" or "OK ​​Only" as options for display on the timeline. Regardless of which option is selected in the rating selection field 16b, highly rated "OK" movies are preferentially displayed on the movie timeline 80.

[0238] In addition, the control unit 210, for example, configures a dynamic image editing screen ( Figure 18 ) is displayed in the material display column 61 (S62). The dynamic image folder 70 obtained by the folder composition process (S62) is shown in FIG. Figure 21 .

[0239] For example, Figure 21 As shown, the movie folder 70 includes clip folders 71, which are folders for each clip, and a hold folder 72 and an OK folder 73, located within each clip folder 71. The hold folder 72 stores movie files with a "hold" rating within the movie material for that clip. The OK folder 73 stores movie files with an "OK" rating within the movie material for that clip. During the folder configuration process in step S62, the control unit 210 automatically stores the directory structure for implementing this structure of the movie folder 70 in the management data. The folder configuration process (S62) will be described in detail later.

[0240] The icon representing the dynamic image file in the dynamic image folder 70 is an example of the dynamic image information in this embodiment. Figure 21 As shown in the example, an icon representing the moving image timeline 80 is displayed together with the moving image folder 70 described above. Figure 21 In the example, the dynamic image icon in the clip folder 71 of the clip number “1” is displayed, but according to the dynamic image folder 70 , the dynamic image icons in other clip folders 71 can also be displayed appropriately by the user operation on the material display column 61 .

[0241] Furthermore, the control unit 210 sets the folders to be displayed in the timeline editing bar 62 ( Figure 18 ) dynamic image timeline 80 (S63). The dynamic image timeline 80 obtained by such timeline setting processing (S63) is shown in FIG. Figure 22 .

[0242] exist Figure 22 In the example, the dynamic image timeline 80 of the following case is illustrated: Figure 21 In the dynamic image folder 70, the number of shots is set to "3" and the display object is set to "OK, keep both" in step S61, and the image is formed in step S62. Figure 21 70 dynamic image folders.

[0243] For example, Figure 22 As shown, movie timeline 80 includes a set number of video tracks 83 and audio tracks 84 corresponding to each of the video tracks 83. Video track 83 includes a video clip 85 representing the video in the movie file for each segment. Audio track 84 includes an audio clip 86 representing the audio in the movie file for each segment. Video / audio clips 85 and 86 are each an example of the movie information in this embodiment.

[0244] exist Figure 22 In the example, video track 83 is positioned above (toward the +V side) audio track 84. In this example, video / audio tracks 83 and 84, track number "1," are adjacent to each other, presumably intended to be used primarily by the user to create a moving image work. In this example, video track 83 has a higher priority the further it is positioned downward (toward the -V side), while audio track 84 has a higher priority the further it is positioned upward (toward the +V side).

[0245] In step S63, the timeline setting process automatically sets the video timeline 80, reflecting the user's ratings. The higher-rated video / audio clips 85 and 86 are assigned to higher-priority video / audio tracks 83 and 84. Thus, for example, the highest-rated video / audio clips 85 and 86 in each clip are assigned to the video / audio track 83 and 84 with track number "1," which is primarily used by the user. The timeline setting process (S63) will be described in detail later.

[0246] The control unit 210 stores the management data (S64) including the results of the above folder configuration process (S62) and the timeline setting process (S63) in the storage unit 220, and ends the process. Figure 19 The processing shown in the flow.

[0247] Based on the above export function processing, the information assistance terminal 200 of this system 10 can provide a dynamic image folder 70 and a dynamic image timeline 80 (S62, S63), which facilitates the process of editing dynamic images of various clips shot by the user along the scene in the dynamic image editing PC 300.

[0248] For example, according to the dynamic image timeline 80 ( Figure 22 ), the highest-rated moving image material in each clip is placed in the video / audio tracks 83 and 84, track number "1," which is primarily used by the user. These video / audio tracks 83 and 84 can be expected to be close to the tracks desired by the user. This allows the user to perform moving image editing while examining the differences with these video / audio tracks 83 and 84, for example. In this case, in the moving image timeline 80, since the highly-rated moving image materials in the same clip are placed in adjacent video / audio tracks 83 and 84, the user can easily examine the differences.

[0249] In addition, according to the dynamic image folder 70 ( Figure 21 For example, among the clip shooting function's shooting results, the moving images of each take with a high "OK / Keep" rating are systematically arranged for each clip according to the evaluation priority. This allows the user to retrieve moving image material from moving image folder 70 and use it for editing, even if it is not allocated to moving image timeline 80 and is evaluated as potentially acceptable. Furthermore, moving images with a low "NG" rating, which indicates a low likelihood of acceptance, are not included in moving image folder 70. This reduces the data volume of moving image folder 70 and reduces the processing load of moving image editing on moving image editing PC 300.

[0250] In the above description of step S61, it is explained that Figure 20 The example of the export settings dialog box 16 is shown. The user settings for the export function (S61) are not limited to the above example. For example, the control unit 210 may also accept a user operation to switch between configuring a single track or multiple tracks in the movie timeline 80. In this case, when the single-track mode is selected, the control unit 210 may disable the user operation in the track number input field 16a and set the track number to "1."

[0251] In the present system 10, the movie folder 70 can have various configurations. For example, the OK folder 73 and the Hold folder 72 may not necessarily be provided for all clips. For example, the OK / Hold folders 73 and 72 may be omitted for additional clips. Furthermore, in the present system 10, a folder for storing movies of takes rated "NG" can be provided within the movie folder 70.

[0252] 2.3.2. Folder structure processing

[0253] use Figures 23 and 24 Come to Figure 19 The details of the folder formation processing in step S62 will be described.

[0254] Figure 23 This is a flowchart illustrating the folder configuration process ( S62 ) in the present system 10 . Figure 24 The data structure of the management data D3 based on the folder configuration process ( S62 ) is exemplified. Figure 24 The management data D3 and Figure 21 The dynamic image folder 70 corresponds to .

[0255] First, the control unit 210 selects one clip from the multiple clips included in the clip segmentation data D1 (S71). Step S71 is performed to set the storage destination for the moving image material corresponding to one clip. For example, normal clips are selected in ascending order of the clip numbers in the clip segmentation data D1. Furthermore, if additional clips are present, the additional clips are selected in ascending order of the additional clip numbers following the normal clips.

[0256] Next, the control unit 210 lists information on each file of the moving image material in the segment in order of evaluation, for example, based on the moving image metadata list of the selected segment in the segment cutting data D1 ( S72 ).

[0257] For example, based on the rating information in the video metadata D2 for each take in the clip, the control unit 210 generates a video material list that lists the video file names of each take in order from the highly rated "OK" to the rated "Reserved." In this video material list, for takes with the same rating, the take number is sorted in ascending order, for example. Furthermore, the video material list may not include the video file names of takes with a low rating of "NG," for example.

[0258] In addition, for example, Figure 24 As shown, the control unit 210 sets the clip folder 71 ( Figure 21), and folders for evaluation such as the OK folder 73 and the reserved folder 72 under the clip folder 71 ( S73 ). For example, each folder 71 to 73 is described as a tag in XML language.

[0259] Next, the control unit 210 sequentially selects, for example, one take of a video file from the video material list obtained in step S72 ( S74 ). In step S74 , for the selected clips, takes other than those rated “NG” are selected in descending order, for example.

[0260] The control unit 210, for example, refers to the evaluation information of the selected take in the moving image metadata D2 and determines a folder from the OK folder 73 and the Hold folder 72 to store the moving image of the take (S75). For example, if the take is evaluated "OK," the control unit 210 determines the storage destination to be the OK folder 73. On the other hand, if the take is evaluated "Hold," the control unit 210 determines the storage destination to be the Hold folder 72.

[0261] Next, the control unit 210 refers to the list of moving image materials from step S72 and determines whether the storage destinations for the moving image files of each take in the selected clip have been determined (S76). If there are moving image files for takes whose storage destinations have not yet been determined (No in S76), the control unit 210 repeats the process from step S74 onward for the undetermined takes. This ensures that all takes in the selected clip that have been rated "OK" or "Retained" are stored in the corresponding clip folder 71.

[0262] Furthermore, for example, when the storage destination of the moving image files for each take in the selected clip has been determined (YES in S76), the control unit 210 determines whether the folder formation process (S71 to S76) has been performed for all clips in the clip cut data D1 (S77). If there are any unprocessed clips (NO in S77), the control unit 210 repeats the process from step S71 onward for the unprocessed clips. In this way, a moving image folder 70 is obtained, which stores the moving image files for all clips in the order of evaluation.

[0263] For example, when the processing for folder configuration is performed for all clips (YES in S77 ), the control unit 210 generates management data D3 for the moving image folder 70 based on a description in, for example, XML ( S78 ).

[0264] For example, Figure 24As shown, the control unit 210 sets a tag for each movie file determined in step S75 below the tag for the OK / hold folders 73 and 72 in each clip folder 71, and describes the movie file name and detailed information (S78). The detailed information of the movie file includes, for example, the duration of the movie material and the timer period.

[0265] By generating the management data D3 for the moving image folder 70, the control unit 210 ends the folder configuration process (S62) and proceeds to the next step. Figure 19 In step S63, for example, automatically record Figure 24 The information on the amount of the dynamic image time line 80 in the management data D3.

[0266] Through the above-described folder formation process (S62), the information support terminal 200 of the present system 10 can prepare a movie folder 70 in which various movie materials are systematically managed, reflecting user ratings and scenes, before the movie editing process. For example, when reading into the editing software, the movie editing PC 300 can store, according to the description of such management data D3, movie data having the same file name as the movie files captured by the digital camera 100 in the OK / hold folders 73 and 72 of each clip folder 71 in the movie folder 70.

[0267] 2.3.3. Timeline setting process

[0268] use Figures 25 and 26 Come to Figure 19 The details of the timeline setting process in step S63 will be described.

[0269] Figure 25 This is a flowchart illustrating the timeline setting process ( S63 ) in the present system 10 . Figure 26 The data structure of the management data D3 based on the timeline setting process ( S63 ) will be exemplified. Figure 26 The management data D3 is related to Figure 22 An example of sequence information corresponding to the timeline 80 .

[0270] First, the control unit 210 selects one clip folder 71 from among the plurality of clip folders 71 (S81), based on, for example, the management data D3 regarding the number of movie folders 70 determined in the folder configuration process (S62). Step S81 determines the placement of the movie assets in the movie timeline 80 for each clip, for example, in ascending order for normal clips.

[0271] Next, the control unit 210 determines whether the number of moving image files in the selected clip folder 71, that is, the number of files, is greater than the number of files specified by Figure 19 The number of tracks is set in step S61 (S82).

[0272] The number of files in step S82 is, for example, the total number of moving image files having shots evaluated as "OK" or "Retained" in a specific segment. Figure 19 In step S61 of FIG. 8 , if the evaluation of the display object in the video timeline 80 is set to “OK only” (see FIG. 8 ), Figure 20 ), in step S82, only the number of files evaluated as "OK" may be used in each segment.

[0273] If the number of files in the selected clip is greater than the number of tracks (Yes in S82), the control unit 210 extracts the target video file arranged in the video timeline 80 in the clip (S83). In step S83, for example, the control unit 210 extracts the target video file from the management data D3 ( Figure 24 )'s clip folder 71 extracts the number of moving image files of tracks in order from high evaluation to low evaluation.

[0274] Next, the control unit 210 manages the time range of the segment in the movie timeline 80, for example, by referring to the detailed information of each extracted movie file (S84). In the present system 10, the time range of the segment in the movie timeline 80 is managed as, for example, the longest playback period among the target movie files. If a timer period is provided, the movie playback period is managed by excluding the timer period from the overall duration of the movie.

[0275] On the other hand, if the number of files in the selected clip folder 71 is less than the number of tracks (No in S82), the control unit 210 selects all the moving image files in the clip folder 71 as the target for placement, and, for example, skips step S83 and proceeds to step S84. In this case, the management of the time range corresponding to the clip in the moving image timeline 80 (S84) is performed with reference to all the moving image files in the clip folder 71.

[0276] Next, the control unit 210 sequentially selects one track from the set number of tracks ( S85 ). Step S85 is a process for arranging a moving image material on each track in the clip.

[0277] For example, the control unit 210 Figure 22 Select the video track 83 (and the audio track 84) in ascending order starting from the track number "1" in the dynamic image timeline 80 (S85). Figure 26As shown, the control unit 210 performs a preview image 64 ( Figure 18 ) is the initial setting of the playback. For example, the video / audio track 83, 84 with track number "1" is set to enable the playback display and audio output, and the other video / audio tracks 83, 84 are set to be invalid.

[0278] Next, the control unit 210 performs the following operations based on the management data D3 ( Figure 24 ), determines the configuration of the dynamic image material in the selected track for the clip (S86).

[0279] In step S86, for example, the control unit 210 first determines the movie file name of the movie material corresponding to the priority order of the track selected for the clip, and then determines various time information related to the movie material. For example, the control unit 210 refers to the timer period of the movie material with the determined file name and determines the start time to be used in the movie timeline 80 for the movie material. For example, if the timer period is "5 seconds," the start time is determined to be 5 seconds after the beginning of the movie material.

[0280] Furthermore, the control unit 210 sets the start and end times for playback of the dynamic image material in the dynamic image timeline 80 (S86). In the present system 10, the playback start time is set to be common among the dynamic image files within a segment. For example, the playback start time for the first segment is set to the start time of the dynamic image timeline 80, while the playback start time for the second and subsequent segments is set to the end of the time range of the preceding segment, that is, the playback end time of the longest dynamic image file. Furthermore, the playback end time of a dynamic image material is set to the time after the playback start time for that file.

[0281] In step S86, the control unit 210 sets the placement of the video clip 85 of the video material in the video track 83 to reflect the various timing settings described above. Similarly to the placement of the video clip 85, the control unit 210 also sets the placement of the audio clip 86 of the video material in the audio track 84 (S86).

[0282] Next, the control unit 210 determines, for example, whether the configuration settings for the dynamic image materials for the clip have been completed (S87). For example, if the configuration settings for the dynamic image materials for the set number of tracks have been completed for the clip, the control unit 210 proceeds to "Yes" in step S87. Alternatively, if the number of dynamic image materials in the clip folder 71 is less than the number of tracks, and the control unit 210 has not yet completed the configuration settings for all dynamic image materials in the clip folder 71, the control unit 210 proceeds to "Yes" in step S87; otherwise, the control unit 210 proceeds to "No."

[0283] If the placement of the moving image materials for the clip is not complete (No in S87), the control unit 210 performs the processing from step S85 onward for the moving image materials not yet placed in the clip. In this way, the video / audio clips 85 and 86 are placed within the number of tracks for the clip (S86).

[0284] Furthermore, if, for example, the placement of the video material corresponding to the clip is complete (YES in S87), the control unit 210 determines whether the placement of all clips to be placed on the video timeline 80 is complete (S88). The determination in step S88 is performed, for example, within the scope of normal clips excluding additional clips.

[0285] If there are unset normal segments in the movie timeline 80 (No in S88), the control unit 210 repeats the process from step S81 onward for the unset normal segments. This results in a movie timeline 80 in which the movie files are arranged in the order of evaluation for all normal segments.

[0286] For example, when the arrangement settings for all normal clips are completed (YES in S88 ), the control unit 210 generates management data D3 for the video timeline 80 based on a description in, for example, XML ( S89 ).

[0287] In step S89, the control unit 210 performs the following operations, for example, Figure 24 In the tag of the dynamic image timeline 80 in the management data D3, as shown in FIG. Figure 26 As shown, tags for each of the video and audio are provided, and tags for each track number are provided in each lower level. Furthermore, the control unit 210 describes information related to the configuration settings, such as the moving image file name and time information of the video / audio clips 85 and 86 determined in step S86, in the tags for each track number for each clip.

[0288] By generating the management data D3 corresponding to the moving image timeline 80, the control unit 210 ends the timeline setting process (S63) and proceeds to the next step. Figure 19 Step S64.

[0289] Through the above-described timeline setting process (S63), the information support terminal 200 of the present system 10 can prepare a moving image timeline 80 with the image / sound clips 85 and 86 arranged along the scenes, reflecting the user's ratings, before the moving image editing process begins. The user can begin moving image editing without having to arrange the moving image material after the clip shooting function along the scenes, and at this point, obtain a moving image timeline 80 reflecting their ratings.

[0290] For example, during the movie editing process, the user can check the video / sound clips 85 and 86 for each clip on the movie timeline 80. In this case, the playback display and sound output of the video / sound clips 85 and 86 for track number "1" are initially enabled, allowing the user to check the movie clips starting with the highest-rated ones. Furthermore, the enabling / disabling of the playback display and sound output can be switched appropriately by the movie editing PC 300 based on user operations on the movie editing screen.

[0291] In the present system 10, management is performed in the prepared moving image timeline 80 so that the time ranges of a plurality of image / sound clips 85, 86 are aligned between different image / sound tracks 83, 84 for each segment (see S84, Figure 22 ). This allows the user to more easily check each clip 85, 86 for each segment during the motion picture editing process, compared to a case where each clip 85, 86 is arranged on each track 83, 84 with a time interval.

[0292] The management of the time range in step S84 is performed based on, for example, the longest playback period of the moving image material arranged in each segment (see Figure 22 ). This can avoid, for example, situations where overlapping cuts occur between adjacent segments, and facilitate confirmation of each segment.

[0293] In this embodiment, the management of the time range of each segment (S84) is not specifically limited to the longest playback period of each segment, but may also be performed based on the various time lengths of the moving image materials for each segment. For example, the control unit 210 may manage the time range of each segment in the moving image timeline 80 so that it is aligned with the playback period of the moving image material with the highest priority (S84).

[0294] 3. Summary

[0295] As described above, in this embodiment, an information assistance terminal 200, which is an example of an electronic device, manages dynamic images in a scene including a plurality of segments. The information assistance terminal 200 includes: a display unit 240 for displaying information; an operation unit 230, which is an example of an input unit for inputting user operations; and a control unit 210 for controlling the display unit 240 according to the operations inputted in the operation unit 230. The control unit 210 causes the display unit 240 to display an evaluation screen (see FIG. 1 ) for prompting evaluation of the dynamic images associated with each of the plurality of segments. Figure 9 as well as Figure 17 (C)), the control unit 210 obtains evaluation information indicating the user's evaluation of the dynamic image from the operation unit 230 (S5, S6). The control unit 210 generates management data D3 based on the obtained evaluation information. The management data D3 is displayed on a dynamic image editing screen ( Figure 18 ), the priority order of the moving image information representing each moving image is managed and arranged (S62 to S64).

[0296] According to the above-described information support terminal 200, by managing the priority order of placement of moving image information on the moving image editing screen based on evaluation information, the user can easily perform the process of scene-based moving image editing based on the results of the clip shooting function. In addition, the present system 10 can improve the processing efficiency of this process by, for example, the moving image editing PC 300 based on the management data D3 such as the moving image timeline 80.

[0297] In the information assistance terminal 200 of this embodiment, the control unit 210 generates management data D3 based on the acquired evaluation information, so that, on the motion picture editing screen, the higher the evaluation information indicates for each segment, the more preferentially the motion picture information is placed on the motion picture editing screen ( S62 , S63 ). This allows the preferential placement of highly rated motion picture information, facilitating motion picture editing.

[0298] In the information support terminal 200 of this embodiment, the management data D3 includes a moving image folder 70 composed of a plurality of folders indicating the priority order according to the evaluation information for each segment, and defines the moving images stored in each of the plurality of folders (see Figure 24 ). The information support terminal 200 of this embodiment can realize the management of the system's dynamic image materials on the dynamic image editing screen by generating the management data D3 of the dynamic image folder 70, thereby facilitating dynamic image editing.

[0299] In the information assistance terminal 200 of this embodiment, the moving image folder 70 in the management data D3 may include folders 72 and 73 corresponding to second ratings (e.g., "Reserved" / "OK"), which are higher than the first rating, rather than a folder corresponding to the first rating (e.g., "NG"). This allows the user's rating to be reflected, reducing the amount of data loaded into the moving image editing PC 300.

[0300] In the information assistance terminal 200 of this embodiment, the management data D3 includes data of a moving image timeline 80 as an example of sequence information in which moving image information is arranged in the order of a plurality of segments in a scene ( Figure 26 The control unit 210 generates sequence information so that the dynamic image information is arranged according to the priority order for each segment (S89). This allows the dynamic image materials to be arranged according to the user's evaluation in the dynamic image timeline 80 on the dynamic image editing screen, which arranges the dynamic image materials for each segment along the scene, facilitating dynamic image editing.

[0301] In the information assistance terminal 200 of this embodiment, the control unit 210 generates sequence information based on the time length of each segment of the moving image, thereby aligning the arrangement of the moving image information for each segment ( S84 , S89 ). This makes it easier for the user to confirm the moving image material for each segment on the moving image timeline 80 , facilitating moving image editing.

[0302] In the information assistance terminal 200 of this embodiment, the control unit 210 receives a user operation (S61) to set a predetermined number of take times for an example through the operation unit 230, and generates sequence information so that no more than the predetermined number of moving image information is arranged for each clip (S82-S87, S89). This allows the number of moving image materials per clip in the moving image timeline 80 to be set within the user's desired range, facilitating moving image editing.

[0303] In the information assistance terminal 200 of this embodiment, the control unit 210 receives a user operation (S31) to set a timer period, an example of a predetermined period, on the operation unit 230, and generates sequence information so that the moving image information is arranged in the moving image excluding the predetermined period (S84, S89). This allows the user to exclude the period of time not intended by the user from the moving image work from the moving image timeline 80, facilitating moving image editing.

[0304] In the information support terminal 200 of the present embodiment, the management data D3 includes the following data structure, for example, by describing tags in XML language, so that the video editing PC 300, which is an example of an external device that displays a video editing screen, displays the video editing screen (see FIG. 1 ) in a state where the video information is arranged according to the priority order. Figure 24 as well as Figure 26 The information assistance terminal 200 of this embodiment manages the arrangement of the moving image information in the moving image editing screen of the external device by using such management data D3, and can facilitate the moving image editing in the external device.

[0305] In this embodiment, the information assistance terminal 200 further includes a storage unit 220 that stores segment segmentation data D1, an example of management information for managing the moving images associated with each of the multiple segments in a scene. The control unit 210 generates management data D3 based on the segment segmentation data D1 and the evaluation information (S62-S64). This facilitates segment-by-segment editing of the moving images based on the segment segmentation data D1.

[0306] In this embodiment, the information support terminal 200 further includes a communication unit 250 for communicating data with a digital camera 100, an example of an imaging device that captures moving images. The control unit 210 manages the moving images captured by the digital camera 100 through data communication via the communication unit 250 (S5). This facilitates the management of moving image capture for each clip in the information support terminal 200, which is independent of the digital camera 100.

[0307] In this embodiment, a dynamic image management method for managing dynamic images in a scene including multiple segments is provided. In this method, the control unit 210 of the information auxiliary terminal 200 causes the display unit 240 to display an evaluation screen (see FIG. 1 ) for prompting evaluation of dynamic images associated with each of the multiple segments. Figure 9 as well as Figure 17 (C)), the control unit 210 obtains evaluation information indicating the user's evaluation of the dynamic image from the operation unit 230 (S5, S6). The control unit 210 generates management data D3 based on the obtained evaluation information. The management data D3 is displayed on a dynamic image editing screen ( Figure 18 ), the priority order of the moving image information representing each moving image is managed and arranged (S62 to S64).

[0308] In this embodiment, a computer program product is provided, including a program for causing the control unit 210 to execute the aforementioned dynamic image management method. This dynamic image management method facilitates editing of dynamic images in scenes comprising multiple segments. In this system, the management of various information per segment is not necessarily limited to individual segments; it can also be performed in segments comprising a group of multiple segments.

[0309] (Other Implementation Methods)

[0310] As described above, the first embodiment is described as an example of the technology disclosed in this application. However, the technology disclosed in this application is not limited thereto and can also be applied to embodiments in which appropriate changes, substitutions, additions, omissions, etc. are made. In addition, the various structural elements described in the above embodiments can also be combined to form new embodiments.

[0311] In the above-mentioned embodiment 1, the information auxiliary terminal 200 is described as an example of an electronic device independent of the camera device, but the present disclosure is not limited to this. The electronic device of this embodiment may also be integrated with the camera device that performs dynamic image shooting. For such a modified example, use Figure 27 Provide explanation.

[0312] Figure 27 A modified example of the digital camera 100 is described below. In this embodiment, the digital camera 100 has various functions such as the clip shooting function of the above-mentioned information auxiliary terminal 200. For example, Figure 27 As shown, the control unit 135 of the digital video camera 100 causes the display monitor 130 to display a clip selection screen including a plurality of clips via the clip list 30 , and accepts user selection of a clip via the operation unit 150 such as a touch panel or operation buttons.

[0313] exist Figure 27 In the example of , the display monitor 130 displays the live view image superimposed on the clip list 30. The control unit 135 of the digital camera 100 generates moving image data by, for example, the image pickup operation of the image sensor 115. The control unit 135 of the digital camera 100 and the processing of the recording mode of the embodiment 1 ( Figure 13 ) Similarly, when a moving image corresponding to the selected segment is captured, the rating screen is displayed on the display monitor 130 ( Figure 9 ), and obtain the user's evaluation information. With this digital camera 100, it is also possible to provide the user with information assistance based on the clip shooting function and the export function, similar to the first embodiment.

[0314] As described above, in this embodiment, the digital camera 100, an example of an electronic device, further includes an image sensor 115, an example of an imaging unit, which captures a subject image and generates image data. The control unit 135 manages the moving images formed by the image data generated by the image sensor 115. This facilitates the management of moving image capture for each clip in the digital camera 100.

[0315] In each of the above-mentioned embodiments, a segment selection screen including a segment list 30 is illustrated, but the selection screen of the present disclosure is not limited to this. The selection screen of this embodiment may not include the segment list 30, or may include multiple segments in a display method different from the segment icon 3. In addition, the selection screen of this embodiment may be displayed as a dialog box or may be displayed overlapping with various display screens. In this embodiment, the segment list 30 may also be an example of a selection screen. In this embodiment, the selection screen of the information auxiliary terminal 200 may also identify and display whether the dynamic image shooting is completed for each segment in various display methods not limited to the above-mentioned examples.

[0316] In each of the above-mentioned embodiments, examples of three types of evaluation information, such as "OK", "Reserved" and "NG", are described, but the evaluation information is not particularly limited to this. In this embodiment, the evaluation information may be three types of evaluation different from those described above, or may be two or more types without being particularly limited to three types. In addition, in this embodiment, the evaluation information may also be a score of a continuous value. The electronic device of this embodiment may also accept user input of such various evaluation information and manage the dynamic image shooting of each segment. For example, the identification display can be performed by appropriately setting a benchmark for whether the dynamic image shooting of the segment has been completed. In addition, the information assistance terminal 200 of this embodiment may also set the priority in the management data D3 so as to reflect such various evaluations.

[0317] In the above-described embodiments, examples were described in which management data D3 included a dynamic image folder 70 and a dynamic image timeline 80. In this embodiment, management data may include either dynamic image folder 70 or dynamic image timeline 80, while omitting the other. Furthermore, in the above-described embodiments, examples were described in which management data D3 was described using XML. In this embodiment, management data D3 is not limited to XML and may be described using various markup languages ​​or data description languages ​​compatible with the editing software used in the dynamic image editing process. Management data D3 may also include various metadata for managing dynamic images in various clips.

[0318] In addition, in each of the above embodiments, the information assistance terminal 200 is described as an example of an electronic device, and the dynamic image editing PC 300 is described as an example of an external device. In this embodiment, the electronic device may also be the dynamic image editing PC 300, and the dynamic image editing PC 300 may also have the various functions of the above-mentioned information assistance terminal 200. In this embodiment, the editing screen is not limited to the external device and can also be displayed on the electronic device. In addition, if the dynamic image editing PC 300 uses an external display device, the external device in this embodiment may also include such a display device.

[0319] In the above embodiments, the digital camera 100 is exemplified as including the optical system 110 and the lens driving unit 112. The imaging device of this embodiment may not include the optical system 110 and the lens driving unit 112, and may be, for example, an interchangeable lens camera.

[0320] Furthermore, in the above embodiments, digital cameras are described as examples of imaging devices, but the present invention is not limited to such devices. The imaging device disclosed herein can be any electronic device with an image capture function (e.g., a video camera, a smartphone, a tablet terminal, etc.). Furthermore, the electronic device disclosed herein does not need to have a specific image capture function and can be any other electronic device.

[0321] (Example)

[0322] Various aspects of the present disclosure are exemplified below.

[0323] The first method involved in the present disclosure is an electronic device for managing dynamic images in a scene including multiple segments. The electronic device includes: a display unit for displaying information; an input unit for inputting user operations; and a control unit for controlling the display unit based on the operations input into the input unit. The control unit causes the display unit to display an evaluation screen that prompts the evaluation of a dynamic image associated with each of the multiple segments, and obtains evaluation information representing the user's evaluation of the dynamic image from the input unit. Based on the obtained evaluation information, management data is generated. The management data manages and configures the priority order of dynamic image information representing each dynamic image in an editing screen for editing multiple dynamic images associated with the multiple segments.

[0324] In the second embodiment, in the electronic device described in the first embodiment, the control unit generates management data based on the acquired evaluation information so that in the editing screen, the higher the evaluation information indicates for each segment of the dynamic image, the higher the dynamic image information is configured on the editing screen.

[0325] In a third aspect, in the electronic device according to the first or second aspect, the management data includes a plurality of folders indicating priority orders according to the evaluation information for each segment, and defines moving images stored in each of the plurality of folders.

[0326] In a fourth embodiment, in the electronic device described in any one of the first to third embodiments, the management data includes sequence information for arranging the dynamic image information in the order of multiple segments in the scene, and the control unit generates the sequence information so that the dynamic image information is arranged according to priority for each segment.

[0327] In a fifth aspect, in the electronic device according to the fourth aspect, the control unit generates the sequence information based on the time length of the moving image of each segment so that the arrangement of the moving image information is aligned for each segment.

[0328] In a sixth aspect, in the electronic device according to the fourth or fifth aspect, the control unit receives a user operation to set a predetermined number on the input unit, and generates sequence information so that moving image information equal to or less than the predetermined number is arranged per segment.

[0329] In a seventh aspect, in the electronic device according to any one of the fourth to sixth aspects, the control unit receives a user operation for setting a predetermined period on the input unit, and generates sequence information so that the moving image information is arranged excluding the predetermined period in the moving image.

[0330] In an eighth aspect, in the electronic device according to any one of the first to seventh aspects, the management data includes a data structure for causing the external device displaying the editing screen to display the editing screen in a state where the moving image information is arranged according to the priority order.

[0331] In a ninth aspect, the electronic device according to any one of the first to eighth aspects further comprises a storage unit storing management information for managing the moving images associated with the plurality of segments in the scene, wherein the control unit generates management data based on the management information and the evaluation information.

[0332] In a tenth aspect, the electronic device according to any one of aspects 1 to 9 further comprises a communication unit for communicating data with an imaging device that captures moving images. The control unit manages the moving images captured by the imaging device through data communication via the communication unit.

[0333] In an eleventh aspect, the electronic device according to any one of the first to ninth aspects further comprises an imaging unit that captures an image of a subject to generate image data, and a control unit that manages a moving image composed of the image data generated by the imaging unit.

[0334] A twelfth embodiment is a method for managing dynamic images in a scene including multiple segments. In this method, a control unit of an electronic device causes a display unit to display an evaluation screen prompting evaluation of a dynamic image associated with each of the multiple segments, obtains evaluation information representing a user's evaluation of the dynamic image from an input unit, and generates management data based on the obtained evaluation information. The management data manages and arranges the priority order of dynamic image information representing each dynamic image in an editing screen for editing the multiple dynamic images associated with the multiple segments.

[0335] In a thirteenth aspect, there is provided a computer program product including a program for causing a control unit to execute the motion picture management method according to the twelfth aspect.

[0336] As described above, the embodiments are described as examples of the technology disclosed herein. Drawings and detailed descriptions are provided for this purpose. Therefore, the structural elements described in the drawings and detailed descriptions include not only those necessary to solve the problems but also those that are not necessary to solve the problems in order to illustrate the above technology.

[0337] Industrial applicability

[0338] The present disclosure can be applied to various applications in which moving image shooting including a plurality of segments is performed.

Claims

1. An electronic device for managing a dynamic image in a scene comprising a plurality of segments, comprising: A display unit for displaying information; An input unit for inputting user operations; and a control unit that controls the display unit according to an operation inputted in the input unit, The control unit performs the following processing: causing the display unit to display an evaluation screen prompting evaluation of a moving image associated with each of the plurality of segments, and acquiring evaluation information indicating a user's evaluation of the moving image from the input unit, Management data is generated based on the acquired evaluation information. The management data manages and arranges the priority order of the moving image information indicating each moving image in an editing screen for editing the plurality of moving images associated with the plurality of segments.

2. The electronic device according to claim 1, wherein The control unit generates the management data based on the acquired evaluation information so that, in the editing screen, for each of the segments, a moving image having a higher evaluation indicated by the evaluation information is arranged on the editing screen with higher priority.

3. The electronic device according to claim 1, wherein The management data includes a plurality of folders indicating a priority order according to the evaluation information for each of the segments, and defines moving images stored in each of the plurality of folders.

4. The electronic device according to claim 1, wherein The management data includes sequence information for arranging the dynamic image information in the order of the plurality of segments in the scene, The control unit generates the sequence information so that the moving image information is arranged according to the priority order for each of the segments.

5. The electronic device according to claim 4, wherein The control unit generates the sequence information based on the time length of the moving image of each segment so that the arrangement of the moving image information is aligned for each segment. The electronic device according to claim 4 , wherein: The control unit performs the following processing: accepting a given number of user operations set in the input unit, The sequence information is generated so that the given number or less of the moving image information is arranged for each of the segments.

7. The electronic device according to claim 4, wherein: The control unit performs the following processing: accepting a user operation of setting a predetermined period in the input unit, The sequence information is generated so that the moving image information is arranged in the moving image except for the predetermined period.

8. The electronic device according to claim 1, wherein The management data includes a data structure for causing an external device that displays the editing screen to display the editing screen in a state where the moving image information is arranged according to the priority order.

9. The electronic device according to claim 1, wherein The electronic device further includes a storage unit that stores management information for managing the moving images associated with the plurality of segments in the scene. The control unit generates the management data based on the management information and the evaluation information.

10. The electronic device according to claim 1, wherein The electronic device further includes: a communication unit for performing data communication with a camera device for performing motion picture shooting; The control unit manages the moving images captured by the imaging device through data communication via the communication unit.

11. The electronic device according to claim 1, wherein The electronic device further comprises: an imaging unit for capturing an image of a subject to generate image data; The control unit manages a moving image composed of image data generated by the imaging unit.

12. A method for managing dynamic images in a scene comprising multiple segments. The control unit of the electronic device performs the following processing: causing a display unit to display an evaluation screen prompting evaluation of a moving image associated with each of the plurality of segments, and acquiring evaluation information indicating a user's evaluation of the moving image from an input unit, Management data is generated based on the acquired evaluation information. The management data manages and arranges the priority order of the moving image information indicating each moving image in an editing screen for editing the plurality of moving images associated with the plurality of segments. 13 . A computer program product comprising a program for causing the control unit to execute the dynamic image management method according to claim 12 .

Images