Electronic apparatus and photography management method

Abstract

To provide an electronic apparatus and a photography management method capable of facilitating management of a failure in photography regarding audio collection in photography of a moving image.SOLUTION: An electronic apparatus (200) which manages photography of a moving image comprises: a display unit (240) which displays information; an input unit (230) which receives a user's operation; and a control unit (210) which controls the display unit in response to the operation received by the input unit. The control unit receives an instruction to photograph a moving image with the input unit and based on audio data in the photographed moving image, acquires a result of a determination process which determines whether the moving image is taken in a failure in photography. In a case where the acquired result of the determination process is the failure in photography, the control unit causes the display unit to display warning information indicating the failure in photography of the moving image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an electronic device that manages video shooting with sound collection and a shooting management method. [Background technology]

[0002] Patent Document 1 discloses a speech processing device for input signals for speech recognition of a speaker. The speech processing device determines whether there is noise and whether the speaker's voice is present, and adjusts the gain of the sound input unit based on the determination result so that the noise level is below a first level. If the voice level falls below a second level as a result, the speech processing device issues a warning to the speaker. This allows the speaker's voice to be at or above the second level when inputting the speaker's voice, and to be below the first level when noise is input, thereby improving the signal-to-noise ratio.

[0003] Patent Document 2 discloses a sound source separation device that separates a target signal from a non-target signal from an acoustic signal. The sound source separation device separates the acoustic signal into the target signal and the non-target signal using various neural networks, with the aim of reducing the amount of calculation of a neural network model for obtaining the target signal from the acoustic signal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-117659 [Patent Document 2] Japanese Patent Publication No. 2022-141581 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides an electronic device and a shooting management method that can easily manage shooting failures regarding sound collection in video shooting. [Means for solving the problem]

[0006] In the present disclosure, an electronic device manages video recording. The electronic device includes a display unit that displays information, an input unit that inputs user operations, and a control unit that controls the display unit in response to the operations input via the input unit. The control unit receives an instruction to record a video via the input unit and acquires the result of a determination process that determines whether or not recording of the video has failed based on audio data in the video being recorded. If the acquired result of the determination process indicates a recording failure, the control unit displays warning information indicating that recording of the video has failed on the display unit.

[0007] In the present disclosure, a shooting management method is a method for managing video shooting, in which a control unit of an electronic device receives an instruction to shoot a video via an input unit, obtains a result of a determination process for determining whether or not the video has been shot successfully based on audio data of the video being shot, and, if the obtained result of the determination process indicates a shooting failure, displays warning information indicating the video has been shot successfully on a display unit. [Effects of the Invention]

[0008] The electronic device and shooting management method according to the present disclosure can facilitate management of shooting failures in sound collection during video shooting. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of the configuration of an imaging system according to a first embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram showing the configuration of a digital camera in an imaging system. [Figure 3] FIG. 1 is a diagram showing the configuration of an information support terminal in an imaging system. [Figure 4] FIG. 10 is a diagram showing an example of a function selection screen displayed on an information support terminal. [Figure 5] FIG. 10 shows an example of a scenario input screen displayed on an information support terminal. [Figure 6]FIG. 10 is a diagram illustrating the data structure of cut data in an information support terminal; [Figure 7] FIG. 10 is a diagram showing an example of a cut selection screen displayed on an information support terminal. [Figure 8] 10 is a flowchart illustrating an example of the operation of the cut photography function in the imaging system; [Figure 9] FIG. 10 is a diagram showing an example of a rating screen displayed on an information support terminal. [Figure 10] A diagram showing an example of a cut list when there is a cut where audio is not allowed [Figure 11] 10 is a flowchart illustrating a process in a recording mode in an imaging system; [Figure 12] FIG. 10 is a diagram showing a display example in a recording mode of the information support terminal. [Figure 13] FIG. 10 is a diagram illustrating an example of the data structure of video metadata in an information support terminal. [Figure 14] 10 is a flowchart illustrating an example of a sound determination process in the imaging system of the first embodiment. [Figure 15] Figure 10 shows an example of a warning message displayed on an information support terminal when audio is not available. [Figure 16] A flowchart illustrating a video playback process with no audio in an imaging system. [Figure 17] Figure 10 shows an example of a video playback screen with no audio output on an information support device. [Figure 18] A diagram showing an example of noise decomposition display on an information support terminal. [Figure 19] A diagram for explaining the operation of setting NG criteria on an information support terminal. [Figure 20] 10 is a flowchart illustrating an example of a sound determination process in the imaging system of the second embodiment. [Figure 21] FIG. 1 is a diagram illustrating a modified example of a digital camera. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0011] (Embodiment 1) In the first embodiment of the present disclosure, a system using an electronic device separate from an imaging device that performs video shooting will be described.

[0012] 1. Configuration An imaging system according to the first embodiment of the present disclosure will be described with reference to FIG.

[0013] 1, the system 10 includes a digital camera 100, an information support terminal 200, and a video editing personal computer (PC) 300. In the system 10, the digital camera 100 and the information support terminal 200 are connected to each other so that data can be communicated therebetween, for example, via wired or wireless communication.

[0014] This system 10 can be used, for example, when a user creates a desired video work by shooting and editing multiple videos using a digital camera 100. This system 10 provides useful information support for a series of workflows, such as when a user creates a scenario that outlines the concept of the video work, repeatedly shoots videos according to multiple cuts that divide the scenario, and edits the multiple shot videos.

[0015] In the present system 10, the information support terminal 200 can, for example, manage the scenario of a video work and control the digital camera 100 to manage video shooting for each cut. For example, a live view image of the digital camera 100 can be confirmed on the information support terminal 200. Video data captured by the digital camera 100 is edited on a video editing PC 300. The present system 10 uses data managed on the information support terminal 200 to facilitate video editing on the video editing PC 300 or the like.

[0016] In the present system 10, the video editing PC 300 may or may not be communicatively connected to one or both of the digital camera 100 and the information support terminal 200. For example, data from the digital camera 100 and / or the information support terminal 200 may be input to the video editing PC 300 via a portable recording medium such as a memory card. The present system 10 does not necessarily have to include the video editing PC 300.

[0017] 1.1.Digital camera configuration The configuration of the digital camera 100 according to this embodiment will be described with reference to FIG.

[0018] 2 is a diagram illustrating an example of the configuration of a digital camera 100 in the system 10. The digital camera 100 is an example of an imaging device in this embodiment. The digital camera 100 of this embodiment includes an image sensor 115, an image processing engine 120, a display monitor 130, and a control unit 135. The digital camera 100 also includes a buffer memory 125, a card slot 140, a flash memory 145, an operation unit 150, a communication module 155, a microphone 160, and a speaker 170. The digital camera 100 also includes, for example, an optical system 110 and a lens driving unit 112.

[0019] The optical system 110 includes a focus lens, a zoom lens, an optical image stabilization lens (OIS), an aperture, a shutter, etc. The focus lens is a lens for changing the focus state of the subject image formed on the image sensor 115. The zoom lens is a lens for changing the magnification of the subject image formed by the optical system. Each of the focus lens, etc. is composed of one or more lenses.

[0020] Lens driving unit 112 drives the focus lens and the like in optical system 110. Lens driving unit 112 includes a motor, and moves the focus lens along the optical axis of optical system 110 under the control of control unit 135. The configuration for driving the focus lens in lens driving unit 112 can be realized by a DC motor, a stepping motor, a servo motor, an ultrasonic motor, or the like.

[0021] The image sensor 115 captures an image of a subject formed via the optical system 110 and generates imaging data. The imaging data constitutes image data representing an image captured by the image sensor 115. The image sensor 115 generates new frame image data at a predetermined frame rate (e.g., 30 frames / second). The timing of generating imaging data and the electronic shutter operation in the image sensor 115 are controlled by the control unit 135. The image sensor 115 can be any of a variety of image sensors, such as a CMOS image sensor, a CCD image sensor, or an NMOS image sensor.

[0022] The image sensor 115 performs operations such as capturing still images and through images. Through images are mainly moving images, and are displayed on the display monitor 130 so that the user can determine the composition for capturing the still image. The through images and still images are each an example of a captured image in this embodiment. The image sensor 115 is an example of an imaging unit in this embodiment.

[0023] The image processing engine 120 performs various processes on the imaging data output from the image sensor 115 to generate image data, and performs various processes on the image data to generate an image to be displayed on the display monitor 130. The various processes include, but are not limited to, white balance correction, gamma correction, YC conversion processing, electronic zoom processing, compression processing, and expansion processing. The image processing engine 120 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer, processor, or the like using a program.

[0024] The display monitor 130 is an example of a display unit that displays various information. For example, the display monitor 130 displays an image (through image) represented by image data captured by the image sensor 115 and processed by the image processing engine 120. The display monitor 130 also displays a menu screen or the like that allows the user to make various settings for the digital camera 100. The display monitor 130 can be configured, for example, by a liquid crystal display device or an organic EL device.

[0025] Operation unit 150 is a general term for hard keys such as operation buttons and operation levers provided on the exterior of digital camera 100, and accepts operations by the user. Operation unit 150 includes, for example, a release button, a mode dial, and a touch panel. When operation unit 150 accepts an operation by the user, it transmits an operation signal corresponding to the user operation to control unit 135.

[0026] The control unit 135 controls the overall operation of the digital camera 100. The control unit 135 includes a CPU and the like, and the CPU executes a program (software) to realize predetermined functions. Instead of a CPU, the control unit 135 may include a processor configured with a dedicated electronic circuit designed to realize the predetermined functions. In other words, the control unit 135 can be realized by various processors such as a CPU, MPU, GPU, DSP, FPGA, ASIC, etc. The control unit 135 may be configured with one or more processors. Furthermore, the control unit 135 may be configured on a single semiconductor chip together with the image processing engine 120, etc.

[0027] The buffer memory 125 is a recording medium that functions as a work memory for the image processing engine 120 and the control unit 135. The buffer memory 125 is realized by a DRAM (Dynamic Random Access Memory) or the like. The flash memory 145 is a non-volatile recording medium. Although not shown, the control unit 135 may have various types of internal memory, for example, a built-in ROM. The ROM stores various programs executed by the control unit 135. The control unit 135 may also have a built-in RAM that functions as a work area for the CPU.

[0028] The card slot 140 is a means for inserting a removable memory card 142. The card slot 140 can electrically and mechanically connect the memory card 142. The memory card 142 is an external memory equipped with a recording element such as a flash memory inside. The memory card 142 can store data such as image data generated by the image processing engine 120.

[0029] The communication module 155 is a module (circuit) that connects to an external device in accordance with a predetermined communication standard for wired or wireless communication. The predetermined communication standard includes, for example, USB, HDMI (registered trademark), IEEE802.11, Wi-Fi, Bluetooth, etc. The digital camera 100 can communicate with other devices via the communication module 155.

[0030] The microphone 160 includes, for example, one or more microphone elements built into the digital camera 100. The microphone 160 outputs an audio signal indicating the picked-up sound to the control unit 135. The digital camera 100 may also use an external microphone. Instead of or in addition to the built-in microphone 160, the digital camera 100 may also include a connection unit such as a terminal for connecting to an external microphone.

[0031] The speaker 170 includes, for example, one or more speaker elements built into the digital camera 100, and outputs sound to the outside of the digital camera 100 under the control of the control unit 135. An external speaker, earphones, or the like may also be used with the digital camera 100. Instead of or in addition to the built-in speaker 170, the digital camera 100 may be provided with a connection unit for connecting to an external speaker, or the like.

[0032] 1.2.Configuration of information support terminal The configuration of the information support terminal 200 in this embodiment will be described with reference to FIG.

[0033] 3 is a diagram illustrating an example of the configuration of information support terminal 200. Information support terminal 200 is an example of an electronic device configured, for example, by a smartphone, a tablet terminal, or a PC. Information support terminal 200 illustrated in FIG. 3 includes a control unit 210, a storage unit 220, an operation unit 230, a display unit 240, a communication unit 250, a microphone 260, and a speaker 270.

[0034] The control unit 210 includes, for example, a CPU or MPU that works in cooperation with software to realize predetermined functions. The control unit 210 controls, for example, the overall operation of the information support terminal 200. The control unit 210 reads data and programs stored in the storage unit 220 and performs various arithmetic processing to realize various functions.

[0035] The control unit 210 executes, for example, a program including a set of instructions for implementing each of the above functions. The program may be provided via a communication network such as the Internet, or may be stored on a portable recording medium. The control unit 210 may also be a hardware circuit such as a dedicated electronic circuit or a reconfigurable electronic circuit designed to implement each of the above functions. The control unit 210 may also be configured with various semiconductor integrated circuits such as a CPU, MPU, GPU, GPGPU, TPU, microcomputer, DSP, FPGA, and ASIC.

[0036] The storage unit 220 is a storage medium that stores programs and data necessary to realize the functions of the information support terminal 200. The storage unit 220 includes a storage unit 221 and a temporary storage unit 222, as shown in FIG.

[0037] The storage unit 221 stores parameters, data, control programs, etc. for realizing predetermined functions. The storage unit 221 is configured, for example, with an HDD or SSD. For example, the storage unit 221 stores the above-mentioned programs and various image data, etc.

[0038] The temporary storage unit 222 is configured with a RAM such as a DRAM or an SRAM, and temporarily stores (i.e., holds) data. For example, the temporary storage unit 222 holds image data that is in the middle of being edited. The temporary storage unit 222 may also function as a work area for the control unit 210, and may be configured as a storage area in the internal memory of the control unit 210.

[0039] The operation unit 230 is a general term for operation members operated by the user. The operation unit 230 is, for example, a touch panel superimposed on the display unit 240 for inputting various touch operations, and is an example of an input unit of the information support terminal 200. The input unit may be a connection software unit that communicates with various external input devices and receives operation signals. The operation unit 230 may be physical buttons or switches provided on the information support terminal 200, or may be a keyboard, mouse, touchpad, or the like. The operation unit 230 may be various GUIs such as virtual buttons, icons, cursors, software keyboards, and objects displayed on the display unit 240.

[0040] The display unit 240 is configured with, for example, a liquid crystal display or an organic EL display. The display unit 240 may display various types of information, such as various GUIs for operating the operation unit 230 and information input from the operation unit 230.

[0041] The communication unit 250 is a module (circuit) that connects to an external device in accordance with a predetermined communication standard for wired or wireless communication. The predetermined communication standard includes, for example, USB, HDMI, IEEE802.11, Wi-Fi, Bluetooth, etc. The communication unit 250 may connect the information support terminal 200 to a communication network such as the Internet. The communication unit 250 is an example of an acquisition unit that receives various information from an external device or a communication network.

[0042] Microphone 260 includes, for example, one or more microphone elements built into information support terminal 200. Microphone 260 outputs an audio signal indicating the picked-up sound to control unit 210. Instead of or in addition to built-in microphone 260, information support terminal 200 may include a connection unit such as a terminal for connecting to an external microphone.

[0043] Speaker 270 includes, for example, one or more speaker elements built into digital camera 100, and outputs sound to the outside of information support terminal 200 under the control of control unit 210. Instead of or in addition to built-in speaker 270, information support terminal 200 may be provided with a connection unit for connecting to an external speaker, earphones, or the like.

[0044] The above-described configuration of information support terminal 200 is an example, and the configuration of information support terminal 200 is not limited to this. For example, various display devices such as a projector and a head-mounted display may be used for display unit 240 of information support terminal 200. Furthermore, when an external display device is used, display unit 240 of information support terminal 200 may be an output interface circuit for a video signal conforming to, for example, the HDMI standard.

[0045] 2. Operation The operation of the system 10 configured as above will be described below.

[0046] In the present system 10, the information support terminal 200 has various functions for sequentially providing information support to the user in the workflow of video production. An example of a screen display for selecting various functions of the information support terminal 200 is shown in FIG.

[0047] The display unit 240 of the information support terminal 200 displays a scenario creation button 11, a shooting button 12, and an export button 13 on the function selection screen illustrated in Fig. 4. Hereinafter, the longitudinal direction of the screen of the display unit 240 is referred to as the X direction, and the width direction is referred to as the Y direction.

[0048] The scenario creation button 11 is a virtual button that accepts user operation to execute a function (i.e., a scenario creation function) that provides information support for the process in which the user creates a scenario before shooting a video in the system 10. The information support terminal 200 of the system 10 manages various information for each cut, such as a shooting section that divides the scenario created in this way. A cut constitutes, for example, a section in multiple video shoots for a scenario.

[0049] The shooting button 12 is a virtual button for executing a function (i.e., a cut shooting function) that supports video shooting of each cut in a scenario created by, for example, the scenario creation function. The number of times a video is shot for one cut is not limited to one take, and multiple takes may be used. In this embodiment, the information support terminal 200 controls video shooting by the digital camera 100 in the cut shooting function and manages the shooting results for each cut.

[0050] The export button 13 is a virtual button for executing a function (i.e., an export function) that performs preprocessing for external output on the management results of video shooting using the cut shooting function and outputs the result. The preprocessing using the export function provides information support that makes it easier to edit multiple video shoot results according to a scenario on the video editing PC 300, for example.

[0051] The information support terminal 200 of this system 10 can provide consistent information support from scenario creation to pre-processing of video editing, for example, by the user sequentially using each function of the scenario creation button 11, shooting button 12, and export button 13.

[0052] In the present system 10, the function selection screen of the information support terminal 200 may further include a delete button for deleting various data in the information support described above. For example, in response to a user operation of the delete button, the information support terminal 200 may collectively delete video files of the same scenario.

[0053] 2.1. About the Scenario Creation Function The scenario creation function of the information support terminal 200 of the present system 10 will be described with reference to FIGS.

[0054] Fig. 5 shows an example of a scenario input screen displayed on the information support terminal 200. When a user operation, such as tapping the scenario creation button 11 on the function selection screen of Fig. 4, is input from the operation unit 230, the control unit 210 of the information support terminal 200 causes the display unit 240 to display the scenario input screen as shown in Fig. 5.

[0055] The scenario input screen is a screen that allows the user to input a scenario into the information support terminal 200 in the scenario creation function of the present system 10. As shown in Fig. 5, for example, the scenario input screen includes a storyboard input field 20 for each cut, a cut edit button 14, and a back button 15. The control unit 210 of the information support terminal 200 accepts various user operations related to the scenario input screen displayed on the display unit 240 at the operation unit 230.

[0056] In the information support terminal 200, the storyboard input field 20 accepts user input of information indicating a storyboard, such as an outline of the scenario concept, for each scene that makes up the scenario. The storyboard input field 20 for each scene includes, for example, a composition field 21, a script field 22, a shooting time field 23, a shooting location field 24, and a memo field 25, as shown in FIG.

[0057] The composition field 21 accepts input of image information indicating the composition of the video of the corresponding cut. The input of image information may be a drawing by a user operation or may be a specification of image data. The script field 22 accepts text input such as a script divided into parts for the corresponding cut in the scenario.

[0058] The shooting time field 23 accepts a numerical input indicating the estimated length of time it will take to shoot the video of the cut. The shooting location field 24 accepts an input of information indicating the location where the video of the cut will be shot. The shooting location may be input as text, or data search may be used as appropriate. The memo field 25 accepts input of various information desired by the user regarding the shooting of the video of the cut, such as the shooting equipment, for example, as text.

[0059] 5, the display unit 240 displays storyboard input fields 20 for two cuts. The control unit 210 acquires storyboard information for each cut in accordance with user input into the various fields 21 to 25 in the storyboard input field 20 for each cut in the scenario. On the scenario input screen of the information support terminal 200, the storyboard input field 20 for the cut displayed on the display unit 240 can be changed, for example, in response to a swipe operation for scrolling in the X direction in which the storyboard input fields 20 for each cut are arranged.

[0060] The cut edit button 14 switches on / off a state in which various user operations can be input, such as adding, deleting, and changing the order of cuts to be included in the scenario. For example, by performing a touch operation while the cut edit button 14 is in the on state, the user can arrange the storyboard input fields 20 for the desired number of cuts in chronological order in the scenario.

[0061] The back button 15 accepts a user operation to go back one screen in the screen transition of the information support terminal 200. For example, the control unit 210 transitions the display unit 240 to the function selection screen (FIG. 4) in response to a user operation of the back button 15 on the scenario input screen (FIG. 5). The control unit 210 of this embodiment generates cut data including storyboard information for each cut as an output of such a scenario creation function and stores the cut data in the storage unit 220. An example of such cut data at the end of the scenario creation function is shown in FIG. 6.

[0062] 6, the cut data D1 manages, for each "cut number," a "script," a "composition," a "shooting time," a "shooting location," a "shooting completion flag," and a "video metadata list" in association with each other. The cut data D1 is an example of management information in this embodiment.

[0063] The control unit 210 of the information support terminal 200 assigns cut numbers indicating identification information of the cuts in the cut allocation data D1 in ascending order in the storyboard input field 20 for each cut arranged on the scenario input screen, for example. Furthermore, if the order of the cuts is changed, the control unit 210 reassigns the cut numbers according to the changed order. For each cut, the control unit 210 records the information entered in the script field 22, composition field 21, shooting time field 23, shooting location field 24, and memo field 25 of the storyboard input field 20 in the "script," "composition," "shooting time," and "shooting location" fields of the cut allocation data D1, respectively.

[0064] In the cut division data D1, the "shooting completion flag" manages whether the shot in question has been shot or not by setting it to ON / OFF. When the scenario creation function is terminated, the shooting completion flag for all shots is set to OFF as the initial setting.

[0065] The "movie metadata list" is a list for storing metadata of the movie that is shot in association with the cut. When the scenario creation function is terminated, the movie metadata list is initially set to null.

[0066] As described above, the scenario creation function of the information support terminal 200 of this system 10 generates cut-by-cut data D1 from user input on the scenario input screen, thereby providing information support for the process of creating a scenario for the user's desired video work cut by cut.

[0067] The scenario creation function of information support terminal 200 is not particularly limited to the above. For example, information support terminal 200 may accept a user instruction to output storyboard information of a scenario entered on a scenario input screen using a data format (e.g., PDF format) that can be shared with another device, and output the data.

[0068] 2.2.About the Cut Shooting Function The operation of the snapshot photography function in the information support terminal 200 of the system 10 will be outlined with reference to FIG.

[0069] 7 shows an example of a cut selection screen displayed on the information support terminal 200. The cut selection screen is a screen for the cut shooting function of the present system 10 to select a desired cut from among cuts provided by, for example, the scenario creation function. The cut selection screen is an example of a selection screen on the information support terminal 200 of this embodiment.

[0070] 7, the cut selection screen includes a cut list 30, a storyboard display field 31, a filter button 32, an add cut button 33, a recording mode button 34, a playback mode button 35, and a back button 15. The cut list 30 is a list that lists various cuts as options that the user can select. The storyboard display field 31 is a display field that displays storyboard information about the selected cut. The cut selection screen will be described in detail later.

[0071] In the cut shooting function of the present system 10, the information support terminal 200 provides information support that makes it easy for the user to comprehensively proceed with video shooting of each cut while checking various cuts, for example, by using a cut selection screen shown in Fig. 7. The user may shoot videos in an order different from the order of the cuts in the scenario, or may shoot multiple takes for one cut of video.

[0072] Therefore, the information support terminal 200 of the present system 10 accepts the user's evaluation of the selected cut as the video of each take is shot, manages whether or not the shooting of the cut has been completed, and visualizes to the user the progress of video shooting for each cut in the cut list 30. The operation of the present system 10 will be described in detail below.

[0073] 2.2.1. Overall operation of the snapshot function The overall operation of the cut photography function in the system 10 will be described with reference to FIGS.

[0074] Fig. 8 is a flowchart illustrating the operation of the shot shooting function in the system 10. Each process shown in the flowchart in Fig. 8 is executed, for example, by the control unit 210 of the information support terminal 200. The process of this flow starts, for example, when the shooting button 12 on the function selection screen (Fig. 4) is operated in a state where shot data D1 generated by the scenario creation function has been stored in the storage unit 220 and a communication connection with the digital camera 100 has been established in the communication unit 250.

[0075] First, the control unit 210 of the information support terminal 200 generates a cut list 30 to be displayed on the cut selection screen (FIG. 7) based on the cut layout data D1 (S1). The cut list generation process (S1) is repeatedly executed in the system 10, for example, while the cut shooting function is being executed, in accordance with the progress of video shooting and various user operations, and the cut list 30 is updated successively.

[0076] Next, the control unit 210 causes the display unit 240 to display a cut selection screen, for example as shown in FIG. 7, based on the generated cut list 30 and the cut layout data D1 (S2).

[0077] 7, the cut list 30 on the cut selection screen includes a plurality of cut icons 3. Each cut icon 3 indicates an individual cut as an option. The cut icon 3 being selected is set to cut number "1" in the initial state, for example.

[0078] For example, the control unit 210 controls the display unit 240 to highlight the cut icon 3 indicating the selected cut (S2). The selected cut icon 3 is highlighted, for example, by displaying it in a larger size than the other cut icons 3 and by surrounding it with a frame in a highlighting color. Furthermore, the control unit 210 references the cut allocation data D1 and causes the storyboard display column 31 to display storyboard information about the cut indicated by the selected cut icon 3 (S2).

[0079] In the example of Fig. 7, the cuts with cut numbers "1" and "4" are in an incomplete state, while the cuts with cut numbers "2" and "3" are in a completed state. In the cut list 30 of this embodiment, the cut icon 3 has a display attribute that distinguishes between a completed state and an incomplete state. Such a display attribute is set, for example, so that the display mode of the completed state highlights the display mode of the incomplete state.

[0080] 7, the control unit 210 accepts various user operations on the operation unit 230, such as a touch panel, when the display unit 240 displays a cut selection screen (S3). The user operations targeted in step S3 include (I) a cut selection operation, (II) an operation to switch to a recording mode, (III) an operation to switch to a playback mode, and (IV) an end operation.

[0081] The cut selection operation ((I) in S3) is a user operation for changing the selected cut, and is, for example, an operation of tapping a cut icon 3 other than the selected cut icon 3 in the cut list 30 displayed on the cut selection screen. The cut selection operation is not limited to this, and for example, a swipe operation in the storyboard display field 31 may be input as a cut selection operation for changing from the selected cut to an adjacent cut.

[0082] When a cut selection operation is input ((I) of S3), the control unit 210 changes the selected cut icon 3 in accordance with the input cut selection operation (S4), and performs the processes from step S2 onwards again. As a result, the selected cut icon 3 is changed on the cut selection screen, and the storyboard display column 31 for the newly selected cut is displayed (S2).

[0083] The transition operation to the recording mode ((II) of S3) is a user operation for transitioning to the recording mode, which is an operating mode for shooting video of the selected cut, and is, for example, a tap operation on the recording mode button 34. In addition to or instead of this, the transition operation may be a swipe operation in a predetermined direction of the ±X directions on the cut selection screen. Also, the recording mode button 34 may be omitted.

[0084] When an operation to switch to recording mode is input ((II) of S3), the control unit 210 executes various processes for recording one take of a moving image in association with the selected cut in recording mode (S5). An example of the display in step S5 is shown in FIG.

[0085] 9 shows an example of a rating screen displayed on information support terminal 200. The rating screen is a screen for prompting the user to rate the video of the taken take. The rating screen is an example of an evaluation screen on information support terminal 200 of this embodiment.

[0086] 9, the rating screen includes an information display field 40 about the filmed video, and rating options including an OK button 41, a KEEP button 42, and an NG button 43. The information display field 40 displays information about the filmed take of the video, including, for example, a thumbnail image of the take, the cut number associated with the take, and the number of takes.

[0087] The OK button 41 indicates, for example, an "OK" rating when the user has decided that the take should be used in the corresponding cut. The KEEP button 42 indicates, for example, a "KEEP" rating when the user is unsure whether to use the take. The NG button 43 indicates, for example, an "NG (No Good)" rating when the user has clearly decided not to use the take. In this embodiment, the rating "NG" is an example of a first rating, and the ratings "OK" and "KEEP" are each examples of a second rating.

[0088] In the recording mode process (S5) in this embodiment, each time one take of video is shot, a rating screen such as that shown in Fig. 9 is displayed to obtain evaluation information indicating the user's evaluation of that take. Details of the process in step S5 will be described later.

[0089] In the information support terminal 200 of this embodiment, the control unit 210 further performs a process of determining whether or not the video of the captured take is NG based on the audio data in the video of the take (S7). In the audio determination process (S7) of this embodiment, a take in which the audio of the performer in the scenario is not sufficiently captured is automatically determined to be NG using a sound source separation technique that separates target sounds emitted by a desired subject, such as a person's voice, from various noises. The process of step S7 will be described in detail later.

[0090] The control unit 210 performs the process of step S1 again, for example, based on the user evaluation information (S5) and the audio evaluation result (S7) obtained for the video of the take. In this way, the cut list 30 is updated to reflect the user's rating results and the automatic audio evaluation results for the new take for the selected cut, and is made visible to the user (see FIG. 10).

[0091] The operation to transition to playback mode ((III) of S3) is a user operation for transitioning to playback mode, which is an operating mode for playing back and displaying the video shot for the selected cut, and is, for example, an operation of the playback mode button 35. In addition to or instead of this, the operation to transition to playback mode may be a swipe operation in the ±X direction of the cut selection screen, opposite to the operation to transition to recording mode. Also, the playback mode button 35 may be omitted.

[0092] When an operation to transition to playback mode is input ((III) of S3), the control unit 210 executes a process to play back various takes of video related to the cut currently selected in playback mode (S6). In the playback mode process (S6) in this embodiment, re-rating can be performed to change the evaluation of the video of each take. Based on the re-rating results of the playback mode process (S6), the control unit 210 performs the cut list generation process (S1) again to update the cut list 30. Details of the process of step S6 will be described later.

[0093] The end operation (S3(IV)) is a user operation for ending the shot shooting function, such as operating the back button 15 on the shot selection screen (FIG. 7). When the end operation is input (S3(IV)), the control unit 210 transitions the display unit 240 from the shot selection screen to the function selection screen (FIG. 4), for example, and ends the processing shown in this flow.

[0094] According to the above process, the user of the system 10 can check various cuts (S4) on the cut selection screen (FIG. 7) in the cut shooting function of the information support terminal 200, and can shoot a video of the desired cut (S5) and play back and display it (S6). In this way, the user can easily manage the video shooting of multiple cuts in a scenario.

[0095] In this embodiment, for example, the audio determination process (S7) in the information support terminal 200 automatically determines whether the audio recorded in the video shot as each take for each cut is NG (i.e., audio NG). FIG. 10 shows an example of a cut list 30 updated in step S1 to reflect the determination results. In this example, since a take with NG audio occurred in the cut with the cut number, an identifier "!" is added to the corresponding cut icon 3 to warn of NG audio. This identifier is an example of warning information of the system 10.

[0096] When shooting audio in videos at various locations, the sound of an actor may not be captured clearly enough due to noise such as the chirps of insects or animals (e.g., cicadas) or wind noise, resulting in a NG. Manually checking the audio in the video to avoid such audio NGs is time-consuming, and it may not be possible to check it thoroughly on set when time is tight. In such cases, the NG audio cut may be discovered during video editing after returning home from the set, resulting in a return trip to the set to reshoot the corresponding cut, resulting in unnecessary additional work and additional hours in the video production process.

[0097] Therefore, the information support terminal 200 of this embodiment automates the determination of shooting failures due to noise contamination through the audio determination process (S7), thereby reducing the burden on the user of checking whether a video is NG. Furthermore, the results of such NG determinations can be visualized to the user, for example, in an updated cut list 30, so that the user can be warned on the set that shooting has not been completed due to audio NG. In this way, the number of steps required for video production by the user of this system 10 can be reduced, contributing to labor savings.

[0098] In the cut selection screen of this embodiment, each cut icon 3 is displayed with a distinguishable indication of whether or not shooting has been completed, thereby preventing the user from forgetting to shoot a cut. The distinguishable indication of whether or not shooting of each cut has been completed is performed in a manner that reflects the user's evaluation of each take of video, making it easy to ensure video quality that meets the user's intentions. Such ratings are performed for each take (S5), and further re-rating can be performed in playback mode (S6). This makes it easy to achieve quality control of video shooting that meets the user's intentions.

[0099] Furthermore, the cut selection screen (FIG. 7) of this embodiment is not limited to updating the cut list 30 (S5, S6, S1) in response to the rating / re-rating of cuts as described above, but may also be updated to filter the displayed items or add cuts. This allows the user to efficiently use the desired cuts on the cut selection screen, for example, while shooting a video, making it easier to use the cut shooting function of this system 10.

[0100] In the shot shooting function of this embodiment, the communication connection with the digital camera 100 may be managed, and for example, a button for managing the communication connection may be provided on the shot selection screen. When the communication connection with the digital camera 100 is not established, the control unit 210 may disable the operation to switch to the recording mode ((II) in S3).

[0101] 2.2.2. Recording Mode The details of the recording mode process in step S5 in FIG. 8 will be described with reference to FIGS.

[0102] Fig. 11 is a flowchart illustrating the recording mode process (S5) in the present system 10. The process shown in the flow of Fig. 11 starts when, for example, an operation to switch to the recording mode is input on the cut selection screen (Fig. 7) (S3 (II)).

[0103] First, the control unit 210 of the information support terminal 200 shifts to a recording mode and causes the display unit 240 to transition to a screen for waiting for video recording (S31). An example of the display on the information support terminal 200 in step S31 is shown in FIG. 12(A).

[0104] 12A, the recording standby screen in step S31 includes a live view image 45 and a recording button 46. The recording button 46 accepts a user operation to start capturing and recording a moving image.

[0105] In this system 10, when the control unit 210 of the information support terminal 200 transitions to, for example, a recording mode, it requests the digital camera 100 to transmit a live view image 45 via the communication unit 250 (S31). For example, in the recording mode, the control unit 210 sequentially receives image data of the live view image 45 from the digital camera 100 via the communication unit 250 and displays the live view image 45. Furthermore, the control unit 210 receives, as appropriate, audio data of the sound picked up by the microphone 160 from the digital camera 100, for example.

[0106] In response to a user operation of the record button 46, the control unit 210 performs various controls to start shooting and recording one take of a moving image associated with the selected cut (S32). For example, in step S32, the control unit 210 instructs the digital camera 100 to start shooting and recording the moving image via the communication unit 250. An example display of the information support terminal 200 in step S32 is shown in FIG. 12(B).

[0107] 12(B), the shooting screen in step S32 includes a live view image 45, a time display field 47, and a recording stop button 46a. The live view image 45 on the shooting screen is highlighted, for example, with a frame or other display to indicate that recording is in progress. The time display field 47 displays, for example, the shooting time of the selected cut in the cut data D1 and the elapsed time since the start of shooting the video of that take, for comparison.

[0108] In step S32, the control unit 210 controls the display unit 240 to switch the display from the shooting standby screen (FIG. 12(A)) to the shooting screen (FIG. 12(B)). The control unit 210 also records, in the storage unit 220 of the information support terminal 200 (S32), a video file showing the live view images 45 sequentially received from the digital camera 100 after the record button 46 is operated. The video file includes, for example, audio data picked up by the microphone 160 in synchronization with the video shooting by the digital camera 100.

[0109] The control unit 210 also determines a file name for the moving image file based on, for example, the cut data D1 and the number of takes already shot for the selected cut. The control unit 210 may include the determined file name in an instruction to the digital camera 100. The control unit 135 of the digital camera 100 starts shooting the moving image in accordance with an instruction from the information support terminal 200 received via, for example, the communication module 155.

[0110] At this time, the control unit 135 repeats the imaging operation of the image sensor 115 and records video data of the imaging results onto the memory card 142, for example, via the card slot 140. The video data includes, for example, audio data collected by the microphone 160. The control unit 135 may start collecting audio in synchronization with video imaging in the digital camera 100 in response to such an imaging instruction. On the imaging screen of FIG. 12(B), the recording stop button 46a accepts a user operation to stop the imaging and recording of the video.

[0111] Thereafter, control unit 210 performs various controls to stop the shooting and recording of the video in response to a user operation of recording stop button 46a (S33). For example, in step S33, control unit 210 instructs digital camera 100 to stop shooting and recording of the video via communication unit 250. Control unit 210 also stops the video recording of live view image 45 in information support terminal 200 (S33). Control unit 135 of digital camera 100 ends video shooting in accordance with the instruction from information support terminal 200.

[0112] Furthermore, the control unit 210 causes the display unit 240 to display a rating screen, as shown in FIG. 9, for example, in order to prompt the user to rate the video of the take shot as described above (S34).

[0113] The control unit 210 receives user operations on various buttons 41 to 43 on a rating screen such as that shown in Fig. 9, and acquires the user's evaluation as a rating result for the video of the taken take (S35). In this embodiment, each time a video of one take is shot, the user can arbitrarily select a desired evaluation from the three types of evaluations "OK," "KEEP," and "NG" for the shot video, without interfering with the evaluations of other videos of the taken.

[0114] The control unit 210 determines, for example, whether the user's evaluation is "NG" or not based on the acquired user's evaluation (S36). For example, if the user's evaluation is "OK" or "KEEP," the determination in step S36 is "NO."

[0115] If the acquired user evaluation is not "NG" (NO in S36), the control unit 210 sets the shooting completion flag of the cut associated with the take in the cut division data D1 (i.e., the selected cut) to "ON" (S37). For example, if the number of takes in the video is "1," or if the number of takes is "2" or more and the evaluation of the video of the existing takes is "NG," the shooting completion flag is switched from "OFF" to "ON" by executing step S37.

[0116] On the other hand, if the acquired user evaluation is "NG" (YES in S36), the control unit 210 proceeds to step S38 without particularly updating the setting of the shooting completion flag. As a result, if the shooting completion flag for the corresponding cut when the video with the evaluation "NG" was shot is OFF, for example, the OFF state is maintained. Also, if the shooting completion flag is ON because a video of a take shot in the past has "KEEP" or "OK", for example, the ON state is maintained.

[0117] The control unit 210 generates metadata for the moving image of the taken image as described above, and records it as cut data D1 in the storage unit 220 (S38), for example. An example of such moving image metadata D2 is shown in FIG.

[0118] 13, the video metadata D2 includes a "video file name," "rating information," and "audio evaluation result." The control unit 210 includes in the video metadata D2 the video file name determined in steps S32 to S33 so as to reflect the number of takes for the video, and the user's rating acquired in step S35. Also, for example, the "audio evaluation result" in the video metadata D2 is null at the time of step S38.

[0119] The control unit 210 stores the generated video metadata D2 in the video metadata list for the cut associated with the video in the cut division data D1 (FIG. 6) (S38). The video metadata D2 is not limited to the above, and may include, for example, the number of takes of the video in addition to or instead of the video file name.

[0120] The control unit 210 ends the recording mode process (S5) by storing the video metadata D2 (S38), for example, and proceeds to step S7 in FIG.

[0121] According to the above-described recording mode process (S5), the system 10 can shoot and record one take of a video for the selected cut and prompt the user to rate it (S32-S35). The system 10 also manages the shooting completion flag for the cut based on the acquired evaluation information (S36, S37). In this way, the user's evaluation information for each take can be appropriately reflected in managing whether or not the cut is in a shooting completion state. Furthermore, according to the recording mode process (S5) of this embodiment, the information support terminal 200 can control the shooting and recording of the video by the digital camera 100, thereby realizing management of video shooting.

[0122] In rating each take of a video (S34-S35), multiple takes associated with the same scene may have the same rating. For example, multiple takes of the same scene may have a rating of "OK."

[0123] The rating screen displayed in step S34 may be displayed as a dialogue. For example, control unit 210 may control display unit 240 to superimpose the dialogue of the rating screen on the display screen before and after step S33.

[0124] The recording standby screen in recording mode (FIG. 12(A)) may further include a back button 15 for returning the screen transition to, for example, the cut selection screen. Such a back operation may be a swipe operation in a predetermined direction among the ±X directions on the video management screen. The information support terminal 200 may also transition to playback mode by a swipe operation in the opposite direction.

[0125] 2.2.3. Audio detection processing The details of the voice determination process in step S7 in FIG. 8 will be described with reference to FIGS.

[0126] Fig. 14 is a flowchart illustrating the audio determination process (S7) in the present system 10. The process shown in the flow of Fig. 14 starts after the process of shooting one take of video (S5 in Fig. 8) is performed in association with the cut currently selected in the recording mode, for example.

[0127] First, for example, the control unit 210 of the information support terminal 200 generates target sound data and noise data based on the audio data of the video file obtained in step S5 so as to separate the target sound component and the noise component from the audio picked up in the video (S41). For example, the target sound data is audio data indicating the target sound component separated in this way, and the noise data is audio data indicating the noise component.

[0128] In step S41, the control unit 210 executes a sound source separation process using, for example, a trained model in which machine learning has been performed in advance on the target sound and noise. Such a sound source separation process can be realized using various known techniques, and can be performed by, for example, converting the target signal feature quantity and the non-target signal feature quantity output by the neural network model of Patent Document 2 from the frequency domain to the time domain, respectively.

[0129] Next, the control unit 210 detects (S42) whether or not there is a noise component in the audio of the video that exceeds a predetermined allowable level, for example, based on the noise data generated by the sound source separation process (S41). The allowable level is set to a volume level that is assumed to be large enough to tolerate noise in the audio data in video editing by the system 10, and is an example of the predetermined level in this embodiment.

[0130] For example, in step S42, the control unit 210 successively compares the volume level of the noise data with the allowable level over the duration of the video recording, and proceeds to "YES" if the noise component exceeding the allowable level continues for a period longer than the predetermined allowable period, and proceeds to "NO" if not. The allowable period is set in advance to a period of time that is assumed to be allowable for noise in video editing by the system 10, for example.

[0131] Such noise detection (S42) may be performed with reference to the target sound data, or may be performed with a focus on the timing when the target sound is being emitted. The acceptable level may be dynamically set according to the volume level of the target sound component, considering that superimposition of noise can make it difficult to ensure the clarity of the target sound. For example, the acceptable level may be set to a volume level that is lower than the volume level of the target sound component by a predetermined margin. This allows the system 10 to tolerate a situation where the volume level of the noise component is high but the volume level of the target sound component is even higher, as long as the difference is within a certain margin, without determining that the filming has failed. Furthermore, time segments in the audio data of a video that are not expected to be used in the final video production may be excluded from the noise detection (S42) target.

[0132] If a noise component exceeding the allowable level is detected (YES in S42), the control unit 210 determines that the video of that take is "audio NG" (S43). Audio NG indicates that the system 10 has determined that the video is NG (unacceptable) in its automatic audio determination.

[0133] On the other hand, if no noise component exceeding the allowable level is detected (NO in S42), the control unit 210 determines that the video of that take is "audio OK" (S44). "Audio OK" indicates that the system 10 has automatically determined that the video is OK (adoptable).

[0134] Next, the control unit 210 stores the determination result for the take, such as audio NG or audio OK, in, for example, video metadata D2 in the storage unit 220 (S45). The control unit 210 also stores the noise data and target sound data for the take in association with the video metadata D2 in the storage unit 220, for example.

[0135] The control unit 210 ends the audio evaluation process (S7) by, for example, storing the evaluation result (S45), and proceeds to step S1 in Fig. 8. In this way, the control unit 210 appropriately updates the cut list 30 based on the evaluation information of the video metadata D2 and the audio evaluation result (see Fig. 10).

[0136] According to the above-described audio determination process (S7), the system 10 uses the sound source separation process (S41) not to suppress noise in the audio data of the video itself, but instead to detect noise components that exceed an acceptable level (S42). This eliminates the need for excessively high quality sound quality required for the audio output in the sound source separation process (S41), and makes it possible to realize such sound quality using, for example, a trained model with reduced computational complexity.

[0137] Furthermore, the system 10 can effectively determine whether audio is NG for both stationary and non-stationary noise by detecting noise components (S42) using noise data obtained as the output of the sound source separation process (S41). The results of the audio determination process (S7) are visualized to the user as warning information about audio NG in, for example, the cut list 30 (S41) that is updated subsequently (FIG. 10). This allows the user to instantly recognize the audio NG cuts on set when a take with audio NG is shot, and to reshoot the cuts on the spot.

[0138] In the information support terminal 200 of this embodiment, the warning information for audio NG is not limited to the example of the cut list 30 described above, and various other information may be presented. Such a modified example will be described with reference to FIG.

[0139] 15 shows an example of how audio NG warning message 48 is displayed on information support terminal 200. In this embodiment, when control unit 210 of information support terminal 200 determines that the take is audio NG in step S43, for example, it may cause display unit 240 to display audio NG warning message 48, as shown in FIG. 15. Warning message 48 in this example may include message information that is an example of warning information, as well as a video playback button 49 that accepts an operation to transition to playback mode for playing the video of the take. In information support terminal 200 of this embodiment, audio NG warning information may be various information presented to the user in playback mode, as will be described later.

[0140] In the above description, the audio evaluation result and the user's evaluation information are stored separately and managed independently in step S45. Instead of such independent management, the information support terminal 200 of this embodiment may update the evaluation information based on the audio evaluation result. For example, for a take with NG audio (S43), the control unit 210 may change the evaluation information to "NG" if the evaluation information in the video metadata D2 at the time of the user's rating (S35 in FIG. 11) is "OK" or "KEEP." In response to such a change in the evaluation information in the video metadata D2, the control unit 210 may manage the shooting completion flag for the cut. For example, if the evaluation information for all takes in the cut is "NG" after the change, the control unit 210 updates the shooting completion flag for the cut from "ON" to "OFF."

[0141] Alternatively, in the present system 10, the audio evaluation process (S7) may be performed only when the user's rating result is "OK" or "KEEP." The control unit 210 of the information support terminal 200 may refer to the video metadata D2 after executing the recording mode process (S5), and skip the audio evaluation process (S7) if the rating information is "NG."

[0142] Furthermore, in the present system 10, the timing at which the audio determination process (S7) is executed is not limited to after the process (S5) of shooting one take of video in the recording mode. For example, the audio determination process (S7) may be executed after shooting a predetermined number of takes of video, or may be executed in response to a user instruction.

[0143] 2.2.4. Playback Mode An example of processing for video playback of a take with NG audio in the playback mode (S6 in FIG. 8) of the system 10 will be described below with reference to FIGS. 16 to 19. FIG.

[0144] Fig. 16 is a flowchart illustrating a process for playing back video with NG audio in the system 10. The process shown in the flow of Fig. 16 is started when an operation to switch to playback mode is input (S3 (III)), for example, on the cut selection screen (Fig. 7) or the warning message 48 (Fig. 15).

[0145] First, the control unit 210 of the information support terminal 200 causes the display unit 240 to display a screen including warning information about a take with NG audio in playback mode (S51), based on the video metadata D2 including the result of the audio determination process (S7). An example of the display on the information support terminal 200 in step S51 is shown in FIG.

[0146] The video playback screen in step S51 includes, for example, as shown in FIG. 17, a cut identification field 51, a video list 50, a playback image 53, a playback control bar 54, an audio graph 60, a menu button 55, a re-rating panel 56, and a back button 15.

[0147] The cut identification field 51 displays identification information of the selected cut. The moving image list 50 includes moving image icons 5 indicating moving images for each take of the selected cut. The moving image icons 5 are configured, for example, by superimposing evaluation information on a thumbnail image of the moving image in the corresponding take. In the playback mode of the present system 10, the moving image of the moving image icon 5 selected in the moving image list 50 is displayed as a playback image 53.

[0148] The playback control bar 54 has a direction (X direction) corresponding to the time axis of the video to be played, for example. The playback control bar 54 indicates the playback timing of the entire video depending on its position in the longitudinal direction, and accepts touch operations to change the playback timing.

[0149] In the example of Figure 17, video icon 5 of take 4 with NG audio is selected in video list 50, so control unit 210 displays audio graph 60 as an example of warning information on the video playback screen (S51).

[0150] The audio graph 60 includes a target sound graph 61, a noise graph 62, and a tolerance line 63. The horizontal axis of the audio graph 60 indicates the time axis similar to that of the playback control bar 54, and the vertical axis indicates the volume level. The target sound graph 61 indicates the volume level for each time in the target sound data of the video to be played. The noise graph 62 indicates the volume level for each time in the noise data of the video. The tolerance line 63 indicates the criteria for determining whether audio is NG (e.g., tolerance level and tolerance period) in the audio determination process (S7).

[0151] According to the audio graph 60 in the information support terminal 200 of this embodiment, the time range in which the noise graph 62 exceeds the tolerance line 63 makes it possible for the user to easily grasp the portion of the video that caused the audio NG, i.e., the NG portion 65. In the example of Fig. 17, the NG portion 65 is highlighted on the playback control bar 54. The NG portion 65 is an example of warning information in this embodiment.

[0152] 17, the menu button 55 accepts a touch operation to switch between displaying and hiding a selection menu 55a in playback mode, for example. The selection menu 55a includes menu items such as "Noise decomposition" and "NG criteria setting" as options that the user can select.

[0153] The re-rating panel 56 includes, for example, a plurality of buttons for re-rating the selected video, similar to the rating screen (FIG. 9). In this example, since a video with bad audio is selected, the re-rating panel 56 is grayed out to be disabled.

[0154] In step S51, the control unit 210 references, for example, the video metadata list for the cut in the cut data D1, and generates each video icon 5 to visualize the evaluation information for each take associated with the cut. The control unit 210 further assigns, for example, an identifier for the take with NG audio to the corresponding video icon 5 (S51). Such a video icon 5 is an example of warning information in the video list 50.

[0155] 17, the control unit 210 accepts various user operations via the operation unit 230 (S52). The user operations that are the subject of step S52 include (I) a video playback operation, (II) a noise resolution operation, (III) an NG criterion setting operation, and (IV) an end operation.

[0156] The video playback operation ((I) of S52) is a variety of user operations that allow the user to play and watch a desired part of the video. For example, as this operation, the user can tap the playback image 53 to switch between playing and pausing the video, or touch the playback control bar 54 to change the playback position.

[0157] When the user inputs the desired video playback operation ((I) of S52), the control unit 210 performs various controls for playing the video on the video playback screen in accordance with the input video playback operation (S53), and makes the judgment of step S52 again.

[0158] For example, the control unit 210 sequentially displays frame-by-frame images of the moving image file stored in the storage unit 220 of the information support terminal 200 as the playback image 53 in accordance with the playback position indicated by the playback control bar 54 (S51). Also, the control unit 210 causes the speaker 270 to output the audio indicated by the audio data of the moving image file, for example, in synchronization with the playback display (S51).

[0159] According to the information support terminal 200 of this embodiment, for example, the playback control bar 54 on the video playback screen (FIG. 17) can be used to prompt the user to selectively play back and view the NG section 65 in the video with NG audio (S51 to S53). In this way, the user of the present system 10 can instantly check the status of the NG section 65 on the shooting site, for example, and can easily identify the cause of the NG audio and consider improvements for future takes when shooting the video.

[0160] Furthermore, the system 10 provides information support such as further decomposing and visualizing audio components such as noise components in videos with bad audio, and applying user settings to the criteria for determining whether audio is bad.

[0161] The noise decomposition operation ((II) of S52) is a user operation that instructs the system 10 to execute information support by decomposing noise components, and can be input by operating the selection menu 55a, for example.

[0162] When the noise decomposition operation ((II) of S52) is input, for example, the control unit 210 of the information support terminal 200 executes a sound analysis process to decompose the sound represented by the noise data of the video into predetermined types of sound components, and obtains the noise decomposition results (S54). The predetermined types are set as various types of noise expected in video shooting, including, for example, the cries of various living creatures, the sound of wind, and the sound of waves.

[0163] Next, the control unit 210 performs the process of step S51 again to update the video playback screen based on the acquired noise decomposition result. An example of a display in which the noise decomposition result is reflected on the video playback screen is shown in FIG.

[0164] FIG. 18 illustrates an example of an audio graph 60 updated in response to a noise decomposition operation ((II) in S52) on the video playback screen illustrated in FIG. 17. In this example, the control unit 210 of the information support terminal 200 updates the audio graph 60 to visualize the distribution of multiple types of audio components, such as "cicadas" and "wind," in the noise graph 62 (S54, S51). The system 10 achieves such visualization through audio analysis that identifies the types of sound sources of audio components contained in noise.

[0165] As the sound analysis process in step S54, for example, the control unit 210 detects the volume levels at which predetermined types of sound components are included in the volume levels of the noise data at each time point. Alternatively or in addition, the control unit 210 may generate sound data so as to separate each type of sound component from the noise data. Such sound analysis process may be performed by data matching using a database containing various types of sound data, or may be performed by a trained model based on machine learning using the database. The types of sound sources are not limited to those shown in FIG. 18 and may include, for example, the sound of waves and the cries of various insects or animals.

[0166] Based on the noise decomposition results acquired in step S54 as described above, the control unit 210 updates the audio graph 60 on the video playback screen, for example as shown in Fig. 18, to visualize the audio components of the noise by type (S51). This allows the system 10 to allow the user to intuitively understand the type of noise that caused the video of the take to be audio NG, making it easier to take measures against noise in future video shoots. Displaying the audio source types of audio components in the noise graph 62 allows the user to understand the source of specific audio components in the noise, making it easier to reshoot the cut.

[0167] The NG criteria setting operation ((III) of S52) is a user operation for changing the setting of the judgment criteria in the voice judgment process (S7). This user operation can be input by, for example, touching the selection menu 55a and the voice graph 60. The NG criteria setting operation in the information support terminal 200 will be described with reference to FIG. 19.

[0168] 19 illustrates the audio graph 60 in a state where the menu item "NG Criteria Settings" is selected from the selection menu 55a on the video playback screen in step S52 (III). At this time, for example, the control unit 210 of the information support terminal 200 receives a drag operation in the Y direction of the tolerance line 63 on the audio graph 60 on the touch panel of the operation unit 230 to acquire a user setting for increasing or decreasing the tolerance level of noise. The control unit 210 also receives a pinch-in / out operation in the X direction of the tolerance line 63 to acquire a user setting for shortening / extending the tolerance period for noise.

[0169] The user of the system 10 can change the settings as described above, taking into consideration, for example, the volume level that can be suppressed by noise reduction processing available in the video editing PC 300 in the post-process, or the degree to which environmental noise, etc., is desired to be retained from the perspective of dramatic effect. The control unit 210 updates the criteria for determining whether audio is NG in accordance with the user settings (S55).

[0170] Next, the control unit 210 performs the same audio determination process as in step S7 on the video of that take based on the updated criteria for determining whether audio is NG (S56), and returns to step S51. As a result, the audio graph 60 on the video playback screen for that take is updated based on the new criteria.

[0171] The end operation ((IV) of S52) is a user operation to return from the playback mode to the function selection screen, for example, the operation of the back button 15 on the video playback screen. The end operation in (IV) of step S52 may be, for example, a swipe operation in a predetermined direction among the ±X directions on the video playback screen. The information support terminal 200 may transition to the recording mode by a swipe operation in the opposite direction.

[0172] For example, when a back operation ((IV) of S52) is input on the video playback screen of FIG. 17, the control unit 210 ends the processing of this flow and returns to step S1 of FIG. 8, for example.

[0173] According to the above process, the information support terminal 200 of this embodiment displays various warning information such as the audio graph 60 and the NG portion 65 on the video playback screen of the take with NG audio in playback mode (S51). This allows the system 10 to prompt the user to check for NG audio in the video of that take.

[0174] For example, the user of the present system 10 can easily check NG points 65 in a video of a take with NG audio. Also, the target sound graph 61 and the noise graph 62 in the audio graph 60 allow the user to check, for example, how the target sound is obstructed by noise.

[0175] Furthermore, in response to the user's noise decomposition operation ((II) of S52), the system 10 visualizes various audio components in a noise graph 62 (S54, S51), for example, as shown in Fig. 18. This allows the user to check the details of the noise components in videos with poor audio quality, making it easier to improve future video shooting.

[0176] Furthermore, the system 10 may accept a user's NG criteria setting operation ((III) in S52) when the information support terminal 200 is in playback mode, for example, and update (S55) the criteria for the audio evaluation process (S7). User setting of the audio NG criteria makes it easier to prevent the system 10 from making frequent automatic audio NG determinations contrary to the user's intention, and makes it easier to realize automatic audio NG determinations that are in line with the user's intention. The information support terminal 200 of this embodiment may accept a NG criteria setting operation ((III) in S52) when not in playback mode.

[0177] In the information support terminal 200 of this embodiment, the video playback screen in step S51 is not particularly limited to the example in Fig. 17. For example, in the example in Fig. 17, the NG point 65 is displayed on the playback control bar 54, but in the present system 10, the NG point 65 may be displayed on the tolerance line 63 of the audio graph 60. Also, in the example in Fig. 17, one NG point 65 is displayed, but if there are multiple points in the video where noise components exceed the tolerance level, multiple NG points 65 may be displayed.

[0178] Furthermore, the audio graph 60 does not have to be displayed on the video playback screen of a take with NG audio in the present system 10. Even in this case, the present system 10 can warn the user of the time range of the NG audio, for example, by displaying the NG portion 65. The present system 10 can also warn the user of the take or cut with NG audio by identifying the NG audio in the video icon 5 or cut icon 3, respectively.

[0179] In the present system 10, the user operations accepted by the information support terminal 200 in playback mode are not limited to (I) to (IV) of steps S52, and other user operations may be accepted. For example, the control unit 210 of the information support terminal 200 may control playback of a selected video in response to a user operation to select a video of a take, not limited to a video with NG audio, in the video list 50 as the video to be played. The control unit 210 may also update the rating information of various takes in response to a user operation on the re-rating panel 56.

[0180] In the above description, the system 10 has been described for the voice analysis (S54) that decomposes noise data, but the system 10 is not particularly limited to this. For example, the system 10 may further perform the above-described voice analysis on the target sound data, or may perform voice analysis on the voice data before sound source separation. This may allow, for example, the type of the target sound component, such as a human voice, to be decomposed and visualized.

[0181] Furthermore, the audio determination process in step S56 is not limited to the video of the take in question, but may also be performed on other takes of the same cut, for example. Furthermore, in the present system 10, audio determination process may be performed on videos of other cuts using new determination criteria. In the present system 10, the user setting (S55) of the criteria for determining whether audio is NG may be performed for each cut, or may be a setting common to all cuts.

[0182] 3. Summary As described above, in this embodiment, information support terminal 200 is an example of an electronic device that manages video capture. Information support terminal 200 includes display unit 240 that displays information, operation unit 230, which is an example of an input unit that inputs user operations, and control unit 210 that controls display unit 240 in response to the operation input via operation unit 230. Control unit 210 receives an instruction to capture a video via operation unit 230 (S5). Control unit 210 acquires the result of a determination process that determines whether or not the video has been unsuccessfully captured, such as due to audio NG, based on the audio data of the captured video, for example, by executing an audio determination process (S7). If the acquired result of the determination process indicates a video capture failure, control unit 210 displays warning information on display unit 240 that warns of the video capture failure (see FIGS. 10, 15, and 17).

[0183] According to the information support terminal 200 described above, by displaying warning information to the user in response to the automatic determination that the audio is NG, it is possible to easily manage failures in capturing audio during video shooting.

[0184] In the information support terminal 200 of this embodiment, the determination process detects noise in the audio data and determines whether or not the video has failed to be recorded based on the noise detection result (S42). This allows the system 10 to automatically determine whether audio is NG due to noise, making it easier to manage video recording failures due to noise.

[0185] In the information support terminal 200 of this embodiment, the determination process separates audio components indicating noise from the audio indicated by the audio data of the video (S41), and detects noise based on the audio components (S42). This allows the system 10 to automatically determine whether audio is NG based on the sound source separation process (S41), making it easier to manage video shooting failures due to noise.

[0186] In the information support terminal 200 of this embodiment, the warning information includes at least one of the timing at which noise was detected in the video (e.g., NG point 65) and the volume level of the noise (e.g., noise graph 62). This allows the system 10 to visualize the noise that caused the audio NG using the warning information, making it easier to manage video shooting failures due to noise.

[0187] In the information support terminal 200 of this embodiment, the control unit 210, based on a user operation input to the operation unit 230 ((II) of S52), for example, acquires the decomposition result of noise data (S54), and controls the display unit 240 to visualize the multiple audio components separated into multiple types of audio represented by the audio data (see FIG. 18). This allows the user of the present system 10 to check the various audio components in the video, making it easier to manage recording failures in video recording.

[0188] In the information support terminal 200 of this embodiment, the control unit 210 sets a criterion for determining that a video has failed to be captured in the determination process (S55) based on the user's operation input to the operation unit 230 ((III) of S52). This allows the system 10 to reflect the user's intention in the criteria for determining whether a video has failed to be captured, making it easier to manage a video capture failure.

[0189] In this embodiment, the information support terminal 200 further includes a storage unit 220 for storing cut data D1, which is an example of management information for managing videos associated with one or more cuts in a scenario. The control unit 210 receives an instruction from the operation unit 230 to shoot a video associated with a specific cut from the one or more cuts (S44), and acquires the result of a determination process for the video based on the audio data in the video shot in association with the specific cut (S7). This enables the system 10 to easily manage audio capture failures in video shooting that includes one or more cuts.

[0190] In this embodiment, the information support terminal 200 further includes a communication unit 250 that performs data communication with the digital camera 100, which is an example of an imaging device that performs video shooting. The control unit 210 manages the video shot by the digital camera 100 through data communication via the communication unit 250 (S5). This makes it easier to manage the video shooting for each cut in the information support terminal 200, which is separate from the digital camera 100.

[0191] In this embodiment, a shooting management method for managing video shooting for a scenario including multiple cuts is provided. In this method, the control unit 210 of the information support terminal 200 receives an instruction to shoot a video from the operation unit 230, acquires the result of a determination process for determining whether or not the video has been shot successfully based on the audio data of the video being shot, and, if the acquired result of the determination process indicates a shooting failure, displays warning information indicating the shooting failure of the video on the display unit 240.

[0192] In this embodiment, a program is provided for causing the control unit 210 to execute the above-described shooting management method. According to this shooting management method, it is possible to easily manage shooting failures in terms of sound collection during video shooting.

[0193] (Embodiment 2) A second embodiment of the present disclosure will be described below with reference to Fig. 20. In the first embodiment, an information support terminal 200 that automatically determines whether audio is NG due to noise has been described. In the second embodiment, an information support terminal 200 that automatically switches between audio that is considered to be noise when determining whether audio is NG will be described.

[0194] Hereinafter, the description of the same configurations and operations as those of the imaging system 10 and the information support terminal 200 according to the first embodiment will be omitted as appropriate, and the imaging system 10 and the information support terminal 200 according to this embodiment will be described.

[0195] FIG. 20 is a flowchart illustrating an example of the audio determination process in the imaging system 10 of the second embodiment. In this system 10, when the information support terminal 200 performs the voice determination process (S7 in FIG. 8) as in the first embodiment, in addition to executing steps S41 to S45 as in FIG. 14, it also performs automatic control of the sound collection target (S46 to S48) as illustrated in FIG. 20.

[0196] For example, a user of the present system 10 may wish to capture a desired sound, such as the chirping of cicadas, in a particular scene in the entire scenario, without treating it as noise. In such a case, the present system 10 includes information about the sound to be captured (hereinafter referred to as "sound capture target information") in the storyboard information for that scene in the scene division data D1, for example, through a user operation in the scenario creation function. An example of processing using such sound capture target information in the present system 10 will be described below.

[0197] For example, in the information support terminal 200 of this embodiment, the control unit 210 refers to the cut division data D1 stored in the memory unit 220 and determines whether the storyboard information of the selected cut includes information on the sound to be collected in the video shooting of that cut (S46).

[0198] For example, the user can enter text information specifying the sound collection target in a memo section or script section of the storyboard information for the desired cut. In this case, in step S46, the control unit 210 performs language recognition processing, such as keyword extraction, on the storyboard information for the selected cut, and if it recognizes the text information as sound collection target information, proceeds to "YES". In this case, the target to be considered as noise in the audio NG (S43) is determined to be, for example, the audio component of noise data other than the sound collection target indicated by the sound collection target information.

[0199] When the control unit 210 determines that there is no sound collection target information in the storyboard information (NO in S46), it performs the processes from step S42 onwards, for example, in the same manner as in the first embodiment.

[0200] On the other hand, when the control unit 210 determines that there is sound collection target information (YES in S46), it detects whether or not there are noise components exceeding the allowable level excluding the sound collection target, for example, based on the noise data obtained in step S41 (S47). The determination in step S47 is made by, for example, the control unit 210 performing a process similar to the sound analysis of noise decomposition (S54 in FIG. 16) and comparing the volume level obtained by subtracting sound components of types matching the sound collection target from the noise data with the allowable level in the same way as in step S42.

[0201] If noise components exceeding the allowable level are detected excluding the target sound collection (YES in S47), the control unit 210 determines that the take is audio NG (S43). On the other hand, if noise components exceeding the allowable level are not detected excluding the target sound collection (NO in S47), the control unit 210 detects whether, for example, the sound components of the target sound collection exceed a predetermined sound collection level (S48).

[0202] Step S48 is performed to determine whether the sound to be collected has been collected sufficiently from the viewpoint of, for example, dramatic effect, etc. The sound collection level is set to a volume level that is expected from the above viewpoint, for example. The processing of step S48 is performed in the same manner as the processing of step S42, using, for example, this sound collection level.

[0203] If no sound components of the sound collection target that exceed the sound collection level are detected (NO in S48), the control unit 210 determines that take as sound NG (S43). On the other hand, if sound components of the sound collection target that exceed the sound collection level are detected (YES in S48), the control unit 210 determines that take as sound OK (S44).

[0204] According to the above process, the information support terminal 200 of this embodiment can utilize, for example, the sound collection target information included in the cut data D1 to automatically determine whether or not a specific sound is to be regarded as noise in accordance with the user's intention (S46). By utilizing such cut data D1, it is possible to make it easier to reflect the sound collection target intended by the user of the present system 10 in the automatic determination.

[0205] In the present system 10, the sound collection target information is not limited to text information, and may be, for example, image information. For example, if the composition in the storyboard information is an illustration of a landscape without including subjects such as performers, it is assumed that the user intends to collect environmental sounds. In such a case, the information support terminal 200 of this embodiment may recognize a composition of only the landscape as the sound collection target information for environmental sounds through image recognition of the storyboard information (YES in S46).

[0206] Furthermore, in this embodiment, the sound collection target information is not limited to the cut data D1, but may be various types of management information stored in the storage unit 220 for video shooting in the system 10.

[0207] Furthermore, the system 10 may use the sound collection target information as an input for the sound source separation process (S41). For example, the control unit 210 may determine the sound collection target information (S46) before step S41, and control the sound source separation process so that the type of sound collection target indicated by the sound collection target information is the target for separation.

[0208] As described above, the information support terminal 200 of this embodiment further includes a storage unit 220 for storing management information for managing video. When the management information includes information indicating the sound of the sound collection target (YES in S46), the control unit 210 determines what is to be regarded as noise in the audio data according to the sound collection target information (S47). As a result, when the sound of the sound collection target is specified, the information support terminal 200 of this embodiment reflects the sound collection target in the detection of noise, performs a process for determining whether or not shooting has failed, and can easily manage shooting failures in terms of sound collection.

[0209] Furthermore, in the information support terminal 200 of this embodiment, in the above case (YES in S46), the control unit 210 may control the judgment process to detect noise by excluding the sound of the target sound from the sound indicated by the audio data, or to determine that shooting has failed depending on the quietness of the sound of the target sound (S47, S48).

[0210] (Other embodiments) As described above, the first and second embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which appropriate modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is also possible to combine the components described in the above embodiments to create new embodiments.

[0211] In the first embodiment, the information support terminal 200 is described as an example of an electronic device separate from an imaging device, but the present disclosure is not limited to this. The electronic device of this embodiment may be integrated with an imaging device that captures video. Such a modification will be described with reference to FIG. 21.

[0212] 21 illustrates a modified example of the digital camera 100. In this embodiment, the digital camera 100 has various functions, such as the cut shooting function of the information support terminal 200 described above. For example, as shown in FIG. 21, the control unit 135 of the digital camera 100 displays a cut selection screen including multiple cuts based on a cut list 30 on the display monitor 130, and accepts cut selection by the user via the operation unit 150, such as a touch panel or operation buttons.

[0213] 21, the display monitor 130 displays a cut list 30 superimposed on a live view image. The control unit 135 of the digital camera 100 generates video data, for example, by the imaging operation of the image sensor 115 and the sound collection operation of the microphone 160. The control unit 135 of the digital camera 100 can automatically determine whether the sound is NG, similar to the sound determination process (S7) in each of the above embodiments, and display various types of warning information on the display monitor 130. With this digital camera 100, similar to the first embodiment, it is possible to provide the user with information support using the cut shooting function.

[0214] As described above, in this embodiment, digital camera 100, which is an example of an electronic device, further includes image sensor 115, which is an example of an imaging unit that captures an image of a subject and generates image data. Control unit 135 manages video made up of the image data generated by image sensor 115. This makes it easier to manage video shooting in digital camera 100.

[0215] In each of the above embodiments, a cut selection screen including a cut list 30 has been exemplified, but the selection screen of the present disclosure is not limited to this. The selection screen of the present embodiment may not include the cut list 30, and may include multiple cuts in a display format different from the cut icon 3. Furthermore, the selection screen of the present embodiment may be a dialog display, or may be displayed superimposed on various display screens. In the present embodiment, the cut list 30 may be an example of a selection screen. In the present embodiment, the selection screen of the information support terminal 200 may distinguish whether or not video shooting has been completed for each cut in various display formats not limited to the above examples.

[0216] In the above embodiments, examples have been described in which the evaluation information is three types, "OK," "KEEP," and "NG," but the evaluation information is not particularly limited to these. In this embodiment, the evaluation information may be three types of evaluation different from the above, or may be two or four or more types instead of three. Furthermore, in this embodiment, the evaluation information may be a continuous score. The electronic device of this embodiment may accept user input of such various types of evaluation information and manage video shooting for each cut. For example, by appropriately setting criteria for whether or not video shooting of a cut has been completed, it is possible to display the completion of the video shooting.

[0217] In the above embodiments, the imaging system 10 has been described as using information based on user ratings, but the present system 10 does not have to use user evaluation information. In this embodiment, the information support terminal 200 may automatically determine whether audio is NG without using user evaluation information. This also allows the present system 10, like the above embodiments, to easily manage recording failures in audio capture during video shooting.

[0218] Furthermore, in each of the above embodiments, an operational example has been described in which the cut data D1 includes multiple cuts, but in the present system 10, the cut data D1 may include only one cut. Furthermore, the information support terminal 200 of this embodiment does not necessarily need to use the cut data D1 or the scenario creation function. Even in such a case, the information support terminal 200 of this embodiment can automatically determine whether audio is NG, for example, according to preset noise and target sounds.

[0219] Furthermore, in each of the above embodiments, an operational example in which the control unit 210 of the information support terminal 200 executes each process, such as the voice determination process (FIG. 14), has been described. However, the present system 10 is not particularly limited to this. For example, in this embodiment, the sound source separation process in step S41 of FIG. 14 may be executed by an external server that communicates with the information support terminal 200. For example, in step S41, the control unit 210 of this embodiment may transmit audio data of the video to the external server via the communication unit 250, receive the results of the sound source separation from the external server, and execute the processes from step S42 onward. Alternatively, the control unit 210 may detect noise that is a voice NG by receiving the voice determination results from the external server. Furthermore, for example, the voice analysis of noise decomposition (S54 of FIG. 16) may also be executed by an external server instead of the information support terminal 200, as described above.

[0220] As described above, the information support terminal 200 of this embodiment may further include a communication unit 250 that performs data communication with an external server that executes the determination process. The control unit 210 may transmit audio data to the external server via the communication unit 250 and receive the result of the determination process for the audio data from the external server via the communication unit. This also enables the present system 10 to easily manage recording failures in audio pickup during video shooting, as in the above embodiments. Such an external server and the information support terminal 200 may form a network system. The external server may also be formed by an information processing device such as a computer equipped with a CPU or the like.

[0221] Furthermore, the present system 10 is not limited to automatically determining whether audio is NG, and may also automatically determine, for example, images in video shooting. The information support terminal 200 of this embodiment may perform image recognition to determine whether or not a person other than the performer or other subject desired by the user is reflected in the video based on, for example, the cut data D1. If such a person is detected, the control unit 210 of the information support terminal 200 may automatically determine that the take is NG. Such image recognition may be performed inside the information support terminal 200 or externally.

[0222] Furthermore, in each of the above embodiments, the digital camera 100 is illustrated as including the optical system 110 and the lens driving unit 112. The imaging device of this embodiment does not necessarily have to include the optical system 110 and the lens driving unit 112, and may be, for example, an interchangeable lens camera.

[0223] Furthermore, in each of the above embodiments, a digital camera has been described as an example of an imaging device, but the present disclosure is not limited to this. The imaging device of the present disclosure may be any electronic device having an image capturing function (for example, a video camera, a smartphone, a tablet terminal, etc.). Furthermore, the electronic device of the present disclosure does not necessarily have to have an image capturing function, and may be any of various electronic devices.

[0224] (Example) Various aspects of the present disclosure will be exemplified below.

[0225] A first aspect of the present disclosure is an electronic device that manages video recording. The electronic device includes a display unit that displays information, an input unit that inputs user operations, and a control unit that controls the display unit in response to the operations input via the input unit. The control unit receives an instruction to record a video via the input unit, and acquires the result of a determination process that determines whether or not the video recording has failed based on audio data in the video being recorded. If the acquired result of the determination process indicates a failure in recording, the control unit displays warning information indicating the failure in recording the video on the display unit.

[0226] In a second aspect, in the electronic device according to the first aspect, the determination process detects noise in the audio data and determines whether or not the video has been unsuccessfully captured based on the noise detection result.

[0227] In a third aspect, in the electronic device described in the second aspect, the judgment process separates audio components indicating noise from the audio indicated by the audio data based on the audio data of the video, and detects noise based on the audio components.

[0228] In a fourth aspect, in the electronic device according to the second or third aspect, the warning information includes at least one of the timing at which noise is detected in the video and the volume level of the noise.

[0229] In a fifth aspect, the electronic device according to any one of the second to fourth aspects further includes a storage unit for storing management information for managing moving images. When the management information includes information indicating a target sound to be collected, the control unit determines what is to be regarded as noise in the audio data according to the information indicating the target sound to be collected.

[0230] In a sixth aspect, in the electronic device described in the fifth aspect, when the management information includes information indicating the audio to be collected, the control unit controls the judgment process so as to detect noise by excluding the audio to be collected from the audio indicated by the audio data.

[0231] In a seventh aspect, in an electronic device described in any of the first to sixth aspects, the control unit controls the display unit to visualize the audio represented by the audio data into multiple audio components separated into multiple types, based on user operations input to the input unit.

[0232] In an eighth aspect, in the electronic device according to any one of the first to seventh aspects, the control unit sets a criterion for determining that video shooting has failed in the determination process, based on a user operation input to the input unit.

[0233] In a ninth aspect, the electronic device according to any one of the first to eighth aspects further includes a memory unit that stores management information for managing moving images associated with one or more cuts in a scenario. The control unit receives an instruction from the input unit to shoot a moving image associated with a specific cut out of the one or more cuts, and acquires a result of a determination process for the moving image based on audio data in the moving image shot in association with the specific cut.

[0234] In a tenth aspect, the electronic device according to any one of the first to ninth aspects further includes a communication unit that performs data communication with an image capture device that captures moving images, and the control unit manages the moving images captured by the image capture device through data communication via the communication unit.

[0235] In an eleventh aspect, the electronic device according to any one of the first to ninth aspects further includes an imaging unit that captures an image of a subject and generates image data, and a control unit that manages a moving image configured from the image data generated by the imaging unit.

[0236] A twelfth aspect is a shooting management method for managing video shooting. In this method, a control unit of an electronic device receives an instruction to shoot a video via an input unit, acquires a result of a determination process for determining whether or not the video has been shot successfully based on audio data of the video being shot, and, if the acquired result of the determination process indicates a shooting failure, displays warning information indicating the video has been shot successfully on a display unit.

[0237] In a thirteenth aspect, there is provided a program for causing a control unit to execute the photography management method according to the twelfth aspect.

[0238] As described above, the embodiments have been described as examples of the technology of the present disclosure. For this purpose, the accompanying drawings and detailed description have been provided. Therefore, among the components described in the accompanying drawings and detailed description, not only components essential for solving the problem but also components that are not essential for solving the problem in order to exemplify the above technology may be included. [Industrial Applicability]

[0239] The present disclosure is applicable to various uses for capturing video including sound. [Explanation of symbols]

[0240] 10. Imaging System 100 digital cameras 200 Information Support Terminal 210 Control Unit 220 Storage section 230 Operation section 240 Display section 250 Communications Department

Claims

1. An electronic device for managing video recording, a display unit that displays information; an input unit for inputting user operations; a control unit that controls the display unit in response to an operation input through the input unit, The control unit receiving an instruction to shoot a video at the input unit; Obtaining a result of a determination process for determining whether or not the video has been unsuccessfully shot based on the audio data of the video being shot; If the acquired result of the determination process is a failure in shooting, a warning message indicating a failure in shooting the video is displayed on the display unit. electronic equipment.

2. The determination process detects noise in the audio data and determines whether or not the video has been unsuccessfully shot based on the noise detection result. The electronic device according to claim 1 .

3. The determination process separates an audio component indicating the noise from the audio indicated by the audio data based on the audio data of the video, and detects the noise based on the audio component. The electronic device according to claim 2 .

4. The warning information includes at least one of the timing at which the noise was detected in the video and the volume level of the noise. The electronic device according to claim 2 .

5. further comprising a storage unit for storing management information for managing the video; When the management information includes information indicating a sound to be collected, the control unit determines an object to be regarded as noise in the sound data according to the information indicating the sound to be collected. The electronic device according to claim 1 .

6. The control unit controls the display unit to visualize a plurality of audio components separated into a plurality of types of audio represented by the audio data based on a user operation input to the input unit. The electronic device according to claim 1 .

7. The control unit sets a criterion for determining that the video has been unsuccessfully captured in the determination process based on a user operation input to the input unit. The electronic device according to claim 1 .

8. a storage unit for storing management information for managing moving images associated with one or more cuts in the scenario; The control unit receiving, via the input unit, an instruction to shoot a moving image in association with a specific cut among the one or more cuts; The result of the determination process for the video is obtained based on audio data in the video that is shot in association with the specific cut. The electronic device according to claim 1 .

9. a communication unit that performs data communication with the imaging device that performs the video shooting, The control unit manages the video captured by the imaging device through data communication via the communication unit. The electronic device according to claim 1 .

10. further comprising an imaging unit that captures an image of a subject and generates image data; The control unit manages a moving image formed from image data generated by the imaging unit. The electronic device according to claim 1 .

11. A shooting management method for managing video shooting, comprising: The control unit of the electronic device An instruction to shoot a video is received at the input unit, Obtaining a result of a determination process for determining whether or not the video has been unsuccessfully shot based on the audio data of the video being shot; If the acquired result of the determination process is a shooting failure, a warning message indicating the shooting failure of the video is displayed on the display unit. Shooting management methods.

12. A program for causing the control unit to execute the photography management method according to claim 11.

Images