Digest creation device, digest creation system, digest creation method, and digest creation program

Abstract

This improves convenience when creating a video digest. [Solution] The digest creation system 1 includes an imaging means capable of capturing moving images and an input means for accepting predetermined input operations, including the use of auxiliary functions to assist users in capturing moving images. The system assigns importance to each predetermined section of moving image data, and generates digest data from the moving image data that is shortened to only the important parts of the moving image, according to the importance assigned to each section. The importance is set based on whether or not the auxiliary functions are used.

Description

[Technical Field]

[0001] The present invention relates to a digest creation device, a digest creation system, a digest creation method, and a digest creation program for creating a digest of a moving image. [Background technology]

[0002] Conventionally, a digest data generation device has been proposed that stores content data including audio data and conference image data, determines feature amounts for each predetermined time section of the content data based on emotion tags input by conference participants, and determines importance from the feature amounts. Furthermore, a digest data generation device generates digest data from the content data by deleting conference image data for unimportant sections based on importance (see Patent Document 1).

[0003] In the digest data generating device described in Patent Document 1, information selected by the user from first information indicating positive emotions and second information indicating negative emotions is used as an evaluation signal, features are determined based on the first information and second information contained in the evaluation signal, and the importance of each section is determined from the features of each section.

[0004] However, the digest data generation device described in Patent Document 1 requires conference participants to input emotion tags in order to generate digest data. In other words, participants are forced to perform actions unrelated to the essential parts of the conference in order to generate digest data, and there is room for improvement in terms of convenience. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-068300 Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the above-mentioned problems, the present invention aims to provide a digest creation device, a digest creation system, a digest creation method, and a digest creation program that can improve convenience when creating a digest of a video. [Means for solving the problem]

[0007] This invention is characterized by a digest creation device, digest creation system, digest creation method, and digest creation program comprising: an imaging means capable of capturing moving images; an input means for accepting predetermined input operations including the use of auxiliary functions intended to assist a user in capturing moving images; a storage means for storing data of the moving images captured by the imaging means; an importance setting means for setting an importance level for each predetermined section of the moving image data; and a generation means for generating digest data from the moving image data that has been shortened to only the important parts of the moving image in accordance with the importance level set for each section, wherein the importance setting means is configured to set the importance level based on whether or not the auxiliary functions are used. [Effects of the Invention]

[0008] According to the present invention, it is possible to improve the convenience when creating a digest of a moving image. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of a digest creation system according to the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a user terminal. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of a management server. [Figure 4] FIG. 10 is an explanatory diagram showing an example of a screen configuration when a video recording function is used. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a screen configuration when using a function for cutting out images for editing while using a video recording function. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a screen configuration when using the drawing function while using the video recording function. [Figure 7] FIG. 10 is an explanatory diagram showing an example of a screen configuration when the pointer function is used while the video recording function is used. [Figure 8] FIG. 10 is an explanatory diagram showing an example of a screen configuration when using a character reading function while using a video recording function. [Figure 9] FIG. 10 is an explanatory diagram showing an example of a screen configuration when character reading is completed when using the video recording function. [Figure 10] FIG. 3 is a flowchart of a digest creation process according to the present invention. [Figure 11] FIG. 10 is an explanatory diagram showing an example of the structure of a report. [Figure 12] FIG. 10 is an explanatory diagram showing another example of the structure of a report. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention will now be described with reference to the drawings. <System configuration>

[0011] Fig. 1 is a block diagram showing an example of the system configuration of a digest creation system 1 of the present invention. Fig. 2 is a block diagram showing an example of the configuration of a user terminal 2. Fig. 3 is a block diagram showing an example of the configuration of a management server 3.

[0012] The digest creation system 1 of the present invention is a system for creating digest moving image data (hereinafter referred to as "digest data") that is abbreviated to only the important parts of moving image (moving image) data that includes audio data and imaging data.

[0013] As shown in Fig. 1, the digest creation system 1 has a user terminal 2 and a management server 3. The user terminal 2 is a terminal used by a user (a person who creates digest data) of the digest creation system 1. The management server 3 is a server for providing a service for creating digest data (digest creation service).

[0014] The user terminal 2 and the management server 3 are connected to each other so that they can communicate with each other via a predetermined communication line 4. Although one user terminal 2 is shown in the figure for the sake of convenience, the digest creation system 1 of the present invention can be used by multiple users and may have multiple user terminals 2.

[0015] The user terminal 2 is a general-purpose computer (user terminal) used by a user of this system. Examples include a desktop PC, a notebook (laptop) PC, a tablet PC, a smartphone, and a feature phone. The management server 3 is configured as a general-purpose server computer (computing device). The management server 3 may be configured as one (single) computing device or multiple computing devices. Furthermore, part or all of the management server 3 may be configured as a cloud server.

[0016] 2, the user terminal 2 includes a control unit 21, an input unit 22, a display unit 23, an imaging unit 24, an audio input unit 25, an audio output unit 26, a communication unit 27, and an auxiliary storage unit 28. The input unit 22, the display unit 23, the imaging unit 24, the audio input unit 25, the audio output unit 26, the communication unit 27, and the auxiliary storage unit 28 are each communicatively connected to the control unit 21. The control unit 21 of the user terminal 2 includes a calculation unit 21a and a main storage unit 21b, and executes various calculations and control operations in the user terminal 2.

[0017] 3, the management server 3 includes a control unit 31, a communication unit 32, and an auxiliary storage unit 33. The communication unit 32 and the auxiliary storage unit 33 are each communicatively connected to the control unit 31. The control unit 31 of the management server 3 includes a calculation unit 36 and a main storage unit 37, and executes various calculations and control operations in the management server 3.

[0018] 2 and 3 are arithmetic processing units including a CPU or an MPU. Each main memory unit (21b, 37) has a RAM (DRAM) or the like. The RAM is used as a work area and a buffer area for the arithmetic units (21a, 36).

[0019] The input unit 22 shown in FIG. 2 has input components that accept operation inputs from the user (user) of the user terminal 2, and an input detection circuit interposed between the input components and the control unit 21. The input components are, for example, a keyboard and / or a computer mouse, and if the input component is a keyboard, it includes hardware operation buttons or operation keys. A touch panel may also be used as the input component. Any type of touch panel can be used. The input detection circuit outputs operation signals or operation data (operation input data) to the control unit 21 in response to the operation of each input component.

[0020] The display unit 23 has a display and a display control circuit interposed between the display and the control unit 21. For example, an LCD (liquid crystal display) or an organic EL display can be used as the display. However, a touch panel display in which a touch panel and a display are integrally formed may also be used. In this case, software keys (operation keys) for accepting input operations are displayed in the display area of the display. Note that software keys refer to keys (icons) reproduced by software on the display surface of a display with a touch panel, for example. Hereinafter, software keys will be referred to as "icons."

[0021] The imaging unit 24 has an imaging element (image sensor), a focus lens, etc., and captures an image by converting imaging light (visible light) captured by the imaging element into an electrical signal. Examples of the imaging element include solid-state imaging elements such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 24 is capable of capturing moving images, and imaging data of the moving images output from the imaging unit 24 is stored in the main memory unit 21b or the auxiliary memory unit 28. In addition, the imaging data output from the imaging unit 24 can be displayed in real time on the display unit 23 (display) as display image data.

[0022] The audio input unit 25 is composed of a sound collector, an A / D converter, and a signal amplifier. The sound collector has a microphone that converts sounds such as voice into electrical signals. The sound input through the sound collector is A / D converted by the A / D converter and then amplified by the signal amplifier, generating digital audio signal data (audio data) corresponding to the input sound. The sound collector may be a general-purpose microphone such as a condenser microphone or a dynamic microphone. The sound collector may also be configured to be externally connected to the audio input unit 25 (a configuration separate from the audio input unit 25, a so-called external microphone). When video is recorded, the audio data generated by the audio input unit 25 is synchronized with the imaging data of the video output from the imaging unit 24 and stored in the main memory unit 21b or the auxiliary memory unit 28 as audio-accompanying video data (hereinafter referred to as "audio-accompanying video data").

[0023] The audio output unit 26 includes a speaker, a D / A converter, an amplifier, and the like. The D / A converter and the amplifier convert a digital audio signal serving as audio data (audio data) into an analog signal, amplify the signal, and output it to the speaker. The speaker receives the analog signal, vibrates the air, and outputs the audio. The speaker may be a general-purpose speaker such as a capacitor speaker or a dynamic speaker. For example, when audio-accompanied video data is played on the user terminal 2, the video is displayed on the display unit 23, and audio is output from the audio output unit 26.

[0024] The communication units (27, 32) shown in Figures 2 and 3 have a communication circuit for connecting to the communication line 4. The communication circuit is a wired communication circuit or a wireless communication circuit, and communicates with an external computer via the communication line 4 in accordance with instructions from the calculation units (21a, 36).

[0025] The auxiliary memory units (28, 33) are composed of non-volatile memory such as HDD, SSD, flash memory, EEPROM, etc., and store programs for each calculation unit (21a, 36) to control the operation of the user terminal 2 and the management server 3, as well as various data necessary for using this system.

[0026] 2, the auxiliary storage unit 28 of the user terminal 2 stores a user program 28a for executing various operations of the user terminal 2 in the system in response to operational inputs from the user, and user data 28b necessary for using the system. The user program 28a and user data 28b are read from the auxiliary storage unit 28 as needed and stored (deployed) in the main storage unit 21b. The operation of the user terminal 2 is realized by the calculation unit 21a executing the user program 28a deployed in the main storage unit 21b.

[0027] The user program 28a includes at least a communication program for accessing (establishing communication with) an external computer (e.g., management server 3), a display program for displaying various screens on display unit 23, an input program for accepting input operations from the user via input unit 22, an imaging program for generating captured imaging data by controlling imaging unit 24, a voice input program for generating voice data from voice input through voice input unit 25, a voice output program for outputting voice from voice output unit 26 in accordance with the input voice data, a voice-accompanied video generation program for simultaneously controlling imaging unit 24 and voice input unit 25 to generate voice-accompanied video data, an auxiliary program for performing the functions of predetermined auxiliary functions (assist functions) when capturing voice-accompanied videos, an event data storage program for storing event data, which is data of events executed when capturing voice-accompanied videos such as the usage history of the auxiliary functions, in association with the voice-accompanied video data, and a program for selecting and executing various functions of user terminal 2. The user data 28b includes at least user data, voice-accompanied video data, etc., required in this system.

[0028] 3, the auxiliary storage unit 33 of the management server 3 stores a management program 34 for executing various operations of the management server 3, and management data 35 necessary for using the system. The management program 34 and management data 35 are read from the auxiliary storage unit 33 as needed and temporarily stored in the main storage unit 37. The operation of the management server 3 is realized when the calculation unit 36 executes the management program 34 temporarily stored in the main storage unit 37.

[0029] The management program 34 includes at least a main processing program 34a for selecting and executing various functions provided by the management server 3, a video acquisition program 34b for acquiring audio-accompanied video data from the user terminal 2, an importance setting program 34c for setting the importance of each specified section of the audio-accompanied video data, a digest generation program 34d for generating digest video data (digest data) that has been shortened to only the important parts of the video data according to the importance set for each section of the audio-accompanied video data, a character conversion program 34e for converting the audio contained in the audio-accompanied video data into character data, a summary character generation program 34f for generating summary character data that summarizes the character data according to the importance of each section to which the audio belongs, and a report output program 34g for outputting report data by adapting the summary character data to a specified format.

[0030] The management data 35 includes audio-accompanied video data 35a obtained from the user terminal 2, event data 35b which is data on events performed during imaging such as the usage history of auxiliary functions, importance data 35c which is data on the importance of each specified section in the audio-accompanied video data, digest data 35d which is data on a digest video, converted text data 35e which is data converted from the audio contained in the video data into text data, summarized text data 35f which summarizes the converted text data according to the importance of each section to which the audio belongs, and report data 35g which is data on a report generated by fitting the summarized text data to a specified format. This predetermined format preferably includes multiple items: a title (in the illustrated example, the work title), a summary, a creation date (the date the video was shot), item names (topics in the illustrated example) when the video is divided into predetermined blocks such as topics, a summary of each item (content in the illustrated example), a digest video title (e.g., "Self-introduction and conversation start" or "Explanation of the water heater problem" in the illustrated example), a link to play the digest video, decisions (items decided in the video), to-dos (items that need to be done in the future in the video), and the next schedule (year, month, and date of the next schedule decided in the video). Saving multiple items as text data in this way makes it easy to later find the desired report data 35g by text search. This also makes it easy to create or find work manuals or instructional videos from the report data 35g. Furthermore, the summary text data 35f contains or is linked to time information (time information, timestamp) indicating the time at which the corresponding audio in the audio-accompanied video belongs. That is, it is possible to identify the playback scene (specified number of seconds) of the audio-accompanying video data corresponding to the summary text data 35f.

[0031] The above-described configuration of the user terminal 2 and the configuration of the digest creation system 1 including the management server 3 are merely examples, and are not limited to these and can be appropriately changed as needed. <System operation example>

[0032] The following describes an example of the operation of the digest creation system 1. First, a brief description will be given of the generation of audio-accompanying video data, which is a premise for explaining the example of the operation of the digest creation system 1.

[0033] For example, audio-accompanied video data is generated by a function of recording audio-accompanied video data provided in accordance with an audio-accompanied video data generation program in the user terminal 2 (hereinafter simply referred to as the "video recording function"). Audio-accompanied video data can be used as explanatory video data for confirming inspection work on electrical equipment, etc., explaining business manuals, explaining how to use electrical equipment, viewing real estate, explaining cooking methods, medical examinations, sports instruction, tourist guides, etc. It is also possible to create digest videos of business manuals, explanatory videos, etc. from video data of inspection work or video data explaining how to use the equipment by using a digest creation process.

[0034] The video recording function of the user terminal 2 is used, for example, when an application for capturing images (a so-called camera application) or an application for video calling (a so-called video calling application) is started.

[0035] When the camera application is activated, for example, the video recording function can be used independently on one user terminal 2. In this case, the audio-accompanying video data captured (recorded) by the camera application (video recording function) is temporarily stored in the user terminal 2. Thereafter, the management server 3 acquires the audio-accompanying video data in response to a user operation or automatically.

[0036] For example, cases in which the video recording function is used independently on one user terminal 2 include creating an explanatory video for a business manual, creating an explanatory video for cooking methods, and creating a video for an administrator to check inspection work on electrical equipment, etc.

[0037] Furthermore, when a video call app is launched, for example, a video recording function can be used on one or both of two or more (plural) user terminals 2. In this case, audio-accompanied video data is generated by recording the video call. At this time, when multiple user terminals 2 make a video call via the management server 3, the management server 3 can automatically acquire the audio-accompanied video data. On the other hand, when multiple user terminals 2 make a video call via peer-to-peer communication, the audio-accompanied video data is temporarily stored in one or both user terminals 2, and the management server 3 acquires the audio-accompanied video data from the user terminal 2 either automatically or in response to a user's operation. For example, examples of cases in which the present invention can be used during a video call include a video call between a field worker (serviceman) and a manager (operator) when inspecting electrical equipment, or a video call between a real estate agent and a user when viewing a property.

[0038] FIG. 4 is an explanatory diagram showing an example of a screen configuration when using the video recording function. FIG. 5 is an explanatory diagram showing an example of a screen configuration when using the editing image cropping function when using the video recording function. FIG. 6 is an explanatory diagram showing an example of a screen configuration when using the drawing function when using the video recording function. FIG. 7 is an explanatory diagram showing an example of a screen configuration when using the pointer function when using the video recording function. FIG. 8 is an explanatory diagram showing an example of a screen configuration when using the character reading function when using the video recording function. FIG. 9 is an explanatory diagram showing an example of a screen configuration when character reading is completed when using the video recording function.

[0039] An embodiment of the present invention will be described below with reference to Figures 4 to 9. In the embodiment shown in Figures 4 to 9, an example will be described in which the video recording function is used independently in one user terminal 2.

[0040] 4, when the video recording function is used, the imaging unit 24 and the audio input unit 25 are activated, and the image display screen 100 is displayed on the display unit 23 (display). When the video recording function is used, audio-accompanying moving image data, which is a recording of part or all of the display area of the image display screen 100, is stored in parallel with the display of the image display screen 100. In this embodiment, audio-accompanying moving image data, which is a recording of the entire display area of the image display screen 100, is stored.

[0041] The image display screen 100 is a screen for displaying moving images or still images according to the imaging data generated by the imaging unit 24. The image display screen 100 has a real-time display section 110 and an editing image display section 120 that is displayed alongside the real-time display section 110.

[0042] The real-time display unit 110 is arranged from the center of the left side of the display area of the image display screen 100, and the image display unit for editing 120 is arranged from the center of the left side of the display area of the image display screen 100 to the right side of the display area of the image display screen 100. In other words, the real-time display unit 110 and the image display unit for editing 120 are arranged side by side on the left and right.

[0043] The real-time display unit 110 displays a real-time moving image (preview image) corresponding to the moving image captured by the imaging unit 24. In this embodiment, the real-time display unit 110 displays a real-time moving image 110a including an image of a predetermined electrical device.

[0044] Also, in the display area of the real-time display unit 110 (in this embodiment, below the display area of the real-time display unit 110), an editing image clipping icon 111 is displayed, to which an editing image clipping function is assigned that clips a still image at a predetermined timing from a moving image being captured as an image for editing. The editing image clipping function is a type of auxiliary function intended to assist the user in capturing a moving image.

[0045] When the editing image cutout function is used (when the editing image cutout icon 111 is selected), the editing image display unit 120 displays an editing image cut out at a predetermined timing from the moving image being captured. That is, the editing image display unit 120 displays a still image cut out as an editing image from the image displayed when the editing image cutout icon 111 was selected. For example, when the editing image cutout icon 111 is selected in the state shown in FIG. 4, the editing image display unit 120 displays an editing image 120a copied from the real-time moving image 110a, as shown in FIG.

[0046] Furthermore, a plurality of auxiliary icons 121 to which auxiliary functions are assigned are displayed within the display area of the editing image display unit 120 (in this embodiment, at the bottom of the display area of the editing image display unit 120). In this embodiment, the auxiliary icons 121 displayed include a drawing icon 121a to which a drawing function for drawing on the editing image 120a is assigned, a pointer display icon 121b to which a pointer display function for displaying a predetermined pointer image is assigned, and a character reading icon 121c to which a character reading function for performing optical character recognition (OCR) processing on characters included in the editing image 120a is assigned. Each of the drawing function, the pointer display function, and the character reading function is a type of auxiliary function.

[0047] When the drawing icon 121a is selected, the drawing function is enabled, allowing freehand drawing of lines and pasting (drawing) of predetermined shapes such as circles, squares, and arrows within the display area of the editing image display unit 120. This drawing function displays the drawing status in real time on the drawing user's terminal as well as the other party's terminal. Therefore, for example, if a line is drawn freehand, the line is drawn in real time on the other party's terminal, and if a shape is selected and placed, the shape is also placed in real time on the other party's terminal. As shown in FIG. 6, the drawn object (drawing object) 122 is displayed superimposed on top of the editing image 120a. Furthermore, when the drawing function is enabled, the drawing icon 121a corresponding to the drawing function is highlighted. Examples of highlighting include adding a pattern to the icon, adding an appropriate color (a color not used elsewhere on the image display screen 100) to the icon, underlining the text of the icon, or displaying the text in bold. The same is true for the pointer display icon 121b and the character reading icon 121c, which are highlighted when the corresponding function is enabled.

[0048] 7, when the pointer display icon 121b is selected, the pointer display function is enabled and a pointer image 123 is displayed in the display area of the editing image display unit 120. The pointer image 123 moves in response to a computer mouse operation or a touch operation. For example, the pointer image 123 is an image of a graphic imitating an arrow or a graphic imitating the shape of a human finger.

[0049] When the character reading icon 121c is selected, the character reading function is enabled. For example, as shown in FIG. 8, assume that the character reading icon 121c is selected while the characters "ABCDE" written on the surface of the subject's electrical equipment are displayed in the display area of the editing image display unit 120. In this case, the characters "ABCDE" are recognized. If the character recognition is successful, a result window 124 showing the character recognition results is displayed as shown in FIG. 9. The result window 124 displays a character recognition result display unit 124a that displays the recognized characters, a copy icon 124b that is assigned a function of copying the recognized characters as text data, and a save icon 124c that is assigned a function of saving (storing) the recognized characters as text data in the auxiliary storage unit 28 or the like.

[0050] In this way, the user of the user terminal 2 can use the auxiliary function as appropriate while using the video recording function. For example, when an important part (important portion) of a subject is displayed, the user can cut out an image for editing and use the drawing function to draw an object indicating the important part so that the important part is easily recognizable in the image for editing, or use the pointer display function to indicate the important part with the pointer image 123. Also, if text is attached to an important part of the subject, the text in the important part can be recognized using the text reading function and the text data can be saved. In other words, the time when the auxiliary function is used and the period before and after that time are likely to be important periods in the audio-accompanying video data.

[0051] When the video recording function is ended (stopped), the audio-accompanying video data is temporarily stored in the auxiliary storage unit 28 of the user terminal 2. At this time, event data, which is data on events that occurred when the video was captured, such as the usage history of the auxiliary functions, is linked to the audio-accompanying video data and stored in the auxiliary storage unit 28. The event data includes data on when the auxiliary functions were used, and is data that can identify which type of auxiliary function was used and when.

[0052] Fig. 10 is a flow diagram of the process of creating digest data (digest creation process) of the present invention. The digest creation process of the present invention will be described below with reference to Fig. 10. The digest creation process is basically performed by the management server 3 (control unit 31), but may also be performed by the user terminal 2 (control unit 21). In this embodiment, the digest creation process is executed by the management server 3 (control unit 31).

[0053] When the management server 3 starts the digest creation process, it acquires audio-accompanied video data from the user terminal 2 (auxiliary memory unit 28) (step S1), and acquires event data linked to the audio-accompanied video data acquired in the previous step S1 from the user terminal 2 (auxiliary memory unit 28) (step S2).

[0054] Next, the audio-accompanying video data is divided into multiple sections, and the importance of each section is set (step S3). Then, digest video data (digest data) is generated that is shortened to only the important parts of the audio-accompanying video data according to the importance set for each section (step S4).

[0055] However, when setting the importance of each section (step S3) and when generating digest data (step S4), known techniques can be used, such as a model with a learning function, so-called AI (artificial intelligence).Since artificial intelligence generally has a learning function to realize intelligent functions such as judgment, it can set more appropriate digest data as it is used more frequently.

[0056] Furthermore, in step S3, the importance is set based on whether or not an auxiliary function was used. That is, a high importance is set for a section in which an auxiliary function was used. For example, when generating digest data using artificial intelligence, various factors such as a change in subject (scene), a change in speaker, and a peak in volume are comprehensively taken into consideration in addition to whether or not an auxiliary function was used. In this embodiment, the use of an auxiliary function is weighted so that it has a greater influence on the setting of the importance than other factors. Therefore, a section in which an auxiliary function was used is more likely to be included in the digest data.

[0057] Furthermore, different weights may be assigned to the importance of each auxiliary function. For example, weights can be set so that the use of an image cropping function for editing or a drawing function contributes more importance than the use of a pointer display function or a character reading function. In this way, for example, a portion of an explanation using a drawing function can be evaluated as important, and a digest video can be generated that does not cut out that portion. This makes it possible to pinpoint the appropriate video when reviewing it later, without any interruptions or other stress.

[0058] 10, following step S4, converted character data is generated from the audio included in the video data (step S5), and summary character data is generated by summarizing the converted character data according to the importance of each segment to which the audio belongs (step S6). However, when generating the converted character data (step S5) and when generating the summary character data (step S6), known techniques, such as artificial intelligence, can be used.

[0059] Next, report data is generated using the summary text data (step S7), and the report data is output (step S8), completing the digest creation process. The report data can be in a format specified by the user and is output in a predetermined electronic document format according to the specified format. Alternatively, the report data and digest data may be linked and output. Furthermore, the report format data includes report configuration data (metadata) such as the title, summary, creation date, topic names (item names) when the video is divided into predetermined blocks such as topics, a summary (content) of each item, the title of the digest video (e.g., "Self-introduction and conversation start" or "Explanation of the water heater problem"), a link to play the digest video, data on each item to be included in the report, such as decisions, to-dos, and next schedule, and location data (placement data) of each item. In step S7, text to be included in each item is selected based on the report configuration data, and the entry column for each item in the report is placed in a predetermined position according to the location information. Furthermore, the report configuration data (data on each item and location data) can be customized by the user.

[0060] FIG. 11 is an explanatory diagram showing an example of the structure of the report 130. FIG. 12 is an explanatory diagram showing another example of the structure of the report 130. For example, as shown in FIG. 11, the report contains summary text for items such as the creation date, subject, summary, reporter, participants, topic and its content, decisions, to-do list, and next schedule. Here, the summary text data is a summary of the converted text data according to the importance of each section to which the audio belongs. For this reason, there is a high possibility that the content of the report 130 will be linked to the digest data.

[0061] Furthermore, when report data and digest data are linked and output, as shown in FIG. 12, in addition to the items shown in FIG. 11, a "digest" item (digest display section) 131 may be added to the report 130. In this case, the contents of the digest display section 131 are displayed in conjunction with the digest data. For example, the digest display section 131 may divide the digest data into multiple sections, displaying text outlining each section and a link (video link) 131a for playing the digest data from the start of each section. That is, the report 130 is configured such that, for each piece of summary text data, a video link 131a is provided for playing a video from the section corresponding to the summary text data. As described above, the summary text data 35f either has time information to which the corresponding audio belongs or is linked to the time information, and the video link 131a specifying the playback scene (specified number of seconds) is generated using this time information.

[0062] It should be noted that if report data is not to be output, that is, if only digest data is to be generated, steps S5 to S8 of the digest creation process can be omitted.

[0063] As described above, the digest creation system 1 of the present invention sets the importance level based on whether or not an auxiliary function was used, so it can automatically generate digest data that is likely to include timings when an auxiliary function was used, which are likely to be important periods in audio-accompanied video data. Furthermore, the auxiliary function is intended solely to assist the user in capturing a video, and is not a function that requires subsequent operation to create digest data. Therefore, the user simply needs to record the video while using the auxiliary function according to their purpose. This improves the convenience of creating video digests.

[0064] Furthermore, according to the present invention, since the importance is determined based on whether or not the editing image cropping function is used, it is possible to automatically generate digest data that is likely to include the timing at which the editing image was cropped. For example, in explanatory videos, the timing at which the editing image was cropped is often an important scene. Therefore, it is possible to easily generate digest data that is likely to include important scenes.

[0065] Furthermore, according to the present invention, since the importance is determined based on whether or not the drawing function is used, digest data that is likely to include the timing of drawing in the image for editing can be automatically generated. In explanatory moving images, the timing of drawing in the image for editing is often an important scene. Therefore, digest data that is likely to include important scenes can be easily generated.

[0066] Furthermore, according to the present invention, summary text data is generated according to the importance of each section to which the audio belongs, and report data is generated using the summary text data, so that report data that is likely to be linked in content to digest data can be generated, i.e., digest data and report data that are likely to include important scenes can be easily generated.

[0067] Furthermore, according to the present invention, a link is provided as part of the report data configuration for playing back video from the section corresponding to the summary text data, so that video can be played back from any scene when viewing the report, which is convenient.

[0068] The digest creation system of this invention corresponds to digest creation system 1 of the above-described embodiment, the user terminal corresponds to user terminal 2, the management server corresponds to management server 3, the imaging means corresponds to imaging unit 24, the input means corresponds to input unit 22, the storage means corresponds to auxiliary storage unit 28 or auxiliary storage unit 33, the importance setting means corresponds to importance setting program 34c and control unit 31 operating in accordance with the same, the generation means corresponds to digest generation program 34d and control unit 31 operating in accordance with the same, the character conversion means corresponds to character conversion program 34e and control unit 31 operating in accordance with the same, the summary character generation means corresponds to summary character generation program 34f and control unit 31 operating in accordance with the same, and the report output means corresponds to report output program 34g and control unit 31 operating in accordance with the same, but this invention is not limited to this embodiment and can be embodied in various other ways. Furthermore, the screens and specific configurations described in the above-described embodiment are merely examples and can be modified as appropriate depending on the actual product.

[0069] For example, in the above embodiment, an example was given in which the video recording function was used independently on one user terminal 2, but the video recording function can also be used when multiple user terminals 2 are making a video call. In this case, the auxiliary function can be used on at least one of the multiple user terminals 2, and the auxiliary function can also be used on all user terminals 2.

[0070] In the above embodiment, the digest creation process is performed by the management server 3 (control unit 31), but the digest creation process may be performed by the user terminal 2 (control unit 21). In this case, the user terminal 2 corresponds to the digest creation device of the present invention. [Industrial Applicability]

[0071] The present invention can be used in industries such as creating digests of moving images. [Explanation of symbols]

[0072] 1. Digest creation system 2...User terminal 3...Administrative Server 31...Control unit 33…Auxiliary storage unit

Claims

1. an imaging means capable of capturing moving images; an input means for receiving a predetermined input operation including the use of an auxiliary function for the purpose of assisting a user in capturing a moving image; a storage means for storing data of moving images captured by the imaging means; an importance setting means for setting an importance level for each predetermined section of the moving image data; generating means for generating digest data from the data of the video, the digest data being shortened to include only important parts of the video, in accordance with the importance level set for each of the segments; the importance setting means is configured to set the importance based on whether or not the auxiliary function is being used; the auxiliary function has an editing image cutting function for cutting out a still image at a predetermined timing from a moving image being captured by the imaging means as an editing image, The importance setting means is configured to determine the importance based on whether or not the editing image cropping function is used. Digest creation device.

2. the auxiliary function has a drawing function for drawing on the image for editing, The importance setting means is configured to determine the importance based on whether or not the drawing function is used.

2. The digest creation device according to claim 1.

3. a character conversion means for converting the voice included in the moving image data into character data; a summary character generating means for generating summary character data by summarizing the character data in accordance with the importance of each of the sections to which the voice belongs; and a report output means for outputting report data by adapting the summarized character data into a predetermined format.

3. The digest creation device according to claim 1 or 2.

4. The report data is configured such that, for each piece of summary character data, a link is provided for playing back a moving image from the section corresponding to the summary character data.

4. The digest creation device according to claim 3.

5. A digest creation system including a user terminal and a management server capable of communicating with the user terminal, The user terminal an imaging means capable of capturing moving images; an input means for receiving a predetermined input operation including the use of an auxiliary function for the purpose of assisting a user in capturing a moving image; The management server a storage means for storing data of moving images captured by the imaging means; an importance setting means for setting an importance level for each predetermined section of the moving image data; generating means for generating digest data from the data of the video, the digest data being shortened to include only important parts of the video, in accordance with the importance level set for each of the segments; the importance setting means is configured to set the importance based on whether or not the auxiliary function is being used; the auxiliary function has an editing image cutting function for cutting out a still image at a predetermined timing from a moving image being captured by the imaging means as an editing image, The importance setting means is configured to determine the importance based on whether or not the editing image cropping function is used. Digest creation system.

6. A digest creation method for a digest creation device including an imaging means capable of capturing a moving image, an input means for receiving a predetermined input operation including use of an auxiliary function for assisting a user in capturing the moving image, and a storage means for storing data of the moving image captured by the imaging means, comprising: a setting step of setting a level of importance for each predetermined section of the moving image data; and generating digest data from the data of the video, the digest data being shortened to include only important parts of the video, in accordance with the importance level set for each section, the auxiliary function has an editing image cutting function for cutting out a still image at a predetermined timing from a moving image being captured by the imaging means as an editing image, In the setting step, the importance is set based on whether or not the editing image cropping function is used. Digest creation method.

7. A computer of a digest creation device including an imaging means capable of capturing a moving image, an input means for receiving a predetermined input operation including the use of an auxiliary function for assisting a user in capturing the moving image, and a storage means for storing data of the moving image captured by the imaging means, an importance setting means for setting an importance level for each predetermined section of the moving image data; a generating unit that generates digest data from the data of the video, the digest data being shortened to include only important parts of the video, in accordance with the importance level set for each of the segments; the auxiliary function has an editing image cutting function for cutting out a still image at a predetermined timing from a moving image being captured by the imaging means as an editing image, The importance setting means is configured to set the importance based on whether or not the editing image cropping function is used. Digest creation program.

Images