Digest creation device, digest creation system, digest creation method, and digest creation program

Abstract

This can improve the convenience of creating video digests. [Solution] The digest creation system 1 comprises an imaging means capable of capturing moving images and an input means that accepts predetermined input operations, including the use of an auxiliary function intended to assist the user in capturing moving images. It sets an importance level for each predetermined section of the moving image data and generates digest data from the moving image data, shortened to include only the important parts of the moving image according to the importance level set for each section. The importance level is set based on whether or not the auxiliary function is used.

Description

Technical Field

[0001] This invention relates to a digest creation device, a digest creation system, a digest creation method, and a digest creation program for creating a digest of moving images.

Background Art

[0002] Conventionally, content data including audio data and conference image data is stored, and based on emotion tags input from conference participants, feature amounts are determined for each predetermined time interval of the content data, and importance levels are determined from the feature amounts. Further, a digest data generation device has been proposed that generates digest data from content data by deleting conference image data in unimportant intervals based on the importance level (see Patent Document 1).

[0003] In the digest data generation device described in Patent Document 1, information selected by the user from the first information indicating positive emotions and the second information indicating negative emotions is used as an evaluation signal, and feature amounts are determined based on the first information and the second information included in the evaluation signal, and the importance level for each interval is determined from the feature amounts for each interval.

[0004] However, in the digest data generation device described in Patent Document 1, it is necessary for conference participants to perform an emotion tag input operation in order to generate digest data. That is, it forces participants to perform an operation that has nothing to do with the essential part of the conference in order to generate digest data, and there is room for improvement from the perspective of convenience.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] In view of the above-mentioned problems, this invention aims to provide a digest creation device, a digest creation system, a digest creation method, and a digest creation program that can improve the convenience of creating video digests. [Means for solving the problem]

[0007] This invention is a digest creation device, digest creation system, digest creation method, and digest creation program comprising: an imaging means capable of capturing moving images; an input means for receiving predetermined input operations, including the use of an auxiliary function intended to assist the user in capturing moving images; a storage means for storing data of moving images captured by the imaging means; an importance setting means for setting the importance of predetermined sections for the data of the moving images; and a generation means for generating digest data shortened to include only the important parts of the moving images according to the importance set for each section, wherein the importance setting means is configured to set the importance based on whether or not the auxiliary function is used. [Effects of the Invention]

[0008] This invention can improve the convenience of creating video digests. [Brief explanation of the drawing]

[0009] [Figure 1] An explanatory diagram showing an example of the configuration of the digest creation system of this invention. [Figure 2] A block diagram showing an example of the configuration of a user terminal. [Figure 3] A block diagram showing an example of the configuration of the management server. [Figure 4] An explanatory diagram showing an example of the screen layout when using the video recording function. [Figure 5] This diagram illustrates an example of the screen layout when using the video recording function and the image extraction function for editing. [Figure 6] An explanatory diagram showing an example of the screen configuration when using the video recording function and the drawing function. [Figure 7] An explanatory diagram showing an example of the screen configuration when using the video recording function and the pointer function. [Figure 8] An explanatory diagram showing an example of the screen configuration when using the video recording function and the text recognition function. [Figure 9] An explanatory diagram showing an example of the screen layout when text recognition is complete while using the video recording function. [Figure 10] Flowchart of the digest creation process of the present invention. [Figure 11] An explanatory diagram showing an example of report structure. [Figure 12] An explanatory diagram showing other examples of report structure. [Modes for carrying out the invention]

[0010] Hereinafter, one embodiment of this invention will be described with reference to the drawings. <System Configuration>

[0011] Figure 1 is a block diagram showing an example of the system configuration of the digest creation system 1 of the present invention. Figure 2 is a block diagram showing an example of the configuration of the user terminal 2. Figure 3 is a block diagram showing an example of the configuration of the management server 3.

[0012] The digest creation system 1 of the present invention is a system for creating digest video data (hereinafter referred to as "digest data") which is shortened to include only the important parts of moving image (video) data, including audio data and image data.

[0013] As shown in Figure 1, the digest creation system 1 has a user terminal 2 and a management server 3. The user terminal 2 is a terminal used by users of the digest creation system 1 (those who create digest data). The management server 3 is a server that provides a service for creating digest data (digest creation service).

[0014] Each of the user terminal 2 and the management server 3 is communicably connected to each other via a predetermined communication line 4. Although only one user terminal 2 is illustrated for convenience of explanation, the digest creation system 1 of the present invention can be used by a plurality of users and may have a plurality of user terminals 2.

[0015] The user terminal 2 is a general-purpose computer (user terminal) used by a user of the present system. For example, it is a desktop PC, a notebook (laptop) PC, a tablet PC, a smartphone, a feature phone, or the like. The management server 3 is composed of a general-purpose server computer (computing device). Note that the management server 3 may be composed of one (single) computing device or may be composed of a plurality of computing devices. Also, part or all of the management server 3 may be composed of a cloud server.

[0016] As shown in FIG. 2, the user terminal 2 includes a control unit 21, an input unit 22, a display unit 23, an imaging unit 24, an audio input unit 25, an audio output unit 26, a communication unit 27, and an auxiliary storage unit 28. Each of the input unit 22, the display unit 23, the imaging unit 24, the audio input unit 25, the audio output unit 26, the communication unit 27, and the auxiliary storage unit 28 is communicably connected to the control unit 21. The control unit 21 of the user terminal 2 has an arithmetic unit 21a and a main storage unit 21b, and executes various arithmetic and control operations in the user terminal 2.

[0017] As shown in FIG. 3, the management server 3 includes a control unit 31, a communication unit 32, and an auxiliary storage unit 33. Each of the communication unit 32 and the auxiliary storage unit 33 is communicably connected to the control unit 31. The control unit 31 of the management server 3 has an arithmetic unit 36 and a main storage unit 37, and executes various arithmetic and control operations in the management server 3.

[0018] The arithmetic units (21a, 36) shown in Figures 2 and 3 are arithmetic processing units including a CPU or MPU. Each main memory unit (21b, 37) has RAM (DRAM), etc. The RAM is used as the work area and buffer area of ​​the arithmetic units (21a, 36).

[0019] The input unit 22 shown in Figure 2 includes an input component that receives operation input from the user of the user terminal 2, and an input detection circuit interposed between the input component and the control unit 21. The input component is, for example, a keyboard and / or a computer mouse, and if the input component is a keyboard, it includes hardware operation buttons or operation keys. Alternatively, a touch panel may be used as the input component. Any type of touch panel can be used. The input detection circuit outputs operation signals or operation data (operation input data) corresponding to the operation of each input component to the control unit 21.

[0020] The display unit 23 includes a display and a display control circuit interposed between the display and the control unit 21. For example, an LCD (liquid crystal display) or an organic EL display can be used as the display. However, a touch panel display, in which the touch panel and the display are integrally formed, may also be used. In this case, software keys (operation keys) for accepting input operations are displayed in the display area of ​​the display. A software key refers to a key (icon) that is software-reproduced on the display surface of a touch panel display, for example. Hereinafter, software keys will be referred to as "icons".

[0021] The imaging unit 24 includes an image sensor and a focus lens, and performs imaging by converting the imaging light, which is visible light captured from the image sensor, into an electrical signal. Examples of image sensors include solid-state image sensors such as CCD (Charge Coupled Device) image sensors or CMOS (Complementary Metal Oxide Semiconductor) image sensors. The imaging unit 24 is capable of capturing moving images, and the imaging data of the moving images output from the imaging unit 24 is stored in the main memory unit 21b or the auxiliary memory unit 28. Furthermore, the imaging data output from the imaging unit 24 can be displayed in real time as display image data on the display unit 23 (display).

[0022] The audio input unit 25 consists of a sound collector, an A / D converter, and a signal amplifier. The sound collector has a microphone that converts sound, such as speech, into electrical signals. The sound input through the sound collector is A / D converted by the A / D converter, and then the signal is amplified by the signal amplifier to generate digital audio signal data (audio data) corresponding to the input speech. As the sound collector, a general-purpose microphone such as a condenser microphone or a dynamic microphone can be used. Alternatively, the sound collector may be connected externally to the audio input unit 25 (a separate configuration from the audio input unit 25, a so-called external microphone). Furthermore, when video is recorded, the audio data generated by the audio input unit 25 is synchronized with the video image data output from the imaging unit 24 and stored in the main memory unit 21b or auxiliary memory unit 28 as video image data with audio (hereinafter referred to as "video image data with audio").

[0023] The audio output unit 26 includes a speaker, a D / A converter, and an amplifier. The D / A converter and amplifier convert the digital audio signal, which is audio data, into an analog signal, amplify it, and output it to the speaker. The speaker receives this analog signal, vibrates the air, and outputs sound. General-purpose speakers such as condenser speakers and dynamic speakers can be used as the speaker. For example, when video data with sound is played on the user terminal 2, the video is displayed on the display unit 23, and sound is output from the audio output unit 26.

[0024] The communication units (27, 32) shown in Figures 2 and 3 have communication circuits for connecting to the communication line 4. The communication circuits are either wired or wireless communication circuits, and they communicate with an external computer via the communication line 4 according to instructions from the arithmetic units (21a, 36).

[0025] The auxiliary storage units (28, 33) consist of non-volatile memory such as HDDs, SSDs, flash memory, and EEPROMs, and each arithmetic unit (21a, 36) stores programs for controlling the operation of the user terminal 2 and the management server 3, as well as various data necessary for using this system.

[0026] As shown in Figure 2, the auxiliary storage unit 28 of the user terminal 2 stores a user program 28a for executing various operations of the user terminal 2 in this system in response to user input, and user data 28b necessary for using this system. The user program 28a and user data 28b are read from the auxiliary storage unit 28 as needed and stored (deployed) in the main storage unit 21b. The operation of the user terminal 2 is realized by the arithmetic unit 21a executing the user program 28a that has been deployed in the main storage unit 21b.

[0027] The user program 28a includes at least a communication program for accessing an external computer (e.g., management server 3) (establishing communication with the external computer), a display program for displaying various screens on the display unit 23, an input program for receiving input operations from the user via the input unit 22, an imaging program for controlling the imaging unit 24 to generate captured imaging data, an audio input program for generating audio data from audio input via the audio input unit 25, an audio output program for outputting audio from the audio output unit 26 according to the input audio data, an audio-enabled video generation program for simultaneously controlling the imaging unit 24 and the audio input unit 25 to generate audio-enabled video data, an auxiliary program for performing predetermined auxiliary functions (assist functions) when capturing audio-enabled video, an event data storage program for storing event data, which is data of events executed when capturing audio-enabled video such as the usage history of auxiliary functions, linked to audio-enabled video data, and a program for selecting and executing various functions provided by the user terminal 2. The user data 28b includes at least user data and audio-enabled video data necessary for this system.

[0028] As shown in Figure 3, the auxiliary storage unit 33 of the management server 3 stores a management program 34 for executing various operations of the management server 3, and management data 35 necessary for using this system. The management program 34 and management data 35 are read from the auxiliary storage unit 33 as needed and temporarily stored in the main memory unit 37. The operation of the management server 3 is realized by the arithmetic unit 36 ​​executing the management program 34 temporarily stored in the main memory unit 37.

[0029] The management program 34 includes at least a main processing program 34a for selecting and executing various functions provided by the management server 3, a video acquisition program 34b for acquiring video data with audio from the user terminal 2, an importance setting program 34c for setting importance levels for predetermined sections of video data with audio, a digest generation program 34d for generating digest video data (digest data) that is shortened to include only the important parts of the video according to the importance level set for each section of the video data with audio, a character conversion program 34e for converting audio contained in video data with audio into character data, a summary character generation program 34f for generating summary character data that summarizes the character data according to the importance level of each section to which the audio belongs, and a report output program 34g for outputting report data in a predetermined format from the summary character data.

[0030] The management data 35 includes video data with audio 35a acquired from the user terminal 2, event data 35b which is data of events performed during imaging such as the usage history of auxiliary functions, importance data 35c which is data of the importance of each predetermined section in the video data with audio, digest data 35d which is data of a digest video, converted text data 35e which is data converted from audio contained in the video data into text data, summarized text data 35f which is a summary of the converted text data according to the importance of each section to which the audio belongs, and report data 35g which is data of a report generated from the summarized text data in a predetermined format. It is preferable to include multiple items in this prescribed format, such as title (work title in the example shown), summary, creation date (date the video was filmed), item names when the video is divided into predetermined blocks such as topics (topics in the example shown), summary of each item (content in the example shown), title of the digest video (e.g., "Self-introduction and start of conversation," "Explanation of the water heater problem"), link to play the digest video, decisions made (decisions made in the video), To-dos (items that need to be done in the future as shown in the video), and next schedule (date and time of the next scheduled date as decided in the video). By saving these multiple items as text data, it becomes easy to find the desired report data 35g later by text search. Therefore, it becomes easy to create and find work manuals and explanatory videos from the report data 35g. In addition, the summary text data 35f contains time information (time information, timestamp) to which the corresponding audio belongs in the audio-attached video data, or the time information is linked to it. In other words, it is possible to identify the playback scene (specified number of seconds) of the audio-equipped video data corresponding to the summarized text data 35f.

[0031] The configuration of user terminal 2 and digest creation system 1, including management server 3, described above are merely examples and are not limited to them; they can be modified as needed. <Example of system operation>

[0032] The following describes an example of the operation of the digest creation system 1. First, we will briefly explain the generation of video data with audio, which is a prerequisite for explaining the operation example of the digest creation system 1.

[0033] For example, video data with audio is generated on user terminal 2 by a function that records video data with audio provided according to a video data generation program (hereinafter simply referred to as the "video recording function"). Video data with audio can be used as explanatory video for purposes such as checking inspection work on electrical equipment, explaining work manuals, explaining how to use electrical equipment, viewing real estate, explaining cooking methods, medical examinations, sports instruction, and tourist guides. Furthermore, it is possible to create digest videos such as work manuals and explanatory videos from video data of inspection work or video data of explanations of how to use equipment through a digest creation process.

[0034] The video recording function of user terminal 2 is used, for example, when launching applications for taking images (so-called camera apps) or applications for making video calls (so-called video call apps).

[0035] When the camera app is launched, for example, the video recording function can be used independently on a single user terminal 2. In this case, the video data with audio captured (recorded) by the camera app (video recording function) is temporarily stored on user terminal 2. Subsequently, the management server 3 retrieves the video data with audio either in response to user operation or automatically.

[0036] For example, cases in which the video recording function is used independently on a single user terminal 2 include creating explanatory videos for work manuals, creating explanatory videos for cooking methods, and creating videos for administrators to verify inspection work on electrical equipment, etc.

[0037] Furthermore, when the video call application is launched, for example, two or more user terminals 2 can use the video recording function on one or both of them. In this case, video data with audio recorded from the video call is generated. When multiple user terminals 2 make a video call via the management server 3, the management server 3 can automatically acquire the video data with audio. On the other hand, when multiple user terminals 2 make a video call via peer-to-peer communication, the video data with audio is temporarily stored on one or both user terminals 2, and the management server 3 acquires the video data with audio from the user terminals 2 either in response to user operation or automatically. For example, cases in which the present invention can be used during video calls include when a field worker (serviceman) and a manager (operator) make a video call when inspecting electrical equipment, etc., or when a real estate agent and a user make a video call when viewing a property.

[0038] Figure 4 is an explanatory diagram showing an example of the screen configuration when using the video recording function. Figure 5 is an explanatory diagram showing an example of the screen configuration when using the image extraction function for editing when using the video recording function. Figure 6 is an explanatory diagram showing an example of the screen configuration when using the drawing function when using the video recording function. Figure 7 is an explanatory diagram showing an example of the screen configuration when using the pointer function when using the video recording function. Figure 8 is an explanatory diagram showing an example of the screen configuration when using the character recognition function when using the video recording function. Figure 9 is an explanatory diagram showing an example of the screen configuration when character recognition is complete when using the video recording function.

[0039] An embodiment of the present invention will be described below with reference to Figures 4 to 9. In the embodiment shown in Figures 4 to 9, the explanation will be given as an example of using the video recording function on a single user terminal 2.

[0040] As shown in Figure 4, when the video recording function is used, the imaging unit 24 and the audio input unit 25 are activated, and the image display screen 100 is displayed on the display unit 23 (display). In addition, when the video recording function is used, in parallel with the display of the image display screen 100, video data with audio, which records part or all of the display area of ​​the image display screen 100, is stored. In this embodiment, video data with audio, which records the entire display area of ​​the image display screen 100, is stored.

[0041] The image display screen 100 is a screen for displaying moving images or still images according to the imaging data generated by the imaging unit 24. The image display screen 100 has a real-time display unit 110 and an editing image display unit 120 which is displayed alongside the real-time display unit 110.

[0042] The real-time display unit 110 is positioned to the left of the center of the display area of ​​the image display screen 100, and the editing image display unit 120 is positioned to the right of the center of the display area of ​​the image display screen 100. In other words, the real-time display unit 110 and the editing image display unit 120 are positioned side by side.

[0043] The real-time display unit 110 displays a real-time video (preview image) corresponding to the video being captured by the imaging unit 24. In this embodiment, the real-time display unit 110 displays a real-time video 110a that includes an image of a predetermined electrical device.

[0044] Furthermore, within the display area of ​​the real-time display unit 110 (in this embodiment, the lower part of the display area of ​​the real-time display unit 110), an editing image extraction icon 111 is displayed, which is assigned to an editing image extraction function that extracts a still image at a predetermined timing from the moving image being captured as an editing image. The editing image extraction function is a type of auxiliary function intended to assist the user in capturing moving images.

[0045] When the editing image extraction function is used (when the editing image extraction icon 111 is selected), the editing image display unit 120 displays an editing image extracted from the moving image being captured at a predetermined timing. In other words, the editing image display unit 120 displays a still image extracted as an editing image from the image that was displayed when the editing image extraction icon 111 was selected. For example, if the editing image extraction icon 111 is selected in the state shown in Figure 4, the editing image display unit 120 displays an editing image 120a that is a copy of the real-time moving image 110a, as shown in Figure 5.

[0046] Furthermore, within the display area of ​​the editing image display unit 120 (in this embodiment, the lower part of the display area of ​​the editing image display unit 120), a plurality of auxiliary icons 121, each assigned an auxiliary function, are displayed. In this embodiment, the auxiliary icons 121 displayed are a drawing icon 121a, which is assigned a drawing function for the editing image 120a; a pointer display icon 121b, which is assigned a pointer display function for displaying a predetermined pointer image; and a character reading icon 121c, which is assigned a character reading function for performing optical character recognition (OCR) processing on characters contained in the editing image 120a. The drawing function, pointer display function, and character reading function are each types of auxiliary functions.

[0047] When the drawing icon 121a is selected, the drawing function is activated, enabling freehand drawing of lines and pasting (drawing) of predetermined shapes such as circles, squares, and arrows within the display area of ​​the editing image display unit 120. This drawing function displays the drawing process in real time not only on the drawing user's terminal but also on the other party's terminal. For example, if a line is drawn freehand, the line is drawn in real time on the other party's terminal, and if a shape is selected and placed, the shape is placed in real time on the other party's terminal. As shown in Figure 6, the drawn object (drawing object) 122 is displayed overlaid in front of the editing image 120a. Also, when the drawing function is activated, the drawing icon 121a corresponding to the drawing function is highlighted. The highlighting can be in the form of a pattern being added to the icon, an appropriate color being added to the icon (a color not used in other parts of the image display screen 100), an underline being added to the icon's text, or the icon's text being displayed in bold. The same highlighting behavior when the corresponding function is activated applies to the pointer display icon 121b and the text reading icon 121c.

[0048] When the pointer display icon 121b is selected, the pointer display function is activated, as shown in Figure 7, and the pointer image 123 is displayed within the display area of ​​the editing image display unit 120. The pointer image 123 moves in response to computer mouse operations or touch operations. For example, the pointer image 123 may be an image of a shape resembling an arrow or an image of a shape resembling a human finger.

[0049] When the character recognition icon 121c is selected, the character recognition function is activated. For example, as shown in Figure 8, suppose the character recognition icon 121c is selected when the letters "ABCDE" attached to the surface of the electrical equipment subject are displayed within the display area of ​​the editing image display unit 120. In this case, the letters "ABCDE" are recognized. If character recognition is successful, a results window 124 showing the character recognition results is displayed, as shown in Figure 9. The results window 124 displays a character recognition result display unit 124a that displays the recognized characters, a copy icon 124b that is assigned the function of copying the recognized characters as text data, and a save icon 124c that is assigned the function of saving (storing) the recognized characters as text data to the auxiliary storage unit 28, etc.

[0050] In this way, the user of user terminal 2 can use auxiliary functions as appropriate while using the video recording function. For example, when an important part of the subject (important part) is displayed, the user can extract an image for editing, use the drawing function to draw an object that indicates the important part in the editing image so that the important part is easier to understand, or use the pointer display function to indicate the important part with pointer image 123. In addition, if there is text attached to the important part of the subject, the user can use the character recognition function to recognize the text in the important part and save the text data. In other words, the timing when the auxiliary function is used and the period before and after that timing are likely to be important periods in the video data with sound.

[0051] When the video recording function is terminated (stopped), the video data with audio is temporarily stored in the auxiliary storage unit 28 of the user terminal 2. At this time, event data, which is data of events performed when the video was captured, such as the usage history of the auxiliary function, is linked to the video data with audio and stored in the auxiliary storage unit 28. The event data includes data on when the auxiliary function was used, and is data that can identify which type of auxiliary function was used and when.

[0052] Figure 10 is a flowchart of the digest data creation process (digest creation process) of the present invention. The digest creation process of the present invention will be described below with reference to Figure 10. The digest creation process is basically performed by the management server 3 (control unit 31), but it may also be performed by the user terminal 2 (control unit 21). In this embodiment, the digest creation process is assumed to be executed by the management server 3 (control unit 31).

[0053] When the management server 3 starts the digest creation process, it acquires video data with audio from the user terminal 2 (auxiliary storage unit 28) (step S1), and acquires event data from the user terminal 2 (auxiliary storage unit 28) that is associated with the video data with audio acquired in the previous step S1 (step S2).

[0054] Next, the video data with audio is divided into multiple sections, an importance level is set for each section (step S3), and according to the importance level set for each section, digest video data (digest data) is generated, which consists only of the important parts of the video data with audio (step S4).

[0055] However, when setting the importance level for each interval (step S3) and when generating the digest data (step S4), known technologies can be used, for example, a model with learning capabilities, so-called AI (artificial intelligence), can be used. Since artificial intelligence generally has learning capabilities to realize intelligent functions such as judgment, it can set more appropriate digest data as it is used more often.

[0056] Furthermore, in step S3, importance is set based on whether or not the auxiliary function was used. That is, sections in which the auxiliary function was used are given a higher importance rating. For example, in the generation of digest data using artificial intelligence, various factors such as the use or non-use of the auxiliary function, as well as subject (scene) changes, speaker changes, and volume peaks, are comprehensively considered. In this embodiment, the use of the auxiliary function is weighted more heavily than other factors in determining the importance rating. Therefore, sections in which the auxiliary function was used are more likely to be included in the digest data.

[0057] Furthermore, the weights contributing to importance can be varied even among auxiliary functions. For example, the use of editing image extraction and drawing functions can be weighted more highly to importance than the use of pointer display or text recognition functions. In this way, for example, a digest video can be generated that does not cut out parts explained using the drawing function, as these parts are evaluated as important. This makes it possible to pinpoint and review the appropriate video without interruptions or other stress when reviewing it later.

[0058] Returning to the flowchart in Figure 10, following step S4, converted character data is generated from the audio contained in the video data (step S5), and summarized character data is generated by summarizing the converted character data according to the importance of each section to which the audio belongs (step S6). However, when generating the converted character data (step S5) and when generating the summarized character data (step S6), known techniques can be used, for example, artificial intelligence can be used.

[0059] Next, report data is generated using the summary text data (step S7), the report data is output (step S8), and the digest creation process is completed. The report data can be in a format specified by the user and is output in a predetermined electronic document data format according to the specified format. The report data and digest data may also be output linked together. Furthermore, the report format data includes report structure data (metadata) such as title, summary, creation date, item names (topics) when the video is divided into predetermined block units such as topics, summary (content) within each item, title of the digest video ("Self-introduction and start of conversation," "Explanation of the water heater problem," etc.), link to play the digest video, decisions, to-dos, and data for each item to be included in the report, as well as location data (placement data) for each item. In step S7, the characters to be included in each item are selected according to the report structure data, and the entry fields for each item in the report are placed in predetermined positions according to the location information. In addition, the report structure data (data for each item and location data) can be customized by the user.

[0060] Figure 11 is an explanatory diagram showing an example of the structure of Report 130. Figure 12 is an explanatory diagram showing another example of the structure of Report 130. For example, as shown in Figure 11, the report includes summary text for items such as creation date, subject, summary, reporter, participants, topic and its content, decisions, to-do list, and next meeting schedule. Here, the summary text data is a summary of the converted text data according to the importance of each section to which the audio belongs. Therefore, the content of Report 130 is highly likely to be linked to the digest data.

[0061] Furthermore, when report data and digest data are output linked together, as shown in Figure 12, in addition to the items shown in Figure 11, a "Digest" item (digest display section) 131 may be added to the report 130. In this case, the contents of the digest display section 131 are displayed in conjunction with the digest data. For example, the digest display section 131 divides the digest data into multiple sections, displays text indicating the outline of each section, and a link (video link) 131a for playback from the start of each section of the digest data. In other words, the report 130 is configured to have a video link 131a for playing a video from the section corresponding to the summary text data for each summary text data. As described above, the summary text data 35f has time information to which the corresponding audio belongs, or time information is linked to it, so a video link 131a is generated that identifies the playback scene (specified number of seconds) using this time information.

[0062] Furthermore, if report data is not to be output, i.e., if only the generation of digest data is to be performed, steps S5 to S8 of the digest creation process can be omitted.

[0063] As described above, the digest creation system 1 of the present invention sets importance based on whether or not auxiliary functions are used, so it can automatically generate digest data that is likely to include the timing when auxiliary functions were used, which is likely to be an important period in the video data with sound. Furthermore, the auxiliary functions are intended solely to assist the user in capturing video, and do not require any subsequent operation to create digest data. Therefore, the user only needs to record video while using the auxiliary functions as needed. Consequently, the convenience of creating video digests can be improved.

[0064] Furthermore, according to the present invention, since importance is determined based on whether or not the image extraction function for editing is used, digest data that is highly likely to include the timing when the image for editing was extracted can be automatically generated. For example, in explanatory videos, the timing when the image for editing is extracted is often an important scene. Therefore, digest data that is highly likely to include important scenes can be easily generated.

[0065] Furthermore, according to the present invention, since importance is determined based on whether or not the drawing function is used, digest data that is likely to include the timing of the drawing on the editing image can be automatically generated. In explanatory videos, the timing of the drawing on the editing image is often the important scene. Therefore, digest data that is likely to include the important scene can be easily generated.

[0066] Furthermore, according to the present invention, summary text data is generated according to the importance of each section to which the audio belongs, and report data is generated using the summary text data. Therefore, it is possible to generate report data that is highly likely to be linked to the digest data. In other words, it is possible to easily generate digest data and report data that are highly likely to contain important scenes.

[0067] Furthermore, according to the present invention, since the report data is structured to include links for playing back video from sections corresponding to summary text data, it is convenient that video can be played back from any scene when viewing the report.

[0068] The digest creation system of this invention corresponds to the digest creation system 1 of the above embodiment, the user terminal corresponds to the user terminal 2, the management server corresponds to the management server 3, the imaging means corresponds to the imaging unit 24, the input means corresponds to the input unit 22, the storage means corresponds to the auxiliary storage unit 28 or auxiliary storage unit 33, the importance setting means corresponds to the importance setting program 34c and the control unit 31 that operates therein, the generation means corresponds to the digest generation program 34d and the control unit 31 that operates therein, the character conversion means corresponds to the character conversion program 34e and the control unit 31 that operates therein, the summary character generation means corresponds to the summary character generation program 34f and the control unit 31 that operates therein, and the report output means corresponds to the report output program 34g and the control unit 31 that operates therein. However, this invention is not limited to this embodiment and can take various other forms. Furthermore, the screens and specific configurations mentioned in the above embodiment are examples and can be appropriately changed according to the actual product.

[0069] For example, in the embodiment described above, the case where the video recording function is used independently on a single user terminal 2 was used as an example, but the video recording function can also be used when multiple user terminals 2 are making a video call. In this case, the auxiliary function can be used on at least one of the multiple user terminals 2, or it can be used on all of the user terminals 2.

[0070] Furthermore, although the above-described embodiment explained the case where the digest creation process is performed on the management server 3 (control unit 31), the digest creation process may also be performed on the user terminal 2 (control unit 21). In this case, the user terminal 2 corresponds to the digest creation device of the present invention. [Industrial applicability]

[0071] This invention can be used in industries such as creating video digests. [Explanation of Symbols]

[0072] 1…Digest creation system 2…User terminal 3…Management Server 31…Control Unit 33…Auxiliary storage unit

Claims

1. An imaging means capable of capturing moving images, An input means that accepts predetermined input operations, including the use of an auxiliary function intended to assist the user in capturing moving images, A storage means for storing data of moving images captured by the aforementioned imaging means, Importance setting means for setting the importance level for each predetermined section of the video data, The system includes a generation means that generates digest data from the video data, shortened to include only the important parts of the video according to the importance set for each section, The aforementioned importance setting means is configured to set the importance level based on whether or not the auxiliary function is used. A digest creation device.

2. The aforementioned auxiliary function includes an editing image extraction function that extracts a still image at a predetermined timing from a moving image being captured by the imaging means as an editing image. The aforementioned importance setting means is configured to determine importance based on whether or not the editing image extraction function is used. The digest preparation apparatus according to claim 1.

3. The aforementioned auxiliary function has a drawing function for drawing on the editing image, The aforementioned importance setting means is configured to determine importance based on whether or not the drawing function is used. The digest preparation apparatus according to claim 2.

4. A character conversion means that converts the audio contained in the video data into character data, A summary character generation means generates summary character data which is a summary of the character data according to the importance of each section to which the aforementioned audio belongs, The system includes a report output means that outputs the summary text data in a predetermined format to produce report data. The digest preparation apparatus according to claim 1, 2, or 3.

5. The report data is configured such that each summary text data has a link to play a video from the section corresponding to that summary text data. The digest preparation apparatus according to claim 4.

6. A digest creation system comprising a user terminal and a management server capable of communicating with said user terminal, The aforementioned user terminal is An imaging means capable of capturing moving images, The system includes an input means that accepts predetermined input operations, including the use of an auxiliary function intended to assist the user in capturing moving images, The aforementioned management server A storage means for storing data of moving images captured by the aforementioned imaging means, Importance setting means for setting the importance level for each predetermined section of the video data, The system includes a generation means that generates digest data from the video data, shortened to include only the important parts of the video according to the importance set for each section, The aforementioned importance setting means is configured to set the importance level based on whether or not the auxiliary function is used. A digest creation system.

7. A digest creation method in a digest creation device comprising: an imaging means capable of capturing moving images; an input means that accepts predetermined input operations, including the use of an auxiliary function intended to assist the user in capturing moving images; and a storage means that stores data of moving images captured by the imaging means, A setting step of setting the importance level for each predetermined section of the video data, The system includes a setting step for generating digest data from the video data, which is shortened to include only the important parts of the video according to the importance set for each section, and a generation means. In the setting step, the importance level is set based on whether or not the auxiliary function is used. How to create a digest.

8. A computer for a digest creation device comprising: an imaging means capable of capturing moving images; an input means that accepts predetermined input operations, including the use of auxiliary functions intended to assist the user in capturing moving images; and a storage means that stores data of moving images captured by the imaging means, Importance setting means for setting the importance level for each predetermined section of the video data, This means functions to generate digest data from the video data, shortened to include only the important parts of the video according to the importance set for each section. The aforementioned importance setting means is configured to set the importance level based on whether or not the auxiliary function is used. A program for creating digests.

Images