Video editing device, video editing program, and video editing method

Abstract

[Problem] To provide a video editing device, a video editing program, and a video editing method with which it is possible to efficiently generate a summary video containing a collected scene in which a viewer will exhibit interest. [Solution] A video editing device (1) divides video content used as a raw material into cut video segments on the basis of predetermined conditions, the cut video segments including a plurality of frame images. The segmented clipped video segments are classified into pre-classified scenes of the video content, then clipped video segments that are impressive to the viewer are extracted, and the extracted clipped video segments are connected together to generate a summary video having a shorter time than the video content.

Description

Technical Field

[0001] The present invention relates to a video editing device, a video editing program, and a video editing method. In particular, the present invention relates to a video editing device, a video editing program, and a video editing method capable of efficiently creating a digest video obtained by collecting scenes that viewers will show interest in. Background Art

[0002] For a long-running televised program such as a sports game, a movie, or a TV shopping program, viewers must take time away from their lives other than watching the program, they may not have time to watch the long program from the beginning to the end, and need to watch a summary program edited to include only the points of the program. Conventionally, usually, a video editor at a broadcast station creates a summary video by manually selecting only scenes in which viewers are considered to show interest in the long program and editing the selected scenes to fall within a predetermined broadcast time frame.

[0003] However, in the case of manually editing a summary video in this way, the program that needs to be made into the summary video has a relatively long time range, and selecting scenes as points from the program takes a lot of time and effort, resulting in a large workload for the video editor. In addition, since the summary video created in this way is a summary including points selected from the viewpoint of the video editor, there is a problem that the summary video does not necessarily reflect the interests of the viewer.

[0004] In view of the above problems, there is a technology disclosed in Patent Document 1 as a technology for efficiently creating a summary video. According to the technical content disclosed in Patent Document 1, the motion state of another viewer is analyzed, the degree of attention to the scene in the program is calculated based on the viewer's interest information, the scene of high interest of the viewer is selected from the video, and the selected scene is connected to create a summary video. This allows efficient creation of a summary video that reflects the viewer's interest.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2012-248987 Summary of the invention

[0008] Problem that the invention aims to solve

[0009] However, only watching scenes that other viewers have paid attention to may create a summary video that does not take into account the context of the content or the context between scenes, and the content may be difficult to understand. For example, in sports broadcasts, although the scoring scene is likely to attract high attention, even if only the scoring scene is used to edit a summary video, the connection between the previous and subsequent videos may be unnatural, and there is a possibility that the viewer cannot fully understand the content of the game.

[0010] The present invention has been made in view of the above points, and an object thereof is to provide a video editing apparatus, a video editing program, and a video editing method capable of efficiently creating a digest video obtained by collecting scenes in which a viewer will show interest.

[0011] Means of solving the problem

[0012] In order to achieve the above-mentioned purpose, the video editing device according to the present invention includes: an image analyzing unit, a cut video creating unit, a cut video selecting unit and a summary video creating unit, the image analyzing unit being configured to perform image analysis on video content used as original material; the cut video creating unit being configured to create a cut video by dividing the video content at predetermined dividing points based on the analysis result of the image analyzing unit; the cut video selecting unit being configured to select a desired cut video from the cut videos based on the analysis result of the image analyzing unit; and the summary video creating unit being configured to create a summary video by connecting the cut videos selected by the cut video selecting unit, the summary video having a shorter video time length than the video content.

[0013] Herein, by including an image analysis unit that performs image analysis on video content serving as an original material, a feature image to be adopted as a digest video can be extracted from the video content serving as an original material by performing image analysis on the video content.

[0014] In addition, by including a cut video creation unit that creates a cut video by dividing the video content based on the analysis result of the image analysis unit, the video content can be divided into a plurality of cut videos. Then, by connecting the desired cut video from the divided cut videos, it is possible to edit a summary video that allows the entire original video to be easily imagined and does not give a sense of incongruity.

[0015] In addition, by including a cut video selection unit that selects a desired cut video from a plurality of cut videos based on the analysis result of the image analysis unit, it is possible to give priority to cut videos that will give a favorable impression to the viewer, such as cut videos having favorable expressions of performers (presenters, competitors, commentators, etc.) in the video content or cut videos in which performers make comments that will leave an impression in the cut videos contained in the scene video and edited videos that will leave an impression on the viewer.

[0016] In addition, a summary video is edited based on the selected feature cut videos by including a summary video creation unit, and the summary video creation unit creates a summary video having a shorter video time length than the video content by connecting the cut videos selected by the cut video selection unit. Therefore, a video that may leave an impression on the viewer and does not give a sense of incongruity can be edited in a short time video.

[0017] In addition, by including a scene video creation unit configured to create a scene video by classifying the cut video into a plurality of attribute groups prepared in advance, the cut video can be classified into the attribute groups classified in advance based on the video content and a scene video can be created. That is, the video content is classified into a plurality of scene videos, and the scene video is selected according to the purpose of the summary video to be edited, so that the connection between the previous and subsequent scene videos is natural, and the summary video can be edited without a sense of incongruity.

[0018] In addition, by including a scene video evaluation unit configured to evaluate scene videos by setting an expected score for each of the scene videos, scene videos that will leave an impression on viewers can be identified by assigning a score to each scene video using, for example, ratings, the number of inquiries from viewers, etc. as parameters among scene videos classified into multiple attribute groups.

[0019] In addition, by including a scene video selection unit configured to select a desired scene video from scene videos based on a score set by a scene video evaluation unit, scene videos that are deemed to attract high attention from viewers can be preferentially selected based on the evaluation results of the above-mentioned scene video evaluation unit, and thus videos that are likely to leave an impression on viewers can be edited.

[0020] In addition, in the following case: the image analysis unit includes a color change amount calculation unit, a text data extraction unit and an OCR (Optical Character Recognition) analysis unit, the color change amount calculation unit is configured to calculate the color change amount between frame images of the video content, the text data extraction unit is configured to recognize audio data contained in the video content and extract the audio data as text data, the OCR analysis unit is configured to extract character information contained in the video content, and the cut video creation unit is configured to use at least one or more parameters among the color change amount, the text data and the character information as feature amounts to set a segmentation point on the time axis for segmenting the video content; the segmentation point of the video content can be accurately determined, and the summary video edited by connecting the cut videos can be edited into a video that does not give a sense of incongruity.

[0021] For example, in the case where the amount of color change in the preceding and succeeding frame images is large, it is estimated that the video scene has been switched, and the point where the amount of color change is greatly changed can be set as the segmentation point of the cut video. In addition, based on the text data obtained by recognizing audio data and converting the audio data into text, the segmentation point of the cut video can be set based on conjunctions or context. In addition, character information displayed on a flip chart displayed in the video content can be extracted by OCR processing, and the point where the character information changes can be set as the segmentation point of the cut video. Based on the three parameters of color change amount, text data and character information, the scene switching is comprehensively determined, so that the segmentation point that allows the cut video to be extracted more accurately can be set.

[0022] In addition, in the following case: the color change amount calculation unit includes an RGB (Red, Green, Blue) change amount calculation unit and a hue shift average value calculation unit, the RGB change amount calculation unit is configured to divide the temporally preceding and succeeding representative frame images in the video content into a plurality of image regions, and calculate the RGB change amount of pixels in each image region of the representative frame image, the hue shift average value calculation unit is configured to calculate the hue shift average value in predetermined adjacent hue regions on the color wheel of the representative frame image, the color change amount calculation unit is configured to calculate the color change amount based on the RGB change amount and the hue shift average value; the color change is determined based on at least two parameters of the RGB change amount and the hue shift average value. Therefore, it is possible to prevent erroneous detection of a segmentation point, reliably detect a segmentation point, and edit a video without a sense of incongruity.

[0023] Further, in the following case: the image analysis unit also includes a morphological analysis unit and an object detection unit, the morphological analysis unit is configured to perform morphological analysis on the text data extracted by the text data extraction unit for each word, the object detection unit is configured to detect the desired object displayed in the video content, and the scene video creation unit is configured to classify the cut video into one of multiple attribute groups based on at least one of the commonality of the context based on the morphological analysis of the morphological analysis unit, the commonality of the character information based on the OCR analysis of the OCR analysis unit, and the presence or absence of movement of the object based on the object detection of the object detection unit; based on the text data based on the morphological analysis, object detection and OCR analysis, the cut video can be accurately classified into the attributes of the scene video pre-classified based on each element of the character information.

[0024] In addition, in the following case: multiple content segments corresponding to the program configuration of the video content (including a beginning segment of content indicating the beginning part of the video content) are stored as an attribute group, and the scene video creation unit is configured to classify the cut video into one of the multiple content segments based on the content of the cut video; the cut video can be classified into content segments corresponding to the program configuration of the video content to form a group of scene videos.

[0025] In addition, in the following case: in the case where the scene video evaluation unit is configured to set a score for each scene video by weighting according to predetermined parameters and time periods, the predetermined parameters indicate the degree of attention from viewers in each scene video of the video content, and the time period corresponds to a pre-arranged program configuration of the video content; by assigning a score to each scene video using, for example, ratings, viewers' reactions, etc. as parameters, it is possible to edit a video that is a scene video that viewers will pay attention to, which is preferentially adopted among multiple scene videos.

[0026] In addition, in the following case: the image analysis unit further includes a volume level detection unit and an expression detection unit, the volume level detection unit is configured to detect the volume level of the video content, the expression detection unit is configured to detect the expression of the performer of the video content, the cut video selection unit is configured to select a desired cut video from a content segment in which the volume of the video content detected by the volume level detection unit exceeds a predetermined threshold, a content segment in which the expression of the performer of the video content detected by the expression detection unit is in a specific state, or a content segment in which the performer of the video content says a specific term based on the text data extracted by the text data extraction unit; it is possible to edit a video that is preferentially adopted from a plurality of cut videos classified as desired scene videos as a cut video with the best appearance for the viewer.

[0027] In order to achieve the above-mentioned purpose, the video editing program according to the present invention enables a computer to perform the following steps: create a cut video by dividing a video content used as an original material based on a predetermined segmentation condition; create a scene video by classifying the cut videos into a plurality of attribute groups prepared in advance; evaluate the scene videos by setting an expected score for each of the scene videos; select a desired scene video from the scene videos based on the score; select a desired cut video from a plurality of cut videos contained in the scene video; and create a summary video having a shorter video time length than the video content by connecting the cut videos selected in the step of selecting the cut videos.

[0028] Herein, by including the step of creating a cut video by dividing the video content used as the original material based on a predetermined division condition, the video content can be divided into a plurality of cut videos. Then, by connecting a desired cut video from the divided cut videos, it is possible to edit a video that allows the entire original video to be easily imagined and does not give a sense of incongruity.

[0029] Furthermore, by including the step of creating a scene video by classifying the cut video into a plurality of attribute groups prepared in advance, it is possible to classify the cut video into attribute groups classified in advance based on the video content and create one scene video.

[0030] Furthermore, by including a step of evaluating scene videos by setting an expected score for each scene video, scene videos that receive high attention from viewers can be identified by assigning a score to each scene video using, for example, ratings, viewers' reactions, etc. as parameters among scene videos classified into a plurality of attribute groups.

[0031] In addition, by including a step of selecting a desired scene video from the scene videos based on the score set in the step of evaluating the scene videos, it is possible to adopt a video that will give a good impression to the viewer by preferentially selecting scene videos that are considered to have high attention from the viewer based on the evaluation results of the above-mentioned scene videos.

[0032] Furthermore, by including a step of selecting a desired cut video from a scene video based on a score, it is possible to prioritize cut videos that will give a favorable impression to a viewer by selecting, from the cut videos included in the scene video, cut videos having favorable expressions of a performer, or cut videos in which a performer makes comments that will leave an impression, and edited videos that will leave an impression on the viewer.

[0033] Furthermore, by including the step of creating a summary video having a shorter video time length than the video content by connecting the cut videos selected in the step of selecting the cut videos, the summary video is edited based on the selected characteristic cut videos. Therefore, it is possible to edit a video that may leave an impression on the viewer and does not give a sense of incongruity in a short time video.

[0034] In addition, in the following case: the step of creating a cut video includes the following steps: calculating the amount of color change between frame images of video content; identifying audio data contained in the video content and extracting the audio data as text data; extracting character information contained in the video content; and using at least one or more parameters among the amount of color change, text data and character information as feature amounts to set a segmentation point on a timeline for segmenting the video content; the segmentation point of the video content can be accurately determined, and a summary video edited by connecting the cut videos can be edited into a video that does not give a sense of incongruity.

[0035] Furthermore, in the following case: the step of calculating the color change amount includes the following steps: dividing the temporally preceding and succeeding representative frame images in the video content into a plurality of image regions, and calculating the RGB change amount of pixels in each image region of the representative frame image; calculating the hue shift average value in predetermined adjacent hue regions on the color wheel of the representative frame image; and calculating the color change amount based on the RGB change amount and the hue shift average value; and determining the color change based on at least two parameters of the RGB change amount and the hue shift average value. Therefore, erroneous detection of the segmentation point can be prevented, the segmentation point can be reliably detected, and a video without a sense of incongruity can be edited.

[0036] In addition, in the following case: the step of creating a scene video includes the following steps: determining the commonality of the context of each cut video based on a morphological analysis of each word of the text data; detecting the expected object displayed in the video content, and determining the presence or absence of movement of the object in each cut video; extracting the character information contained in the video content based on OCR analysis, and determining the commonality of the character information of each cut video; classifying the cut videos into one of a plurality of attribute groups based on at least one of the commonality of the context, the presence or absence of movement of the object, or the commonality of the character information; and according to the text data based on morphological analysis, object detection, and OCR analysis, the cut videos can be classified into the attributes of the scene video pre-classified based on each element of the character information.

[0037] In addition, in the following case: the step of selecting a cut video includes the following steps: determining whether the volume level of the video content is equal to or greater than a predetermined threshold; determining whether the expression of a performer of the video content is in a specific state; determining whether a word spoken by the performer of the video content is a specific term; and selecting a desired cut video from content clips whose volume level is equal to or greater than a predetermined threshold, content clips whose performer's expression is in a specific state, and content clips whose performer speaks a specific term; a cut video having the best appearance for a viewer can be selected from a plurality of cut videos classified as desired scene videos.

[0038] Effects of the Invention

[0039] The video editing apparatus, the video editing program, and the video editing method according to the present invention can efficiently create a digest video obtained by collecting scenes in which viewers will show interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] [ Figure 1 ] is a diagram showing the overall configuration of a video editing device according to an embodiment of the present invention.

[0041] [ Figure 2 ] is a diagram showing the overall configuration of the image analysis unit.

[0042] [ Figure 3 ] is a flowchart of a video editing method according to an embodiment of the present invention.

[0043] [ Figure 4 ] is a processing flow chart of the color change calculation unit.

[0044] [ Figure 5 ] is a diagram showing an example of text data extracted by a text data extraction unit.

[0045] [ Figure 6 ] is a processing flow chart of the cut video creation unit.

[0046] [ Figure 7 ] is a processing flow chart of the scene video creation unit.

[0047] [ Figure 8 ] is a diagram showing an example of a framework of a summary video prepared in advance. DETAILED DESCRIPTION

[0048] Hereinafter, embodiments of the present invention related to a video editing apparatus, a video editing program, and a video editing method will be described with reference to the accompanying drawings, and the embodiments of the present invention are provided for understanding of the present invention.

[0049] First, refer to Figure 1 The overall configuration of the video editing apparatus 1 according to the embodiment of the present invention is described. The video editing apparatus 1 according to the embodiment of the present invention is mainly configured with an image analyzing unit 10, a cut video creating unit 20, a scene video creating unit 30, a scene video evaluating unit 40, a scene video selecting unit 50, a cut video selecting unit 60, a summary video creating unit 70 and a storage unit 80.

[0050] 1 Image analysis unit

[0051] The image analysis unit 10 has Figure 2 The image analysis unit 10 mainly has the following configuration.

[0052] [Color change calculation unit]

[0053] The color change amount calculation unit 11 has the following functions: calculates the color change amount of the temporally preceding and succeeding frame images in the video content that reduces the image by a predetermined amount, and extracts the frame image whose color change amount exceeds the predetermined value as a segmentation point indicating video switching, and includes an RGB change amount calculation unit 11a and a hue shift average value calculation unit 11b. The RGB change amount calculation unit 11a calculates the RGB values ​​(0 to 255) of the pixels at the same position in the preceding and succeeding frame images segmented into a plurality of pixels (image areas), identifies the pixels that have changed by a predetermined threshold value (e.g., 20%) or more in the succeeding frame compared to the preceding frame, counts the pixels as changed pixels and a count, and extracts the frame images whose count number of RGB change amounts of the entire screen is equal to or greater than the predetermined number.

[0054] Herein, the threshold value of the amount of color change between the preceding and succeeding frames is not necessarily limited to 20%, and the threshold value can be freely selected. In addition, it is not necessarily necessary to adopt a reduced image as a frame image for comparison. However, by adopting a reduced image as a frame image for comparison, the sensitivity to a slight color change in the frame image can be reduced, which makes it possible to prevent over-determination of the switching of the video.

[0055] In addition, the hue shift average calculation unit 11b obtains the content of each color in the previous and next frame images of the video content based on a well-known color circle indicating continuous hue changes of 360 colors, further classifies the 360 ​​colors in the color circle into adjacent 16 color areas, and calculates the shift average of the 16 colors. Then, for each color of the calculated shift average, the count number of hue changes of the entire screen is calculated by comparing the previous and next frames and counting the colors showing a predetermined change amount, and determines whether there is a switching of the video based on the determination of whether the count number is equal to or greater than a threshold value.

[0056] Herein, in the calculation of the hue shift average value, it is not necessary to calculate the shift average value of the adjacent areas of the 16 colors classified in the color wheel. However, as described above, from the perspective of preventing excessive determination of video switching, in order to suppress the spike of each color and the small change of hue, it is preferable to calculate the content of each color not only based on the color wheel but also based on the shift average value of the hue of the adjacent predetermined area.

[0057] For example, according to Figure 4 According to the process shown in FIG. 1 , the determination of video switching is performed based on the color change amount calculated by the color change amount calculation unit 11. First, it is determined whether the RGB change amount C1 of the preceding and succeeding frame images calculated by the RGB change amount calculation unit 11a is equal to or greater than a first threshold value T1 (e.g., 24%) (step 11).

[0058] If it is determined in step 11 that the RGB change amount C1 is equal to or greater than the first threshold value T1, it is determined whether the value obtained by subtracting the average value C2 of the RGB change amount at a predetermined time from the RGB change amount C1 is equal to or greater than the second threshold value T2 (for example, 20%) (step 12). In the case of "yes" in step 12, it is determined that the RGB change amount C1 of the previous and next frame images is large, and the determination flag of video switching is set (step 13).

[0059] Furthermore, it is determined whether the hue shift average value C3 calculated by the hue shift average value calculation unit 11b exceeds the third threshold value T3 (step 14). If it is determined that the hue shift average value C3 is equal to or greater than the third threshold value T3, it is determined that the color component has changed significantly, and it is determined to switch the video (step 15). If it is determined to be "no" in steps 11, 12, and 14, it is determined that the color change amount is small and the video is not switched, and the processing ends.

[0060] As described above, the switching of the video is determined by the color change amount based on both the RGB change amount and the hue shift average value. For example, in the case of capturing an image by combining zoom, pan or tilt of a camera as a capturing method of an image in the video content, the RGB change amount of the previous and subsequent frame images temporarily changes significantly, but in the case where the threshold is not reached based on the hue shift average value, it is determined that the video is switched, thereby preventing the switching of the video from being excessively determined in advance.

[0061] [Text data extraction unit]

[0062] The text data extraction unit 12 includes a voice recognition engine (not shown) that recognizes the voice of a performer of the video content (the voice of a commentator in the case of a sports broadcast), and has a function of transcribing the voice output from the video content (i.e., converting the voice data into text data). Figure 5 As shown, text data extracted by transcription is associated with the cut video of the video content and stored together with the time code in the storage unit 80. The text data is used for morphological analysis in the morphological analysis unit 14 to be described later, for example, and determination of video switching is performed.

[0063] [OCR analysis unit]

[0064] The OCR analysis unit 13 has the following functions: identifying all information of characters and numbers displayed on the screen through OCR analysis, such as character information displayed in video content, for example, character information displayed on an activity chart used by performers in the video content to present explanations to viewers, or information displayed as superimposed text displayed in the video content (in the case of sports programs, subtitle information including player information and game scores), and storing the information as text data in the storage unit 80.

[0065] [Morphological Analysis Unit]

[0066] The morphological analysis unit 14 performs morphological analysis on the character information extracted by the text data extraction unit 12, and stores the analysis result in the storage unit 80. For example, the morphological analysis unit 14 divides the character information into morpheme units, and estimates the part of speech, conjugated form, or context of each sentence for each morpheme. Then, for each morpheme, the morphological analysis unit 14 stores the morphological analysis result including the character string divided into the morpheme units and information such as the estimated part of speech, conjugated form, etc. in the storage unit 80.

[0067] In the storage unit 80, words such as "now", "then", "good", "next", "so", and "meanwhile" that are frequently used when the video is switched, or words such as "yes", "I think", and "thank you" that are frequently used at the end of a sentence immediately before the video scene is switched are registered in advance. In the case where these words are included at the beginning or end of a sentence as a result of the morphological analysis by the morphological analysis unit 14, it is determined that there is a high possibility that the video scene has been switched, which makes it possible to prevent determination of the switching of the video scene from being made at an unnatural point.

[0068] [Object Detection Unit]

[0069] The object detection unit 15 performs object detection on each object contained in the frame image acquired from the video content. In the object detection performed by the object detection unit 15, a pre-prepared object detection model is stored in the storage unit 80, and the detection object detected by the object detection unit 15 is identified by comparing the object with the object detection model stored in the storage unit 80. Then, for example, the object detection model is updated as needed by an image learning device (not shown).

[0070] The object detection unit 15 is also used in the above-mentioned OCR analysis. First, the object detection unit 15 detects character information displayed in the video content as a target object to be subjected to OCR analysis based on machine learning. For example, in order to recognize an activity flipchart displayed in the video content, the shape and size of a commonly used activity flipchart are registered as data in advance, and when the pre-registered data matches the shape and size of the object displayed in the video, it is determined that the object is an activity flipchart, and the character information displayed on the activity flipchart can be subjected to OCR analysis by the OCR analysis unit 13.

[0071] In addition, the object detection unit 15 detects another object displayed in the video content (for example, in the case where the video content is a sports broadcast, a competitor introduced in the content), and detects a competitor appearing in the video content after matching with the pre-registered competitor information. Then, it can be determined that there is no switching of the video when the detected competitor is displayed in the video content, and in the case where a different competitor is detected, it can be determined that there is a switching of the video or a switching of the video scene.

[0072] [Volume level detection unit]

[0073] The volume level detection unit 16 detects the sound output from the video content and identifies the volume level. For example, based on the volume level detected by the volume level detection unit 16, a scene in the video content whose volume level exceeds a predetermined threshold can be determined as a highlight scene. Note that the volume level detection unit 16 can be incorporated into the text data extraction unit 12 and configured to detect the volume level based on the speech recognition engine of the text data extraction unit 12.

[0074] [Expression Detection Unit]

[0075] The expression detection unit 17 has a function of detecting the face of a person who is a performer appearing in the video content and detecting the expression thereof. In the detection of the expression performed by the expression detection unit 17, similarly to the object detection unit 15, a plurality of human expression models prepared in advance are registered, the movement of each part of the face is detected, the human emotion is classified into, for example, happiness, anger, sadness, joy, surprise, fear, and emptiness, and the expression of the human can be analyzed. As a result, a cut video with the expression best viewed by the viewer can be extracted.

[0076] 2Cut video creation unit

[0077] The cut video creation unit 20 has the function of identifying switching points (division points) of video content serving as original material based on the analysis result of the above-mentioned image analysis unit 10 and dividing the video content into a plurality of cut videos based on the identified division points.

[0078] based on Figure 6 First, the color change amount calculation unit 11 calculates the color change amount of the extracted previous and subsequent frame images, and sets the segmentation point where the color change amount is equal to or greater than a predetermined value (step 21). It should be noted that based on the above Figure 3 The flow of calculation is the calculation processing of the color change amount calculation unit 11.

[0079] When the color change amount is equal to or greater than a predetermined value, a more detailed segmentation point is set based on the OCR analysis (step 22) of the OCR analysis unit 13, the object detection (step 23) of the object detection unit 15, and the text analysis (step 24) of the text data extraction unit 12, and a cut video is created (step 25). That is, the segmentation point is roughly set by the color change amount calculation unit 11, and the front and rear positions of the segmentation point are adjusted by the OCR analysis, the image analysis based on the object detection, and the text analysis, so that the segmentation point can be prevented from being set as a switching point of an incomplete scene.

[0080] Note that the segmentation points for creating cut videos can be set based on the amount of color change, but in order to prevent unnatural connections between before and after each cut video in the edited summary video, the segmentation points are preferably set through OCR analysis, image analysis based on object detection, and further text analysis.

[0081] 3-Scene Video Creation Unit

[0082] The scene video creation unit 30 has the following functions: determining scene changes of the video content used as the original material through the image analysis of the image analysis unit 10, classifying the video content into a plurality of attribute groups divided by time zones (a plurality of content segments according to the program configuration of the video content), assigning the cut videos created by the cut video creation unit 20 to the classified attribute groups, and creating a set of scene videos. Figure 7 The flow shown in FIG. 1 processes the creation of a scene video by the scene video creation unit 30 .

[0083] First, the character information displayed on the flipchart or the like displayed in the video content is subjected to OCR analysis by the OCR analysis unit 13, and it is determined whether they can be grouped into a scene video of a content segment of a group (step 31). Specifically, it is determined whether the character information extracted by the OCR analysis changes with time, and the cut video during the time when the character information does not change is classified into a predetermined content segment as a group of scene videos.

[0084] Next, the object detection unit 15 detects objects (e.g., performers, exercise tools, buildings, etc.) in the image shown in the video content, and determines whether they can be grouped into a scene video of a content segment of one group (step 32). When the object in the image does not change in a certain period of time, the cut video in the period of time is classified as a predetermined content segment as a scene video of one group.

[0085] Then, based on the text data extracted by the text data extraction unit 12 through transcription and the morphological analysis by the morphological analysis unit 14, it is determined whether they can be contextually grouped into a group of scene images (step 33).

[0086] Based on the above-mentioned process, the cut video created by the cut video creating unit 20 is classified into preset content segments, and a group of scene videos is created (step 34).

[0087] Note that creation of scene video does not need to be determined based on three elements of OCR analysis by the OCR analysis unit 13, object detection by the object detection unit 15, and text data extracted by the text data extraction unit 12, and may be created using either or a combination of the two.

[0088] 4-Scene Video Evaluation Unit

[0089] The scene video evaluating unit 40 has a function of setting a score for each of the attribute groups (scene videos) composed of a plurality of content segments created by the scene video creating unit 30. In setting of the score, a score weighted under a predetermined condition is given to each scene video.

[0090] In the score setting of the scene video evaluation unit 40, first, the editor selects a frame of the summary video to be edited. In this frame, in the case where the video content used as the original material of the summary video to be edited is a sports broadcast, such as Figure 8 As shown, according to the time of the summary video to be edited (e.g., 10 minutes (longer highlights), 5 minutes (shorter highlights), 3 minutes (more scoring scenes), and 3 minutes (only scoring scenes), a desired one is selected from the pre-prepared scene video templates.

[0091] For example, in the case where "10 minutes (longer highlights)" is selected as the framework of the summary video, a series of content segments starting from "Beginning", followed by "Athlete Introduction", "Scoring Scene 1", "State of the Audience", "Scoring Scene 2" and "Athlete Interviews of the Game" are set as its program configuration.

[0092] Then, the score setting of each scene video can be freely set in the framework of the selected scene video. For example, in the case of selecting a template with a relatively long program configuration, set weighting so that the score is evenly distributed to each scene video. On the contrary, in the case of selecting a framework with a relatively short program configuration and many scoring scenes, set weighting so that the scene video of the scoring scene has a high score.

[0093] In addition, the score can be set by adding weight to a scene video that has greater audience rating data or has caused many reactions from viewers through SNS (Social Networking Service) when broadcasting video content used as original material. In addition, the score can be set based on past viewing data by adding weight to scene videos in time zones that are likely to be adopted as summary videos according to video content (for example, in the case of sports broadcasts, time zones shortly after the start of a game or time zones in the final stages close to the end time of a game).

[0094] 5-Scene Video Selection Unit

[0095] The scene video selection unit 50 has a function of selecting a scene video to be adopted as a summary video based on the evaluation result of the above-mentioned scene video evaluation unit 40. Specifically, for the frame of the summary video to be edited, a scene video with a high score is selected from the score value set by the scene video evaluation unit 40 and stored in the storage unit 80.

[0096] 6 Cut video selection unit

[0097] The cut video selection unit 60 selects a desired cut video for the summary video from the cut videos included in the scene videos selected by the scene video selection unit 50. The cut video is freely selected based on the results extracted or detected by the above-mentioned text data extraction unit 12, volume level detection unit 16 and expression detection unit 17.

[0098] For example, in the text data extraction unit 12, a cut video using pre-registered positive words (e.g., "excellent", "beautiful", "goal", "home run", etc.) in the words spoken by the performer is preferentially selected. In addition, among the volumes detected by the volume level detection unit 16, a cut video having a particularly high volume due to cheering or applause from the audience is preferentially selected. Furthermore, the expression detection unit 17 preferentially selects a cut video having a good expression of the performer.

[0099] As described above, since a cut video that will be most attractive is selected from scene videos based on text data, volume, expression, etc., a digest video that will leave an impression on viewers even in a short time can be edited.

[0100] 7Summary Video Creation Unit

[0101] The summary video production unit 70 has a function of connecting the cut videos extracted from the selected scene videos to create a summary video according to the purpose. Specifically, when only the cut videos that will leave an impression on the viewer are connected by the above-mentioned cut video selection unit 60, the connection between the scene videos becomes unnatural. In order to solve this problem, in the summary video creation unit 70, for example, a cut video that prevents the context connection of the preceding and succeeding cut videos from becoming unnatural is inserted using morphological analysis, so that the summary video can be edited without a sense of incongruity as a whole.

[0102] The above is the main configuration of the video editing device 1. Figure 2 , describing a video editing method according to an embodiment of the present invention implemented in the video editing apparatus 1. It is to be noted that the video editing apparatus 1 executes the processing shown in the drawings according to a program.

[0103] First, the cut video creation unit 20 creates a cut video by dividing the video content used as the original material based on a predetermined division condition (step 1). Figure 4 and Figure 6 The process of creating a cut video.

[0104] The cut videos created in step 1 are classified into a plurality of attribute groups prepared in advance, and a set of scene videos is created by the scene video creation unit 30 (step 2). Figure 7 Then, the scene video evaluation unit 40 sets a score for each created scene video (step 3), and selects a desired scene video based on the score value (step 4).

[0105] Next, the cut video selection unit 60 selects the most attractive cut video from the cut videos included in the scene videos as a video (step 5), and the summary video creation unit 70 creates a summary video by connecting the cut videos and inserting appropriate cut videos so as not to form an unnatural video (step 6).

[0106] Although the video editing device, video editing program and video editing method according to the embodiments of the present invention are described above, the technical scope of the present invention is not limited to the above embodiments, and various inventions can be made by appropriately combining the multiple components disclosed in the above embodiments without departing from the main purpose of the present invention.

[0107] For example, in the above-described embodiment, the scene video creation unit 30, the scene video evaluation unit 40, the scene video selection unit 50, and the cut video selection unit 60 have independent configurations. However, the scene video creation unit 30, the scene video evaluation unit 40, and the scene video selection unit 50 may each be included in the cut video selection unit 60, or some components may be deleted from the components described in other embodiments.

[0108] As described above, the video editing apparatus, the video editing program, and the video editing method according to the present invention can efficiently create a digest video obtained by collecting scenes in which a viewer will show interest.

[0109] Reference Numbers List

[0110] 1: Video editing device

[0111] 10: Image analysis unit

[0112] 11: Color change calculation unit

[0113] 11a: RGB change calculation unit

[0114] 11b: Hue shift average calculation unit

[0115] 12: Text data extraction unit

[0116] 13: OCR analysis unit

[0117] 14: Morphological Analysis Unit

[0118] 15: Object Detection Unit

[0119] 16: Volume level detection unit

[0120] 17: Expression detection unit

[0121] 20: Cut video creation unit

[0122] 30: Scene video creation unit

[0123] 40: Scene Video Evaluation Unit

[0124] 50: Scene video selection unit

[0125] 60: Cut video selection unit

[0126] 70: Summary Video Creation Unit

[0127] 80: Storage unit

Claims

1. A video editing device, comprising: an image analysis unit configured to perform image analysis on video content used as raw material; a cut video creation unit configured to create a cut video by segmenting the video content at predetermined segmentation points based on the analysis result of the image analysis unit; a cut video selection unit configured to select a desired cut video from the cut videos based on the analysis result of the image analysis unit; as well as A summary video creating unit is configured to create a summary video by connecting the cut videos selected by the cut video selecting unit, the summary video having a shorter video time length than the video content.

2. The video editing device according to claim 1, wherein: The cut video selection unit comprises: a scene video creating unit configured to create a scene video by classifying the cut video into a plurality of pre-prepared attribute groups; a scene video evaluation unit configured to evaluate the scene videos by setting an expected score for each of the scene videos; and a scene video selection unit configured to select a desired scene video from the scene videos based on the score set by the scene video evaluation unit, and The cut video selection unit is configured to select a desired cut video from a plurality of cut videos included in the scene video selected by the scene video selection unit.

3. The video editing device according to claim 1 or 2, wherein: The image analysis unit comprises: A color change amount calculation unit, configured to calculate the color change amount between frame images of the video content; a text data extraction unit configured to identify audio data contained in the video content and extract the audio data as text data; and An OCR analysis unit is configured to extract character information contained in the video content, and The cut video creation unit is configured to set a division point for dividing the video content on a time axis using at least one or more parameters among the color change amount, the text data, and the character information as a feature amount.

4. The video editing device according to claim 3, wherein: The color change amount calculation unit comprises: an RGB variation calculation unit configured to divide the temporally preceding and succeeding representative frame images in the video content into a plurality of image regions, and calculate the RGB variation of pixels in each image region of the representative frame image; and a hue shift average value calculation unit configured to calculate a hue shift average value in predetermined adjacent hue regions on a color wheel of the representative frame image, and The color change amount calculation unit is configured to calculate the color change amount based on the RGB change amounts and the hue shift average value.

5. The video editing device according to claim 3, wherein: The image analysis unit further comprises: a morphological analysis unit configured to perform morphological analysis on the text data extracted by the text data extraction unit for each word; and an object detection unit configured to detect a desired object displayed in the video content, and The scene video creation unit is configured to classify the cut video into one of the multiple attribute groups based on at least one of the following: commonality of context based on morphological analysis by the morphological analysis unit, commonality of character information based on OCR analysis by the OCR analysis unit, and presence or absence of movement of an object based on object detection by the object detection unit.

6. The video editing device according to claim 2, wherein: A plurality of content segments corresponding to the program configuration of the video content are stored as the attribute group, the plurality of content segments including a beginning segment indicating content of a beginning portion of the video content, and The scene video creation unit is configured to classify the cut video into one of the plurality of content segments based on content of the cut video.

7. The video editing apparatus according to claim 2, wherein: The scene video evaluation unit is configured to set a score for each scene video by weighting according to a predetermined parameter and a time period, wherein the predetermined parameter indicates a degree of attention from a viewer in each scene video of the video content, and the time period corresponds to a pre-arranged program configuration of the video content.

8. The video editing apparatus according to claim 3, wherein: The image analysis unit further comprises: a volume level detection unit configured to detect the volume level of the video content; and an expression detection unit configured to detect an expression of a performer of the video content, and the cut video selection unit configured to select a desired cut video from: a content segment in which the volume of the video content detected by the volume level detection unit exceeds a predetermined threshold, a content segment in which the expression of the performer of the video content detected by the expression detection unit is in a specific state, or a content segment in which the performer of the video content speaks a specific term based on the text data extracted by the text data extraction unit.

9. A video editing program for causing a computer to perform the following steps: creating a cut video by segmenting video content serving as an original material based on a predetermined segmentation condition; Creating a scene video by classifying the cut video into a plurality of pre-prepared attribute groups; evaluating the scene videos by setting an expected score for each of the scene videos; selecting a desired scene video from the scene videos based on the score; Selecting a desired cut video from a plurality of cut videos included in the scene video; as well as By connecting the cut videos selected in the step of selecting the cut videos, a digest video having a shorter video time length than the video content is created.

10. The video editing program according to claim 9, wherein: The step of creating the cut video comprises the following steps: Calculating the color change between frame images of the video content; identifying audio data contained in the video content and extracting the audio data as text data; extracting character information contained in the video content; and At least one or more parameters among the color change amount, the text data, and the character information are used as feature amounts to set division points for dividing the video content on a time axis.

11. The video editing program according to claim 9 or 10, wherein: The step of calculating the color change amount comprises the following steps: The representative frame images that are temporally preceding and succeeding in the video content are divided into a plurality of image regions, and the RGB variation of pixels in each image region of the representative frame images is calculated; Calculating an average value of hue shifts in predetermined adjacent hue regions on a color wheel of the representative frame image; and The color change amount is calculated based on the RGB change amounts and the hue shift average value.

12. The video editing program according to claim 9 or 10, wherein: The steps of creating the scene video include the following steps: Determining commonalities of contexts of each cut video based on morphological analysis of each word of the text data; detecting a desired object displayed in the video content and determining the presence or absence of movement of the object in each cut video; extracting character information contained in the video content based on OCR analysis, and determining commonality of the character information of each cut video; and The cut videos are classified into one of the plurality of attribute groups based on at least one of the commonality of the contexts, the presence or absence of movement of the objects, and the commonality of the character information.

13. The video editing program according to claim 9 or 10, wherein: The step of selecting the video to be cut comprises the following steps: determining whether a volume level of the video content is equal to or greater than a predetermined threshold; determining whether an expression of a performer of the video content is in a specific state; determining whether a word spoken by a performer of the video content is a specific term; and A desired cut video is selected from the content segment in which the volume level is equal to or greater than a predetermined threshold, the content segment in which the performer's expression is in a specific state, and the content segment in which the performer utters a specific term.

14. A video editing method comprising the following steps: creating a cut video by segmenting video content serving as an original material based on a predetermined segmentation condition; Creating a scene video by classifying the cut video into a plurality of pre-prepared attribute groups; evaluating the scene videos by setting an expected score for each of the scene videos; selecting a desired scene video from the scene videos based on the score; Selecting a desired cut video from a plurality of cut videos included in the scene video; as well as By connecting the cut videos selected in the step of selecting the cut videos, a digest video having a shorter video time length than the video content is created.

Images