Video subtitle summarization method and device, equipment and storage medium

Abstract

The invention discloses a video subtitle summarizing method, device and equipment and a storage medium, and relates to the field of data processing, and the method comprises the steps: obtaining the video duration and / or subtitle number of a target video, grouping the subtitles of the target video based on the video duration and / or subtitle number, obtaining a plurality of subtitle groups, subtitle summarization is carried out on each subtitle group; the subtitles of the target video are grouped based on the video duration and / or the subtitle number, and then the subtitles of each subtitle group are summarized, so that the subtitle summarizing efficiency is effectively improved, a user can know the video content before watching the video, and the user experience is improved. The problem that the time of the user is wasted due to the fact that the video summary is inconsistent with the video content is effectively avoided.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a video subtitle summarization method, device, equipment and storage medium. Background Art

[0002] With the development of video platforms and the increase in individual creators, the number of videos on the Internet is increasing. However, the titles of many videos may be inconsistent with their actual content. Therefore, users are easily misled by the titles of videos. It is currently impossible to effectively summarize the content of the video, resulting in users being unable to identify the actual content of the video before watching it, wasting a lot of users' time.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a video subtitle summarization method, device, equipment and storage medium, aiming to solve the technical problem that the prior art cannot effectively summarize the video content, resulting in the user being unable to identify the actual content of the video before watching the video, wasting a lot of user time.

[0005] To achieve the above object, the present invention provides a video subtitle summarization method, which comprises the following steps:

[0006] Get the video length and / or subtitle quantity of the target video;

[0007] Grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups;

[0008] Provide a subtitle summary for each subtitle group.

[0009] Optionally, grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups includes:

[0010] Determine the number of subtitle groups for the target video based on the video duration and / or the number of subtitles;

[0011] The subtitles of the target video are grouped based on the number of subtitle groups to obtain multiple subtitle groups.

[0012] Optionally, grouping the subtitles of the target video based on the number of subtitle groups to obtain multiple subtitle groups includes:

[0013] Obtaining the subtitle time of each subtitle in the target video;

[0014] Determining a segment type of each subtitle based on the subtitle moment;

[0015] The subtitles of the target video are grouped based on the subtitle grouping quantity and the segment type to obtain multiple subtitle groups.

[0016] Optionally, before grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups, the method further includes:

[0017] Determine whether the video duration exceeds a preset duration;

[0018] If the duration of the video does not exceed the preset duration, grouping the subtitles of the target video;

[0019] If the video duration exceeds a preset duration, the step of grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups is performed.

[0020] Optionally, summarizing the subtitles for each subtitle group includes:

[0021] Perform word segmentation on each subtitle group to obtain multiple initial words;

[0022] Screening the multiple initial words to obtain keywords in each subtitle group;

[0023] Obtain the video type of the target video;

[0024] Subtitle summaries are performed for each subtitle group based on the video type and the keywords.

[0025] Optionally, the screening of the multiple initial words to obtain keywords in each subtitle group includes:

[0026] Get the part of speech of each initial word;

[0027] Screening the multiple initial words based on the parts of speech to obtain multiple candidate words;

[0028] Perform word frequency statistics on each candidate word to obtain the occurrence frequency of each candidate word;

[0029] Determine whether the occurrence frequency of each candidate word exceeds a preset frequency;

[0030] Based on the judgment result, keywords in each subtitle group are screened out from the multiple candidate words.

[0031] Optionally, obtaining the video duration and / or the number of subtitles of the target video includes:

[0032] Get the video length of the target video;

[0033] Extracting a target image frame containing subtitles in the target video based on the video duration;

[0034] The number of subtitles of the target video is obtained based on the target image frame.

[0035] Optionally, extracting a target image frame containing subtitles in the target video based on the video duration includes:

[0036] Decomposing the target video into a plurality of initial image frames based on the video duration;

[0037] Performing image processing on the multiple initial image frames to obtain multiple image frames to be identified;

[0038] Performing subtitle recognition on the multiple image frames to be recognized;

[0039] A target image frame containing subtitles is extracted from the plurality of image frames to be identified based on the subtitle recognition result.

[0040] Optionally, performing image processing on the multiple initial image frames to obtain multiple image frames to be identified includes:

[0041] Obtaining image resolutions of a plurality of initial image frames;

[0042] determining a target size based on the image resolution;

[0043] Adjusting the image sizes of the plurality of initial image frames to the target size;

[0044] Performing noise reduction processing on the multiple initial image frames after image size adjustment to obtain multiple candidate image frames;

[0045] Contrast adjustment is performed on the multiple candidate image frames to obtain multiple image frames to be identified.

[0046] Optionally, the step of performing contrast adjustment on the multiple candidate image frames to obtain multiple image frames to be identified includes:

[0047] Dividing each candidate image frame into a plurality of sub-regions;

[0048] Determining a region to be adjusted among the multiple sub-regions;

[0049] Obtaining grayscale distribution information of the area to be adjusted;

[0050] Based on the grayscale distribution information, contrast adjustment is performed on the areas to be adjusted in the multiple candidate image frames to obtain multiple image frames to be identified.

[0051] In addition, to achieve the above object, the present invention also proposes a video subtitle summarizing device, the video subtitle summarizing device comprising:

[0052] A subtitle acquisition module, used to obtain the video duration and / or number of subtitles of a target video;

[0053] A subtitle grouping module, used for grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups;

[0054] The subtitle summary module is used to summarize the subtitles of each subtitle group.

[0055] Optionally, the subtitle grouping module is further used to determine the number of subtitle groups of the target video based on the video duration and / or the number of subtitles; and group the subtitles of the target video based on the number of subtitle groups to obtain multiple subtitle groups.

[0056] Optionally, the subtitle grouping module is further used to obtain the subtitle time of each subtitle in the target video; determine the segmentation type of each subtitle based on the subtitle time; and group the subtitles of the target video based on the number of subtitle groups and the segmentation type to obtain multiple subtitle groups.

[0057] Optionally, the subtitle grouping module is also used to determine whether the video length exceeds a preset length; if the video length does not exceed the preset length, the subtitles of the target video are grouped; if the video length exceeds the preset length, the subtitles of the target video are grouped based on the video length and / or the number of subtitles to obtain a plurality of subtitle groups.

[0058] Optionally, the subtitle summarizing module is further used to perform word segmentation processing on each subtitle group to obtain multiple initial words; filter the multiple initial words to obtain keywords in each subtitle group; obtain the video type of the target video; and perform subtitle summarization on each subtitle group based on the video type and the keywords.

[0059] Optionally, the subtitle summarizing module is further used to obtain the part of speech of each initial word; screen the multiple initial words based on the part of speech to obtain multiple candidate words; perform word frequency statistics on each candidate word to obtain the frequency of occurrence of each candidate word; determine whether the frequency of occurrence of each candidate word exceeds a preset frequency; and screen out keywords in each subtitle group from the multiple candidate words based on the determination result.

[0060] Optionally, the subtitle acquisition module is further used to acquire the video duration of the target video; extract target image frames containing subtitles in the target video based on the video duration; and acquire the number of subtitles of the target video based on the target image frames.

[0061] Optionally, the subtitle acquisition module is further used to decompose the target video into multiple initial image frames based on the video length; perform image processing on the multiple initial image frames to obtain multiple image frames to be identified; perform subtitle recognition on the multiple image frames to be identified; and extract target image frames containing subtitles from the multiple image frames to be identified based on the subtitle recognition results.

[0062] In addition, to achieve the above-mentioned purpose, the present invention also proposes a video subtitle summarizing device, which includes: a memory, a processor, and a video subtitle summarizing program stored in the memory and executable on the processor, wherein the video subtitle summarizing program is configured to implement the steps of the video subtitle summarizing method described above.

[0063] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a video subtitle summary program is stored, and when the video subtitle summary program is executed by a processor, the steps of the video subtitle summary method described above are implemented.

[0064] The present invention obtains the video length and / or the number of subtitles of a target video, groups the subtitles of the target video based on the video length and / or the number of subtitles, obtains multiple subtitle groups, and summarizes the subtitles for each subtitle group. Since the present invention groups the subtitles of the target video based on the video length and / or the number of subtitles, and then summarizes the subtitles for each subtitle group, the efficiency of subtitle summarization is effectively improved, thereby ensuring that the user can understand the video content before watching the video, and effectively avoiding the problem of inconsistency between the video summary and the video content, which leads to a waste of user time. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a structural diagram of a video subtitle summarizing device in a hardware operating environment involved in an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of the flow chart of the first embodiment of the video subtitle summarization method of the present invention;

[0067] Figure 3 A schematic diagram of a flow chart of a second embodiment of a video subtitle summarization method according to the present invention;

[0068] Figure 4 A schematic diagram of a flow chart of a third embodiment of a video subtitle summarizing method according to the present invention;

[0069] Figure 5 It is a structural block diagram of the first embodiment of the video subtitle summarizing device of the present invention.

[0070] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0072] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of the structure of a video subtitle summary device in a hardware operating environment according to an embodiment of the present invention.

[0073] like Figure 1 As shown, the video subtitle summary device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0074] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the video subtitle summarization device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0075] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a video subtitle summary program.

[0076] exist Figure 1 In the video subtitle summarizing device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the video subtitle summarizing device of the present invention can be set in the video subtitle summarizing device, and the video subtitle summarizing device calls the video subtitle summarizing program stored in the memory 1005 through the processor 1001, and executes the video subtitle summarizing method provided by the embodiment of the present invention.

[0077] The embodiment of the present invention provides a video subtitle summarization method, referring to Figure 2 , Figure 2 The figure is a flow chart of a first embodiment of a method for summarizing video subtitles according to the present invention.

[0078] In this embodiment, the video subtitle summarization method includes the following steps:

[0079] Step S10: Obtain the video length and / or number of subtitles of the target video.

[0080] It should be understood that the executor of the method of this embodiment can be a video subtitle summarizing device with data processing, network communication and program running functions, such as a computer, or other devices or equipment that can achieve the same or similar functions. The above-mentioned video subtitle summarizing device (hereinafter referred to as the subtitle summarizing device) is used as an example for explanation here.

[0081] It should be noted that the target video may be a video such as a movie, a short video, or an animation that needs to be summarized with subtitles. For example, if the page currently displayed on the user terminal contains video A, video B, and video C, then the above video A, video B, and video C are the target videos. The above video duration may be the total duration from the start time to the end time of the target video. The above number of subtitles may be the number of non-repeated subtitles contained in all video frames in the target video.

[0082] It should be understood that the subtitle summarizing device determines the target video that needs to be subtitled by obtaining the video contained in the current page or the current interface, obtains the video attribute information of the target video based on the page information or the interface information, obtains the start time and the end time of the target video based on the video attribute information, thereby determining the video length of the target video, obtains the subtitle file of the target video, and obtains the number of subtitles of the target video based on the subtitle file.

[0083] In a specific implementation, the subtitle summarizing device can also parse the video frames of the target video, obtain multiple video frames, identify the subtitle frames containing subtitles in each video frame, eliminate the subtitle frames with repeated subtitles, obtain the target subtitle frames, and determine the number of subtitles based on the number of target subtitle frames.

[0084] Step S20: grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups.

[0085] It should be noted that the subtitle summarizing device can group subtitles based on the video length of the target video, can group subtitles based on the number of subtitles in the target video, or can group subtitles of the target video based on the video length and the number of subtitles.

[0086] It should be understood that this embodiment can divide the subtitles of the video into one or more segments for summary based on the video length of the target video. For example, if the video does not exceed one minute, the subtitles are directly summarized in full without segmentation; if the video exceeds one minute, the subtitles are divided into multiple segments or parts based on the length for segmented summary, thereby improving the summarization efficiency.

[0087] For example, if the video length is t, if 0min≤t<1min, directly summarize the entire video without considering the time period; if 1min≤t<5min, divide all subtitles into three parts for summary; if 5min≤t<10min, divide all subtitles into five parts for summary; if 10min≤t<20min, divide all subtitles into six parts for summary; if 20≤t, divide all subtitles into seven parts for summary.

[0088] It can be understood that this embodiment can summarize the subtitles of the target video into one or more segments based on the number of subtitles of the target video. For example, if the number of video subtitles is less than 10, all subtitles are directly summarized without segmentation; if the number of video subtitles exceeds 10, the subtitles are divided into multiple segments for summary.

[0089] For example, the subtitle summary device can set the segmentation number interval to segment the subtitles, with 10 subtitles as the segmentation number interval. If the number of video subtitles is less than 10, no segmentation summary will be performed; if the number of video subtitles is 11, the subtitles will be summarized in two segments, with the first 10 being divided into one segment and the last being divided into one segment; if the number of video subtitles is 19, the subtitles will be summarized in two segments, with the first 10 being divided into one segment and the last 9 being divided into one segment; if the number of video subtitles is 29, the subtitles will be summarized in three segments, with the first 10 being divided into one segment, the middle 10 being divided into one segment, and the last 9 being divided into one segment for summary.

[0090] It should be understood that the minimum value of subtitles is 10 subtitles. For 1min≤t<5min, 5min≤t<10min, 10min≤t<20min, 20≤t, each time period must be summarized into at least two sections. If t≤10, the time period is not summarized. If it is 11, it is divided into two sections: 10 and 1; if it is 19, it is divided into two sections: 10 and 9.

[0091] Furthermore, in order to effectively group the subtitles, the above step S20 may include:

[0092] Step S201: determining the number of subtitle groups of the target video based on the video duration and / or the number of subtitles;

[0093] Step S202: grouping the subtitles of the target video based on the number of subtitle groups to obtain multiple subtitle groups.

[0094] It should be noted that the number of subtitle groups can be the number of groupings of the target number of subtitles. For example, if the number of subtitle groups is 2, the subtitles of the target video are divided into two groups of subtitles; if the number of subtitle groups is 1, the subtitles of the target video are not grouped and the subtitles are directly summarized.

[0095] It should be understood that the subtitle summarizing device can determine whether to group and summarize the subtitles of the target video based on the video length and / or the number of subtitles. If the video length of the target video is too short or the number of subtitles is too small, there is no need to group the subtitles and the subtitles can be summarized directly. For example, if the video length of the target video is less than 1 minute or the number of subtitles of the target video is less than 10, there is no need to group the subtitles and the subtitles can be summarized directly.

[0096] Further, to accurately determine the subtitle segmentation method of the target video, the above step S202 may include:

[0097] Obtaining the subtitle time of each subtitle in the target video;

[0098] Determining a segment type of each subtitle based on the subtitle moment;

[0099] The subtitles of the target video are grouped based on the subtitle grouping quantity and the segment type to obtain multiple subtitle groups.

[0100] It should be noted that the subtitle time may be the time when each subtitle included in the target video appears in the video. For example, if subtitle A appears in the target video at 23 minutes and 15 seconds, then the subtitle time of subtitle A is 23 minutes and 15 seconds.

[0101] It should be understood that the present implementation plans groups for each subtitle based on the subtitle moment of each subtitle, obtains the total duration of the target video, generates a subtitle sequence for the target video based on the total duration and the subtitle moment of each subtitle, and determines the grouping order of each subtitle based on the subtitle sequence. For example, the target video includes subtitles A, B, C, and D. Based on the subtitle moment of each subtitle, the grouping order of each subtitle is determined, and based on the grouping order and the number of groups, subtitles A and B are grouped into the same group, and subtitles C and D are grouped into the same group.

[0102] In a specific implementation, the subtitle summarizing device can generate at least one subtitle group based on the number of subtitle groups, and configure a summary order for each subtitle group, obtain the subtitle time of each subtitle, determine the segmentation type of each subtitle based on the subtitle time, generate a subtitle sequence based on the segmentation type, and assign each subtitle to each subtitle group based on the subtitle sequence to obtain multiple subtitle groups.

[0103] Furthermore, in order to improve the efficiency of subtitle grouping, before the above step S20, the following steps may be included:

[0104] Determine whether the video duration exceeds a preset duration;

[0105] If the duration of the video does not exceed the preset duration, grouping the subtitles of the target video;

[0106] If the video duration exceeds a preset duration, the step of grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups is performed.

[0107] It should be noted that the preset duration can be a pre-set judgment duration for determining the video duration type. For example, the preset duration can be 1 minute, 2 minutes, etc. If the video duration does not exceed the preset duration, it is determined that the target video duration is too short. There is no need to group based on the video duration, and grouping can be performed directly based on the number of subtitles of the target video.

[0108] In the specific implementation, if the video length of the target video does not exceed the preset length, continue to determine whether the number of subtitles of the target video exceeds the preset number. If it does not exceed the preset number, it is determined that the subtitles of the target video do not need to be grouped, and the subtitles are directly summarized. Because the number of subtitles of the target video is too small and the video length is too short, it is more efficient to directly summarize the subtitles.

[0109] Step S30: Summarize the subtitles for each subtitle group.

[0110] It can be understood that the subtitle summarizing device summarizes each subtitle group to obtain summary information of each subtitle group, performs content analysis on the target video based on the summary information to obtain content information of the target video, extracts a summary of the content information to obtain a content summary of the target video, and generates a video title of the target video based on the content summary.

[0111] This embodiment obtains the video length and / or the number of subtitles of the target video, groups the subtitles of the target video based on the video length and / or the number of subtitles, obtains multiple subtitle groups, and summarizes the subtitles for each subtitle group. Since this embodiment groups the subtitles of the target video based on the video length and / or the number of subtitles, and then summarizes the subtitles for each subtitle group, the efficiency of subtitle summarization is effectively improved, thereby ensuring that the user can understand the video content before watching the video, and effectively avoiding the problem of inconsistency between the video summary and the video content, which leads to a waste of user time.

[0112] refer to Figure 3 , Figure 3 FIG. 4 is a flow chart of a second embodiment of a video subtitle summarization method according to the present invention.

[0113] Based on the above first embodiment, in this embodiment, step S30 includes:

[0114] Step S31: performing word segmentation processing on each subtitle group to obtain a plurality of initial words.

[0115] It should be noted that the initial word may be a word obtained by segmenting a subtitle group to obtain a plurality of characters and then matching each character with a preset dictionary.

[0116] It should be understood that this embodiment obtains a subtitle segmentation strategy, obtains a segmentation granularity based on the subtitle segmentation strategy, segments each subtitle group based on the segmentation granularity to obtain multiple original characters, matches each original character with a dictionary of the subtitle segmentation strategy, and obtains multiple initial words.

[0117] Step S32: Screening the multiple initial words to obtain keywords in each subtitle group.

[0118] It should be noted that the keyword may be a word with important meaning contained in each vocabulary of the subtitle group, and the subtitle summarizing device may determine the meaning of the subtitle based on the meaning of the keyword.

[0119] It should be understood that the subtitle summarizing device obtains a stop word dictionary, performs stop word filtering on the multiple initial words based on the stop word dictionary, obtains candidate words, counts the frequency of each candidate word in the subtitles, obtains the word frequency of each candidate word, determines whether the word frequency of each candidate word exceeds a preset threshold, and obtains keywords in each subtitle group based on the determination result.

[0120] Furthermore, in order to accurately identify the keywords in each subtitle group, the above step S32 may include:

[0121] Step S321: Obtain the part of speech of each initial word;

[0122] Step S322: screening the multiple initial words based on the parts of speech to obtain multiple candidate words;

[0123] Step S323: Perform word frequency statistics on each candidate word to obtain the occurrence frequency of each candidate word;

[0124] Step S324: determining whether the occurrence frequency of each candidate word exceeds a preset frequency;

[0125] Step S325: filtering out keywords in each subtitle group from the plurality of candidate words based on the determination result.

[0126] It should be noted that the subtitle summarizing device obtains a stop word dictionary, performs stop word filtering on multiple initial words based on the stop word dictionary and the part of speech of each initial word, removes meaningless stop words, thereby effectively reducing word noise, and obtains multiple candidate words. It performs word frequency statistics on each candidate word to obtain the frequency of occurrence of each candidate word, sorts the words after statistics based on the frequency of occurrence, generates a word sequence, inputs the word sequence into a pre-constructed keyword extraction model, evaluates the importance of each candidate word, and screens out keywords in each subtitle group.

[0127] Step S33: Obtain the video type of the target video.

[0128] It should be noted that the video type may be the video length type, subtitle quantity type, and video style type of the target video. For example, the video style type may include movies, documentaries, animations, and self-media videos.

[0129] Step S34: summarizing the subtitles for each subtitle group based on the video type and the keywords.

[0130] It can be understood that the subtitle summarizing device obtains the summarizing strategy based on the video type, obtains the meaning of the keywords in each subtitle group, and summarizes the subtitles for each subtitle group based on the keyword meaning and the summarizing strategy.

[0131] This embodiment performs word segmentation processing on each subtitle group to obtain multiple initial words, screens the multiple initial words to obtain keywords in each subtitle group, obtains the video type of the target video, and summarizes the subtitles for each subtitle group based on the video type and the keywords. Since this embodiment performs word segmentation processing on each subtitle group and screens keywords from multiple initial words, the video type of the target video is accurately obtained, the accuracy of the subtitle summary is improved, and the problem of inaccurate summary misleading users is effectively avoided.

[0132] refer to Figure 4 , Figure 4 FIG. 4 is a flow chart of a third embodiment of a method for summarizing video subtitles according to the present invention.

[0133] Based on the above first embodiment, in this embodiment, step S10 includes:

[0134] Step S11: Obtain the video duration of the target video.

[0135] It can be understood that, in this embodiment, by obtaining the start time and the end time of the target video, the video duration of the target video is calculated based on the start time and the end time.

[0136] In a specific implementation, the subtitle summarizing device can identify meaningless invalid frames in the target video, determine the invalid duration of the invalid frames, obtain the start time and end time of the target video, calculate the total duration of the target video based on the start time and end time, subtract the invalid duration from the total video duration, and obtain the video duration of the target video containing valid pictures, thereby effectively avoiding the problem of invalid pictures in the video reducing the efficiency of subtitle summarization.

[0137] Step S12: extracting target image frames containing subtitles from the target video based on the video duration.

[0138] It should be noted that the target image frame may be an image frame including subtitles among the image frames of the target video.

[0139] It should be understood that, in this embodiment, multiple image frames contained in the target video are extracted based on the video duration, each image frame is identified respectively, and the target image frame containing the subtitle is determined.

[0140] Furthermore, in order to accurately extract the target image frame, the above step S12 may include:

[0141] Step S121: decomposing the target video into a plurality of initial image frames based on the video duration;

[0142] Step S122: performing image processing on the multiple initial image frames to obtain multiple image frames to be identified;

[0143] Step S123: performing subtitle recognition on the multiple image frames to be recognized;

[0144] Step S124: extracting a target image frame containing subtitles from the plurality of image frames to be identified based on the subtitle recognition result.

[0145] It should be noted that the initial image frame can be a total of image frames contained in the target video. For example, if the target video is a 24-frame refresh rate video, then a 1-second frame of the target video is decomposed into 24 initial image frames based on the video duration. The image frame to be identified can be an image frame obtained after image processing (such as noise reduction, grayscale adjustment, etc.) of the initial image frame.

[0146] It should be understood that the subtitle summarizing device decomposes the target video into multiple initial image frames based on the video length and the number of frames of the target video, performs noise reduction and grayscale adjustment on the multiple initial image frames to obtain multiple images to be identified, performs subtitle recognition on the multiple image frames to be identified, and extracts the target image frame containing subtitles from the multiple image frames to be identified based on the subtitle recognition results.

[0147] Furthermore, in order to improve the recognition efficiency of the image frame, the above step S122 may include:

[0148] Obtaining image resolutions of a plurality of initial image frames;

[0149] determining a target size based on the image resolution;

[0150] Adjusting the image sizes of the plurality of initial image frames to the target size;

[0151] Performing noise reduction processing on the multiple initial image frames after image size adjustment to obtain multiple candidate image frames;

[0152] Contrast adjustment is performed on the multiple candidate image frames to obtain multiple image frames to be identified.

[0153] It should be noted that the image resolution may be the resolution of each initial image frame. The target size may be the screen size of each image frame adjusted in the same manner.

[0154] It can be understood that this embodiment improves image processing efficiency by uniformly adjusting each initial image frame to a target size based on the image resolution of each initial image frame, and then performs noise reduction processing on each initial image frame to reduce image noise to obtain multiple candidate image frames, and performs contrast adjustment on the candidate image frames to enhance image contrast, thereby improving recognition efficiency.

[0155] Furthermore, in order to improve the recognition accuracy of the image frames, the contrast of the multiple candidate image frames is adjusted to obtain multiple image frames to be recognized, including:

[0156] Dividing each candidate image frame into a plurality of sub-regions;

[0157] Determining a region to be adjusted among the multiple sub-regions;

[0158] Obtaining grayscale distribution information of the area to be adjusted;

[0159] Based on the grayscale distribution information, contrast adjustment is performed on the areas to be adjusted in the multiple candidate image frames to obtain multiple image frames to be identified.

[0160] It should be noted that, since subtitles usually appear at the top or bottom of the picture, this embodiment adjusts the picture frame by region, thereby effectively reducing the workload of image processing and improving recognition efficiency. The above-mentioned region to be adjusted can be a region in the candidate image frame containing subtitles.

[0161] In a specific implementation, the subtitle summarizing device obtains the picture resolution of the candidate image frame, divides the candidate image frame based on the picture resolution to obtain multiple sub-regions, determines the area to be adjusted in the sub-region where subtitles may exist based on the subtitle distribution information of the target video, obtains the grayscale distribution information of the area to be adjusted, and adjusts the contrast of the area to be adjusted in the multiple candidate image frames based on the grayscale distribution information to obtain multiple image frames to be identified.

[0162] Step S13: Acquire the number of subtitles of the target video based on the target image frame.

[0163] It should be understood that this embodiment obtains valid image frames by identifying subtitles in target image frames, extracting repeated frames with repeated subtitles in target image frames, and determining the number of subtitles in the target video based on the number of valid image frames.

[0164] This embodiment obtains the video duration of a target video, extracts target image frames containing subtitles in the target video based on the video duration, and obtains the number of subtitles in the target video based on the target image frames. Since this embodiment accurately obtains the number of subtitles in the target video by extracting target image frames containing subtitles in the target video, the efficiency of subtitle grouping is improved, thereby improving the efficiency of subtitle summarization.

[0165] In addition, an embodiment of the present invention further provides a storage medium, on which a video subtitle summarizing program is stored. When the video subtitle summarizing program is executed by a processor, the steps of the video subtitle summarizing method described above are implemented.

[0166] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0167] Reference Figure 5 , Figure 5 It is a structural block diagram of the first embodiment of the video subtitle summarizing device of the present invention.

[0168] like Figure 5 As shown, the video subtitle summarizing device proposed in the embodiment of the present invention includes:

[0169] A subtitle acquisition module 10 is used to acquire the video duration and / or the number of subtitles of a target video;

[0170] A subtitle grouping module 20, configured to group the subtitles of the target video based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups;

[0171] The subtitle summarizing module 30 is used to summarize the subtitles of each subtitle group.

[0172] Furthermore, the subtitle grouping module 20 is further configured to determine the number of subtitle groups of the target video based on the video duration and / or the number of subtitles; and to group the subtitles of the target video based on the number of subtitle groups to obtain a plurality of subtitle groups.

[0173] Furthermore, the subtitle grouping module 20 is also used to obtain the subtitle time of each subtitle in the target video; determine the segment type of each subtitle based on the subtitle time; group the subtitles of the target video based on the number of subtitle groups and the segment type to obtain multiple subtitle groups.

[0174] Furthermore, the subtitle grouping module 20 is also used to determine whether the video length exceeds a preset length; if the video length does not exceed the preset length, the subtitles of the target video are grouped; if the video length exceeds the preset length, the subtitles of the target video are grouped based on the video length and / or the number of subtitles to obtain a plurality of subtitle groups.

[0175] Furthermore, the subtitle summarizing module 30 is also used to perform word segmentation processing on each subtitle group to obtain multiple initial words; filter the multiple initial words to obtain keywords in each subtitle group; obtain the video type of the target video; and perform subtitle summarization on each subtitle group based on the video type and the keywords.

[0176] Furthermore, the subtitle summarizing module 30 is also used to obtain the part of speech of each initial word; screen the multiple initial words based on the part of speech to obtain multiple candidate words; perform word frequency statistics on each candidate word to obtain the frequency of occurrence of each candidate word; determine whether the frequency of occurrence of each candidate word exceeds a preset frequency; and screen out keywords in each subtitle group from the multiple candidate words based on the determination result.

[0177] Furthermore, the subtitle acquisition module 10 is also used to acquire the video duration of the target video; extract the target image frame containing subtitles in the target video based on the video duration; and acquire the number of subtitles of the target video based on the target image frame.

[0178] Furthermore, the subtitle acquisition module 10 is also used to decompose the target video into multiple initial image frames based on the video length; perform image processing on the multiple initial image frames to obtain multiple image frames to be identified; perform subtitle recognition on the multiple image frames to be identified; and extract target image frames containing subtitles from the multiple image frames to be identified based on the subtitle recognition results.

[0179] Furthermore, the subtitle acquisition module 10 is also used to obtain image resolutions of multiple initial image frames; determine a target size based on the image resolution; adjust the image sizes of the multiple initial image frames to the target size; perform noise reduction processing on the multiple initial image frames after the image size adjustment to obtain multiple candidate image frames; and perform contrast adjustment on the multiple candidate image frames to obtain multiple image frames to be identified.

[0180] Furthermore, the subtitle acquisition module 10 is also used to divide each candidate image frame into multiple sub-regions; determine the area to be adjusted in the multiple sub-regions; obtain grayscale distribution information of the area to be adjusted; and adjust the contrast of the area to be adjusted in the multiple candidate image frames based on the grayscale distribution information to obtain multiple image frames to be identified.

[0181] This embodiment obtains the video length and / or the number of subtitles of the target video, groups the subtitles of the target video based on the video length and / or the number of subtitles, obtains multiple subtitle groups, and summarizes the subtitles for each subtitle group. Since this embodiment groups the subtitles of the target video based on the video length and / or the number of subtitles, and then summarizes the subtitles for each subtitle group, the efficiency of subtitle summarization is effectively improved, thereby ensuring that the user can understand the video content before watching the video, and effectively avoiding the problem of inconsistency between the video summary and the video content, which leads to a waste of user time.

[0182] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0183] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0184] In addition, for technical details not fully described in this embodiment, reference may be made to the video subtitle summarization method provided in any embodiment of the present invention, and will not be repeated here.

[0185] The present invention proposes A1, a video subtitle summarization method, the video subtitle summarization method comprising:

[0186] Get the video length and / or subtitle quantity of the target video;

[0187] Grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups;

[0188] Provide a subtitle summary for each subtitle group.

[0189] A2. The video subtitle summarization method as described in A1, wherein the subtitles of the target video are grouped based on the video duration and / or the number of subtitles to obtain multiple subtitle groups, including:

[0190] Determine the number of subtitle groups for the target video based on the video duration and / or the number of subtitles;

[0191] The subtitles of the target video are grouped based on the number of subtitle groups to obtain multiple subtitle groups.

[0192] A3. The video subtitle summarizing method as described in A2, wherein the subtitles of the target video are grouped based on the number of subtitle groups to obtain multiple subtitle groups, including:

[0193] Obtaining the subtitle time of each subtitle in the target video;

[0194] Determining a segment type of each subtitle based on the subtitle moment;

[0195] The subtitles of the target video are grouped based on the subtitle grouping quantity and the segment type to obtain multiple subtitle groups.

[0196] A4, as described in A2, the video subtitle summarization method, before grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups, also includes:

[0197] Determine whether the video duration exceeds a preset duration;

[0198] If the duration of the video does not exceed the preset duration, grouping the subtitles of the target video;

[0199] If the video duration exceeds a preset duration, the step of grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups is performed.

[0200] A5. The video subtitle summarizing method as described in A1, wherein summarizing subtitles for each subtitle group comprises:

[0201] Perform word segmentation on each subtitle group to obtain multiple initial words;

[0202] Screening the multiple initial words to obtain keywords in each subtitle group;

[0203] Obtain the video type of the target video;

[0204] Subtitle summaries are performed for each subtitle group based on the video type and the keywords.

[0205] A6. The video subtitle summarizing method as described in A5, wherein the multiple initial words are screened to obtain keywords in each subtitle group, including:

[0206] Get the part of speech of each initial word;

[0207] Screening the multiple initial words based on the parts of speech to obtain multiple candidate words;

[0208] Perform word frequency statistics on each candidate word to obtain the occurrence frequency of each candidate word;

[0209] Determine whether the occurrence frequency of each candidate word exceeds a preset frequency;

[0210] Based on the judgment result, keywords in each subtitle group are screened out from the multiple candidate words.

[0211] A7, as described in A1, the video subtitle summarization method, wherein the step of obtaining the video duration and / or the number of subtitles of the target video comprises:

[0212] Get the video length of the target video;

[0213] Extracting a target image frame containing subtitles in the target video based on the video duration;

[0214] The number of subtitles of the target video is obtained based on the target image frame.

[0215] A8. The video subtitle summarization method as described in A7, wherein extracting the target image frame containing subtitles in the target video based on the video duration comprises:

[0216] Decomposing the target video into a plurality of initial image frames based on the video duration;

[0217] Performing image processing on the multiple initial image frames to obtain multiple image frames to be identified;

[0218] Performing subtitle recognition on the multiple image frames to be recognized;

[0219] A target image frame containing subtitles is extracted from the plurality of image frames to be identified based on the subtitle recognition result.

[0220] A9. The video subtitle summarization method as described in A8, wherein the image processing is performed on the multiple initial image frames to obtain multiple image frames to be identified, including:

[0221] Obtaining image resolutions of a plurality of initial image frames;

[0222] determining a target size based on the image resolution;

[0223] Adjusting the image sizes of the plurality of initial image frames to the target size;

[0224] Performing noise reduction processing on the multiple initial image frames after image size adjustment to obtain multiple candidate image frames;

[0225] Contrast adjustment is performed on the multiple candidate image frames to obtain multiple image frames to be identified.

[0226] A10. The video caption summarizing method as described in A9, wherein the contrast of the multiple candidate image frames is adjusted to obtain multiple image frames to be identified, including:

[0227] Dividing each candidate image frame into a plurality of sub-regions;

[0228] Determining a region to be adjusted among the multiple sub-regions;

[0229] Obtaining grayscale distribution information of the area to be adjusted;

[0230] Based on the grayscale distribution information, contrast adjustment is performed on the areas to be adjusted in the multiple candidate image frames to obtain multiple image frames to be identified.

[0231] B11. A video subtitle summarizing device, the video subtitle summarizing device comprising:

[0232] A subtitle acquisition module, used to obtain the video duration and / or number of subtitles of a target video;

[0233] A subtitle grouping module, used for grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups;

[0234] The subtitle summary module is used to summarize the subtitles of each subtitle group.

[0235] B12. In the video subtitle summarizing device as described in B11, the subtitle grouping module is further used to determine the number of subtitle groups of the target video based on the video duration and / or the number of subtitles; and group the subtitles of the target video based on the number of subtitle groups to obtain multiple subtitle groups.

[0236] B13. In the video subtitle summarizing device as described in B12, the subtitle grouping module is further used to obtain the subtitle time of each subtitle in the target video; determine the segmentation type of each subtitle based on the subtitle time; and group the subtitles of the target video based on the number of subtitle groups and the segmentation type to obtain multiple subtitle groups.

[0237] B14. In the video subtitle summarizing device as described in B12, the subtitle grouping module is further used to determine whether the video duration exceeds a preset duration; if the video duration does not exceed the preset duration, the subtitles of the target video are grouped; if the video duration exceeds the preset duration, the subtitles of the target video are grouped based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups.

[0238] B15. In the video subtitle summarizing device as described in B11, the subtitle summarizing module is further used to perform word segmentation processing on each subtitle group to obtain multiple initial words; screen the multiple initial words to obtain keywords in each subtitle group; obtain the video type of the target video; and perform subtitle summarization on each subtitle group based on the video type and the keywords.

[0239] B16. In the video subtitle summarizing device as described in B15, the subtitle summarizing module is further used to obtain the part of speech of each initial word; screen the multiple initial words based on the part of speech to obtain multiple candidate words; perform word frequency statistics on each candidate word to obtain the appearance frequency of each candidate word; determine whether the appearance frequency of each candidate word exceeds a preset frequency; and screen out keywords in each subtitle group from the multiple candidate words based on the determination result.

[0240] B17. In the video subtitle summarizing device as described in B11, the subtitle acquisition module is further used to obtain the video duration of the target video; extract the target image frame containing subtitles in the target video based on the video duration; and obtain the number of subtitles of the target video based on the target image frame.

[0241] B18. In the video subtitle summarizing device as described in B17, the subtitle acquisition module is further used to decompose the target video into multiple initial image frames based on the video duration; perform image processing on the multiple initial image frames to obtain multiple image frames to be identified; perform subtitle recognition on the multiple image frames to be identified; and extract target image frames containing subtitles from the multiple image frames to be identified based on the subtitle recognition results.

[0242] The present invention also discloses C19, a video subtitle summarizing device, which includes: a memory, a processor, and a video subtitle summarizing program stored in the memory and executable on the processor, wherein the video subtitle summarizing program is configured to implement the video subtitle summarizing method as described above.

[0243] The present invention also discloses D20, a storage medium, on which a video subtitle summary program is stored, and when the video subtitle summary program is executed by a processor, the video subtitle summary method as described above is implemented.

[0244] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0245] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0246] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0247] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video subtitle summarization method, characterized in that: The video subtitle summarization method comprises: Get the video length and / or subtitle quantity of the target video; Grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups; Provide a subtitle summary for each subtitle group.

2. The video subtitle summarization method according to claim 1, characterized in that: The step of grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups includes: Determine the number of subtitle groups for the target video based on the video duration and / or the number of subtitles; The subtitles of the target video are grouped based on the number of subtitle groups to obtain multiple subtitle groups.

3. The video subtitle summarization method according to claim 2, characterized in that: The step of grouping the subtitles of the target video based on the number of subtitle groups to obtain a plurality of subtitle groups includes: Obtaining the subtitle time of each subtitle in the target video; Determining a segment type of each subtitle based on the subtitle moment; The subtitles of the target video are grouped based on the subtitle grouping quantity and the segment type to obtain multiple subtitle groups.

4. The video subtitle summarization method according to claim 2, characterized in that: Before grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups, the method further includes: Determine whether the video duration exceeds a preset duration; If the duration of the video does not exceed the preset duration, grouping the subtitles of the target video; If the video duration exceeds a preset duration, the step of grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain a plurality of subtitle groups is performed.

5. The video subtitle summarization method according to claim 1, characterized in that: The subtitle summarizing of each subtitle group includes: Perform word segmentation on each subtitle group to obtain multiple initial words; Screening the multiple initial words to obtain keywords in each subtitle group; Obtain the video type of the target video; Based on the video type and the keywords, subtitles are summarized for each subtitle group.

6. The video subtitle summarization method according to claim 5, characterized in that: The screening of the multiple initial words to obtain keywords in each subtitle group includes: Get the part of speech of each initial word; Screening the multiple initial words based on the parts of speech to obtain multiple candidate words; Perform word frequency statistics on each candidate word to obtain the occurrence frequency of each candidate word; Determine whether the occurrence frequency of each candidate word exceeds a preset frequency; Based on the judgment result, keywords in each subtitle group are screened out from the multiple candidate words.

7. The video subtitle summarization method according to claim 1, characterized in that: The step of obtaining the video duration and / or the number of subtitles of the target video includes: Get the video length of the target video; Extracting a target image frame containing subtitles in the target video based on the video duration; The number of subtitles of the target video is obtained based on the target image frame.

8. A video subtitle summarizing device, characterized in that: The video subtitle summarizing device comprises: A subtitle acquisition module, used to obtain the video duration and / or number of subtitles of a target video; A subtitle grouping module, used for grouping the subtitles of the target video based on the video duration and / or the number of subtitles to obtain multiple subtitle groups; The subtitle summary module is used to summarize the subtitles of each subtitle group.

9. A video subtitle summarizing device, characterized in that: The video subtitle summarizing device comprises: a memory, a processor, and a video subtitle summarizing program stored in the memory and executable on the processor, wherein the video subtitle summarizing program is configured to implement the video subtitle summarizing method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a video subtitle summarizing program, and when the video subtitle summarizing program is executed by the processor, the video subtitle summarizing method according to any one of claims 1 to 7 is implemented.

Images