Short video production method based on intelligent identification

Through intelligent recognition technology, video manuscripts and segmented video clips are extracted and short video materials are generated, which solves the problem that a large amount of manual participation is required for the production of short videos in the existing technology, and an efficient and objective short video production process is achieved.

CN118972677APending Publication Date: 2024-11-15HANGZHOU ARCVIDEO TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410890117.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, short video production requires a lot of manual participation, resulting in increased costs and mechanized production process, and it is impossible to efficiently process long video content to generate short video summary.

Method used

Through intelligent recognition technology, the voice content in the video is extracted into a manuscript, and the abstract is generated using natural language processing, combined with transition detection and segmented video clips, and extracted video clips through keyword association to generate the final short video material.

Benefits of technology

It greatly simplifies the difficulty of short video production, shortens production cycle, reduces labor costs, and objectively reflects the video content, reducing the subjective influence of the producers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972677A_ABST
    Figure CN118972677A_ABST
Patent Text Reader

Abstract

The invention discloses a short video production method based on intelligent recognition, which comprises the following steps of: extracting voice content in an original video, and converting a language in the original video into a manuscript of the original video through voice recognition; taking the manuscript content of the original video as input information, extracting the manuscript content into summary information of the original video, and marking corresponding keywords; carrying out content identification on the original video, confirming each transition time point, and segmenting the video into a plurality of segments; further comparing with the original manuscript, and associating with the keyword; determining corresponding video clips in the manuscript of the original video through the keywords, and extracting the corresponding video clips conforming to the text abstract; synchronously presenting the video content and the abstract through a playing tool, manually checking the matching degree of the video content and the abstract, and confirming the final effect of the video corresponding to the abstract; and splicing the selected videos, and processing the text abstract by manual voice to generate a final short video material.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of video processing, and in particular relates to a short video production method based on intelligent recognition. Background Art

[0002] Short videos refer to high-frequency video content played on various new media platforms, suitable for viewing in mobile and short leisure states, ranging from a few seconds to a few minutes. The content integrates themes such as skill sharing, humor, fashion trends, social hot spots, street interviews, public welfare education, advertising creativity, and commercial customization. Because the content is short, it can be a separate film or a series of columns. Short videos have become the main form of Internet content dissemination. Naturally, many original long-length report video content, film and television works, and news programs often use the method of intercepting part of the video content as short videos, and then post-produce them into corresponding short video programs as summaries, explanations, highlights, etc. of these film and television works, and the demand for this is increasing day by day. At present, this kind of work still mainly requires manual participation. As the demand for programs increases, the cost of labor is also increasing simultaneously. These relatively mechanical production processes, but they require manual participation to confirm the content of the video, can speed up the production process through intelligent recognition. Summary of the invention

[0003] In view of the above problems, the present invention provides a short video production method based on intelligent recognition, which is used for intelligent processing of video content. After extracting the original manuscript, summary information is generated, and the video content is intercepted in combination with the summary. Finally, these video contents are spliced ​​into short videos to improve video processing efficiency.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A short video production method based on intelligent recognition comprises the following steps:

[0006] Extract the speech content in the original video and convert the language in the original video into the text of the original video through speech recognition;

[0007] The manuscript content of the original video is used as input information, language and text processing is performed based on natural language processing, the manuscript content is extracted into summary information of the original video, and corresponding keywords are marked;

[0008] Perform content recognition on the original video, confirm each transition time point, and divide the video into multiple segments; further compare it with the original manuscript and associate it with keywords; confirm the corresponding video segments in the manuscript of the original video through keywords, and extract the corresponding video segments that meet the text summary;

[0009] The video content and summary are presented synchronously through the playback tool. The matching degree between the video content and the summary is manually reviewed, and the final effect of the video corresponding to the summary is confirmed. The selected videos are spliced ​​together, and the text summary is processed by manual voice to generate the final short video material.

[0010] In a possible implementation manner, extracting the manuscript content into summary information of the original video specifically includes:

[0011] Preprocess the manuscript, including word segmentation and removal of stop words;

[0012] Extract key information from the text, including using the term frequency-inverse document frequency (TF-IDF) method, that is, calculating the frequency of each word in the manuscript and the inverse document frequency in the entire corpus, and selecting words with higher TF-IDF values ​​as keywords; or using a fast automatic keyword extraction method to segment the text into candidate keyword phrases; calculating the term frequency and word co-occurrence of each word; calculating the score of each candidate keyword phrase, and selecting keywords with higher scores;

[0013] Use the extracted keywords to score each sentence; the more keywords a sentence contains, the higher the score; select the sentences with the highest scores as the summary.

[0014] In a possible implementation manner, the step of confirming each transition time point and dividing the video into multiple segments includes the following steps:

[0015] The video is tested using a transition detection algorithm, the detected transitions are time-stamped, the specific time point of each transition is recorded, and the original video is divided into multiple segments based on the recorded transition time points. The start and end points of each segment correspond to two adjacent transition time points respectively; FFmpeg is used to cut the video according to time points to generate multiple independent video segment files.

[0016] In a possible implementation, the transition detection algorithm uses pixel difference-based detection to detect transitions by comparing pixel differences between consecutive frames. If the difference between two frames exceeds a set threshold, it is considered that a transition exists.

[0017] In one possible implementation, the transition detection algorithm adopts feature point-based detection, uses computer vision technology to detect feature points within a frame, and tracks the movement of the feature points between consecutive frames. If the distribution and number of feature points change significantly, it is determined that a transition exists.

[0018] In a possible implementation, the transition detection algorithm uses optical flow-based detection to calculate the optical flow between consecutive frames. If the optical flow changes greatly, a transition may exist.

[0019] In a possible implementation manner, the summary information of the original video corresponds to the number of words in 3-5 minutes of speech.

[0020] The present invention has the following beneficial effects: by combining video intelligent recognition and document processing capabilities, the key content of the video can be extracted from the original video as the content material of the short video. It has an excellent improvement effect on the short video production of the original video content, especially reporting materials, product descriptions, documentaries, etc., long videos of single character descriptions, greatly simplifies the production difficulty of such short videos, shortens the production cycle, greatly reduces the labor cost, and, combined with computer recognition and processing capabilities, can more objectively reflect the video content and avoid the influence of the subjective consciousness of the production staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of the steps of a short video production method based on intelligent recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] See also Figure 1 , which is a flowchart of a short video production method based on intelligent recognition according to an embodiment of the present invention, comprising the following steps:

[0024] S10, extracting the voice content in the original video, and converting the language in the original video into the text of the original video through voice recognition; for better effect, the recognition errors in the text can be further corrected to ensure that the text is consistent with the voice content.

[0025] S20, taking the manuscript content of the original video as input information, performing language and text processing based on natural language processing, extracting the manuscript content into summary information of the original video, and marking corresponding keywords; specifically, the summary information of the original video corresponds to the number of words in 3-5 minutes of speech.

[0026] S30, performing content recognition on the original video, confirming each transition time point, and dividing the video into multiple segments; and further comparing with the original manuscript, and associating with the keywords; confirming the corresponding video segments in the manuscript of the original video through the keywords, and extracting the corresponding video segments that meet the text summary;

[0027] S40, through the playback tool, the video content and the summary are synchronously presented, and the matching degree of the video content and the summary is manually reviewed, and the final effect of the video corresponding to the summary is confirmed; the selected videos are spliced, and the text summary is processed by manual voice to generate the final short video material.

[0028] In a short video production method based on intelligent recognition according to another embodiment of the present invention, extracting the manuscript content into summary information of the original video in S20 specifically includes:

[0029] Preprocess the manuscript, including word segmentation and removal of stop words;

[0030] Extract key information from the text, including using the term frequency-inverse document frequency (TF-IDF) method, that is, calculating the frequency of each word in the manuscript and the inverse document frequency in the entire corpus, and selecting words with higher TF-IDF values ​​as keywords; or using a fast automatic keyword extraction method to segment the text into candidate keyword phrases; calculating the term frequency and word co-occurrence of each word; calculating the score of each candidate keyword phrase, and selecting keywords with higher scores;

[0031] Use the extracted keywords to score each sentence; the more keywords a sentence contains, the higher the score; select the sentences with the highest scores as the summary.

[0032] Specifically, the formula for the term frequency-inverse document frequency TF-IDF method can be as follows\(\text{TF-IDF}(t,d,D)=\text{TF}(t,d)\times\text{IDF}(t,D)\)

[0033] Among them, \(\text{TF}(t,d)\) is the frequency of word \(t\) in document \(d\), and \(\text{IDF}(t,D)\) is the inverse document frequency of word \(t\) in all document sets \(D\).

[0034] In another embodiment of the present invention, a short video production method based on intelligent recognition is provided. In S30, each transition time point is confirmed, and the video is divided into multiple segments, including the following steps:

[0035] The video is tested by the transition detection algorithm, the detected transitions are time-stamped, the specific time point of each transition is recorded, and the original video is divided into multiple segments according to the recorded transition time points. The starting point and end point of each segment correspond to two adjacent transition time points respectively; FFmpeg is used to cut the video according to the time points to generate multiple independent video segment files. Through the above steps, the transition time points in the video can be effectively detected and the video can be divided into multiple segments. This method combines image processing and computer vision technology, which can improve processing efficiency while ensuring accuracy.

[0036] Furthermore, the transition detection algorithm adopts detection based on pixel difference, and detects transition by comparing the pixel difference between consecutive frames. If the difference between two frames exceeds a set threshold, it is considered that a transition exists.

[0037] Alternatively, the transition detection algorithm adopts feature point-based detection, uses computer vision technology to detect feature points within the frame, and tracks the movement of the feature points between consecutive frames. If the distribution and number of feature points change significantly, it is determined that a transition exists.

[0038] Alternatively, the transition detection algorithm uses optical flow-based detection to calculate the optical flow between consecutive frames. If the optical flow changes greatly, there may be a transition. When determining the threshold, a portion of the video data set can be used for verification, and the mean and standard deviation of the optical flow change in the entire video can be calculated, and a certain multiple of the standard deviation can be used as the threshold. For video scenes with drastic changes, the threshold can be adjusted dynamically, for example, dynamically setting the threshold based on the standard deviation of the local optical flow change.

[0039] Through the above-mentioned intelligent recognition-based short video production method, combined with video intelligent recognition and manuscript processing capabilities, the key content of the video is extracted from the original video as the content material of the short video. For the original video content, especially reporting materials, product descriptions, documentaries, etc., the short video production of long videos of single-character descriptions has an excellent improvement effect, greatly simplifying the difficulty of producing such short videos, shortening the production cycle, and greatly reducing labor costs. In addition, combined with computer recognition and processing capabilities, it can more objectively reflect the video content and avoid the influence of the subjective consciousness of the production staff.

[0040] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention are described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A short video production method based on intelligent recognition, characterized in that: The following steps are involved: Extract the speech content in the original video and convert the language in the original video into the text of the original video through speech recognition; The manuscript content of the original video is used as input information, language and text processing is performed based on natural language processing, the manuscript content is extracted into summary information of the original video, and corresponding keywords are marked; Perform content recognition on the original video, confirm each transition time point, and split the video into multiple clips; And further compare it with the original manuscript, and associate it with keywords; through the keywords, confirm the corresponding video clips in the manuscript of the original video, and extract the corresponding video clips that meet the text summary; The video content and summary are presented synchronously through the playback tool. The matching degree between the video content and the summary is manually reviewed, and the final effect of the video corresponding to the summary is confirmed. The selected videos are spliced ​​together, and the text summary is processed by manual voice to generate the final short video material.

2. The short video production method based on intelligent recognition according to claim 1, characterized in that: The step of extracting the manuscript content into summary information of the original video specifically includes: Preprocess the manuscript, including word segmentation and removal of stop words; Extract key information from the text, including using the term frequency-inverse document frequency (TF-IDF) method, that is, calculating the frequency of each word in the manuscript and the inverse document frequency in the entire corpus, and selecting words with higher TF-IDF values ​​as keywords; or using a fast automatic keyword extraction method to segment the text into candidate keyword phrases; calculating the term frequency and word co-occurrence of each word; calculating the score of each candidate keyword phrase, and selecting keywords with higher scores; Use the extracted keywords to score each sentence; the more keywords a sentence contains, the higher the score; select the sentences with the highest scores as the summary.

3. The short video production method based on intelligent recognition according to claim 1, characterized in that: Determining each transition time point and dividing the video into multiple segments includes the following steps: The video is tested using a transition detection algorithm, the detected transitions are time-stamped, the specific time point of each transition is recorded, and the original video is divided into multiple segments based on the recorded transition time points. The start and end points of each segment correspond to two adjacent transition time points respectively; FFmpeg is used to cut the video according to time points to generate multiple independent video segment files.

4. The short video production method based on intelligent recognition according to claim 3, characterized in that: The transition detection algorithm adopts detection based on pixel difference, and detects transition by comparing the pixel difference between consecutive frames. If the difference between two frames exceeds a set threshold, it is considered that there is a transition.

5. The short video production method based on intelligent recognition according to claim 3, characterized in that: The transition detection algorithm adopts feature point-based detection, uses computer vision technology to detect feature points within a frame, and tracks the movement of the feature points between consecutive frames. If the distribution and number of feature points change significantly, it is determined that a transition exists.

6. The short video production method based on intelligent recognition according to claim 3, characterized in that: The transition detection algorithm adopts detection based on optical flow to calculate the optical flow between consecutive frames. If the change amplitude of the optical flow is large, there may be a transition.

7. The short video production method based on intelligent recognition according to claim 1, characterized in that: The summary information of the original video corresponds to the number of words in 3-5 minutes of speech.