Method and device for adjusting playing speed of voice content in video, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a method for adjusting the playing speed of voice content in a video, and the method comprises the steps: obtaining a video file containing to-be-adjusted voice content, carrying out the voice recognition of an audio track of the video file, and converting the audio data of the voice content into text data; dividing the text data into a plurality of sentences or phrases; identifying keywords in each sentence or phrase, performing emotion content classification on the sentences or phrases containing the keywords, and determining the importance level of each sentence or phrase; according to the emotion content and the importance level, adjusting the playing speed of the corresponding voice content, and generating adjusted audio data; aligning the adjusted audio data with the video image to generate synchronous audio and video data; and a final video is generated. Through automatic voice recognition, sentiment analysis and playing speed adjustment, the steps of manual participation are reduced, and the video editing efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a method, device, equipment and storage medium for adjusting the playback speed of voice content in a video. Background Art

[0002] In the financial sector, the application of video and audio processing technologies is becoming more and more widespread, especially in scenarios such as intelligent customer service, financial education, marketing and customer communication. Financial institutions usually need to record a large amount of video content to convey product information, financial services and customer support. However, the recording and playback speed adjustment of video lines is still an important technical problem.

[0003] Current technologies rely on manual post-processing of recorded video dialogues. One of the most common tasks is to manually adjust the playback speed of the dialogue to match the duration and rhythm of the video. The video content of financial institutions is often long and involves complex financial terms and explanations, which requires extremely high precision. Manually adjusting the speed of the dialogue is not only time-consuming, but also prone to errors, especially when faced with the need for frequent updates or multi-language support. This method is much less efficient.

[0004] Some automated tools have been introduced for speech recognition and speed adjustment, but these tools still have obvious deficiencies in speech recognition accuracy, especially when dealing with financial videos involving professional terms. In addition, when adjusting the speech playback speed, existing automated tools are difficult to fully consider the length of the video, emotional expression, and the importance of key information. This often leads to reduced coherence of video content and information delivery, and fails to meet the expected quality standards.

[0005] In summary, the existing technologies for automatically processing the recording and speed adjustment of video lines have problems such as inaccurate voice recognition and inability to match the rhythm and emotional expression of the video content. Therefore, there is an urgent need for a solution that can accurately adjust the playback speed of video lines to help financial industry users improve video production efficiency while ensuring the accuracy and fluency of the content. Summary of the invention

[0006] The main purpose of the present invention is to provide a method, device, equipment and storage medium for adjusting the playback speed of voice content in a video, aiming to solve the technical problem in the prior art that the playback speed of voice content in a video cannot be automatically adjusted to match the video content and rhythm, and manual editing is required, resulting in a long time and low efficiency.

[0007] To achieve the above object, the present invention provides a method for adjusting the playback speed of voice content in a video, comprising:

[0008] Acquire an original video containing voice content to be adjusted, perform voice recognition on an audio track of the original video, and convert audio data of the voice content into text data;

[0009] Performing word segmentation processing on the text data to divide the text data into multiple sentences or phrases;

[0010] Identify the keywords in each sentence or phrase, classify the sentiment content of the sentences or phrases containing the keywords, and determine the importance level of each sentence or phrase;

[0011] According to the emotional content and importance level of each sentence or phrase, adjusting the playback speed of the corresponding voice content to generate audio data with adjusted playback speed;

[0012] Aligning the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data;

[0013] The synchronized audio and video data are edited and synthesized to generate a final video with the playback speed of the voice content adjusted.

[0014] Furthermore, in order to achieve the above object, the present invention provides a device for adjusting the playback speed of voice content in a video, comprising:

[0015] A speech recognition module, used to obtain an original video containing speech content to be adjusted, perform speech recognition on an audio track of the original video, and convert the audio data of the speech content into text data;

[0016] A text processing module, used for performing word segmentation processing on the text data, dividing the text data into a plurality of sentences or phrases;

[0017] Sentiment analysis and importance assessment module, which is used to identify keywords in each sentence or phrase, classify the sentiment content of sentences or phrases containing keywords, and determine the importance level of each sentence or phrase;

[0018] An audio adjustment module, used to adjust the playback speed of the corresponding voice content according to the emotional content and importance level of each sentence or phrase, and generate audio data with adjusted playback speed;

[0019] An audio and video synchronization module, used to align the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data;

[0020] The video editing and synthesis module is used to edit and synthesize the synchronized audio and video data to generate a final video with the playback speed of the voice content adjusted.

[0021] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a program for adjusting the playback speed of voice content in a video that is stored in the memory and can be run on the processor. When the program for adjusting the playback speed of voice content in a video is executed by the processor, the steps of the method for adjusting the playback speed of voice content in a video as described above are implemented.

[0022] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which is stored a program for adjusting the playback speed of voice content in a video, and when the program for adjusting the playback speed of voice content in a video is executed by a processor, the steps of the method for adjusting the playback speed of voice content in a video as described above are implemented.

[0023] Beneficial effects: The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a method for adjusting the playback speed of voice content in a video. The method comprises the following steps: obtaining a video file containing the voice content to be adjusted, performing voice recognition on the audio track of the video file, and converting the audio data of the voice content into text data; performing word segmentation processing on the text data and dividing it into multiple sentences or phrases; identifying keywords in each sentence or phrase, classifying the emotional content of the sentences or phrases containing the keywords, and determining the importance level of each sentence or phrase; adjusting the playback speed of the corresponding voice content according to the emotional content and the importance level, and generating adjusted audio data; aligning the adjusted audio data with the video image to generate synchronized audio and video data; and editing and synthesizing the synchronized audio and video data to generate a final video. The present invention reduces the steps of manual participation through automatic voice recognition, emotional analysis and playback speed adjustment, and greatly improves the efficiency of video editing. Through emotional analysis and importance evaluation, the voice playback speed can be dynamically adjusted, highly matched with the rhythm and emotional expression of the video content, and improves the viewing experience of the video and the accuracy of content communication. The need for manual operation is reduced through automated processing, and errors that may occur during manual adjustment are avoided, ensuring consistent video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0025] Figure 1 A schematic diagram of an application environment of a method for adjusting the playback speed of voice content in a video according to an embodiment of the present invention;

[0026] Figure 2 A flow chart of an embodiment of a method for adjusting the playback speed of voice content in a video according to the present invention;

[0027] Figure 3A schematic diagram of functional modules of a preferred embodiment of a device for adjusting the playback speed of voice content in a video of the present invention;

[0028] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0029] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0030] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0031] The method for adjusting the playback speed of voice content in a video provided by the embodiment of the present invention can be applied to Figure 1 In an application environment, the user end communicates with the server end through a network. The server end can obtain a video file containing voice content to be adjusted through the user end, perform voice recognition on the audio track of the video file, and convert the audio data of the voice content into text data; perform word segmentation processing on the text data and divide it into multiple sentences or phrases; identify the keywords in each sentence or phrase, classify the emotional content of the sentences or phrases containing the keywords, and determine the importance level of each sentence or phrase; adjust the playback speed of the corresponding voice content according to the emotional content and the importance level, and generate adjusted audio data; align the adjusted audio data with the video image to generate synchronized audio and video data; edit and synthesize the synchronized audio and video data to generate the final video. The present invention reduces the steps of manual participation through automatic voice recognition, emotional analysis and playback speed adjustment, and greatly improves the efficiency of video editing. Through emotional analysis and importance evaluation, the voice playback speed can be dynamically adjusted, highly matched with the rhythm and emotional expression of the video content, and the video's viewing experience and the accuracy of content communication are improved. The need for manual operation is reduced through automated processing, and errors that may occur during manual adjustment are avoided, ensuring consistent video quality. The user end may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server end may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0032] See also Figure 2 , Figure 2 The flowchart is a schematic diagram of an embodiment of a method for adjusting the playback speed of voice content in a video provided by the present invention. It should be noted that although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0033] like Figure 2As shown, the method for adjusting the playback speed of voice content in a video proposed by the present invention comprises the following steps:

[0034] S10, obtaining an original video containing voice content to be adjusted, performing voice recognition on an audio track of the original video, and converting audio data of the voice content into text data;

[0035] In this embodiment, a video file containing the voice content to be processed is obtained. The video includes not only images, but also audio tracks that need to be identified and adjusted. The video format can be diversified, such as MP4, AVI, MKV, etc. The video file is obtained from local storage or a server. A video processing tool (such as FFmpeg) is used to read the video content and extract the audio track in the video. If the video format is incompatible, the video format is first converted to ensure that the audio track can be extracted smoothly.

[0036] The audio track is the sound data stored independently in the video. The speech is converted into text using speech recognition technology (such as ASR system). Use tools (such as FFmpeg) to extract the audio track in the video into a separate audio file (such as WAV, MP3, etc.). Input the audio file into the speech recognition system (such as Google Speech-to-Text API, IBM Watson, Microsoft Azure, etc.), recognize the audio content through automatic speech recognition technology (ASR), and generate the corresponding text data. If the accuracy of speech recognition is not high, a customized vocabulary or pre-trained model can be introduced for professional terms or specific fields (such as finance) to improve recognition accuracy.

[0037] The speech content is converted into text data for subsequent analysis and processing through the speech recognition system. The text data can be used for further word segmentation, sentiment analysis and speech speed adjustment. The text data output by the speech recognition system can be saved in the form of a text file (such as TXT, JSON, etc.). The generated text data is preliminarily checked and processed, including removing irrelevant words (such as "hmm", "ah", etc.) to improve the text quality. It is also possible to further manually correct certain words or terms based on the speech recognition results, especially when the recognition results contain a large number of professional terms (such as financial terms).

[0038] Example: In the financial field, a financial institution produced a video to explain the latest investment products to customers. The video is 30 minutes long and contains a lot of financial terms and digital interpretations. After using automated speech recognition technology: the audio data in the video is extracted, and the audio content is converted into accurate text data using a customized speech recognition model. Financial terms are specially processed during the speech recognition process so that the recognition results meet the needs of the field. After the text data is generated, these texts can be segmented, sentiment analyzed, and speed adjusted, and finally a video that is highly matched with the original video content and has been precisely adjusted is generated.

[0039] Through the above steps, the efficiency and accuracy of speech recognition of video content can be improved, especially for complex and professional terms in the financial field. Automated methods can effectively reduce manual operations and achieve efficient audio-to-text conversion. Combined with customized vocabulary or pre-trained models, it can also improve the recognition accuracy in specific fields, especially for scenarios such as financial education and data analysis that require high-precision term recognition.

[0040] S20, performing word segmentation processing on the text data to divide the text data into a plurality of sentences or phrases;

[0041] In this embodiment, the continuous text data generated by speech recognition is segmented and divided into smaller semantic units, such as sentences, phrases, etc., to facilitate subsequent sentiment analysis and speech speed adjustment. Load a text processing tool (such as natural language processing libraries such as NLTK and SpaCy) and input the text into the word segmentation tool. According to the language characteristics, select a suitable word segmentation model (Chinese is usually a character-based word segmentation model, and English is a word-based word segmentation model). Apply preprocessing steps, such as removing excess spaces and normalizing punctuation marks, to ensure that the word segmentation tool can accurately divide the text content.

[0042] Language features refer to the unique properties and structures of different languages ​​in terms of grammar, vocabulary, syntax, etc. These features require special consideration when processing natural language (such as word segmentation, speech recognition, text analysis, etc.). The following are some of the main language features:

[0043] Grammatical structure: Chinese: Chinese does not have clear word boundaries (unlike English, which has spaces as separators), so word segmentation is a key step when processing Chinese. The grammatical structure of Chinese is usually in the form of subject-predicate-object, and the use of punctuation in sentences is relatively concise. English: English has clear word boundaries, with spaces separating words. The grammatical structure is relatively complex, including tense, voice, word order, etc. Word segmentation is relatively simple, but grammatical rules such as tense and subject-verb agreement need to be considered during syntactic analysis. Other languages: For example, German, French, etc., verb changes and the gender, number, and case of nouns will also affect language processing strategies.

[0044] Vocabulary characteristics: Chinese: Chinese vocabulary is mostly composed of single characters, each of which can express a certain meaning independently or become part of a multi-character word. The meaning of vocabulary can change according to the context, so it is necessary to use word segmentation tools to identify the actual meaning of the words. English: English vocabulary is mostly in the form of roots plus affixes, with clear part-of-speech classification (such as nouns, verbs, adjectives, etc.), and new words can be formed through changes in prefixes and suffixes. Other languages: such as Arabic, which has complex word form changes, not only requires word segmentation, but also needs to deal with word form changes and derivation rules.

[0045] Use of punctuation marks: Chinese: There are relatively few types of punctuation marks, mainly commas, periods, question marks, exclamation marks, etc., and the role of punctuation marks is relatively simple, used to separate sentences or semantic paragraphs. English: Punctuation marks are not only used to separate sentences, but also can affect the structure of sentences, such as commas for enumeration, brackets for inserting information, quotation marks for quotations, etc. The use of punctuation marks is richer and the rules are more complex. Other languages: such as Japanese and Korean, the combination of punctuation marks and words is different, and punctuation marks are also used to divide the structure of sentences.

[0046] Parts of speech and word order: Chinese: Parts of speech do not change much, and different meanings are often expressed through the order and collocation of words. The word order is relatively fixed, and the subject-verb-object structure is the main one. English: Parts of speech change more, and nouns, verbs, adjectives, pronouns, conjunctions and other parts of speech are very clear, and the word order is relatively fixed. Other languages: For example, the word order of German is more complex than that of English. Verbs are often at the end of the sentence, and the structure of the sentence changes according to the context and the emphasis.

[0047] Context dependence: Chinese: Chinese is highly dependent on context, and the meaning of a word or character often needs to be judged by the context. Especially when dealing with Chinese word segmentation, the relationship between phrases and sentences needs special attention. English: Relatively speaking, English is less context-dependent, and the meaning of words is independent in most cases, but in some fixed collocations and phrasal verbs, the context still needs to be considered.

[0048] In the process of natural language processing (NLP) or speech recognition, it is very important to understand the characteristics of different languages. Word segmentation tools or speech recognition models need to be adjusted according to the characteristics of the language to accurately process text or speech data. For example:

[0049] When processing Chinese, you need to use a word segmentation tool to separate single-word and multi-word words.

[0050] When processing English, you can directly segment words based on spaces, but you need to consider changes in parts of speech and tense.

[0051] When processing German, it is also necessary to handle grammatical changes such as gender, number, and case of nouns and adjust the processing rules.

[0052] Therefore, language characteristics refer to the uniqueness of different languages ​​in grammar, vocabulary, syntax, punctuation, etc. These characteristics determine the specific strategies to be adopted in the natural language processing process.

[0053] Based on the processing results of the word segmentation tool, the text is divided into sentences or phrases. Accurate segmentation is performed based on grammatical structure, punctuation marks, and the logical structure of phrases to ensure that each segment has independent semantics. The word segmentation tool is used to perform preliminary division of sentences based on punctuation marks (such as periods, commas, question marks, etc.). For longer sentences, a phrase separation strategy is used to further subdivide complex sentences into multiple phrases. This strategy is based on grammatical and semantic analysis, and extracts phrases separately by identifying the subject-verb-object structure or other modifying relationships in the sentence. The generated word segmentation results can be saved as structured text (such as a list form) for subsequent sentiment analysis and speed adjustment operations.

[0054] Example: In the financial field, a bank produced a video to explain the new financial policy terms. The lines in the video were generated into text data through voice recognition. Through word segmentation, the system divides the continuous text into multiple phrases and sentences, which is convenient for subsequent sentiment analysis and speed adjustment. For example: long sentences are separated into phrases to ensure that key information such as interest rate adjustment terms are extracted separately. Through word segmentation, the text is further accurately divided to facilitate the identification of financial terms and key terms.

[0055] By segmenting text data and dividing it into phrases, the text can be more structured, making subsequent sentiment analysis and speech rate adjustment more accurate. Segmentation not only simplifies the text processing process, but also enhances the accuracy of analysis. Especially in complex sentence structures, the introduction of phrase separation strategies can better parse important information in the text. For video content in the financial field, such as interpretation of contract terms and news broadcasts, segmentation can ensure that the content is clearly conveyed and avoid semantic ambiguity.

[0056] S30, identifying keywords in each sentence or phrase, and classifying the sentiment content of the sentences or phrases containing the keywords, and determining the importance level of each sentence or phrase;

[0057] In this embodiment, in the text data after word segmentation, the recognition of keywords is to extract the core words in the sentence or phrase to help understand the main content of the text. Keywords are usually words that express the core meaning of a sentence or phrase, such as nouns, verbs, adjectives, etc., especially in the financial field, which may include terms, numerical values, specific financial product names, etc.

[0058] A method based on TF-IDF (Term Frequency-Inverse Document Frequency) can be used to identify words that appear frequently and have important information.

[0059] For specific fields (such as finance), common financial terms (such as "interest rate", "equity", "bonds", etc.) can be preferentially extracted in combination with predefined keyword lexicons.

[0060] You can also use natural language processing tools (such as SpaCy, NLTK) to perform part-of-speech tagging on the word segmentation results and extract keywords such as nouns, verbs, and adjectives. Especially in the financial field, nouns (such as the names of financial products) and numerical values ​​are often key information.

[0061] The purpose of sentiment classification is to identify the emotional tendency (such as positive, negative or neutral) in the text, so as to help understand the emotional expression of a sentence or phrase. This is very important in scenarios such as financial analysis, market reports, and news interpretation, especially the analysis of market sentiment.

[0062] You can use pre-trained sentiment analysis models (such as VADER, BERT, etc.) to classify the sentiment of the identified sentences or phrases. These models can determine whether the sentence expresses positive, negative or neutral emotions based on the sentiment tendency of the keywords.

[0063] Since a single keyword may have multiple meanings, sentiment analysis models can classify it in context. For example, “up” in financial news may convey positive sentiment, while “loss” may convey negative sentiment.

[0064] In response to the special needs of the financial field, sentiment analysis models can be combined with domain vocabulary libraries to specifically process financial news, market reports, and other content to improve the accuracy of sentiment analysis.

[0065] The importance level is determined based on the classification of sentiment content and the identification of keywords to quantitatively assess the importance of sentences or phrases. In financial analysis videos, some sentences may contain core or key information that needs to be presented at a slower speed, while other minor information can be quickly skipped.

[0066] Combining the weight of keywords (such as the frequency of occurrence of financial terms) and the results of sentiment analysis, each sentence or phrase is assigned an importance level. For example, sentences with strong emotions (such as "significant losses" and "stock price plummets") can be assigned a higher importance level.

[0067] Analyze the position of the sentence in the entire text. The opening paragraph or the closing paragraph may have a higher importance. In addition, sentences containing summary information are usually given a higher ranking.

[0068] In the financial field, sentences or phrases involving financial data, market forecasts, risk warnings, etc. usually have a higher level of importance. The model can adjust the priority judgment according to the domain rules.

[0069] Example description: In the financial field, a financial analyst is making a video about market trends. The video needs to adjust the playback speed of the lines according to different changes in the market: First, through keyword extraction, the system identifies key information such as "market fluctuations" and "stock price increases". The sentiment analysis model further classifies sentences into positive emotions ("stock price increases") and negative emotions ("market fluctuations"). Based on keyword and sentiment classification, the system marks sentences describing market fluctuations as high priority and plays them at a slower speed so that viewers can better understand the complex changes in the market.

[0070] Through keyword extraction, sentiment classification and importance level determination, accurate analysis of text data can be achieved. It can help quickly identify and prioritize key information, and improve the efficiency and accuracy of video processing. Through automated analysis, the time cost of manual analysis can be greatly reduced, while ensuring accuracy and comprehensiveness when processing financial news, market analysis and contract terms.

[0071] S40, adjusting the playback speed of the corresponding voice content according to the emotional content and importance level of each sentence or phrase, and generating audio data with adjusted playback speed;

[0072] In this embodiment, the emotional content and importance level of each sentence or phrase are based on the results generated in the previous steps. The emotional content describes the emotional tendency (positive, negative, neutral, etc.) of the sentence or phrase, while the importance level indicates the relative weight of the sentence or phrase in the overall content.

[0073] The system receives the sentiment classification results and importance level of each sentence or phrase after sentiment analysis and importance evaluation. According to pre-defined criteria, the emotional content (such as strong emotional fluctuations) is combined with the importance level (such as key terms, core information) to generate a comprehensive weight. The greater the emotional fluctuation and the higher the importance level of the sentence, the greater the adjustment of the playback speed. After combining the emotional content and the importance level, it is decided in priority order which sentences or phrases need to be slowed down or accelerated.

[0074] Based on the evaluation results of emotional content and importance level, the playback speed of the voice content is dynamically adjusted so that important sentences or sentences with strong emotions are played slower, while sentences with less important information are played faster.

[0075] Different playback speed adjustment ranges (such as ±20% speed adjustment) can be set according to emotion and importance. For example, emotionally strong sentences can be slowed down by 20%, while less important sentences can be sped up by 10%.

[0076] You can also use the Time-Scale Modification (TSM) algorithm to adjust the playback speed of the audio clip. TSM technology maintains the pitch when changing the playback speed, so it can achieve dynamic changes in speech speed without causing sound distortion.

[0077] During playback, the system can also dynamically adjust the playback speed according to the emotion and importance of the sentence, and render the audio data after the speed change in real time.

[0078] The final output audio data is a version with adjusted playback speed. This audio data not only retains the original voice content, but also optimizes the playback speed according to the importance and emotion of different sentences, ensuring that the audience can focus on important information and smoothly receive the overall content.

[0079] Generates audio data with adjusted speed, usually output in common audio formats (such as WAV, MP3). These formats can be easily synchronized with video content later. Save the generated audio file to provide input data for subsequent audio and video synchronization, editing and synthesis steps.

[0080] Example description: A financial company produced an analysis video about market trends. The video content includes analysis and interpretation of market fluctuations, risk warnings, stock price fluctuations, etc.: The sentiment analysis system identified that the "market fluctuations" section has a high negative sentiment, and the system marked this section as high priority and slowed down the speech speed by 20% so that the audience can carefully understand the risks of the market. The general market trends in the video are classified as secondary content, and the system speeds up the playback of these sentences by 10% so that the audience can quickly browse this section. The audio data with adjusted playback speed is applied to the video, allowing the audience to focus on important content and improve the absorption of market risk information.

[0081] By adjusting the speech playback speed according to the emotional content and importance level, the audience's understanding and attention to important information can be improved. This dynamic adjustment method avoids the monotony of the speech content, ensures that the audience can focus on the core information and quickly browse the secondary content, and improves the effectiveness of information delivery and user experience.

[0082] Especially in the financial field, many video contents contain complex terms, key market changes and important financial data. By adjusting the speech speed, the audience can understand the complex content more clearly, while saving time to quickly process minor information, thereby improving the overall efficiency of financial data interpretation and contract interpretation videos.

[0083] S50, aligning the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data;

[0084] In this embodiment, the audio data whose playback speed has been adjusted is synchronized with the video image so that the audio and video content are consistent on the time axis. Since the playback speed has changed, the duration of the audio may be different from the original video, so the time axis alignment process is required to ensure audio and video synchronization.

[0085] Use a tool (such as FFmpeg) to extract the timeline information of the original video and obtain the timestamp of each video frame. Obtain the timeline information of the adjusted audio data and calculate the timestamp of each audio segment, including the effect of the change in playback speed on the duration of the audio. Match the adjusted audio timestamp with the timestamp of the video frame, and ensure that the audio and video content are consistent on the timeline by inserting, scaling, or smoothing. Specifically, the adjustment of the audio playback speed may make the audio length longer or shorter, so the timeline needs to be precisely aligned during synchronization.

[0086] By precisely aligning the timelines of audio and video, the final synchronized audio and video data is generated to ensure that the audio and image content are coordinated during video playback. At this point, each frame of audio and video content is completely synchronized in time so that the final video content can be played seamlessly.

[0087] The aligned audio data is merged with the original video image to generate a synchronized audio and video file. Commonly used synthesis tools include FFmpeg, which can re-encode the adjusted audio and video files into a complete audio and video file. Of course, other synthesis tools such as Adobe Premiere Pro, DaVinci Resolve, Avid Media Composer, CyberLink Power Director, etc., or self-developed audio and video synthesis tools can also be used, and the present invention does not limit this. Select a suitable video output format (such as MP4, MOV, etc.) to ensure that the output audio and video data is compatible on different devices and platforms. Synchronous verification is performed through the player to check whether the generated audio and video files are fully matched on the timeline, and to ensure that the effect of the adjusted audio playback speed is closely integrated with the video content.

[0088] Example: In the financial sector, a financial institution produced a video explaining a new financial policy. Due to the complexity of the policy content, the speed of the interpretation was dynamically adjusted to highlight important clauses: after the audio content was analyzed for sentiment and importance, the speed of some important clauses was slowed down. The timeline information of the video was extracted, and the adjusted audio data was realigned with the video image according to the timeline. Synthesis tools were used to generate synchronized audio and video data, and the final video playback effect was natural and smooth, ensuring that the audio interpretation content was displayed synchronously with the corresponding policy chart. Through the precise synchronization of audio and video, the audience can better absorb important information and ensure the accuracy and clarity of the interpretation of financial policies.

[0089] By aligning the adjusted audio data with the original video image on the timeline, the synchronous playback of audio and video can be ensured, avoiding the phenomenon of audio and video being out of sync. Especially in the financial field, accurate synchronization of audio and video is crucial, because the content in financial news, market analysis or contract interpretation videos needs to be clearly and accurately conveyed to the audience. Through the automated audio and video alignment process, video producers do not need to manually adjust the correspondence between each audio clip and video frame, which greatly improves efficiency.

[0090] S60, editing and synthesizing the synchronized audio and video data to generate a final video with the playback speed of the voice content adjusted.

[0091] In this embodiment, the synchronized audio and video data is further edited to remove unnecessary parts, adjust the duration, or perform fine editing according to content requirements. Through editing, the overall structure of the video can be optimized to make it more concise and coherent.

[0092] First, analyze the video content and identify the clips that need to be edited. These editing points can be useless interruptions, repeated information, or irrelevant content. The selection of editing points can be marked manually or detected by intelligent algorithms, such as automatically marking editing points by identifying silent clips or irrelevant video frames. Use video editing software (such as Adobe PremierePro, Final Cut Pro, FFmpeg) to crop and delete the video. You can choose to crop out the unnecessary parts and adjust the connection between the remaining clips to maintain the smoothness of the video.

[0093] After the audio and video editing is completed, the synthesis step recombines all the edited and adjusted audio and video data to generate a complete video file. The synthesis process needs to ensure that the audio and video data are synchronized and output into a playable file format.

[0094] During the editing process, the duration of audio and video may change, so you need to make final adjustments to the timeline before synthesis to ensure that the audio and video data match exactly on the timeline. Select a suitable output format (such as MP4, AVI, MOV, etc.) and use video editing tools to synthesize the edited audio and video. Ensure that the audio and video frames correspond accurately during synthesis, and export the final file through the tool.

[0095] The edited and synthesized audio and video data will generate a complete video file containing the adjusted speech content. This file contains the audio with adjusted speech speed and the image synchronized with it, and has been edited and optimized for better viewing.

[0096] Select the appropriate video file format according to your needs and export the final audio and video files. The output video files can be played directly or uploaded to a video platform for sharing. After generating the video files, you can selectively perform additional post-processing operations, such as adding subtitles, special effects, or enhancing video clarity, to ensure that the final output meets your expectations.

[0097] Example description: In the financial field, a financial institution produced a new policy interpretation video, which contains detailed policy interpretation and market forecasts. Since the video content is long, the production team decided to edit the redundant content and adjust the playback speed of the voice: During the editing process, the clips irrelevant to the policy interpretation were removed, while the detailed interpretation of the policy points was retained. The speech speed of the policy interpretation part was adjusted so that the audience can clearly understand the details of each policy. Finally, through the synthesis tool, high-quality synchronized audio and video content was generated, and the audience can quickly and accurately understand the impact of policy changes on the market in this video.

[0098] By editing and synthesizing the synchronized audio and video data, the structure of the video content can be optimized to make it more concise and coherent while ensuring that important content is preserved. Redundant parts can be eliminated to improve the watchability of the video and ensure that the audience can focus on key information. In addition, the synthesis step can accurately combine the adjusted audio data with the video content to ensure that the final output video content is clear and professional.

[0099] The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a method for adjusting the playback speed of voice content in a video. The method comprises the following steps: obtaining a video file containing voice content to be adjusted, performing voice recognition on the audio track of the video file, converting the audio data of the voice content into text data; performing word segmentation processing on the text data, and dividing it into multiple sentences or phrases; identifying keywords in each sentence or phrase, classifying the sentences or phrases containing keywords by emotional content, and determining the importance level of each sentence or phrase; adjusting the playback speed of the corresponding voice content according to the emotional content and the importance level, and generating adjusted audio data; aligning the adjusted audio data with the video image to generate synchronized audio and video data; and editing and synthesizing the synchronized audio and video data to generate a final video. The present invention reduces the steps of manual participation through automatic voice recognition, emotional analysis and playback speed adjustment, and greatly improves the efficiency of video editing. Through emotional analysis and importance evaluation, the voice playback speed can be dynamically adjusted, highly matched with the rhythm and emotional expression of the video content, and improves the viewing quality of the video and the accuracy of content communication. The need for manual operation is reduced through automated processing, and errors that may occur during manual adjustment are avoided, ensuring consistent video quality.

[0100] In one embodiment, in the above S30, determining the importance level of each sentence or phrase includes:

[0101] S301, determining the keyword density of each sentence or phrase based on the keyword recognition result;

[0102] S302, analyzing the subject content of the original video, and evaluating the relevance of each sentence or phrase to the subject content;

[0103] S303: assigning a corresponding importance level to each sentence or phrase based on the keyword density and the relevance to the subject content.

[0104] In this embodiment, keyword density refers to the proportion of keywords (such as core terms, data, etc.) in a sentence or phrase relative to the entire sentence or phrase. Sentences with higher keyword density usually contain more key information and may therefore have higher importance.

[0105] Use natural language processing (NLP) tools (such as TF-IDF, part-of-speech tagging, etc.) to extract keywords from sentences or phrases. Keywords are usually nouns, verbs, etc., especially core terms in the financial field, such as "interest rate", "risk", "market volatility", etc. The keyword density of each sentence or phrase is determined by calculating the ratio of the number of times the keyword appears in the sentence to the length of the sentence. The following formula can be used:

[0106]

[0107] Different keywords can be given different weights. For example, keywords in the financial field (such as "investment" and "market changes") are given higher weights to ensure that the keyword density reflects its importance.

[0108] The topic content refers to the core discussion point or main information in the video. By analyzing the relevance of a sentence or phrase to the topic content, you can determine its importance in the overall content. The more relevant a sentence is to the topic content, the higher its importance.

[0109] Use text topic modeling methods (such as LDA, Latent Dirichlet Allocation) to extract topics from the text data in the video and identify the main content or key topics of the video. Compare each sentence or phrase with the extracted topic to evaluate the relevance of its content to the topic. The relevance score can be calculated using similarity measurement methods such as cosine similarity and Jaccard similarity coefficient:

[0110]

[0111] In the financial field, specific terms or key financial data may be more relevant to the topic, so the relationship between these sentences and the video topic should be analyzed to determine their importance in the content.

[0112] Finally, an importance level is assigned to each sentence or phrase by combining keyword density and topic relevance. The importance level is used to determine the relative priority of the sentence in the overall content. The importance level can be divided into high, medium, and low levels so that subsequent processing (such as speech speed adjustment) can prioritize important information.

[0113] The keyword density and topic relevance are weighted to generate a comprehensive score for each sentence. The following formula can be used for weighting:

[0114] Comprehensive score = α × keyword density + β × topic relevance;

[0115] Among them, α and β are weight parameters used to adjust the importance of keyword density and topic relevance.

[0116] According to the comprehensive score, sentences or phrases are divided into different levels. For example, sentences with high scores are classified as "high importance", while sentences with low scores are classified as "low importance". The final importance level of the sentence is used as an important basis for subsequent speech speed adjustment and content editing. After the level division is completed, you can further decide how to deal with these sentences (such as slower playback, emphasis, etc.).

[0117] Example description: A financial institution produced a video explaining the latest market policies. In order to highlight the key clauses in the policy, the system performs analysis according to the following process: extract keywords in the video, such as "tax policy", "enterprise exemption", "investment incentives", etc., and calculate the density of these words in each sentence. Analyze the theme of the entire video, identify the core content as "tax reform", and evaluate the relevance of each sentence to the theme. Sentences that contain high-density keywords and are highly relevant to the core topic are assigned a "high importance" level. These sentences will be presented at a slower speed when the video is played to ensure that the audience can focus on understanding the policy changes.

[0118] This embodiment can more accurately evaluate the importance of each sentence or phrase in the video by combining keyword density and relevance to the subject content, which can help the system identify key information and prioritize important content in subsequent video processing (such as playback speed adjustment), ensuring that the audience can focus on understanding the core information in the video.

[0119] In one embodiment, the above S40 includes:

[0120] S401, determining a playback speed adjustment coefficient of each sentence or phrase according to the emotional content and importance level of each sentence or phrase;

[0121] S402, adjusting the playback speed of the voice segment corresponding to each sentence or phrase based on the playback speed adjustment coefficient;

[0122] S403: Merge the adjusted voice segments to generate audio data with adjusted playback speed.

[0123] In this embodiment, the playback speed adjustment coefficient is determined by the emotional content and the importance level. The emotional content refers to the emotional intensity of the sentence, while the importance level represents the weight of the sentence in the overall content. The adjustment coefficient is used to control the playback speed of each sentence, so that sentences with high importance and strong emotions are played slower, and less important sentences are played faster.

[0124] The sentiment intensity and importance level of the sentence are obtained from the sentiment analysis model and the importance assessment results. Each sentence or phrase is assigned an adjustment factor based on the intensity of the sentiment content and the importance level. Sentences with stronger sentiment (such as those with positive or negative sentiment) are assigned a slower playback speed adjustment factor, while sentences with weaker sentiment and lower importance are assigned a faster playback speed. The formula can set the influence factor based on the sentiment content and importance level separately, for example:

[0125] Playing speed adjustment factor = emotion factor·emotion intensity+level factor·importance level;

[0126] Among them, the emotional factor and the grade factor can be flexibly set according to the specific scenario, instead of having to use fixed weights.

[0127] The playback speed of each voice clip is dynamically controlled through the playback speed adjustment coefficient, ensuring that sentences with different emotions and importance can be presented at an appropriate speed without affecting the sound quality.

[0128] Use the Time Stretch Method (TSM) to apply a playback speed adjustment factor to each speech segment to ensure that changes in playback speed do not affect sound quality and speech clarity. Each sentence or phrase will be processed independently according to its adjustment factor, with sentences of high importance and strong emotions played at a slower speed and less important sentences played at a faster speed. The adjusted speed ratio should ensure the coherence of the entire audio content and the effectiveness of information delivery.

[0129] All adjusted voice clips will be re-merged in order to generate complete audio data. This process ensures seamless connection between clips and smooth playback. Through the audio processing tool, all adjusted clips are spliced ​​in the original order to ensure natural transition between clips. Select the appropriate audio format (such as MP3, WAV) and export the merged audio data as the final file.

[0130] Example description: In a video about financial market analysis, there are a lot of data interpretations and market forecasts. By performing sentiment analysis on the text data of the video, the system identifies the following two different types of sentences: Major changes in the market (such as "stock market fell sharply") are identified as content with strong emotions and a high level of importance. The system assigns a lower playback speed adjustment factor (0.85x) to such sentences to ensure that the audience can clearly understand the severity of market changes. Routine data updates (such as "daily market fluctuations") are identified as content with weaker emotions and lower importance, and the system assigns a higher playback speed adjustment factor (1.15x) to speed up playback and save audience time.

[0131] This embodiment uses a two-factor model of emotional content and importance level to determine the playback speed adjustment coefficient of each sentence or phrase, which can achieve more accurate voice playback speed control. For content with strong emotions or high importance, the playback speed is slowed down, and the audience can have more time to understand and absorb key information, which is particularly suitable for the interpretation of complex content in the financial field. For minor or emotionally weak sentences, the playback speed is accelerated, which effectively reduces the overall length of the video and improves the audience's viewing efficiency without affecting the communication of important information. By dynamically adjusting the speech rate, the rhythm of the audio content is enhanced, avoiding the monotony of a fixed speech rate throughout the process, making the audience's viewing experience richer and more vivid.

[0132] In one embodiment, the above S50 includes:

[0133] S501, obtaining timeline information of the original video, including a timestamp of each video frame;

[0134] S502, obtaining time axis information of the audio data whose playing speed has been adjusted, including the timestamp of each audio segment;

[0135] S503, using a synchronization module to match the timestamp of the audio segment with the timestamp of the corresponding video frame;

[0136] S504, adjusting the time axis of the original video according to the matching result, so that the time axis of the adjusted video matches the duration of the adjusted audio data;

[0137] S505, synthesizing the adjusted audio data and the corresponding video image in the adjusted video to generate synchronized audio and video data.

[0138] In this embodiment, the timestamp information of each video frame is extracted from the original video to construct a video timeline, which is used to mark the specific time point of each frame of the video during playback, providing a basis for subsequent audio and video synchronization.

[0139] Use video parsing tools (such as FFmpeg, OpenCV) to extract the timestamp data of each video frame from the original video. The tool records the playback time of each frame by analyzing the encoding structure of the video file. The timestamp information is usually recorded in milliseconds to build a timeline map so that each video frame has a unique time stamp on the timeline.

[0140] Extract the timestamp of each audio segment from the adjusted audio data and build the audio timeline. This step is similar to building the video timeline, but the timestamps of the audio segments need to be recalculated according to the adjusted playback speed.

[0141] Through the time stretching technique (TSM), the timestamps of each adjusted audio clip are recalculated to ensure that the adjusted timeline can reflect the playback time after the speech rate change. The calculated timestamps can be used to match the timestamps of the video frames. Use audio parsing tools (such as Audacity, FFmpeg) to obtain the timestamps of the adjusted audio clips and map them to the adjusted audio timeline.

[0142] Use the synchronization module to find the matching points between audio and video by comparing the timestamps of audio clips and video frames. This process ensures that the audio and video can be played in sync.

[0143] Through the time stamp alignment algorithm (such as linear interpolation or time series-based matching algorithm), the time stamps of the audio clips and video frames are matched to find the exact time point of each audio clip and the corresponding video frame. The time stamp of the audio clip is inserted into the video timeline so that the corresponding video frame is played synchronously with the audio clip.

[0144] Based on the matching results of the audio and video timelines, adjust the video timeline to match the adjusted audio duration. This step ensures that when the audio playback speed changes, the video playback rhythm is also adjusted accordingly.

[0145] If the audio duration changes (such as slowing down or speeding up), the video timeline needs to be adjusted accordingly. By modifying the display duration of each video frame, the video and audio are kept in sync. Based on the matching results, the video timeline is adjusted using time stretching technology to ensure that the video and audio play at the same length.

[0146] Finally, the adjusted audio data is synthesized with the video frame to generate synchronized audio and video data. The synthesized audio and video data should ensure that the audio and video images are completely consistent in time.

[0147] Use tools such as FFmpeg and Adobe Premiere Pro to reassemble the adjusted audio and video data into a file (such as MP4, AVI, etc.), and ensure seamless synchronization of audio and video frames. Output the synthesized file to ensure that the audio and video are consistent during playback, and the sound quality and picture quality remain stable.

[0148] In the financial sector, an institution produced a video explaining the latest market policy. Due to the complexity of the policy, the speed of the interpretation was adjusted to highlight the key points. The following steps are used to synchronize the audio and video: The system extracts the timestamp of each video frame from the original video and extracts the timestamp from the adjusted audio. The timestamps of the audio and video are matched through the synchronization module, and the timeline of the video is adjusted to accommodate the adjusted audio duration. The adjusted audio is synthesized with the corresponding video frame to output the final synchronized policy interpretation video.

[0149] This embodiment can ensure that the audio and video content are played synchronously by aligning the adjusted audio data with the time axis of the video image, thereby effectively avoiding the problem of audio and video being out of sync. It can ensure that the important interpretation in the video is highly consistent with the audio information, helping the audience to better understand the information when watching.

[0150] In one embodiment, the above S20 includes:

[0151] S201, preprocessing the text data, removing spaces and special symbols, and standardizing punctuation marks;

[0152] S202, selecting a word segmentation tool that matches the language characteristics of the text data, and initializing the word segmentation tool;

[0153] S203, using the word segmentation tool to segment the preprocessed text data, and preliminarily divide the text data into multiple sentences or phrases based on grammatical structure and punctuation marks;

[0154] S204, applying a phrase separation strategy to the text portion containing multiple phrases in the sentence, separating the phrases from the sentence, and generating a final sentence or phrase.

[0155] In the present embodiment, text data preprocessing is a step before word segmentation, the purpose of which is to clean up irrelevant information in the text, such as extra spaces and special symbols, to ensure that the word segmentation tool can correctly parse the text. At the same time, punctuation marks are standardized for subsequent sentence and phrase processing. Regular expressions or text cleaning tools are used to remove extra spaces, line breaks, and special symbols (such as @, #, $, etc.) in the text to ensure that the text is concise and interference-free. Through text formatting tools, all punctuation marks are converted to standard formats (such as full-width punctuation marks in Chinese are converted to half-width, or vice versa), and the punctuation mark types in the text are unified.

[0156] Different languages ​​have different characteristics, so you need to choose a word segmentation tool that is suitable for that language. When choosing a tool, you need to consider the grammatical structure, vocabulary characteristics, and characteristics of the text data (such as whether it contains domain-specific vocabulary).

[0157] Choose the appropriate word segmentation tool based on the language characteristics of the text. For example, Chinese requires a word segmentation tool based on a statistical model (such as Jieba), while English can use a rule or dictionary matching tool (such as SpaCy). Initialize the word segmentation tool based on the language characteristics of the text. For Chinese, you can load a dictionary in a specific field (such as a term library in the financial field) to improve the accuracy of word segmentation; for English, you can choose a POS (part of speech) tagging model to assist in word segmentation.

[0158] The preprocessed text is initially divided by the word segmentation tool. This step divides the text into multiple sentences or phrases based on the punctuation and grammatical structure in the text, which serves as the basis for the subsequent separation strategy.

[0159] Based on the preprocessed punctuation marks, the word segmentation tool automatically identifies sentence boundaries and divides them into sentences or phrases according to grammatical rules. The word segmentation tool combines grammatical rules to make preliminary divisions of long sentences, and reasonably divides the structures within each sentence into phrases to ensure the grammatical integrity of the phrases. For example, the syntax tree is used to analyze the subject, predicate, and object structure of a sentence.

[0160] In the case of a sentence containing multiple phrases, the sentence is further subdivided into independent phrases through the phrase separation strategy. The purpose of phrase separation is to make each phrase structurally independent, which is convenient for subsequent analysis and processing.

[0161] Determine the phrase boundaries in a sentence based on the grammatical structure and punctuation of the sentence. For complex sentences, use dependency parsing to separate modifiers from the main sentence to make the phrase independent. Use grammatical and semantic analysis techniques to identify the modifiers, supplementary information, or clauses in the sentence, and extract these components separately to generate the final sentence or phrase. The generated phrase has a high degree of semantic independence.

[0162] Example description: In the financial field, an investment company prepared a market trend analysis report, which contained multiple complex sentences and phrases. The system preprocessed and segmented the text data to complete the following tasks: clean up extra spaces and special symbols in the text to ensure that the text data is concise and usable. Use the Jieba word segmentation tool in combination with the financial terminology vocabulary to segment the report text. For complex sentences (such as "global economic slowdown, market volatility intensified"), the system automatically separates them into two independent phrases for subsequent analysis. Through word segmentation and phrase separation, structured text data is generated, which is convenient for adjusting the speech speed and content classification of each phrase.

[0163] This embodiment can significantly improve the text structure by preprocessing, word segmentation and phrase separation of text data, especially in financial reports and contract interpretation, which can help better analyze and process information. The word segmentation and phrase separation strategy can break down complex text into clear phrases and sentences, which is convenient for subsequent sentiment analysis, importance assessment and speech speed adjustment.

[0164] In one embodiment, after the above S60, the method further includes:

[0165] S701, displaying a waveform of the adjusted audio data and determining a change in the playback speed of the voice content;

[0166] S702, based on the playback speed change, generating and displaying a playback speed change curve of the voice content;

[0167] S703, receiving a playback speed change instruction from a user, and adjusting the playback speed of the voice content based on the playback speed change instruction;

[0168] S704, recording the user's playback speed adjustment operation and generating user adjustment data.

[0169] In this embodiment, the adjusted audio data is presented intuitively by displaying an audio waveform graph, which not only reflects the amplitude change of the audio, but also can display the adjustment of the playback speed of the voice content, helping the user understand the overall change of the audio playback speed.

[0170] Use audio processing tools (such as Python's matplotlib or other audio processing libraries) to convert the adjusted audio data into a waveform graph. Use visualization tools to display the audio waveform to show the fluctuations in the speech content and the playback rhythm. Mark the changes in playback speed at different time points on the waveform graph. For example, use colors or lines to mark the parts where the speech speed changes, and intuitively reflect which segments are slowed down or sped up.

[0171] Based on the playback speed changes of each time segment in the audio data, a playback speed change curve is generated. This curve can reflect the playback speed dynamics in different time periods, so that users can further understand the specific situation of the playback adjustment.

[0172] Through the audio processing algorithm, the playback speed data of each time segment is extracted to generate a time series of speed changes. By comparing with the standard speed (such as 1x speed), the actual playback speed of each time period is calculated. Use visualization tools to draw a playback speed change curve, with the X-axis representing time and the Y-axis representing playback speed. The fluctuation of the curve reflects the changes in playback speed at different time points.

[0173] The system allows users to manually adjust the playback speed, and dynamically updates the playback speed of the audio clip by receiving instructions from the user. Users can speed up or slow down the playback speed of a specific clip according to actual needs.

[0174] Provide an interactive method through the user interface (such as a slider, button), allowing users to select a specific time segment and adjust the playback speed of the segment. The user can directly enter the speed value or select the adjustment range through the slider. After receiving the user's adjustment instruction, the system immediately adjusts the speed of the selected voice segment and updates the displayed waveform and speed change curve. The adjusted segment can maintain consistent pitch through the time stretching algorithm (TSM).

[0175] Record each playback speed adjustment operation of the user and generate the user's playback speed adjustment data. The adjustment data can be used to analyze the user's preferences and operation modes to further optimize the audio playback experience.

[0176] Whenever a user adjusts the playback speed of a certain clip, the system records the adjustment time, the adjusted speed, and the corresponding clip information. The generated adjustment data will be saved in a structured form (such as JSON or a database). The recorded adjustment data is stored in the user log, which can be used to analyze the user's playback speed preferences later, understand which clips the user tends to adjust the playback speed, and provide a basis for further optimizing the audio playback experience.

[0177] Example description: In the financial field, an institution produced a market interpretation video, which included complex market analysis and policy changes. Users want to interpret different content at different speeds: the system generates an audio waveform chart, marking the playback speed changes in each time period, and users can clearly see which clips are accelerated or slowed down. The user viewed the voice playback speed change curve and understood the speed changes of each analysis clip. The user used the slider to adjust the speed of the "stock market fluctuations" part to 0.9x, and accelerated the "regular market data" to 1.2x. Each speed adjustment operation of the user is recorded in the system, which is convenient for subsequent analysis of the user's operating habits and optimization of future playback experience.

[0178] This embodiment provides a flexible and user-friendly interactive platform by realizing the display of audio waveforms, the generation of playback speed change curves, and the reception and recording of user speed adjustment operations. Users can intuitively see the changes in voice playback speed and flexibly adjust the playback speed according to personal needs.

[0179] In one embodiment, after the above S704, the method further includes:

[0180] S801, storing the user adjustment data, and establishing a user adjustment history record based on the user adjustment data;

[0181] S802, analyzing the user adjustment history records to identify user preferences and common adjustment modes;

[0182] S803, generating a personalized playback speed adjustment suggestion based on the user preference and the common adjustment mode;

[0183] S804: Based on the personalized playback speed adjustment suggestion, regularly update the setting logic of the playback speed adjustment coefficient, where the playback speed adjustment coefficient is used to adjust the playback speed of the voice segment.

[0184] In this embodiment, the system needs to store the operation data of each time the user adjusts the playback speed, and generate a user adjustment history record based on the data. The user adjustment history record will include the user's adjustment time, adjustment range and frequency for different clips.

[0185] Each time the user adjusts the playback speed, the system will record the adjustment data (including the adjusted segment, adjustment range, adjustment time, etc.) and save it to the database. Based on the stored data, the system will automatically generate the user's adjustment history, showing the user's adjustment of different segments in different time periods in the form of a timeline. The history records can be saved by date, playback speed, content type, etc.

[0186] By analyzing the user's adjustment history, the system can identify the user's preferences and common patterns in adjusting the playback speed. For example, the user may tend to slow down the playback speed when hearing a certain type of information, or speed up the playback speed when interpreting minor content.

[0187] The system uses data analysis algorithms to conduct in-depth analysis of the user's adjustment history and identify common adjustment patterns, such as which types of clips the user often slows down or speeds up. Based on the analysis results, the system generates the user's playback speed adjustment preferences and clarifies the user's adjustment tendencies under different content types. The analysis can be based on specific content categories (such as news, contract interpretation, etc.) or adjustment behaviors in specific time periods.

[0188] Based on the user's playback speed adjustment preferences, the system generates personalized playback speed adjustment suggestions. These suggestions are based on the user's historical adjustment patterns, helping the user to automatically apply the appropriate playback speed when playing similar content in the future.

[0189] The system generates personalized speed adjustment suggestions based on the user's adjustment patterns in the history. For example, if a user is used to adjusting the playback speed of the market fluctuation analysis section to 0.9x, the system will automatically suggest that the user make the same adjustment for similar content in the future. When the user plays new content, the system can automatically prompt the user whether to apply the personalized playback speed adjustment suggestions based on the previous adjustment patterns. The user can choose to accept or ignore these suggestions.

[0190] Based on the user's personalized suggestions and common adjustment modes, the system regularly updates the setting logic of the playback speed adjustment coefficient to ensure that future content playback can better meet user needs.

[0191] The system will regularly adjust the calculation logic of the playback speed adjustment coefficient based on the user's personalized suggestions. For example, the system can dynamically adjust the range or weight of the coefficient to make it more in line with the user's actual usage preferences. When the user plays similar content again, the system will automatically set the appropriate playback speed adjustment coefficient for the specific clip based on the new adjustment logic, making the user's playback experience more intelligent and personalized.

[0192] Example: In the financial field, a user repeatedly adjusted the playback speed of the "Stock Market Changes" section in a financial policy analysis video, slowing it down to 0.9x. The system recorded the user's adjustment operations, and in subsequent analysis found that the user tended to slow down the playback speed in market changes and forecasts. Based on this preference, the system generated personalized recommendations for the user and prompted the user to use the same slowed playback settings in similar videos in the future. In addition, the system also regularly updates the logic of the playback speed adjustment coefficient to make it more in line with the user's viewing habits and improve the user experience.

[0193] By storing and analyzing the user's playback speed adjustment data, the system can identify the user's common adjustment mode and preference, and then provide the user with personalized playback speed adjustment suggestions. Regularly updating the setting logic of the playback speed adjustment coefficient can ensure that the system adapts to the user's usage habits and improves the user experience.

[0194] In one embodiment, a device for adjusting the playback speed of voice content in a video is provided, and the device for adjusting the playback speed of voice content in a video corresponds to the method for adjusting the playback speed of voice content in a video in the above embodiment. Figure 3 , Figure 3 The functional module diagram of a preferred embodiment of the device for adjusting the playback speed of voice content in a video of the present invention is shown in FIG. A voice recognition module 10, a text processing module 20, a sentiment analysis and importance assessment module 30, an audio adjustment module 40, an audio and video synchronization module 50, and a video editing and synthesis module 60. The functional modules are described in detail as follows:

[0195] The speech recognition module 10 is used to obtain an original video containing speech content to be adjusted, perform speech recognition on the audio track of the original video, and convert the audio data of the speech content into text data;

[0196] A text processing module 20, configured to perform word segmentation processing on the text data, and divide the text data into a plurality of sentences or phrases;

[0197] Sentiment analysis and importance assessment module 30, for identifying keywords in each sentence or phrase, and classifying the sentiment content of the sentences or phrases containing the keywords, and determining the importance level of each sentence or phrase;

[0198] The audio adjustment module 40 is used to adjust the playback speed of the corresponding voice content according to the emotional content and importance level of each sentence or phrase, and generate audio data with adjusted playback speed;

[0199] An audio and video synchronization module 50, used to align the audio data with the adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data;

[0200] The video editing and synthesis module 60 is used to edit and synthesize the synchronized audio and video data to generate a final video with the playback speed of the voice content adjusted.

[0201] In one embodiment, the sentiment analysis and importance assessment module 30 is specifically used to:

[0202] Based on the keyword recognition results, determine the keyword density of each sentence or phrase;

[0203] Analyzing the subject content of the original video and evaluating the relevance of each sentence or phrase to the subject content;

[0204] Based on the keyword density and relevance to the subject content, each sentence or phrase is assigned a corresponding importance level.

[0205] In one embodiment, the audio adjustment module 40 is specifically configured to:

[0206] Determining a playback speed adjustment factor for each sentence or phrase based on the emotional content and importance level of each sentence or phrase;

[0207] Based on the playback speed adjustment coefficient, adjusting the playback speed of the voice segment corresponding to each sentence or phrase;

[0208] The adjusted voice segments are merged to generate audio data with adjusted playback speed.

[0209] In one embodiment, the audio and video synchronization module 50 is specifically used for:

[0210] Obtaining timeline information of the original video, including the timestamp of each video frame;

[0211] Get the timeline information of the audio data with adjusted playback speed, including the timestamp of each audio segment;

[0212] Use a synchronization module to match the timestamps of the audio segments with the timestamps of the corresponding video frames;

[0213] According to the matching result, the time axis of the original video is adjusted so that the time axis of the adjusted video matches the duration of the adjusted audio data;

[0214] The adjusted audio data and the corresponding video image in the adjusted video are synthesized to generate synchronized audio and video data.

[0215] In one embodiment, the text processing module 20 is specifically used for:

[0216] Preprocessing the text data to remove spaces and special symbols and standardize punctuation;

[0217] Selecting a word segmentation tool that matches the language characteristics of the text data, and initializing the word segmentation tool;

[0218] Using the word segmentation tool, segmenting the preprocessed text data, and preliminarily dividing the text data into a plurality of sentences or phrases based on grammatical structure and punctuation marks;

[0219] Apply phrase separation strategy to the text part containing multiple phrases in the sentence, separate the phrases from the sentence, and generate the final sentence or phrase.

[0220] In one embodiment, the video editing and synthesis module 60 is specifically used for:

[0221] Displaying a waveform of the adjusted audio data and determining a change in the playback speed of the voice content;

[0222] Based on the playback speed change, generate and display a playback speed change curve of the voice content;

[0223] receiving a playback speed change instruction from a user, and adjusting the playback speed of the voice content based on the playback speed change instruction;

[0224] Record the user's playback speed adjustment operation and generate user adjustment data.

[0225] In one embodiment, the video editing and synthesis module 60 is specifically used for:

[0226] storing the user adjustment data, and establishing a user adjustment history record based on the user adjustment data;

[0227] Analyze the user adjustment history to identify user preferences and common adjustment patterns;

[0228] Generating personalized playback speed adjustment suggestions based on the user preferences and common adjustment modes;

[0229] Based on the personalized playback speed adjustment suggestion, the setting logic of the playback speed adjustment coefficient is regularly updated, and the playback speed adjustment coefficient is used to adjust the playback speed of the voice segment.

[0230] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal via a network connection. When the computer program is executed by the processor, the functions or steps on the server side of a method for adjusting the playback speed of voice content in a video are realized.

[0231] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the user side of a method for adjusting the playback speed of voice content in a video

[0232] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0233] Acquire an original video containing voice content to be adjusted, perform voice recognition on an audio track of the original video, and convert audio data of the voice content into text data;

[0234] Performing word segmentation processing on the text data to divide the text data into multiple sentences or phrases;

[0235] Identify the keywords in each sentence or phrase, classify the sentiment content of the sentences or phrases containing the keywords, and determine the importance level of each sentence or phrase;

[0236] According to the emotional content and importance level of each sentence or phrase, adjusting the playback speed of the corresponding voice content to generate audio data with adjusted playback speed;

[0237] Aligning the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data;

[0238] The synchronized audio and video data are edited and synthesized to generate a final video with the playback speed of the voice content adjusted.

[0239] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0240] Acquire an original video containing voice content to be adjusted, perform voice recognition on an audio track of the original video, and convert audio data of the voice content into text data;

[0241] Performing word segmentation processing on the text data to divide the text data into multiple sentences or phrases;

[0242] Identify the keywords in each sentence or phrase, classify the sentiment content of the sentences or phrases containing the keywords, and determine the importance level of each sentence or phrase;

[0243] According to the emotional content and importance level of each sentence or phrase, adjusting the playback speed of the corresponding voice content to generate audio data with adjusted playback speed;

[0244] Aligning the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data;

[0245] The synchronized audio and video data are edited and synthesized to generate a final video with the playback speed of the voice content adjusted.

[0246] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0247] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0248] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0249] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for adjusting the playback speed of voice content in a video, characterized in that: The following steps are involved: Acquire an original video containing voice content to be adjusted, perform voice recognition on an audio track of the original video, and convert audio data of the voice content into text data; Performing word segmentation processing on the text data to divide the text data into multiple sentences or phrases; Identify the keywords in each sentence or phrase, classify the sentiment content of the sentences or phrases containing the keywords, and determine the importance level of each sentence or phrase; According to the emotional content and importance level of each sentence or phrase, adjusting the playback speed of the corresponding voice content to generate audio data with adjusted playback speed; Aligning the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data; The synchronized audio and video data are edited and synthesized to generate a final video with the playback speed of the voice content adjusted.

2. The method for adjusting the playback speed of voice content in a video according to claim 1, characterized in that: Determine the importance level of each sentence or phrase, including: Based on the keyword recognition results, determine the keyword density of each sentence or phrase; Analyzing the subject content of the original video and evaluating the relevance of each sentence or phrase to the subject content; Based on the keyword density and relevance to the subject content, each sentence or phrase is assigned a corresponding importance level.

3. The method for adjusting the playback speed of voice content in a video according to claim 1, characterized in that: According to the emotional content and importance level of each sentence or phrase, the playback speed of the corresponding voice content is adjusted to generate audio data with adjusted playback speed, including: Determining a playback speed adjustment factor for each sentence or phrase based on the emotional content and importance level of each sentence or phrase; Based on the playback speed adjustment coefficient, adjusting the playback speed of the voice segment corresponding to each sentence or phrase; The adjusted voice segments are merged to generate audio data with adjusted playback speed.

4. The method for adjusting the playback speed of voice content in a video according to claim 1, characterized in that: The audio data with adjusted playback speed is aligned with the video image of the original video according to the time axis to generate synchronized audio and video data, including: Obtaining timeline information of the original video, including the timestamp of each video frame; Get the timeline information of the audio data with adjusted playback speed, including the timestamp of each audio segment; Use a synchronization module to match the timestamps of the audio segments with the timestamps of the corresponding video frames; According to the matching result, the time axis of the original video is adjusted so that the time axis of the adjusted video matches the duration of the adjusted audio data; The adjusted audio data and the corresponding video image in the adjusted video are synthesized to generate synchronized audio and video data.

5. The method for adjusting the playback speed of voice content in a video according to claim 1, characterized in that: Performing word segmentation processing on the text data to divide the text data into multiple sentences or phrases includes: Preprocessing the text data to remove spaces and special symbols and standardize punctuation; Selecting a word segmentation tool that matches the language characteristics of the text data, and initializing the word segmentation tool; Using the word segmentation tool, segmenting the preprocessed text data, and preliminarily dividing the text data into a plurality of sentences or phrases based on grammatical structure and punctuation marks; Apply phrase separation strategy to the text part containing multiple phrases in the sentence, separate the phrases from the sentence, and generate the final sentence or phrase.

6. The method for adjusting the playback speed of voice content in a video according to claim 1, characterized in that: After editing and synthesizing the synchronized audio and video data to generate a final video with the playback speed of the voice content adjusted, the method further includes: Displaying a waveform of the adjusted audio data and determining a change in the playback speed of the voice content; Based on the playback speed change, generate and display a playback speed change curve of the voice content; receiving a playback speed change instruction from a user, and adjusting the playback speed of the voice content based on the playback speed change instruction; Record the user's playback speed adjustment operation and generate user adjustment data.

7. The method for adjusting the playback speed of voice content in a video as claimed in claim 6, characterized in that: After recording the user's playback speed adjustment operation and generating the user adjustment data, it also includes: storing the user adjustment data, and establishing a user adjustment history record based on the user adjustment data; Analyze the user adjustment history to identify user preferences and common adjustment patterns; Generating personalized playback speed adjustment suggestions based on the user preferences and common adjustment modes; Based on the personalized playback speed adjustment suggestion, the setting logic of the playback speed adjustment coefficient is regularly updated, and the playback speed adjustment coefficient is used to adjust the playback speed of the voice segment.

8. A device for adjusting the playback speed of voice content in a video, characterized in that: The device for adjusting the playback speed of the voice content in the video includes: A speech recognition module, used to obtain an original video containing speech content to be adjusted, perform speech recognition on an audio track of the original video, and convert the audio data of the speech content into text data; A text processing module, used for performing word segmentation processing on the text data, dividing the text data into a plurality of sentences or phrases; Sentiment analysis and importance assessment module, which is used to identify keywords in each sentence or phrase, classify the sentiment content of sentences or phrases containing keywords, and determine the importance level of each sentence or phrase; An audio adjustment module, used to adjust the playback speed of the corresponding voice content according to the emotional content and importance level of each sentence or phrase, and generate audio data with adjusted playback speed; An audio and video synchronization module, used to align the audio data with adjusted playback speed with the video image of the original video according to the time axis to generate synchronized audio and video data; The video editing and synthesis module is used to edit and synthesize the synchronized audio and video data to generate a final video with the playback speed of the voice content adjusted.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a program for adjusting the playback speed of voice content in a video that is stored in the memory and can be run on the processor. When the program for adjusting the playback speed of voice content in a video is executed by the processor, the steps of the method for adjusting the playback speed of voice content in a video as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program for adjusting the playback speed of voice content in a video, and when the program for adjusting the playback speed of voice content in a video is executed by a processor, the steps of the method for adjusting the playback speed of voice content in a video as described in any one of claims 1-7 are implemented.

Images