Method and system for quickly generating short video by combining clone sound with AI copywriting

Abstract

The invention provides a method and a system for quickly generating a short video by combining cloned sound and AI copywriting, and relates to the technical field of data processing, the method comprises the following steps: acquiring sound data of a target person, extracting tone characteristics and volume characteristics of the target person, obtaining sound characteristics, acquiring text information of the AI copywriting, and identifying emotion words to obtain a short video; analyzing the sentiment type of the sentence, calculating the sentiment intensity of the sentiment type according to the modifier of the sentiment word, analyzing the value of the sound feature of each character to obtain sentiment expression data, determining a pause point in the text information, determining the speech speed of the sentence according to the pause point and the sentiment expression data to obtain text expression data, and determining the speech speed of the sentence according to the sentiment expression data. And extracting the sound of each character from the sound data, splicing all the sound data according to the text expression data to obtain audio data, and synthesizing the audio data with the video data to generate a short video. According to the invention, the audio can be generated by combining the cloned sound with the AI copywriting, and the short video can be quickly produced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for quickly generating short videos by cloning sounds and combining them with AI text. Background Art

[0002] In the field of short video creation, the combination of sound cloning technology and AI copy generation technology has become an important tool for content creation. By converting AI copy from text to speech, the tedious process of manual recording is simplified. However, in scenarios that require complex emotional expression, existing technologies may not be able to achieve a natural fusion of audio content with copy emotions and video images.

[0003] For example, in travel short videos, different scenes need to convey different emotions, but existing technologies may not be able to adjust the audio speed and pauses in combination with the emotions in the copy, resulting in abrupt switching between audio and video, making the short video lose coherence, thus affecting the audience's viewing experience and emotional resonance. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for quickly generating short videos by cloning sound and combining AI text, aiming to solve the problems mentioned in the background technology.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In the first aspect, a method for quickly generating a short video by cloning sound and combining AI copywriting, the method comprising:

[0007] Acquire the target person's voice data, analyze the changes in its fundamental frequency and energy, and obtain the pitch change data and volume change data;

[0008] By performing time domain analysis on the pitch change data and the volume change data, the pitch characteristics and volume characteristics of the target person are extracted to obtain the sound characteristics;

[0009] Obtain the text information of the AI ​​copywriting and divide it into sentences. Use the sentiment dictionary to filter the words in the sentences, identify the sentiment words in the sentences, and obtain the sentiment word data.

[0010] According to the sentiment word data, the sentiment type of the sentence is analyzed to obtain the text sentiment data;

[0011] By analyzing the sentiment words and their sentences, identifying the modifiers of the sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence based on the modifiers, the sentiment intensity data is obtained;

[0012] According to the emotion intensity data, the value of the sound feature of each word in each sentence in the text emotion data is analyzed to obtain the emotion expression data;

[0013] According to different emotion types in the text information, the pause points in the text information are determined, and the emotion changes at the pause points are smoothed to obtain the pause data;

[0014] According to the pause point and emotional expression data, the speaking speed of sentences with different emotional types is obtained, and it is fused with the pause data to obtain text expression data;

[0015] According to the emotional expression data, the sound of each word in the text information is extracted from the sound data, and all of them are spliced ​​together according to the text expression data to obtain audio data;

[0016] The prepared video data is obtained and synthesized with the audio data to ensure that the audio data and the visual data are aligned on the timeline to generate a short video.

[0017] Furthermore, the text information of the AI ​​copy is obtained and divided into sentences. The words in the sentences are screened through the sentiment dictionary, and the sentiment words in the sentences are identified to obtain the sentiment word data, including:

[0018] The text information is divided by commas and periods in the text information to obtain sentences;

[0019] The words in the sentence are screened through the sentiment dictionary, the sentences containing sentiment words are identified, and the initial sentiment word data is obtained;

[0020] Analyze the initial sentiment word data, identify the connection between sentiment words and context, and obtain the sentence context;

[0021] According to the sentence context, the sentiment words related to the sentence context are screened to obtain the sentiment word data.

[0022] Furthermore, based on the sentiment word data, the sentiment type of the sentence is analyzed to obtain text sentiment data, including:

[0023] According to the sentiment word data, each sentence is classified and marked as intense sentiment information and bland sentiment information, and classification mark data is obtained;

[0024] By analyzing the sentences with strong emotional information in the classified labeled data, the types of emotional words and sentence contexts are identified, and the emotional labels of the sentences are obtained;

[0025] Refine the sentiment label of each sentence, analyze the correlation between sentiment words and the subject of the sentence, identify the sentiment type of the sentence, and obtain sentiment data;

[0026] According to the sentiment data, the relationship between non-sentiment words and sentiment words in the sentence is analyzed to obtain the sentiment correlation data of the sentence;

[0027] The sentiment type data and sentiment correlation data are integrated to obtain the text sentiment data.

[0028] Furthermore, by analyzing the sentiment words and their sentences, identifying the modifiers of the sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence based on the modifiers, the sentiment intensity data is obtained, including:

[0029] According to the text sentiment data, analyze the words in each sentence and extract the words that modify the sentiment words to obtain the modifying words;

[0030] By analyzing the correlation between modifiers and sentiment words, the type and scope of modifiers are determined, and modifier data is obtained;

[0031] According to the modifier data, the emotional intensity of the emotional type in the sentence is calculated. The calculation formula of emotional intensity is: ,

[0032] in, For the The emotional intensity of the emotional type in the sentence, is the index of the sentence, is the index of sentiment words, For the The total number of sentiment words in the sentence, is the type strength of sentiment words, is the correlation between the modifier and the sentiment word, is the distance between the modifier and the sentiment word, is the correlation between the sentiment word and the subject of the sentence, is the correlation between sentiment words and sentence context, For the The total number of characters in the sentence, For sentiment words Relative position in a sentence, are coefficients respectively.

[0033] Furthermore, based on the emotion intensity data, the values ​​of the sound features of each word in each sentence in the text emotion data are analyzed to obtain the emotion expression data, including:

[0034] According to the emotional intensity data, the sound characteristics of each word in the sentence are analyzed and the pitch value and volume value are calculated;

[0035] The formula for calculating the pitch value is: ,

[0036] in, For the In the sentence The pitch value of the word, is the index of the text, For the The emotional intensity of the emotional type in the sentence, is the fundamental frequency of the characters in the sound data, is the frequency of the sine function of the tone in the sound data, is the natural base, is the period of the sine function of the tone in the sound data, For the importance of word parts in grammar, are the coefficients, is the integration variable;

[0037] The volume value is calculated as: ,

[0038] in, For the In the sentence The volume value of the text. is the index of the text, For the The emotional intensity of the emotional type in the sentence, For the importance of word parts in grammar, is the energy of the characters in the sound data, are coefficients respectively.

[0039] Furthermore, according to different emotion types in the text information, the pause points in the text information are determined, and the emotion changes at the pause points are smoothed to obtain pause data, including:

[0040] According to the different emotional types in the text emotional data, extract sentences containing different emotional types and mark pause points between adjacent sentences;

[0041] According to the pause points, the emotional changes between adjacent sentences are analyzed to determine the difference in emotional intensity and obtain the change difference;

[0042] When the sentiment types between adjacent sentences are different, a pause node is inserted between them;

[0043] When the emotion types between adjacent sentences are the same, the emotion change range between adjacent sentences is judged according to the change difference and the preset difference. When the change difference is greater than the preset difference, a transition node is inserted between adjacent sentences.

[0044] Extract features of pause nodes to obtain the length of sentences before and after the pause nodes, the emotion type and intensity, and determine the pause duration of the pause nodes based on them;

[0045] A smooth curve is generated by extracting the pitch value and volume value of the text before and after the transition node;

[0046] The transition nodes are processed according to the smooth curve to eliminate the abruptness of the transition nodes and obtain the transition data;

[0047] The pause data is generated according to the pause duration and the transition data.

[0048] Furthermore, according to the pause points and emotional expression data, the speech speed of sentences with different emotional types is obtained, and it is fused with the pause data to obtain text expression data, including:

[0049] Determine the speaking speed range based on the emotional type of the sentence;

[0050] According to the number of words and emotional intensity of the sentence, the speed of the sentence is determined by matching it from the speed range;

[0051] According to the pause data, determine the connection between adjacent sentences and the transition method of emotional changes in sentences;

[0052] According to the above results, the expression method of the text information is determined to obtain the text expression data.

[0053] In the second aspect, a short video rapid generation system combining voice cloning and AI copywriting, the system includes:

[0054] The sound data module is used to obtain the sound data of the target person, analyze the changes in its fundamental frequency and energy, and obtain the pitch change data and volume change data;

[0055] A sound feature module is used to extract the tone feature and volume feature of the target person by performing time domain analysis on the tone change data and the volume change data to obtain the sound feature;

[0056] The text analysis module is used to obtain the text information of the AI ​​copywriting, segment it into sentences, filter the words in the sentences through the sentiment dictionary, identify the sentiment words in the sentences, and obtain the sentiment word data;

[0057] Sentiment analysis module, used to analyze the sentiment type of sentences based on sentiment word data to obtain text sentiment data;

[0058] The sentiment intensity module is used to analyze sentiment words and their sentences, identify the modifiers of sentiment words, and calculate the sentiment intensity of the sentiment type in the sentence according to the modifiers to obtain sentiment intensity data;

[0059] The emotion expression module is used to analyze the value of the sound feature of each word in each sentence in the text emotion data according to the emotion intensity data to obtain the emotion expression data;

[0060] The pause analysis module is used to determine the pause points in the text information according to different emotional types in the text information, and to smooth the emotional changes of the pause points to obtain pause data;

[0061] The text expression module is used to obtain the speech rate of sentences of different emotional types according to the pause points, and fuse it with the pause data to obtain text expression data;

[0062] The audio synthesis module is used to extract the sound of each word in the text information from the sound data according to the emotion expression data, and splice them all together according to the text expression data to obtain the audio data;

[0063] The video synthesis module is used to obtain the prepared video data and synthesize it with the audio data to ensure that the audio data and the visual data are aligned on the timeline to generate a short video.

[0064] The above solution of the present invention includes at least the following beneficial effects:

[0065] The present invention can extract the characteristics of pitch and volume by analyzing the fundamental frequency and energy changes of the target person's voice data, thereby obtaining accurate voice feature data, combining the emotional words and modifiers in the AI ​​copy, and conducting in-depth analysis to obtain the emotional type and intensity of the text, and applying it to the regulation of voice features, ensuring a high match between the voice and the copy emotion. The emotion expressed in the copy can be achieved by adjusting the fundamental frequency and energy values, making the audio and scene visual effects more natural and coordinated. This deep fusion of emotion and audio not only makes up for the rough processing of emotional expression by traditional methods, but also significantly improves the user experience and prompts the audience to have a stronger emotional resonance.

[0066] The present invention can analyze sentiment words and modifiers through sentiment dictionaries, and can not only identify sentiment intensity, but also dynamically adjust sound features, including pitch, volume, and speaking speed. In the prior art, sound can often only be adjusted based on a single dimension, resulting in insufficient performance in complex emotional scenes. By combining sentiment intensity data with text sentiment data, each word in each sentence can reflect emotional changes with precise pitch and volume. By fine-tuning the tone and emotional fluctuations of each sentence, highly customized voice output can be achieved. This dynamic adjustment mechanism realizes dynamic adjustment of multi-dimensional emotional expression, so that the generated short video can convey complex emotional information in a more natural way.

[0067] The present invention can automatically process the sound features and text information of the target person, thereby reducing the tedious process of manually recording voice or editing audio. This automated method can complete multiple links including voice generation and video synthesis in a short time. The automated method can quickly generate voice that matches the emotion of the new text, and generate new short videos in combination with visual materials, thereby shortening the production cycle and improving content production efficiency. This method is not only suitable for large-scale content creation, but also particularly suitable for scenarios where content iteration is required.

[0068] The present invention can greatly improve the audiovisual synchronization of short videos by ensuring the alignment of audio and video data on the timeline. By analyzing and smoothing the pause points of the text, the coherence of the audio and screen switching is further enhanced. The switching of different screens often requires different voice intonations and pause times. The pause data is dynamically adjusted according to the emotional state of the text to generate audio that is more in line with the rhythm of the picture. This improvement in synchronization not only reduces the problem of audio and video screen asynchrony in the prior art, but also can optimize the audio generation strategy according to the screen content, so that the audience has a more immersive viewing experience.

[0069] The present invention can greatly improve the viewing experience of short videos through the natural integration of audio generation and video synthesis. When watching short videos, the audience can not only feel the intention of the copy through the pictures, but also further enhance their understanding and emotional identification of the content through highly matching audio. Through the precise adjustment of different emotional intensities and pause nodes, a more dramatic and narrative viewing experience can be created. This function significantly enhances the attractiveness of the content and the audience's viewing stickiness, and provides short video creators with higher quality content output capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a flowchart of a method for quickly generating short videos by cloning sounds and combining them with AI texts provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0072] like Figure 1 As shown, an embodiment of the present invention proposes a method for quickly generating a short video by cloning sound and combining it with AI text, and the method includes:

[0073] Acquire the target person's voice data, analyze the changes in its fundamental frequency and energy, and obtain the pitch change data and volume change data;

[0074] By performing time domain analysis on the pitch change data and the volume change data, the pitch characteristics and volume characteristics of the target person are extracted to obtain the sound characteristics;

[0075] Obtain the text information of the AI ​​copywriting and divide it into sentences. Use the sentiment dictionary to filter the words in the sentences, identify the sentiment words in the sentences, and obtain the sentiment word data.

[0076] According to the sentiment word data, the sentiment type of the sentence is analyzed to obtain the text sentiment data;

[0077] By analyzing the sentiment words and their sentences, identifying the modifiers of the sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence based on the modifiers, the sentiment intensity data is obtained;

[0078] According to the emotion intensity data, the value of the sound feature of each word in each sentence in the text emotion data is analyzed to obtain the emotion expression data;

[0079] According to different emotion types in the text information, the pause points in the text information are determined, and the emotion changes at the pause points are smoothed to obtain the pause data;

[0080] According to the pause point and emotional expression data, the speaking speed of sentences with different emotional types is obtained, and it is fused with the pause data to obtain text expression data;

[0081] According to the emotional expression data, the sound of each word in the text information is extracted from the sound data, and all of them are spliced ​​together according to the text expression data to obtain audio data;

[0082] The prepared video data is obtained and synthesized with the audio data to ensure that the audio data and the visual data are aligned on the timeline to generate a short video.

[0083] In an embodiment of the present invention, the sound data of the target person is obtained, and the changes in its fundamental frequency and energy are analyzed to obtain pitch change data and volume change data. By capturing the natural changes in the sound, a foundation is laid for subsequent audio generation. By performing time domain analysis on the pitch change data and the volume change data, the pitch characteristics and volume characteristics of the target person are extracted to obtain the sound characteristics. The extraction of such sound characteristics not only improves the quality of the subsequent synthesized speech, but also enables the generated speech to dynamically adapt to the requirements of different texts in terms of pitch and volume.

[0084] The text information of AI copywriting is obtained and divided into sentences. The words in the sentences are screened through the sentiment dictionary, the sentiment words in the sentences are identified, and the sentiment word data is obtained, which effectively simplifies the processing process of complex copywriting, so that subsequent sentiment analysis can be carried out on a single sentence, and the sentiment keywords are accurately located for each sentence, providing a solid foundation for generating emotion-rich speech; according to the sentiment word data, the sentiment type of the sentence is analyzed to obtain text sentiment data, and the sentiment type of the sentence is determined by the classification of sentiment words.

[0085] By analyzing sentiment words and their sentences, identifying the modifiers of sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence based on the modifiers, we can obtain sentiment intensity data. This combines the type intensity of sentiment words, the scope of the modifiers, and the contextual relevance to achieve the quantification of sentiment intensity. Based on the sentiment intensity data, we analyze the values ​​of the sound features of each word in each sentence in the text sentiment data to obtain sentiment expression data. Then, through the sentiment intensity, we determine the tone and volume of the words in each sentence to express the corresponding emotion.

[0086] According to the different emotion types in the text information, the pause points in the text information are determined, and the emotional changes at the pause points are smoothed to obtain pause data. Dynamic adjustments are made by analyzing the emotion types and their change intensity. When the emotion types or intensities of adjacent sentences are very different, the abruptness of the speech is eliminated by inserting transition nodes. According to the pause points and emotional expression data, the speaking speed of sentences with different emotional types is obtained, and it is merged with the pause data to obtain text expression data. The dynamic adjustment of the speaking speed is based on the number of words, emotional type and intensity in the sentence to ensure that the expression rhythm and emotion of each sentence are consistent.

[0087] According to the emotional expression data, the sound of each word in the text information is extracted from the sound data, and all of them are spliced ​​together according to the text expression data to obtain audio data. This process ensures that the generated voice highly matches the text content in terms of pitch, volume and speaking speed. Compared with the traditional method of voice splicing in units of sentences, this method further improves the naturalness of the voice through the refined processing of word-by-word splicing; obtain the prepared video data and synthesize it with the audio data to ensure that the audio data and visual data are aligned on the timeline to generate a short video. The synthesis of audio and video is based on audio and volume, and the screen switching duration and rhythm are dynamically adjusted to make the fusion of audio and picture more natural. The short video finally generated shows excellent audio-visual synchronization.

[0088] The target person's voice data is obtained, and the changes in its fundamental frequency and energy are analyzed to obtain the pitch change data and volume change data, which specifically includes:

[0089] Use a high-sensitivity microphone or other audio acquisition equipment to record the target person's voice samples in a quiet environment to ensure that the collected audio signal has a high signal-to-noise ratio. During the recording process, the target person's various tones and speaking speeds should be included to cover the sound characteristics of different emotional expressions.

[0090] The autocorrelation method is used to extract the fundamental frequency of the collected speech signal, the audio signal is divided into frames, and a fast Fourier transform is performed on each frame to calculate the spectrum energy distribution and extract the corresponding fundamental frequency value. The fundamental frequency reflects the main frequency component of the sound and is used to describe the pitch. The time domain energy of each frame of the audio signal is calculated to obtain the frame energy, and the energy values ​​of all frames are serialized to form an energy value sequence.

[0091] The fundamental frequency value and energy value are normalized to remove the environmental impact during the audio recording process, and generate pitch change data and volume change data based on the time axis. The pitch change data is presented with time as the horizontal axis and the fundamental frequency as the vertical axis, and the volume change data is presented with time as the horizontal axis and the frame energy as the vertical axis, forming basic data for subsequent analysis.

[0092] Among them, by performing time domain analysis on the pitch change data and the volume change data, the pitch characteristics and volume characteristics of the target person are extracted to obtain the sound characteristics, which specifically include:

[0093] The pitch change data and volume change data are smoothed using a sliding average algorithm to eliminate noise and mutation points in the data.

[0094] The extraction of pitch features includes:

[0095] Pitch Range: Calculates the difference between the maximum and minimum values ​​of the pitch variation data, reflecting the high and low span of the target person's voice.

[0096] Pitch Trend: By fitting a curve to the pitch change data, the rising and falling trends of the pitch are extracted, marking the pitch change pattern within a specific time period.

[0097] Pitch stability: Calculates the variance of pitch variation data to assess the smoothness or volatility of the target person’s voice. Voices with higher pitch stability are usually suitable for narrative scenes, while voices with larger fluctuations are suitable for expressing emotions.

[0098] The extraction of volume features includes:

[0099] Average Volume: Calculates the average volume change data to describe the average volume level of the target person throughout the audio.

[0100] Volume Fluctuation: Calculates the difference between the peak and trough of the volume change data to evaluate the dynamic range of the target person's voice.

[0101] Volume stability: By calculating the standard deviation of volume change data, we can analyze whether the loudness of the sound is stable. Highly stable volume is suitable for long-term narration, while volume with a large dynamic range is suitable for emotional expression.

[0102] The pitch features and volume features are combined to form the target person's voice feature data, including pitch range, trend, stability, as well as multi-dimensional information such as the average value, dynamic range and stability of the volume. These voice feature data will be used in the subsequent speech synthesis module to lay the foundation for generating voices that meet the characteristics of the target person.

[0103] Among them, according to the emotional expression data, the sound of each word in the text information is extracted from the sound data, and all of them are spliced ​​together according to the text expression data to obtain audio data, which specifically includes:

[0104] According to the content of each word in the text information, a corresponding sound segment is extracted from the sound data of the target person, and the extraction of the sound segment is based on the pitch value and volume value provided in the emotion expression data;

[0105] The audio clip of each word must meet the following characteristics:

[0106] Tone matching: Based on the quantified results of the text’s emotional intensity, adjust the fundamental frequency of the word so that its pitch is consistent with the emotional intensity.

[0107] Volume adjustment: Adjust the volume value based on the grammatical function and emotional expression intensity of the text.

[0108] For each word, a separate audio segment is extracted and generated from the sound data. The duration of each segment is determined by the speaking rate range. The speaking rate is adjusted according to the emotion type. The duration of the audio segment is processed by a time stretching algorithm to make it consistent with the rhythm and pause time defined in the text expression data.

[0109] According to the sentence order provided in the text expression data, all the audio clips of the single words are spliced ​​one by one. When splicing, the audio transition is processed, especially the pause points between adjacent words. The abrupt changes in pitch and volume are reduced to ensure the fluency of speech. The pause time is determined according to the pause nodes marked in the text expression data. When the emotion type changes or the difference in the emotion intensity of the previous and next sentences is significant, a more obvious pause segment is inserted to reflect the natural transition of tone.

[0110] The spliced ​​audio data covers the complete text content and is stored in sentences to form a complete voice file for subsequent synthesis with video data.

[0111] The prepared video data is obtained and synthesized with the audio data to ensure that the audio data and the visual data are aligned on the time axis to generate a short video, which specifically includes:

[0112] The prepared video data can be an existing video clip in the short video material library, or it can be content obtained through editing and shooting. According to the sentence sequence and time length in the audio data, the time axis of the video data is adjusted to determine the start time and end time of each sentence in the audio data and match them to the corresponding video clip;

[0113] The video clips and audio clips are synchronized sentence by sentence on the timeline to ensure that the audio of each sentence can accurately correspond to the picture in the video. For the inconsistency between the rhythm of the picture and the audio, the following processing methods are adopted:

[0114] If the video clip is long, it is trimmed to match the audio sentence;

[0115] If the video clip is short, extend its length by video time stretching or inserting transition shots;

[0116] The video screen is dynamically adjusted according to the emotional intensity of the audio data. For the emotional change processing at the pause point, a soft transition effect can be introduced in the screen switching, such as fade-in and fade-out, gradient switching, etc., to enhance the continuity of the short video. The synchronized audio and video data are synthesized. During the synthesis process, the superposition effect of audio and video is verified in real time to ensure that the starting and ending points of the video screen are consistent with the voice to avoid problems with early or lagging pictures. After completing the preliminary synthesis, the short video can be optimized according to needs, such as adding subtitles, background music or visual effects to enhance the overall effect, generate a complete short video file, and export the final short video file to ensure that it meets the expected playback time, resolution and audio-visual effects.

[0117] In a preferred embodiment of the present invention, the text information of the AI ​​copy is obtained and divided into sentences. The words in the sentences are screened through the sentiment dictionary, and the sentiment words in the sentences are identified to obtain the sentiment word data, including:

[0118] The text information is divided by commas and periods in the text information to obtain sentences;

[0119] The words in the sentence are screened through the sentiment dictionary, the sentences containing sentiment words are identified, and the initial sentiment word data is obtained;

[0120] Analyze the initial sentiment word data, identify the connection between sentiment words and context, and obtain the sentence context;

[0121] According to the sentence context, the sentiment words related to the sentence context are screened to obtain the sentiment word data.

[0122] In the embodiments of the present invention, the text information is segmented by commas and full stops in the text information to obtain sentences, effectively dividing the semantic units of the text information and avoiding inaccurate emotional expressions caused by the overall analysis of the text; the words in the sentences are screened through an emotion dictionary to identify the sentences containing emotion words, obtaining initial emotion word data, and by comparing the words in the sentences character by character, the key words expressing emotions in the sentences can be effectively identified; the initial emotion word data is analyzed to identify the connection between the emotion words and the context, obtaining the sentence context, and through context analysis, the emotional misjudgment caused by the isolated extraction of single emotion words is avoided; according to the sentence context, the emotion words related to the sentence context are screened to obtain emotion word data, and by removing redundant data, the emotion word data is made more expressive.

[0123] Among them, analyzing the initial emotion word data to identify the connection between the emotion words and the context, obtaining the sentence context specifically includes:

[0124] For each emotion word marked in the sentence, a certain number of words are respectively selected forward and backward from the position where the emotion word is located to form a context window. For example, if the size of the context window is set to 3, the three words before and after the emotion word will be extracted as context information; the context extraction range can be dynamically adjusted according to the sentence length. For short sentences, the window range can be expanded to cover the whole sentence, and for long sentences, only local content is extracted with the emotion word as the center;

[0125] The extracted context words and the emotion word are input into a pre-trained word vector model together to calculate the semantic similarity between the emotion word and the context words, and the semantic similarity is quantified by the vector inner product or cosine similarity; if the similarity is high, it indicates that the context words have a strong semantic influence on the emotion word. If the similarity is low, the relevance between the context words and the emotion word is weak;

[0126] The words semantically related to the emotion word in the context are weighted and combined to form the semantic representation of the sentence. The weight can be comprehensively determined according to the semantic similarity and the importance of the part of speech. For example, the weights of the subject and the modifier can be higher; a language model is used to generate the overall context embedding vector of the sentence to ensure that the emotion type of the emotion word is consistent with the overall context logic of the sentence;

[0127] If there are multiple emotion words in the sentence, an independent context window and semantic representation are respectively constructed for each emotion word, and the results are merged to form a unified context description. When optimizing, high-frequency and low-correlation words such as "de" (的) and "le" (了) are removed to avoid interfering with the context analysis results.

[0128] Among them, according to the sentence context, screening the emotion words related to the sentence context to obtain the emotion word data specifically includes:

[0129] The word vector of the sentiment word and the embedding vector of the sentence context are calculated to obtain the correlation between each sentiment word and the overall context. The correlation is measured by semantic similarity, and the result is between 0 and 1. If the correlation of a sentiment word is higher than the preset threshold, it is considered that the sentiment word is consistent with the context. If it is lower than the threshold, it is considered that the sentiment word has a low contribution to the overall sentiment of the current sentence.

[0130] According to the calculated relevance, sentiment words with low relevance are eliminated, and only words that are consistent with the context are retained. For sentences with multiple sentiment words, they are sorted from high to low by relevance, and only the top n sentiment words are retained. n is dynamically adjusted according to the complexity of the context. For example, for sentences with rich emotional expressions, n can be 3, and for simple sentences, n can be 1.

[0131] For the retained sentiment words, their modifiers and contextual effects are further analyzed, and the screening results of the sentiment words are double-checked to ensure the consistency of the context analysis and screening results; the screened sentiment words and their sentiment types, sentiment intensity, modification information, etc. are combined into sentiment word data. The data storage form includes sentiment word text, type identifier and context label to provide complete data support for subsequent steps.

[0132] In a preferred embodiment of the present invention, the sentiment type of the sentence is analyzed according to the sentiment word data to obtain text sentiment data, including:

[0133] According to the sentiment word data, each sentence is classified and marked as intense sentiment information and bland sentiment information, and classification mark data is obtained;

[0134] By analyzing the sentences with strong emotional information in the classified labeled data, the types of emotional words and sentence contexts are identified, and the emotional labels of the sentences are obtained;

[0135] Refine the sentiment label of each sentence, analyze the correlation between sentiment words and the subject of the sentence, identify the sentiment type of the sentence, and obtain sentiment data;

[0136] According to the sentiment data, the relationship between non-sentiment words and sentiment words in the sentence is analyzed to obtain the sentiment correlation data of the sentence;

[0137] The sentiment type data and sentiment correlation data are integrated to obtain the text sentiment data.

[0138] In an embodiment of the present invention, each sentence is classified according to the sentiment word data and marked as intense sentiment information and bland sentiment information, and classified marking data is obtained, thereby realizing the preliminary classification of sentiment and stratifying the sentences according to the sentiment intensity, laying a foundation for subsequent analysis; by analyzing the sentences with intense sentiment information in the classified marking data, the type of sentiment words and the sentence context are identified, and the sentiment label of the sentence is obtained, and the sentiment label provides a clear sentiment classification for each sentence; the sentiment label of each sentence is refined, the correlation between the sentiment words and the subject of the sentence is analyzed, the sentiment type of the sentence is identified, and the sentiment data is obtained. By analyzing the correlation between the sentiment words and the subject, the main sentiment of the sentence can be more accurately identified; according to the sentiment data, the relationship between the non-sentiment words and the sentiment words in the sentence is analyzed to obtain the sentiment correlation data of the sentence, and the extracted sentiment correlation data can fully reflect the emotional expression in the sentence; the sentiment type data and the sentiment correlation data are integrated to obtain the text sentiment data, and the text sentiment data finally generated is output in the form of a data table, including the sentiment type and correlation analysis results of each sentence.

[0139] Among them, the sentiment label of each sentence is refined, the correlation between the sentiment word and the subject of the sentence is analyzed, the sentiment type of the sentence is identified, and the sentiment data is obtained, which specifically includes:

[0140] Extract the sentiment label and subject of each sentence, analyze whether the sentiment word directly modifies the subject or indirectly acts on the subject through the modifier, adjust or confirm the sentiment type of the sentence according to the correlation between the sentiment word and the subject, and further refine the sentiment label in combination with the sentence structure to obtain sentiment data.

[0141] In a preferred embodiment of the present invention, by analyzing sentiment words and their sentences, identifying the modifiers of the sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence according to the modifiers, the sentiment intensity data is obtained, including:

[0142] According to the text sentiment data, analyze the words in each sentence and extract the words that modify the sentiment words to obtain the modifying words;

[0143] By analyzing the correlation between modifiers and sentiment words, the type and scope of modifiers are determined, and modifier data is obtained;

[0144] According to the modifier data, the emotional intensity of the emotional type in the sentence is calculated. The calculation formula of emotional intensity is: ,

[0145] in, For the The emotional intensity of the emotional type in the sentence, is the index of the sentence, is the index of sentiment words, For the The total number of sentiment words in the sentence, is the correlation between the modifier and the sentiment word, is the distance between the modifier and the sentiment word, is the correlation between the sentiment word and the subject of the sentence, is the correlation between sentiment words and sentence context, For the The total number of characters in the sentence, For sentiment words Relative position in a sentence, are coefficients respectively.

[0146] In an embodiment of the present invention, based on the text sentiment data, the words in each sentence are analyzed, and the words that modify the sentiment words are extracted to obtain the modifiers. By extracting the modifiers and classifying them, this step can refine the semantic table of the sentiment words; by analyzing the correlation between the modifiers and the sentiment words, the type and scope of the modifiers are determined, the modifier data is obtained, and the influence of the modifiers on the sentiment words is quantified; based on the modifier data, the sentiment intensity of the sentiment type in the sentence is calculated, and by accumulating the modifying effect of each sentiment word in the sentence, the overall sentiment intensity of the sentence is obtained.

[0147] in, is the correlation between the modifier and the sentiment word, indicating the degree of modification of the sentiment word by the modifier. The specific value is determined by the modifier itself. The larger the value, the stronger the modification degree. It is the correlation between the sentiment word and the subject of the sentence, indicating whether the sentiment word is related to the core theme of the current sentence. The higher the value, the stronger the correlation. It is the correlation between sentiment words and sentence context, which is used to measure whether the sentiment words are reasonable in the sentence context. The higher the value, the higher the degree of integration with the context.

[0148] Among them, by analyzing the correlation between the modifier and the sentiment word, the type and scope of the modifier are determined, and the modifier data is obtained, which specifically includes:

[0149] Sentences are segmented to obtain each word and its part of speech in the sentence. After matching sentiment words with sentiment dictionaries, the words around the sentiment words are analyzed to screen out modifiers with grammatical functions such as limiting, emphasizing, and weakening. According to the grammatical functions of the modifiers, they are classified into emphasizing, weakening, or limiting types.

[0150] In a preferred embodiment of the present invention, based on the emotion intensity data, the value of the sound feature of each word in each sentence in the text emotion data is analyzed to obtain the emotion expression data, including:

[0151] According to the emotional intensity data, the sound characteristics of each word in the sentence are analyzed and the pitch value and volume value are calculated;

[0152] The formula for calculating the pitch value is: ,

[0153] in, For the In the sentence The pitch value of the word, is the index of the text, For the The emotional intensity of the emotional type in the sentence, is the fundamental frequency of the characters in the sound data, is the frequency of the sine function of the tone in the sound data, is the natural base, is the period of the sine function of the tone in the sound data, For the importance of word parts in grammar, are the coefficients, is the integration variable;

[0154] The volume value is calculated as: ,

[0155] in, For the In the sentence The volume value of the text. is the index of the text, For the The emotional intensity of the emotional type in the sentence, For the importance of word parts in grammar, is the energy of the characters in the sound data, are coefficients respectively.

[0156] in, is the frequency of the sine function of the pitch in the sound data, determined by the pitch variation data, and represents the periodic fluctuation of the pitch; The importance of the word part of speech in grammar measures the role of the word in the grammatical structure of the sentence. The higher the value, the greater its contribution to the semantics of the sentence. It is the energy of the text in the sound data, determined by the volume change data, and represents the volume information of the target person's text at the acoustic level.

[0157] In an embodiment of the present invention, this step solves the problem that traditional methods can only generate speech based on a single emotion type. Through word-by-word analysis, the system can refine the sound characteristics of each character, so that the generated speech can better reflect the emotional delicacy of the text content. For example, when emphasizing certain emotionally strong characters, the pitch and volume of the character can be dynamically adjusted to enhance the appeal of the speech.

[0158] In a preferred embodiment of the present invention, according to different emotion types in the text information, the pause points in the text information are determined, and the emotion changes of the pause points are smoothed to obtain the pause data, including:

[0159] According to the different emotional types in the text emotional data, extract sentences containing different emotional types and mark pause points between adjacent sentences;

[0160] According to the pause points, the emotional changes between adjacent sentences are analyzed to determine the difference in emotional intensity and obtain the change difference;

[0161] When the sentiment types between adjacent sentences are different, a pause node is inserted between them;

[0162] When the emotion types between adjacent sentences are the same, the emotion change range between adjacent sentences is judged according to the change difference and the preset difference. When the change difference is greater than the preset difference, a transition node is inserted between adjacent sentences.

[0163] Extract features of pause nodes to obtain the length of sentences before and after the pause nodes, the emotion type and intensity, and determine the pause duration of the pause nodes based on them;

[0164] A smooth curve is generated by extracting the pitch value and volume value of the text before and after the transition node;

[0165] The transition nodes are processed according to the smooth curve to eliminate the abruptness of the transition nodes and obtain the transition data;

[0166] The pause data is generated according to the pause duration and the transition data.

[0167] In an embodiment of the present invention, sentences containing different emotion types are extracted according to different emotion types in text emotion data, and pause points are marked between adjacent sentences, which can dynamically identify emotion changes and provide a basis for subsequent pause optimization; according to the pause points, the emotion changes between adjacent sentences are analyzed, the difference in emotion intensity is determined, and the change difference is obtained. A transition node is introduced by the change difference, which solves the problem of unnatural emotion switching in traditional audio generation; feature extraction is performed on the pause node to obtain the length of the previous and next sentences, the emotion type and the intensity of the pause node, and the pause duration of the pause node is determined based on it. Through feature extraction and duration calculation, this step can ensure that the pause node is coordinated with the content of the previous and next sentences.

[0168] By extracting the pitch and volume values ​​of the text before and after the transition node and generating a smooth curve, the abruptness of the voice at the pause node can be eliminated and the fluency of emotional expression can be enhanced; the transition node is processed according to the smooth curve to eliminate the abruptness of the transition node and obtain transition data. Through the generation and integration of transition data, the overall coherence and naturalness of the short video audio can be further improved; pause data is generated based on the pause duration and transition data. The generated pause data can fully match the emotional needs of the text content and the time rhythm of the visual content.

[0169] The transition nodes are processed according to the smooth curve to eliminate the abruptness of the transition nodes and obtain transition data, which specifically includes:

[0170] According to the difference in pitch and volume of the pause node, a smooth curve is constructed, and by calculating the gradient parameters of the connection between the sentences before and after the pause node of the smooth curve, transition data is generated, including the gradient characteristics of pitch and volume. The transition data is integrated into the pause node to eliminate the audio abruptness of the pause node and ensure a natural transition.

[0171] In a preferred embodiment of the present invention, the speech rate of sentences of different emotion types is obtained according to the pause points and the emotion expression data, and the speech rate is merged with the pause data to obtain text expression data, including:

[0172] Determine the speaking speed range based on the emotional type of the sentence;

[0173] According to the number of words and emotional intensity of the sentence, the speed of the sentence is determined by matching it from the speed range;

[0174] According to the pause data, determine the connection between adjacent sentences and the transition method of emotional changes in sentences;

[0175] According to the above results, the expression method of the text information is determined to obtain the text expression data.

[0176] In an embodiment of the present invention, the speaking speed range is determined according to the emotion type of the sentence, and the corresponding relationship between the emotion type and the speaking speed range can be used to match the speaking speed; the speaking speed of the sentence is determined by matching it within the speaking speed range according to the number of words and the emotion intensity of the sentence, so that dynamic adjustment of the speaking speed can be achieved; the connection method between adjacent sentences and the transition method of emotional changes in the sentence are determined according to the pause data, and the transition between sentences appears more natural by optimizing the connection method; based on the above results, the expression method of the text information is determined to obtain text expression data, and the generation of text expression data directly affects the final performance of the short video audio.

[0177] Among them, according to the pause data, the connection method between adjacent sentences and the transition method of emotional changes in sentences are determined, which specifically includes:

[0178] Obtain the sentiment type and sentiment intensity data of adjacent sentences from the sentiment analysis module, calculate the change difference of the sentiment intensity of adjacent sentences, and determine whether to insert a transition node based on it. When the change difference is greater than the preset change threshold, insert a transition node between the two sentences. If the sentiment types of the two sentences are different, directly insert a sentiment turning transition node. If the sentiment types are the same but the change difference is large, insert a sentiment smoothing node.

[0179] According to the pause duration, emotional intensity difference and emotional type of the previous and next sentences, the pause time of the transition node is determined, the pitch value, volume value and speech speed of the sentences before and after the transition node are extracted, and the curve is generated through the smoothing algorithm. The smoothing algorithm can use the cubic spline interpolation method to ensure the natural transition of the audio;

[0180] Analyze the characteristics of pause points in adjacent sentences, including the relevance of sentence length, semantic relationship, and emotional type, and select the appropriate pause processing method:

[0181] For scenes with dramatic emotional changes, such as from cheerfulness to depression, extend the pauses and gradually slow down the speech speed;

[0182] For scenes with gradual emotional changes, such as from lighthearted to warm, keep the pauses short and adjust the tone of voice transitions slightly.

[0183] According to the above results, the expression mode of the text information is determined to obtain text expression data, which specifically includes:

[0184] According to the number of words and emotional intensity in the sentence, the speech rate range is matched, and the speech rate and pause time are combined to generate the speech rate control parameters of the sentence;

[0185] Record relevant data for each pause point, including: pause time, pitch change range of the preceding and following sentences, emotional smoothing curve parameters, and transition type;

[0186] Based on the speech rate and pause point data of each sentence, a timeline mapping table is generated for the entire text. The timeline records the start and end time of each word. On the timeline, the smooth curve of the emotional change of the transition node is inserted between the sentences, and the time continuity of all nodes is ensured;

[0187] Integrate the sentence speed data, pause point data and timeline mapping table to generate complete text expression data. For example, for the text "The sun shines on the beach, and the breeze blows gently", according to the emotional smooth transition curve, the speed of each word is dynamically adjusted to 200 milliseconds to 300 milliseconds, the pause point is set to 1.5 seconds, and the timeline mapping is:

[0188] Sentence 1 start time: 0 milliseconds; end time: 5000 milliseconds;

[0189] Pause node start time: 5000 milliseconds; end time: 6500 milliseconds;

[0190] Sentence 2 start time: 6500 milliseconds; end time: 12000 milliseconds;

[0191] The matching degree between the timeline and the speech flow is verified through simulation tools to ensure the coordination and consistency of emotional changes and speech expressions.

[0192] An embodiment of the present invention further provides a short video rapid generation system combining cloned sound with AI text, the system comprising:

[0193] The sound data module is used to obtain the sound data of the target person, analyze the changes in its fundamental frequency and energy, and obtain the pitch change data and volume change data;

[0194] A sound feature module is used to extract the tone feature and volume feature of the target person by performing time domain analysis on the tone change data and the volume change data to obtain the sound feature;

[0195] The text analysis module is used to obtain the text information of the AI ​​copywriting, segment it into sentences, filter the words in the sentences through the sentiment dictionary, identify the sentiment words in the sentences, and obtain the sentiment word data;

[0196] Sentiment analysis module, used to analyze the sentiment type of sentences based on sentiment word data to obtain text sentiment data;

[0197] The sentiment intensity module is used to identify the modifiers of sentiment words by analyzing the sentiment words and their sentences, and calculate the sentiment intensity of the sentiment type in the sentence based on the modifiers;

[0198] The emotion expression module is used to analyze the value of the sound feature of each word in each sentence in the text emotion data according to the emotion intensity data to obtain the emotion expression data;

[0199] The pause analysis module is used to determine the pause points in the text information according to different emotional types in the text information, and to smooth the emotional changes of the pause points to obtain pause data;

[0200] The text expression module is used to obtain the speech rate of sentences of different emotional types according to the pause points, and fuse it with the pause data to obtain text expression data;

[0201] The audio synthesis module is used to extract the sound of each word in the text information from the sound data according to the emotion expression data, and splice them all together according to the text expression data to obtain the audio data;

[0202] The video synthesis module is used to obtain the prepared video data and synthesize it with the audio data to ensure that the audio data and the visual data are aligned on the timeline to generate a short video.

[0203] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0204] The embodiment of the present invention further provides a computing device, including: a processor, a memory storing a computer program, and when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0205] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0206] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for quickly generating short videos by cloning sound and combining AI copywriting, characterized in that: The method comprises: Acquire the target person's voice data, analyze the changes in its fundamental frequency and energy, and obtain the pitch change data and volume change data; By performing time domain analysis on the pitch change data and the volume change data, the pitch characteristics and volume characteristics of the target person are extracted to obtain the sound characteristics; Obtain the text information of the AI ​​copywriting and divide it into sentences. Use the sentiment dictionary to filter the words in the sentences, identify the sentiment words in the sentences, and obtain the sentiment word data. According to the sentiment word data, the sentiment type of the sentence is analyzed to obtain the text sentiment data; By analyzing the sentiment words and their sentences, identifying the modifiers of the sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence based on the modifiers, the sentiment intensity data is obtained; According to the emotion intensity data, the value of the sound feature of each word in each sentence in the text emotion data is analyzed to obtain the emotion expression data; According to different emotion types in the text information, the pause points in the text information are determined, and the emotion changes at the pause points are smoothed to obtain the pause data; According to the pause point and emotional expression data, the speaking speed of sentences with different emotional types is obtained, and it is fused with the pause data to obtain text expression data; According to the emotional expression data, the sound of each word in the text information is extracted from the sound data, and all of them are spliced ​​together according to the text expression data to obtain audio data; The prepared video data is obtained and synthesized with the audio data to ensure that the audio data and the visual data are aligned on the timeline to generate a short video.

2. The method for quickly generating short videos by cloning sounds and combining AI text according to claim 1 is characterized in that: Get the text information of AI copywriting and divide it into sentences. Use the sentiment dictionary to filter the words in the sentences, identify the sentiment words in the sentences, and obtain the sentiment word data, including: The text information is divided by commas and periods in the text information to obtain sentences; The words in the sentence are screened through the sentiment dictionary, the sentences containing sentiment words are identified, and the initial sentiment word data is obtained; Analyze the initial sentiment word data, identify the connection between sentiment words and context, and obtain the sentence context; According to the sentence context, the sentiment words related to the sentence context are screened to obtain the sentiment word data.

3. The method for quickly generating short videos by cloning sounds and combining AI text according to claim 2 is characterized in that: According to the sentiment word data, the sentiment type of the sentence is analyzed to obtain text sentiment data, including: According to the sentiment word data, each sentence is classified and marked as intense sentiment information and bland sentiment information, and classification mark data is obtained; By analyzing the sentences with strong emotional information in the classified labeled data, the types of emotional words and sentence contexts are identified, and the emotional labels of the sentences are obtained; Refine the sentiment label of each sentence, analyze the correlation between sentiment words and the subject of the sentence, identify the sentiment type of the sentence, and obtain sentiment data; According to the sentiment data, the relationship between non-sentiment words and sentiment words in the sentence is analyzed to obtain the sentiment correlation data of the sentence; The sentiment type data and sentiment correlation data are integrated to obtain the text sentiment data.

4. The method for quickly generating short videos by cloning sounds and combining them with AI text according to claim 3, characterized in that: By analyzing sentiment words and their sentences, identifying the modifiers of sentiment words, and calculating the sentiment intensity of the sentiment type in the sentence based on the modifiers, we can obtain sentiment intensity data, including: According to the text sentiment data, analyze the words in each sentence and extract the words that modify the sentiment words to obtain the modifying words; By analyzing the correlation between modifiers and sentiment words, the type and scope of modifiers are determined, and modifier data is obtained; According to the modifier data, the emotional intensity of the emotional type in the sentence is calculated. The calculation formula of emotional intensity is: ， in, For the The emotional intensity of the emotional type in the sentence, is the index of the sentence, is the index of sentiment words, For the The total number of sentiment words in the sentence, is the correlation between the modifier and the sentiment word, is the distance between the modifier and the sentiment word, is the correlation between the sentiment word and the subject of the sentence, is the correlation between sentiment words and sentence context, For the The total number of characters in the sentence, For sentiment words Relative position in a sentence, are coefficients respectively.

5. The method for quickly generating short videos by cloning sounds and combining them with AI text according to claim 4, characterized in that: According to the emotion intensity data, the values ​​of the sound features of each word in each sentence in the text emotion data are analyzed to obtain the emotion expression data, including: According to the emotional intensity data, the sound characteristics of each word in the sentence are analyzed and the pitch value and volume value are calculated; The formula for calculating the pitch value is: ， in, For the In the sentence The pitch value of the word, is the index of the text, For the The emotional intensity of the emotional type in the sentence, is the fundamental frequency of the characters in the sound data, is the frequency of the sine function of the tone in the sound data, is the natural base, is the period of the sine function of the tone in the sound data, For the importance of word parts in grammar, are coefficients, is the integral variable; The volume value is calculated as: ， in, For the In the sentence The volume value of the text, is the index of the text, For the The emotional intensity of the emotional type in the sentence, For the importance of word parts in grammar, is the energy of the characters in the sound data, are coefficients respectively.

6. The method for quickly generating short videos by cloning sounds and combining AI text according to claim 5, characterized in that: According to different emotional types in the text information, the pause points in the text information are determined, and the emotional changes of the pause points are smoothed to obtain pause data, including: According to the different emotional types in the text emotional data, extract sentences containing different emotional types and mark pause points between adjacent sentences; According to the pause points, the emotional changes between adjacent sentences are analyzed to determine the difference in emotional intensity and obtain the change difference; When the sentiment types between adjacent sentences are different, a pause node is inserted between them; When the emotion types between adjacent sentences are the same, the emotion change range between adjacent sentences is judged according to the change difference and the preset difference. When the change difference is greater than the preset difference, a transition node is inserted between adjacent sentences. Extract features of pause nodes to obtain the length of sentences before and after the pause nodes, the emotion type and intensity, and determine the pause duration of the pause nodes based on them; A smooth curve is generated by extracting the pitch value and volume value of the text before and after the transition node; The transition nodes are processed according to the smooth curve to eliminate the abruptness of the transition nodes and obtain the transition data; The pause data is generated according to the pause duration and the transition data.

7. The method for quickly generating short videos by cloning sounds and combining them with AI text according to claim 6, characterized in that: According to the pause point and emotional expression data, the speaking speed of sentences with different emotional types is obtained, and it is fused with the pause data to obtain text expression data, including: Determine the speaking speed range based on the emotional type of the sentence; According to the number of words and emotional intensity of the sentence, the speed of the sentence is determined by matching it from the speed range; According to the pause data, determine the connection between adjacent sentences and the transition method of emotional changes in sentences; According to the above results, the expression method of the text information is determined to obtain the text expression data.

8. A short video rapid generation system combining voice cloning and AI copywriting, characterized by: The system comprises: The sound data module is used to obtain the sound data of the target person, analyze the changes in its fundamental frequency and energy, and obtain the pitch change data and volume change data; A sound feature module is used to extract the tone feature and volume feature of the target person by performing time domain analysis on the tone change data and the volume change data to obtain the sound feature; The text analysis module is used to obtain the text information of the AI ​​copywriting, segment it into sentences, filter the words in the sentences through the sentiment dictionary, identify the sentiment words in the sentences, and obtain the sentiment word data; Sentiment analysis module, used to analyze the sentiment type of sentences based on sentiment word data to obtain text sentiment data; The sentiment intensity module is used to analyze sentiment words and their sentences, identify the modifiers of sentiment words, and calculate the sentiment intensity of the sentiment type in the sentence according to the modifiers to obtain sentiment intensity data; The emotion expression module is used to analyze the value of the sound feature of each word in each sentence in the text emotion data according to the emotion intensity data to obtain the emotion expression data; The pause analysis module is used to determine the pause points in the text information according to different emotional types in the text information, and to smooth the emotional changes of the pause points to obtain pause data; The text expression module is used to obtain the speech rate of sentences of different emotional types according to the pause points, and fuse it with the pause data to obtain text expression data; The audio synthesis module is used to extract the sound of each word in the text information from the sound data according to the emotion expression data, and splice them all together according to the text expression data to obtain the audio data; The video synthesis module is used to obtain the prepared video data and synthesize it with the audio data to ensure that the audio data and the visual data are aligned on the timeline to generate a short video.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images