Intelligent podcast generation method and apparatus, device, and storage medium

Abstract

The application discloses an intelligent podcast generation method and device, equipment and storage medium, and relates to the technical field of artificial intelligence, which comprises the following steps: obtaining a podcast material uploaded by a user, wherein the podcast material comprises a theme text and description information; determining a dialogue text of the user according to the theme text and the description information, and analyzing the emotional features of the dialogue text to obtain an emotional vector; performing voice synthesis according to the pitch, volume and speech rate in the emotional vector to obtain a voice stream of the user; determining background sound of the dialogue text according to the audio energy in the voice stream; and generating a podcast audio according to the voice stream and the background sound. The application can create a coherent and immersive auditory space, make the emotional trend of the voice naturally fit the environmental soundscape of the background sound, make the overall listening experience smooth and unified, enhance the real sense of communication of the dialogue, and improve the infectivity of the podcast audio.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to methods, apparatus, devices and storage media for generating intelligent podcasts. Background Technology

[0002] With the rapid growth of audio content consumption such as podcasts and audiobooks, users' demands for auditory experience are gradually increasing. They expect the generated audio to have natural and fluent speech, to express emotions such as joy, anger, sorrow, and happiness according to the content plot, and to be accompanied by environmental sound effects consistent with the semantics, forming an immersive listening experience. Therefore, there is a need to quickly transform text materials into high-quality conversational podcasts.

[0003] Currently, the existing approach involves users uploading text files, which are then segmented using simple natural language processing to extract speech synthesis instructions. A pre-defined speech library is then used to read the text sentence by sentence, with a fixed background music added. However, this approach fails to understand the text content, cannot summarize the material, ignores the deeper emotional information within the text, and uses the same rhythm and tone for all content, resulting in poor emotional impact. Furthermore, the fixed background music cannot dynamically match ambient sounds according to semantic changes, creating a strong sense of disjointedness in the auditory experience. Therefore, how to intelligently generate high-quality dialogue content suitable for podcast expression based on different materials has become an urgent problem to be solved. Summary of the Invention

[0004] The main purpose of this application is to provide a method, apparatus, device and storage medium for generating intelligent podcasts, aiming to solve the technical problem of how to intelligently generate high-quality dialogue content suitable for podcast expression based on different materials.

[0005] To achieve the above objectives, this application proposes a method for generating smart podcasts, the method comprising: Obtain podcast materials uploaded by users, including topic text and descriptive information; Based on the topic text and the description information, the user's dialogue text is determined, and the sentiment characteristics of the dialogue text are analyzed to obtain a sentiment vector; Speech synthesis is performed based on the pitch, volume, and speech rate in the emotion vector to obtain the user's speech stream; The background sound of the dialogue text is determined based on the audio energy in the speech stream; Podcast audio is generated based on the speech stream and the background noise.

[0006] In one embodiment, the step of determining the user's dialogue text based on the topic text and the description information includes: Based on the topic text and the description information, the user's dialogue elements are determined, including name, gender, dialogue content, and emotional fluctuation type. The user's dialogue text is obtained based on the name, gender, dialogue content, and emotional fluctuation type.

[0007] In one embodiment, the step of analyzing the sentiment features of the dialogue text to obtain a sentiment vector includes: The emotional dimension is obtained, which includes pleasure, arousal and dominance; Based on the pleasure level, arousal level, and dominance level, the emotional features of the dialogue text are analyzed to obtain an emotional vector.

[0008] In one embodiment, the step of synthesizing speech based on pitch, volume, and speech rate in the emotion vector to obtain the user's speech stream includes: Obtain the user's baseline timbre hyperparameters; Based on the timbre hyperparameters corresponding to the emotion vector, the baseline timbre hyperparameters are adjusted to obtain the speech hyperparameters; Speech synthesis is performed based on the pitch, volume, and speech rate in the speech hyperparameters to obtain the user's speech stream.

[0009] In one embodiment, the step of determining the background noise of the dialogue text based on the audio energy in the speech stream includes: Based on the audio energy in the speech stream, determine the gain attenuation factor corresponding to the audio energy; The background sound of the dialogue text is determined based on the gain attenuation factor.

[0010] In one embodiment, the step of determining the gain attenuation factor corresponding to the audio energy based on the audio energy in the speech stream includes: Get the slice length; The speech stream is divided according to the slice length to determine audio frames, and the corresponding audio energy is calculated based on the audio frames. The corresponding gain attenuation factor is determined based on the audio energy.

[0011] In one embodiment, the step of determining the background audio of the dialogue text based on the gain attenuation factor includes: Environmental keywords are identified based on the dialogue text; Based on the environmental keywords, the corresponding initial background sound track is retrieved from the preset background sound database. The initial background sound track includes white noise and ambient sound. The initial background sound is adjusted according to the gain attenuation factor to obtain the background sound.

[0012] Furthermore, to achieve the above objectives, this application also proposes a smart podcast generation device, which includes: The acquisition module is used to acquire podcast materials uploaded by users, including topic text and description information; The processing module is used to determine the user's dialogue text based on the topic text and the description information, and to obtain an emotion vector based on the dialogue text; The processing module is also used to perform speech synthesis based on the pitch, volume and speech rate in the emotion vector to obtain the user's speech stream; An execution module is used to determine the background sound of the dialogue text based on the audio energy in the speech stream; The execution module is also configured to generate podcast audio based on the speech stream and the background noise.

[0013] In addition, to achieve the above objectives, this application also proposes a smart podcast generation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the smart podcast generation method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the smart podcast generation method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This embodiment proposes a smart podcast generation method. The method involves acquiring podcast material uploaded by a user, including topic text and descriptive information; determining the user's dialogue text based on the topic text and descriptive information, and analyzing the emotional features of the dialogue text to obtain an emotional vector; synthesizing speech based on the pitch, volume, and speech rate in the emotional vector to obtain the user's speech stream; determining the background sound of the dialogue text based on the audio energy in the speech stream; and generating podcast audio based on the speech stream and the background sound. This application automatically generates dialogue text by analyzing user-input podcast material, performs emotional analysis to extract emotional vectors, synthesizes a speech stream using the emotional vectors, and dynamically matches environmental background sound corresponding to the semantic content based on the audio energy of the speech stream. This results in podcast audio that combines natural emotional appeal and an immersive auditory atmosphere, creating a coherent and immersive auditory space. The emotional direction of the speech naturally matches the environmental soundscape of the background sound, resulting in a smooth and unified overall listening experience, enhancing the realism of the dialogue, and improving the appeal of the podcast audio. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the smart podcast generation method of this application. Figure 2 This is a flowchart of the intelligent podcast generation method system in this application; Figure 3 This is a flowchart illustrating Embodiment 2 of the intelligent podcast generation method of this application; Figure 4 This is a schematic diagram of the module structure of the intelligent podcast generation device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the smart podcast generation method in this application embodiment.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is as follows: Acquire podcast material uploaded by a user, the podcast material including topic text and description information; determine the user's dialogue text based on the topic text and description information, and analyze the emotional features of the dialogue text to obtain an emotional vector; perform speech synthesis based on the pitch, volume, and speech rate in the emotional vector to obtain the user's speech stream; determine the background sound of the dialogue text based on the audio energy in the speech stream; and generate podcast audio based on the speech stream and the background sound.

[0023] In this embodiment, for ease of description, the following description will focus on identifying the smart podcast generation device as the execution subject.

[0024] Because existing technology cannot understand the text content, it cannot summarize the material, ignores the deep emotional information in the text, uses the same rhythm and tone for all content, has poor appeal, and the background music is fixed and cannot dynamically match the ambient sound with semantic changes, resulting in a strong sense of auditory disconnect.

[0025] This application provides a solution that automatically generates dialogue text by analyzing user-input podcast material, performs sentiment analysis to extract sentiment vectors, synthesizes speech streams using sentiment vectors, and dynamically matches environmental background sounds corresponding to the semantic content based on the audio energy of the speech stream. This results in podcast audio that combines natural emotional appeal and an immersive auditory atmosphere, thereby creating a coherent and immersive auditory space. The emotional trajectory of the speech naturally matches the environmental soundscape of the background sound, resulting in a smooth and unified overall listening experience, enhancing the realism of the dialogue, and improving the appeal of the podcast audio.

[0026] Based on this, embodiments of this application provide a method for generating smart podcasts, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the smart podcast generation method of this application.

[0027] In this embodiment, the smart podcast generation method includes steps S10 to S40: Step S10: Obtain podcast materials uploaded by the user, wherein the podcast materials include topic text and description information; It should be noted that the podcast material mentioned is the original input material for podcast audio, that is, the original content that the user expects to be converted into podcast audio.

[0028] It is understood that the topic text is a summary of the core content provided by the user, which can be expressed as a short keyword, a one-sentence summary, or a clear creative direction, such as "Talk about the importance of R&D security". The descriptive information is supplementary explanatory material, which can be a detailed outline, background information, references, etc.

[0029] In a specific embodiment, podcast materials can be obtained based on user-uploaded material descriptions or specified themes. The podcast materials include theme text and descriptive information. The theme text is used to characterize the core topic or creative direction of the podcast, and the descriptive information is used to characterize supplementary background or stylistic descriptions related to the topic, thereby obtaining user-uploaded podcast materials.

[0030] Step S20: Based on the topic text and the description information, determine the user's dialogue text, and analyze the sentiment features of the dialogue text to obtain a sentiment vector; It should be noted that the dialogue text is a sequentially arranged text that simulates the natural communication of multiple podcast characters around a target topic, used to represent the emotional speaking style of the selected characters in the podcast.

[0031] It is understood that the sentiment vector is a numerical vector obtained by quantifying the sentiment features of the dialogue text, and is used to characterize the emotional state and its changes at different locations in the dialogue text.

[0032] In a specific embodiment, the user's dialogue elements are determined based on the topic text and the description information. These dialogue elements include name, gender, dialogue content, and emotion type. Based on the name, gender, dialogue content, and emotion type, the user's dialogue text is obtained. This involves semantic parsing based on the topic text and description information to automatically summarize structured dialogue content, ensuring each round of speech revolves around the topic and aligns with the user's role. Each sentence of the dialogue content is analyzed to determine the emotion type (joy, anger, sorrow, happiness, normal). The name, gender, dialogue content, and emotion type are then assembled into a complete dialogue text according to the dialogue sequence. The dialogue text is represented as follows:

[0033] At this point, the dialogue text can be represented as dialogue content with persona, content, and emotional tags, enabling intelligent summarization and targeted enhancement of the podcast's expressiveness and immersiveness.

[0034] The emotional dimensions are obtained, including pleasure, arousal, and dominance. Based on these dimensions, the emotional features of the dialogue text are analyzed to obtain an emotional vector, i.e., the emotional dimensions (pleasure, arousal, and dominance) are acquired. The dialogue text is then segmented into semantic fragments, and each fragment is input into a pre-trained BERT model. Contextual semantic encoding yields a high-dimensional semantic feature vector, which is then compressed into a three-dimensional emotional space composed of pleasure, arousal, and dominance through a fully connected mapping layer, thus forming a quantized emotional vector E. The BERT model is a pre-trained language model capable of capturing the contextual semantic information of each word in the text simultaneously from left to right and from right to left. Through pre-training on large-scale unlabeled corpora, it learns general language representations. Downstream tasks only require fine-tuning based on the pre-trained model to achieve excellent results. For example, BERT can encode text fragments into high-dimensional semantic vectors and then map them to emotional dimensions such as pleasure, arousal, and dominance. Here, let the text fragment... The emotion vector is , Represented as:

[0035] in, These represent emotional dimensions, such as: Valence (pleasure), Arousal (arousivity), and Dominance (dominance).

[0036] In one feasible implementation, step S20 may include steps A11-A12: Step A11: Based on the topic text and the description information, determine the user's dialogue elements, including name, gender, dialogue content, and emotional fluctuation type; It should be noted that the dialogue elements are information units used to characterize the user's dialogue features, which may include name, gender, dialogue content, and emotional fluctuation type. Here, name is the user's identity identifier, gender is the user's social attribute feature, dialogue content is the specific semantic information expressed by the user, and emotional fluctuation type is the category of emotional state changes during the dialogue, such as joy, anger, sadness, calmness, etc.

[0037] Step A12: Obtain the user's dialogue text based on the name, gender, dialogue content, and emotional fluctuation type.

[0038] Understandably, names are used to identify the subjects of the conversation, gender is used to define the user's language style and way of addressing them, the content of the conversation is used to represent the core information that the user actually wants to convey and is the semantic subject of the conversation text, and the type of emotional fluctuation is used to represent the trajectory of the user's emotional ups and downs in the process of expression and determine the overall tone and emotional color of the text.

[0039] In another feasible implementation, step S20 may include steps B11-B12: Step B11: Obtain the emotional dimensions, which include pleasure, arousal, and dominance. It should be noted that the emotional dimension is a quantitative representation of complex emotional experiences by breaking them down into several measurable basic directions.

[0040] Understandably, pleasure measures the positive or negative nature of an emotion, ranging from pain to pleasure; arousal measures the intensity of emotional activation, ranging from calm to excitement; and dominance measures an individual's sense of control over a situation or interaction, ranging from compliance to dominance.

[0041] Step B12: Analyze the emotional features of the dialogue text based on the pleasure level, the arousal level, and the dominance level to obtain an emotional vector.

[0042] It is understood that the emotional features are the subjective emotional attributes contained in the dialogue text, including the type of emotion, the intensity of emotion, and the fluctuation trend of emotion as the dialogue progresses. They can be identified and extracted from text semantics, interjections, punctuation marks, and contextual relationships. The richer the emotional features, the more accurately the generated emotional vector can represent the emotional changes in the dialogue process.

[0043] Step S30: Speech synthesis is performed based on the pitch, volume, and speech rate in the emotion vector to obtain the user's speech stream; It should be noted that the speech stream is a continuous sequence of output acoustic signals, which is the audio representation of the text content. It can be unfolded on the time axis in the order of pronunciation, carrying rhythmic information such as pitch fluctuations, loudness changes, and speech rate, and can be converted into audible sound.

[0044] Understandably, pitch is the fundamental frequency of sound vibration, representing the sharpness or depth of the sound, and is often used to express excitement, tension, or calmness. Volume is the amplitude of sound vibration, reflecting the strength of the sound, and is often used to express anger, excitement, or gentleness. Speech rate is the number of syllables pronounced per unit of time, representing the speed and rhythm of speech, and is often used to express urgency, composure, or hesitation. Therefore, by adjusting timbre hyperparameters such as pitch, volume, and speech rate, the emotional expressiveness of speech can be shaped, making synthesized speech sound more like real human emotional expression.

[0045] In a specific embodiment, the user's baseline timbre hyperparameter is obtained, that is, the baseline timbre hyperparameter can be expressed as: ,in, The average pitch (in Hz) of synthesized speech determines the high and low baseline of the sound. This represents the volume energy variance (unit: dB²), describing the dynamic fluctuation range of sound loudness. The larger the variance, the more pronounced the volume fluctuations. It represents the speech rate ratio (dimensionless), that is, the ratio of the synthesized speech rate to the user's baseline speech rate.

[0046] Based on the timbre hyperparameters corresponding to the emotion vector, the baseline timbre hyperparameters are adjusted to obtain speech hyperparameters. Speech synthesis is then performed based on the pitch, volume, and speech rate in the speech hyperparameters to obtain the user's speech stream. The emotion vector includes Valence, Arousal, and Dominance. The emotion vector can be converted into corresponding adjustment coefficients for the baseline timbre hyperparameters. Adjustments are made to synthesize the user's speech stream, thereby recognizing the positive and negative emotional polarities of the text. This allows the synthesized speech to transition naturally between complex emotions such as joy, anger, sadness, and tension, presenting a nuanced emotional fluctuation close to that of a human broadcaster. For example, fear corresponds to... The coefficient value is 1.2. With a coefficient value of 1.3, adjusting the baseline timbre hyperparameters at this point is equivalent to automatically increasing Pitch_mean by 20% and Speed_ratio by 30%, thereby synthesizing a high-pitched, rapid-fire, and frightening voice, greatly enhancing the podcast's appeal and immersive listening experience.

[0047] In one feasible implementation, step S30 may include steps C11-C13: Step C11: Obtain the user's baseline timbre hyperparameters; It should be noted that the reference timbre hyperparameters are a set of acoustic parameters that characterize the inherent identity attributes of a user's voice. They do not change with emotional changes and may include the lower limit of the fundamental frequency, formant distribution, glottal wave parameters, etc., to ensure that the synthesized speech is highly consistent with the user's own timbre.

[0048] Step C12: Adjust the baseline timbre hyperparameters according to the timbre hyperparameters corresponding to the emotion vector to obtain the speech hyperparameters; Understandably, timbre hyperparameters are incremental control parameters derived from emotion vector decoding used to adjust the timbre of a voice. By changing the range of pitch fluctuations, the contrast between loud and soft volumes, and the rhythm of speech, a neutral human voice can be given emotional colors such as the sharpness of fear, the dullness of sadness, or the leaps of joy.

[0049] Step C13: Speech synthesis is performed based on the pitch, volume, and speech rate in the speech hyperparameters to obtain the user's speech stream.

[0050] It is understandable that the speech hyperparameters represent the average pitch, volume range, and speech rate of the synthesized speech, which directly determine the emotional appeal and naturalness of the synthesized speech stream.

[0051] Step S40: Determine the background sound of the dialogue text based on the audio energy in the speech stream; It should be noted that the background sounds are used to create an atmosphere for the dialogue scene, and are expressed as background music or environmental sound effects. Their style and intensity should change with the emotional fluctuations of the voice, thereby strengthening the emotional context that the text wants to convey.

[0052] It is understood that the audio energy is a measure of the intensity of sound wave vibration of a speech signal per unit time, usually expressed in decibels. It represents the volume of the speaker's voice and the degree of emotional excitement. When the audio energy in the speech stream is high, it indicates that the current speech is in a high arousal state, such as fear, anger, or excitement. At this time, a background sound with a tight rhythm and strong dynamics can be selected to create a tense or exciting atmosphere. When the audio energy is low, it indicates that the speech tends to be calm or depressed. A soothing and gentle background sound can be selected to match a gentle or sad emotion, thereby achieving a natural auditory fusion between the background sound and the speech stream, significantly enhancing the sense of scene and appeal of the dialogue text.

[0053] In a specific embodiment, a gain attenuation factor corresponding to the audio energy in the speech stream is determined; based on the gain attenuation factor, the background sound of the dialogue text is determined, that is, based on environmental keywords identified from the dialogue text, such as "forest," "coffee shop," and "space," the initial background sound track of the dialogue text is retrieved from a preset background sound database. The initial background audio track can be white noise or ambient sound. Audio energy (Energy_speech(t)) is extracted from the speech stream by frame segmentation and combined with a sensitivity coefficient. The sensitivity coefficient is used to control the response intensity of background noise as speech energy changes, and the gain attenuation factor corresponding to the audio energy can be obtained. And utilize the gain attenuation factor With the initial background music track It can be obtained , represented as:

[0054] At this time, when the anchor's voice energy is high, the background noise is automatically reduced exponentially to prevent the human voice from being masked by ambient noise. When the voice pauses or the volume decreases, the background noise automatically rises to maintain the continuity of the scene atmosphere, thereby significantly enhancing the immersiveness, emotional expressiveness and overall auditory experience of the dialogue text.

[0055] Step S50: Generate podcast audio based on the speech stream and the background noise.

[0056] It should be noted that the podcast audio is the final podcast audio file generated, which can be output in MP3 or AAC formats, with human voice as the foreground and background sounds that change in real time with the energy of the voice and the semantics of the text as the background, so as to give the auditory immersion while ensuring that the voice is clear and understandable.

[0057] In a specific embodiment, when generating podcast audio based on the voice stream and the background noise, a real-time streaming output architecture that generates and plays simultaneously can be constructed using WebRTC or WebSocket protocols. WebRTC is a standardized browser-side real-time communication technology that can maintain low-latency continuous playback in weak network environments. WebSocket is a full-duplex communication protocol that establishes a persistent connection through a handshake and transmits data bidirectionally in an orderly and reliable manner in frames. The protocol is simple and highly compatible, facilitating message push and chunked transmission between the server and the client. Therefore, the synthesized podcast audio can be pushed in real time via WebRTC or WebSocket, and the client can receive and decode the audio while playing it, thereby achieving a virtually imperceptible latency experience for the user.

[0058] Simultaneously, based on the user's customized voice profile and personality parameters, matching timbres can be retrieved and cloned from the user-customized timbre data association library to generate a consistent speech stream. The background sound of the dialogue text can be determined based on the audio energy in the speech stream. In other words, users can choose to customize their persona or use the default data, and the timbres can be cloned according to the user's choice. This enables customized persona and timbre cloning, giving podcast audio personalized voice characteristics and emotional appeal, while maintaining professional-grade audio quality.

[0059] This embodiment proposes a smart podcast generation method. The method involves acquiring podcast material uploaded by a user, including topic text and descriptive information; determining the user's dialogue text based on the topic text and descriptive information, and analyzing the emotional features of the dialogue text to obtain an emotional vector; synthesizing speech based on the pitch, volume, and speech rate in the emotional vector to obtain the user's speech stream; determining the background sound of the dialogue text based on the audio energy in the speech stream; and generating podcast audio based on the speech stream and the background sound. This application automatically generates dialogue text by analyzing user-input podcast material, performs emotional analysis to extract emotional vectors, synthesizes a speech stream using the emotional vectors, and dynamically matches environmental background sound corresponding to the semantic content based on the audio energy of the speech stream. This results in podcast audio that combines natural emotional appeal and an immersive auditory atmosphere, creating a coherent and immersive auditory space. The emotional direction of the speech naturally matches the environmental soundscape of the background sound, resulting in a smooth and unified overall listening experience, enhancing the realism of the dialogue, and improving the appeal of the podcast audio.

[0060] It is understandable that, such as Figure 2 As shown, Figure 2The flowchart of the intelligent podcast generation method system in this application shows that after the user inputs topic text or description file, intelligent dialogue content can be automatically generated. During the generation process, semantic slicing, entity recognition, and emotional feature extraction are performed simultaneously. Specifically, based on semantic segmentation and entity information, corresponding music and sound effects are matched from a dynamic background sound library, and emotional tendencies and intensities are extracted to generate acoustic model parameters to obtain the corresponding speech. The multi-track audio synthesis engine integrates speech, music, and sound effects, and establishes a real-time streaming output link for simultaneous generation and playback using WebRTC or WebSocket protocols. Low-latency real-time speech playback is achieved on the client side, while text subtitles are pushed simultaneously. At this point, the user can download or publish the complete synthesized speech content, thus completing the generation of the entire intelligent podcast.

[0061] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment can be referred to the above description, and will not be repeated hereafter.

[0062] In this embodiment, refer to Figure 3 , Figure 3 This is a flowchart illustrating Embodiment 2 of the intelligent podcast generation method of this application. Step S40 specifically includes steps S41 to S42: Step S41: Determine the gain attenuation factor corresponding to the audio energy based on the audio energy in the speech stream; It should be noted that the gain attenuation factor is a coefficient used to dynamically adjust the background noise, which can maintain the continuity of the background noise atmosphere while ensuring the clarity of speech.

[0063] In a specific embodiment, the slice length is obtained, the speech stream is divided according to the slice length, audio frames are determined, and the corresponding audio energy is calculated based on the audio frames. The corresponding gain attenuation factor is determined based on the audio energy. Thus, the slice length is set to 0.5 seconds to 1 second according to the sampling rate of the speech stream and the real-time processing requirements to ensure that subsequent gain adjustments can be synchronized with the semantic rhythm of the speech stream. At this time, the speech stream is divided according to the slice length to obtain several consecutive audio frames, and the short-time energy of each audio frame is calculated as the corresponding audio energy. The audio energy is used to characterize the intensity of the speech signal in the current frame, thereby characterizing the intensity of the broadcaster's voice. When the voice energy is detected to be high, a smaller gain attenuation factor is output to reduce the background noise volume. When the voice energy is low or the voice is paused, a larger gain attenuation factor is output to restore the background noise volume. This can prevent the background noise from masking the human voice and maintain the continuity and atmosphere of the environmental sound effects, thereby significantly improving the clarity and immersive listening experience of podcast audio.

[0064] In one feasible implementation, step S41 may include steps D11 to D13: Step D11: Obtain the slice length; It should be noted that the slice length is the length of the time window used to divide the continuous speech stream into several short processing units on the time axis. It can be set to 0.5 seconds to 1 second. If the slice length is too short, the audio energy calculation will be too fragmented, resulting in frequent jitter of the gain attenuation factor. If the slice length is too long, the energy change will lag behind the emotional fluctuations of the speech, making it impossible to achieve real-time synchronization between background sound and speech.

[0065] Step D12: Divide the speech stream according to the slice length, determine the audio frames, and calculate the corresponding audio energy based on the audio frames; It should be noted that the audio frame is a short signal segment obtained after the speech stream is sliced. Each audio frame corresponds to a fixed time interval. For each audio frame, its short-time energy can be calculated as the audio energy of that frame. For example, the average of the squared amplitudes of all sampling points in the frame can be obtained and then converted into decibel values ​​to obtain a quantitative index that can characterize the loudness of the current speech, effectively characterizing the volume and emotional excitement of the anchor's speech.

[0066] Step D13: Determine the corresponding gain attenuation factor based on the audio energy.

[0067] Understandably, the gain attenuation factor and audio energy are inversely related. When the audio energy is high, it means that the host is speaking loudly or is in a highly aroused emotional state. At this time, a smaller gain attenuation factor is generated to automatically reduce the background noise volume and prevent the ambient noise from masking the voice. When the audio energy is low or there is a pause in the speech, a larger gain attenuation factor is generated to naturally increase the background noise volume and maintain the continuity of the scene atmosphere.

[0068] Step S42: Determine the background sound of the dialogue text based on the gain attenuation factor.

[0069] In a specific embodiment, environmental keywords are identified based on the dialogue text; the corresponding initial background sound track, including white noise and ambient sound, is retrieved from a preset background sound database based on the environmental keywords; the initial background sound track is adjusted according to the gain attenuation factor to obtain the background sound. This allows for the extraction of scene-related words from the dialogue text, such as "forest," "coffee shop," "rainy night," and "space," as environmental keywords. When retrieving the corresponding initial background sound track from the preset background sound database based on the environmental keywords, the preset background sound database pre-stores multitracks categorized by scene tags. Sound effects materials, including white noise (such as fan noise, ocean wave noise) and ambient sounds (such as birdsong, keyboard typing, and vehicle sounds), are used to match and call the most suitable initial background sound track. The original amplitude of the initial background sound track is adjusted according to the gain attenuation factor, so that the volume of the background sound changes in real time with the audio energy of the speech stream. That is, the background sound is automatically lowered when the speech is loud and automatically restored when the speech pauses or weakens. This generates a background sound that is consistent with the semantics of the dialogue content and dynamically fluctuates, avoiding the auditory disconnect caused by the disconnect between the ambient sound effects and the semantics, and making the human voice always clear and prominent, thereby significantly improving the scene immersion and emotional appeal of podcast audio.

[0070] In one feasible implementation, step S42 may include steps E11 to E13: Step E11: Identify environmental keywords based on the dialogue text; It should be noted that the environmental keywords are words or phrases that characterize the current narrative scene, such as "rainy night," "forest," "café," and "space station," to ensure that the background sound is consistent with the dialogue content at the semantic level, avoid the abruptness caused by the sound being out of sync with the narrative, and enhance the listener's sense of immersion and understanding of the scene's atmosphere.

[0071] Step E12: Based on the environmental keywords, retrieve the corresponding initial background sound track from the preset background sound database. The initial background sound track includes white noise and ambient sound. It should be noted that the initial background audio track is the original audio material corresponding to environmental keywords stored in the preset background sound database, which is composed of white noise and ambient sound layered and mixed.

[0072] It is understandable that white noise is a stable random signal with a uniform power spectrum, such as the sound of a fan turning or the continuous background noise of ocean waves, used to provide a continuous and uniform acoustic foundation and avoid a hollow feeling in the background sound. Ambient sound, on the other hand, is a distinctive sound event in a specific scene, such as birdsong, traffic noise, keyboard typing, or raindrops falling, used to create spatial orientation and narrative realism.

[0073] Step E13: Adjust the initial background sound according to the gain attenuation factor to obtain the background sound.

[0074] Understandably, when the voice energy is high, the gain attenuation factor is small, and the overall amplitude of the initial background sound track is proportionally reduced, thus preventing the background sound from masking the human voice. When the voice energy is low or there is a pause, the gain attenuation factor increases, and the amplitude of the background sound returns to its original level, maintaining the continuity of the environmental atmosphere and preserving the background sound's role in enhancing the scene's atmosphere and emotional tension, resulting in a more professional, immersive, and natural listening experience.

[0075] This embodiment proposes a smart podcast generation method that determines a gain attenuation factor corresponding to the audio energy in the speech stream; and determines the background sound of the dialogue text based on the gain attenuation factor. This application determines the gain attenuation factor corresponding to the audio energy in the speech stream, and determines the background sound of the dialogue text based on the gain attenuation factor, achieving a natural fusion of background sound and speech. When the speaker's volume increases or their emotions become agitated, the background sound is automatically reduced to ensure clear and intelligible speech; when the speaker's volume decreases or there is a pause, the background sound is automatically restored, thereby maintaining the continuity and immersion of the scene atmosphere and improving the overall listening quality of the audio work.

[0076] This application also provides a smart podcast generation device, please refer to... Figure 4 The smart podcast generation device includes: The acquisition module 10 is used to acquire podcast materials uploaded by users, wherein the podcast materials include topic text and description information; Processing module 20 is used to determine the user's dialogue text based on the topic text and the description information, and to obtain an emotion vector based on the dialogue text; The processing module 20 is further configured to perform speech synthesis based on the pitch, volume and speech rate in the emotion vector to obtain the user's speech stream; Execution module 30 is used to determine the background sound of the dialogue text based on the audio energy in the speech stream; The execution module 30 is also configured to generate podcast audio based on the speech stream and the background noise.

[0077] The processing module 20 is further configured to determine the user's dialogue elements based on the topic text and the description information, wherein the dialogue elements include name, gender, dialogue content and emotional fluctuation type; The user's dialogue text is obtained based on the name, gender, dialogue content, and emotional fluctuation type.

[0078] The processing module 20 is also used to acquire the emotional dimension, which includes pleasure, arousal and dominance. Based on the pleasure level, arousal level, and dominance level, the emotional features of the dialogue text are analyzed to obtain an emotional vector.

[0079] The processing module 20 is also used to obtain the user's baseline timbre hyperparameters; Based on the timbre hyperparameters corresponding to the emotion vector, the baseline timbre hyperparameters are adjusted to obtain the speech hyperparameters; Speech synthesis is performed based on the pitch, volume, and speech rate in the speech hyperparameters to obtain the user's speech stream.

[0080] The execution module 30 is further configured to determine the gain attenuation factor corresponding to the audio energy based on the audio energy in the speech stream; The background sound of the dialogue text is determined based on the gain attenuation factor.

[0081] The execution module 30 is also used to obtain the slice length; The speech stream is divided according to the slice length to determine audio frames, and the corresponding audio energy is calculated based on the audio frames. The corresponding gain attenuation factor is determined based on the audio energy.

[0082] The execution module 30 is further configured to identify environmental keywords based on the dialogue text; Based on the environmental keywords, the corresponding initial background sound track is retrieved from the preset background sound database. The initial background sound track includes white noise and ambient sound. The initial background sound is adjusted according to the gain attenuation factor to obtain the background sound.

[0083] The intelligent podcast generation device provided in this application, employing the intelligent podcast generation method described in the above embodiments, can solve the technical problem of how to intelligently generate high-quality dialogue content suitable for podcast expression based on different materials. Compared with the prior art, the beneficial effects of the intelligent podcast generation device provided in this application are the same as those of the intelligent podcast generation method provided in the above embodiments, and other technical features in the intelligent podcast generation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0084] This application provides a smart podcast generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the smart podcast generation method in Embodiment 1 above.

[0085] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the smart podcast generation device of the embodiments of this application. The smart podcast generation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The smart podcast generation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0086] like Figure 5 As shown, the smart podcast generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the smart podcast generation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the smart podcast generating device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show smart podcast generating devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0087] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0088] The intelligent podcast generation device provided in this application, employing the intelligent podcast generation method described in the above embodiments, can solve the technical problem of how to intelligently generate high-quality dialogue content suitable for podcast expression based on different materials. Compared with the prior art, the beneficial effects of the intelligent podcast generation device provided in this application are the same as those of the intelligent podcast generation method provided in the above embodiments, and other technical features in this intelligent podcast generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0089] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0091] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the smart podcast generation method in the above embodiments.

[0092] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0093] The aforementioned computer-readable storage medium may be included in the smart podcast generating device; or it may exist independently and not assembled into the smart podcast generating device.

[0094] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a smart podcast generation device, cause the smart podcast generation device to: acquire podcast material uploaded by a user, the podcast material including topic text and descriptive information; determine the user's dialogue text based on the topic text and the descriptive information, and analyze the emotional features of the dialogue text to obtain an emotional vector; perform speech synthesis based on the pitch, volume, and speech rate in the emotional vector to obtain the user's speech stream; determine the background sound of the dialogue text based on the audio energy in the speech stream; and generate podcast audio based on the speech stream and the background sound.

[0095] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0098] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described intelligent podcast generation method. This solves the technical problem of how to intelligently generate high-quality dialogue content suitable for podcast expression based on different materials. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the intelligent podcast generation method provided in the above embodiments, and will not be repeated here.

[0099] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating intelligent podcasts, characterized in that, The method includes: Obtain podcast materials uploaded by users, including topic text and descriptive information; Based on the topic text and the description information, the user's dialogue text is determined, and the sentiment characteristics of the dialogue text are analyzed to obtain a sentiment vector; Speech synthesis is performed based on the pitch, volume, and speech rate in the emotion vector to obtain the user's speech stream; The background sound of the dialogue text is determined based on the audio energy in the speech stream; Podcast audio is generated based on the speech stream and the background noise.

2. The method as described in claim 1, characterized in that, The step of determining the user's dialogue text based on the topic text and the description information includes: Based on the topic text and the description information, the user's dialogue elements are determined, including name, gender, dialogue content, and emotional fluctuation type. The user's dialogue text is obtained based on the name, gender, dialogue content, and emotional fluctuation type.

3. The method as described in claim 1, characterized in that, The step of analyzing the sentiment features of the dialogue text to obtain the sentiment vector includes: The emotional dimension is obtained, which includes pleasure, arousal and dominance; Based on the pleasure level, arousal level, and dominance level, the emotional features of the dialogue text are analyzed to obtain an emotional vector.

4. The method as described in claim 1, characterized in that, The step of synthesizing speech based on pitch, volume, and speech rate in the emotion vector to obtain the user's speech stream includes: Obtain the user's baseline timbre hyperparameters; Based on the timbre hyperparameters corresponding to the emotion vector, the baseline timbre hyperparameters are adjusted to obtain the speech hyperparameters; Speech synthesis is performed based on the pitch, volume, and speech rate in the speech hyperparameters to obtain the user's speech stream.

5. The method as described in claim 1, characterized in that, The step of determining the background noise of the dialogue text based on the audio energy in the speech stream includes: Based on the audio energy in the speech stream, determine the gain attenuation factor corresponding to the audio energy; The background sound of the dialogue text is determined based on the gain attenuation factor.

6. The method as described in claim 5, characterized in that, The step of determining the gain attenuation factor corresponding to the audio energy based on the audio energy in the speech stream includes: Get the slice length; The speech stream is divided according to the slice length to determine audio frames, and the corresponding audio energy is calculated based on the audio frames. The corresponding gain attenuation factor is determined based on the audio energy.

7. The method as described in claim 5, characterized in that, The step of determining the background noise of the dialogue text based on the gain attenuation factor includes: Environmental keywords are identified based on the dialogue text; Based on the environmental keywords, the corresponding initial background sound track is retrieved from the preset background sound database. The initial background sound track includes white noise and ambient sound. The initial background sound is adjusted according to the gain attenuation factor to obtain the background sound.

8. A smart podcast generation device, characterized in that, The device includes: The acquisition module is used to acquire podcast materials uploaded by users, including topic text and description information; The processing module is used to determine the user's dialogue text based on the topic text and the description information, and to obtain an emotion vector based on the dialogue text; The processing module is also used to perform speech synthesis based on the pitch, volume and speech rate in the emotion vector to obtain the user's speech stream; An execution module is used to determine the background sound of the dialogue text based on the audio energy in the speech stream; The execution module is also configured to generate podcast audio based on the speech stream and the background noise.

9. A smart podcast generation device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the smart podcast generation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the smart podcast generation method as described in any one of claims 1 to 7.

Images