Audio signal processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an audio signal processing method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a start instruction when the start instruction sent by a user is received, and obtaining at least one piece of personal sound audio information at the current moment; obtaining target human voice audio information at the current moment based on a pre-obtained sound effect guidance text and the human voice audio information at the current moment; determining target audio information at the current moment according to the target human voice audio information at the current moment and predetermined background audio information at the current moment; and generating and playing a target audio based on the target audio information. According to the method and the device, the voice audio information in the text deduction scene is flexibly adjusted in real time to be matched with scene changes, so that subtle changes and emotional fluctuations in the performance process can be flexibly dealt with, the audio content better fits the actual deduction condition, and the overall auditory experience and immersion of a user are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of audio signal processing, and in particular to an audio signal processing method, device, electronic device and storage medium. Background Art

[0002] In the modern multimedia and entertainment industry, it has become an important task to generate corresponding audio according to complex scenes and multi-character texts to enrich the user's auditory experience. The implementation of this technology involves many aspects, including natural language processing, emotional computing, and audio signal processing.

[0003] Current audio generation methods mainly rely on text analysis of texts that describe scenes in detail, and generate audio corresponding to the text based on the results of text analysis, pre-recorded audio clips, and manual editing. However, this method can only generate fixed background audio based on the determined scene text. For example, when some artificial intelligence characters or actors perform live scenes, they can only play some fixed background audio or human voice audio according to a certain schedule. However, the performance of the actors during the performance or the emotions during the actual scene interpretation may change slightly, making this method inflexible and difficult to respond to changes in the interpretation process in real time, thus affecting the overall auditory experience and immersion. Summary of the invention

[0004] The embodiments of the present invention provide an audio signal processing method, device, electronic device and storage medium, which can flexibly adjust human voice audio information in a text interpretation scene in real time, capture subtle changes in the interpretation process, and improve the user's overall auditory experience and immersion.

[0005] In a first aspect, an embodiment of the present invention provides an audio signal processing method, including:

[0006] When receiving a start instruction sent by a user, acquiring at least one human voice audio information at the current moment in response to the start instruction;

[0007] Based on the pre-acquired sound effect guidance text and the audio information of each human voice at the current moment, obtaining the target human voice audio information at the current moment;

[0008] Determining the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment;

[0009] Generate and play target audio based on the target audio information.

[0010] In a second aspect, an embodiment of the present invention provides an audio signal processing device, the device comprising:

[0011] An information acquisition module, configured to, upon receiving a start instruction sent by a user, acquire at least one human voice audio information at a current moment in response to the start instruction;

[0012] A first determination module, configured to obtain the target human voice audio information at the current moment based on the pre-acquired sound effect guidance text and the human voice audio information at the current moment;

[0013] A second determination module is used to determine the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment;

[0014] The audio playing module is used to generate and play the target audio based on the target audio information.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an audio signal processing method as described in any one of the embodiments of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the audio signal processing method as described in any one of the embodiments of the present invention.

[0017] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the audio signal processing method as described in any one of the embodiments of the present invention.

[0018] In an embodiment of the present invention, when a start instruction is received from a user, at least one human voice audio information at the current moment is acquired in response to the start instruction; based on the pre-acquired sound effect guidance text and each human voice audio information at the current moment, the target human voice audio information at the current moment is acquired; the target audio information at the current moment is determined according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment; and the target audio is generated and played based on the target audio information. The method of an embodiment of the present invention can flexibly respond to subtle changes and emotional fluctuations in the performance process by flexibly adjusting the human voice audio information in the text interpretation scene in real time to match the scene changes, so as to make the audio content more in line with the actual interpretation situation and improve the user's overall auditory experience and immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A first flow chart of an audio signal processing method provided by an embodiment of the present invention;

[0021] Figure 2 A second flow chart of an audio signal processing method provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the structure of an audio signal processing device provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] Figure 1 This is the first flow chart of an audio signal processing method provided in an embodiment of the present invention. The method in the embodiment of the present invention can flexibly adjust the human voice audio information in the text interpretation scene in real time, capture subtle changes in the interpretation process, and improve the user's overall auditory experience and immersion. The method can be executed by an audio signal processing device provided in an embodiment of the present invention, and the device can be implemented in software and / or hardware. The following embodiments will be described by taking the device integrated in an electronic device as an example. The electronic device can be a server or computer device for implementing an audio signal processing method, etc. Figure 1 , the method may specifically include the following steps:

[0026] Step 101: upon receiving a start instruction sent by a user, obtaining at least one human voice audio information at the current moment in response to the start instruction.

[0027] The start instruction is an instruction sent by the user to the server to instruct the server to start adjusting the human voice audio in the scene interpretation process in real time. The human voice audio can be the sound audio produced by the actor or artificial intelligence character in the scene interpretation process. When the server receives the start instruction, it can start to obtain all the human voice audio in the scene interpretation process in real time in response to the start instruction.

[0028] In an optional implementation, a corresponding voice collection microphone for collecting human voices may be arranged around each character, and each microphone has an independent channel. The server may collect the audio of each human voice by only enabling or monitoring the channel of each voice collection microphone. Alternatively, a pre-configured audio interface for connecting each voice collection microphone is used to directly obtain the audio of each human voice through its audio interface.

[0029] Step 102: based on the pre-acquired sound effect guidance text and the audio information of each human voice at the current moment, obtain the target human voice audio information at the current moment.

[0030] Among them, the sound effect guidance text includes the content that needs to be performed in each time period during the overall performance process, as well as the scene types corresponding to each performance content. Scene types include battle scenes, dialogue scenes, and chase scenes. In this solution, the sound effect guidance text can also include character role information, line information, and emotional prompts, such as scripts or scripts. The target human voice audio information is determined based on the sound effect guidance text and the human voice audio collected in real time, and is used to determine the audio information of the target audio that is finally played for the user. The target human voice audio information includes information such as the volume, timbre, and speech speed of the human voice audio.

[0031] Specifically, the scene interpretation process is a dynamic process. The emotions or speech speed of each character in the scene interpretation process are uncertain, and the sound effect guidance text may only include a rough guidance emotion. For example, in a dialogue scene, the sound effect knows that the emotion specified in the text is "anger", but it cannot quantify the "degree of anger". Or, the timbre of some artificial intelligence characters can only express emotions through volume, and their timbre will not change with emotions. However, in actual scenes, a character may show a rough and tense voice when angry, and some trembling voices when sad. Therefore, in order to improve the user's overall auditory experience and immersion, so that the character's life audio is more in line with the interpretation scene, the human voice audio information can be adjusted according to the sound effect guidance text to obtain the target human voice audio information. For example, in the application scenario of artificial intelligence characters, when the artificial intelligence character changes from "angry" to "very angry", the volume and timbre of the artificial intelligence voice can be adjusted in real time. In the application scenario of personnel roles, when the performer changes from "angry" to "very angry", the volume of the human voice audio signal can be increased in real time to adjust the human voice audio.

[0032] In an optional implementation, after obtaining the audio information of each human voice at the current moment, determine the target scene content in the sound effect guidance text at the current moment, and perform role analysis on the target scene content to obtain the characteristics of each role of the target scene content; perform sentiment analysis on the current human voice audio information according to the characteristics of each role and a pre-set sentiment analysis model to obtain the sentiment analysis result of the current human voice audio information; wherein the sentiment analysis result includes the corresponding sentiment type and sentiment depth; adjust the current human voice audio information based on the sentiment analysis result to obtain the target human voice audio information at the current moment.

[0033] Step 103: Determine the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment.

[0034] The target audio information includes the information of the audio that will eventually be played for the user. Background audio is the audio used as background music or ambient sound effects during the scene interpretation process. Different scene contents correspond to different background audio. For example, the deep forest scene includes background audio such as birdsong, wind blowing leaves, and stream flow; the beach scene includes background audio such as waves, seagulls, and wind; the city street scene includes background audio such as traffic, crowd conversations, and construction site noise.

[0035] Specifically, after obtaining the sound effect guidance text, the background audio corresponding to the scene content of each time period can be pre-determined according to the sound effect guidance text. In an optional implementation, for the scene content of each time period, a text analysis is performed on the current scene content of the current time period to obtain the current scene type corresponding to the current scene content; all background audio signals corresponding to the current scene type are searched in a predetermined background sound effect library; and the background audio information at the current moment is determined according to all background audio signals. During the scene interpretation process, since the human voice audio may have changed, it is necessary to adjust the background audio information at the current moment according to the target human voice audio information at the current moment to obtain the target background audio information at the current moment; audio fusion is performed on the target human voice audio information at the current moment and the target background audio information at the current moment to obtain the target audio information at the current moment.

[0036] Step 104: Generate and play target audio based on the target audio information.

[0037] Specifically, after obtaining the target audio information, the playback mode of the target audio can be determined in combination with the environmental information of the performance space obtained in advance. The environmental information includes the position and number of playback devices, and the playback mode includes surround playback or one-sided playback. Different target audio information corresponds to different playback modes. For example, if the target audio includes bird calls, etc., the playback mode can be determined as the playback device located above the performance space plays the background audio in the target audio, while the playback device located in the middle of the performance space plays the human voice audio in the target audio. For example, if the scene type is a war scene, and the target audio includes the sound of chasing footsteps, the playback mode can be determined as surround playback, that is, all playback devices are used to surround play the target audio in the order of arrangement to improve the user's overall immersion.

[0038] The technical solution of this embodiment, upon receiving a start instruction sent by the user, obtains at least one human voice audio information at the current moment in response to the start instruction; obtains the target human voice audio information at the current moment based on the pre-acquired sound effect guidance text and the human voice audio information at the current moment; determines the target audio information at the current moment based on the target human voice audio information at the current moment and the predetermined background audio information at the current moment; generates and plays the target audio based on the target audio information. The technical solution of this embodiment, by flexibly adjusting the human voice audio information in the text interpretation scene in real time to match the scene changes, can flexibly respond to subtle changes and emotional fluctuations during the performance, making the audio content more in line with the actual interpretation situation, and improving the user's overall auditory experience and immersion.

[0039] Figure 2 The second flow chart of the audio signal processing method provided in the embodiment of the present invention is a refinement of the above embodiment. The specific method can be as follows Figure 2 As shown, the method may include the following steps:

[0040] Step 201: For the scene contents in each time period, text analysis is performed on the current scene content in the current time period to obtain the current scene type corresponding to the current scene content.

[0041] Specifically, after obtaining the sound effect guidance text, the sound effect guidance text is cleaned, including removing irrelevant characters and punctuation marks, to obtain the preprocessed sound effect guidance text. The preprocessed sound effect guidance text is segmented to segment the scene content text into words or characters, and the pre-trained word vector model is used to convert the words into vector representations to obtain the text vector of the sound effect guidance text.

[0042] After obtaining the text vector, the pre-trained scene type determination model can be used to process the text vector to obtain the scene type corresponding to the scene content. In an optional embodiment, the scene type determination model can be a BiLSTM model. After obtaining the text vector, the text vector is input into the input layer of the scene type determination model. The two LSTM layers of the BiLSTM model process the input sequence from the forward and backward directions respectively. Each LSTM layer contains memory units, which can capture the contextual information in the text and solve the gradient disappearance problem. The outputs of the bidirectional LSTM layers are merged and further processed by one or more fully connected layers. The number of neurons in the fully connected layer can be adjusted according to the complexity of the sound effect guidance text. The output layer uses a softmax function to generate category predictions. Assuming there are n scene types, the output layer will have n neurons, each neuron representing the probability distribution of a scene type. Further, the scene type corresponding to the scene content is determined based on the output data of the BiLSTM model.

[0043] Step 202: Search for all background audio signals corresponding to the current scene type in a predetermined background sound effect library, and determine the background audio information at the current moment based on all background audio signals.

[0044] Among them, the background sound effect library stores all audio signals corresponding to each scene type. In an optional implementation, after determining the scene type of each time period in the sound effect guidance text, all background audio signals corresponding to each scene type are found in the background sound effect library. If there is a type of scene that cannot be found, a prompt message is generated according to the type of scene that cannot be found, and the prompt message is sent to the staff, so that the staff can supplement the background audio signal of the scene type that cannot be found in time.

[0045] In actual applications, there are many types of background audio signals corresponding to a scene type. For example, a deep forest scene includes background audio such as birdsong, wind blowing leaves, and stream flow; a beach scene includes background audio such as waves, seagulls, and wind; a city street scene includes background audio such as traffic, crowd conversations, and construction site noise. However, not every background audio signal will be used in the sound effect guidance text. Therefore, after obtaining all the background audio signals corresponding to the scene type, the scene content corresponding to the scene type can be contextually analyzed, and all background audio can be screened based on the analysis results.

[0046] In an optional implementation, after obtaining all background audio signals of the current scene type, contextual learning technology (such as In-Context Learning, etc.) can be used to perform contextual analysis on the scene content corresponding to the current scene type: understand factors such as character actions, plot progression, and environmental changes in the scene content. Based on the results of the contextual analysis, all background audio signals are evaluated for relevance to determine whether each background audio signal matches the content, atmosphere, and emotion of the current scene. Based on the results of the relevance evaluation, background audio signals with relevance greater than a preset value are screened out. After the background audio signal is screened out, the background audio signal can be further optimized in parameters, such as adjusting the volume, frequency, and rhythm of the background audio signal, to ensure that the background audio signal is closely coordinated with other elements of the scene content (such as visual images and dialogue content, etc.) to obtain the background audio information at the current moment.

[0047] Exemplarily, the current scene type is a deep forest adventure type, and all the corresponding background audio signals include background audio such as birdsong, wind blowing leaves, and streams flowing. The scene content of the scene type is contextually analyzed to determine that the characters in the scene content are carefully walking through the forest while paying attention to the surrounding movements. All background audio signals are evaluated for relevance to determine whether the bird song is too frequent or loud to distract the audience from the character's actions; at the same time, consider whether the sound of wind blowing leaves is natural enough, and whether the sound of streams flowing adds the necessary sense of tranquility to the scene. Based on the evaluation results of each background audio signal, choose to reduce the volume or frequency of the bird song, or directly delete the bird song, or enhance the sound of wind blowing leaves and streams flowing to better create the atmosphere of the deep forest.

[0048] Step 203: upon receiving a start instruction sent by the user, obtaining at least one human voice audio information at the current moment in response to the start instruction.

[0049] Step 204: determine the target scene content in the sound effect guidance text at the current moment, and perform role analysis on the target scene content to obtain the characteristics of each role of the target scene content.

[0050] The target scene content is the scene content in the sound effect guidance text at the current moment. Specifically, the sound effect guidance text includes scene content corresponding to each time period. For example, the sound effect guidance text includes 3 time periods, the first time period is 6 minutes, the second time period is 5 minutes, and the third time period is 8 minutes. If the current moment is 7 minutes away from the moment when the start instruction is received, it can be determined that the scene content at the current moment is the scene content corresponding to the second time period.

[0051] Character features are various attributes and characteristics of characters in scene content extracted through text analysis. These features together constitute the unique identity and behavior pattern of the character. Different character features can correspond to different human voice audio. The character features in this solution include appearance features, personality characteristics, emotional state, background story, social relationships, and growth changes.

[0052] In an optional implementation, the sound effect guidance text generally includes all character introductions. After obtaining the sound effect guidance text, the character name in the character introduction can be set as the role identifier. After determining the target scene content, the character in the target scene content can be determined according to the role identifier. Alternatively, after determining the target scene content, match specific keywords or phrases (such as "said", "shouted" or "thinking", etc.) to locate the character in the scene content. Further, combined with the context information of the scene content, the basic characteristics of the character such as age and gender are determined, thereby obtaining the characteristics of each character in the target scene content.

[0053] Step 205: Based on the characteristics of each character in the target scene content and the audio information of each human voice at the current moment, obtain the target human voice audio information at the current moment.

[0054] Specifically, the scene interpretation process is a dynamic process. The emotions or speech speed of each character in the scene interpretation process are uncertain, and the sound effect guidance text may only include a rough guidance emotion. For example, in a dialogue scene, the sound effect knows that the emotion specified in the text is "anger", but the "degree of anger" cannot be quantified. Therefore, in order to improve the user's overall auditory experience and immersion, and make the character's life audio more consistent with the interpreted scene, the human voice audio information can be adjusted according to the sound effect guidance text to obtain the target human voice audio information. In this solution, optionally, based on the characteristics of each character in the target scene content and the human voice audio information at the current moment, obtaining the target human voice audio information at the current moment includes the following steps A1-A2:

[0055] Step A1: Perform sentiment analysis on the current human voice audio information according to the characteristics of each character and a preset sentiment analysis model to obtain a sentiment analysis result of the current human voice audio information.

[0056] Among them, the sentiment analysis result includes the corresponding sentiment type and sentiment depth. The sentiment analysis model can be a Transformer model. In an optional implementation, after obtaining the character features, the character features and the current human voice audio information are input into the Transformer model. The Transformer model can process the character features and the current human voice audio information through a multi-layer Transformer encoder. The Transformer encoder can use the volume of the human voice audio as an influencing factor affecting the depth of emotion. After processing the character features and the human voice audio information, the loss function value is calculated based on the processed fusion features, and the sentiment analysis result of the human voice audio information is determined based on the loss function value. The loss function in this scheme is: L total =L cross-entropy +γL regression , among which, L cross-entropy is the cross entropy loss of sentiment type prediction, L regression is the regression loss (mean square error or mean absolute error) of sentiment depth prediction, and γ is a pre-set weight coefficient used to balance the importance of the two loss terms.

[0057] Step A2: According to a predetermined emotional audio information rule, determine the target audio volume corresponding to the emotional analysis result of the current human voice audio information; adjust the audio volume of the current human voice audio based on the target audio volume to obtain the target human voice audio information at the current moment.

[0058] Among them, the emotional audio information rules are used to determine the audio volumes corresponding to different emotion types and emotional depths. For example, for emotions of the anger type, the greater the emotional depth, the greater the corresponding audio volume. For emotions of the fear type, the greater the emotional depth, the smaller the corresponding audio volume. After determining the emotional analysis result of the current human voice audio information, the target audio volume corresponding to the current human voice audio information is determined according to the emotional audio information rules. And according to the target audio volume, the audio volume of the current human voice audio is adjusted so that the audio volume of the current human voice audio is closer to the target audio volume to ensure that the audio output matches the emotional state of the character.

[0059] Through the above steps, the volume of the vocal audio can be dynamically adjusted according to the character's characteristics and emotional state, thereby creating a more realistic and immersive sound experience.

[0060] Step 206: Adjust the background audio information at the current moment based on the target human voice audio information at the current moment to obtain the target background audio information at the current moment.

[0061] The target background audio information includes the adjusted background audio data and related parameter settings (such as volume and frequency, etc.). Specifically, since the human voice audio may have changed, it is necessary to adjust the background audio information at the current moment according to the target human voice audio information at the current moment to obtain the target background audio information at the current moment. After obtaining the target human voice audio information, extract the key features of the target human voice audio, such as pitch, rhythm, and emotional intensity. Adjust the background audio according to its key features to ensure the consistency of the background audio with the human voice audio in emotion and atmosphere. Including adjusting the volume of the background audio accordingly according to the volume of the target human voice audio. For example, if the volume of the human voice audio is large, the volume of the background audio may need to be lowered to avoid sound mixing and interference. Adjust the frequency response of the background audio according to the frequency distribution of the target human voice audio. For example, if the human voice audio is concentrated in the low-frequency area, the high-frequency components in the background audio can be appropriately enhanced to maintain the balance and clarity of the audio.

[0062] Step 207: Perform audio fusion on the target human voice audio information and the target background audio information at the current moment to obtain the target audio information at the current moment.

[0063] After obtaining the target human voice audio information and the target background audio information, the target human voice audio information and the target background audio information are fused according to a predetermined audio fusion method. The audio fusion method can be predetermined according to the specific interpretation environment, and the audio fusion method includes superposition mixing, dynamic range compression, and equalizer adjustment. Superposition mixing is to directly superimpose the human voice audio and the background audio together. Dynamic range compression is to control the dynamic range of the audio signal through compression technology, so that the human voice audio and the background audio are better integrated. Equalizer adjustment is to use an equalizer to adjust the gain of different frequency bands to ensure that the human voice and background sound effects have good performance at all frequencies.

[0064] Step 208: Generate and play target audio based on the target audio information.

[0065] In the technical solution of this embodiment, for the scene content of each time period, the current scene content of the current time period is subjected to text analysis to obtain the current scene type corresponding to the current scene content. All background audio signals corresponding to the current scene type are searched in the predetermined background sound effect library, and the background audio information at the current moment is determined according to all background audio signals. When a start instruction sent by the user is received, at least one human voice audio information at the current moment is obtained in response to the start instruction. The target scene content at the current moment in the sound effect guidance text is determined, and the role analysis of the target scene content is performed to obtain the characteristics of each role of the target scene content. Based on the characteristics of each role of the target scene content and the human voice audio information at the current moment, the target human voice audio information at the current moment is obtained. Based on the target human voice audio information at the current moment, the background audio information at the current moment is adjusted to obtain the target background audio information at the current moment; the target human voice audio information at the current moment and the target background audio information are subjected to audio fusion to obtain the target audio information at the current moment. The target audio is generated and played based on the target audio information. The technical solution of this embodiment can analyze the sound effect guidance text through context learning technology, and accurately filter out the background audio signal that matches the current scene content, atmosphere and emotion from the sound effect library, which ensures the consistency and coordination of the background audio and the scene content. According to the characteristics of the role corresponding to the scene content, the current human voice audio information can be emotionally analyzed to accurately obtain the emotional type and emotional depth. According to the emotional analysis results, the human voice audio is adjusted so that the audio output is closer to the emotional state of the role, enhancing the user's auditory experience. The background audio is adjusted based on the target human voice audio information to ensure the consistency of the background audio and the human voice audio in emotion and atmosphere. Improve the overall audio quality. That is, the technical solution of this embodiment realizes the accurate capture and expression of the role emotion and scene atmosphere by combining role analysis, emotional analysis, audio adjustment and audio fusion technology. Not only the quality and consistency of audio generation are improved, but also the user's immersion and auditory experience are enhanced.

[0066] Figure 3 FIG. 1 is a schematic diagram of the structure of an audio signal processing device provided in an embodiment of the present invention, and the device is suitable for executing the audio signal processing method provided in an embodiment of the present invention. Figure 3 As shown, the device may specifically include:

[0067] The information acquisition module 301 is used for, upon receiving a start instruction sent by a user, acquiring at least one human voice audio information at the current moment in response to the start instruction;

[0068] The first determination module 302 is used to obtain the target human voice audio information at the current moment based on the pre-acquired sound effect guidance text and the human voice audio information at the current moment;

[0069] A second determination module 303 is used to determine the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment;

[0070] The audio playing module 304 is used to generate and play the target audio based on the target audio information.

[0071] Optionally, the sound effect guidance text includes scene contents of each time period. Before receiving the start instruction sent by the user, the second determination module 303 is specifically used to: for the scene contents of each time period, perform text analysis on the current scene content of the current time period to obtain the current scene type corresponding to the current scene content;

[0072] Searching for all background audio signals corresponding to the current scene type in a predetermined background sound effect library;

[0073] The background audio information at the current moment is determined according to all background audio signals.

[0074] Optionally, the first determination module 302 is specifically used to: determine the target scene content in the sound effect guidance text at the current moment, and perform role analysis on the target scene content to obtain each role feature of the target scene content;

[0075] Based on the characteristics of each character in the target scene content and the audio information of each human voice at the current moment, the target human voice audio information at the current moment is obtained.

[0076] Optionally, the first determination module 302 is further used to: perform sentiment analysis on the current human voice audio information according to the characteristics of each character and a preset sentiment analysis model to obtain a sentiment analysis result of the current human voice audio information; wherein the sentiment analysis result includes a corresponding sentiment type and sentiment depth;

[0077] The current human voice audio information is adjusted based on the emotion analysis result to obtain the target human voice audio information at the current moment.

[0078] Optionally, the first determination module 302 is further used to: determine a target audio volume corresponding to the emotion analysis result of the current human voice audio information according to a predetermined emotion audio information rule;

[0079] The audio volume of the current human voice audio is adjusted based on the target audio volume to obtain the target human voice audio information at the current moment.

[0080] Optionally, the second determination module 303 is specifically configured to: adjust the background audio information at the current moment based on the target human voice audio information at the current moment to obtain the target background audio information at the current moment;

[0081] Audio fusion is performed on the target human voice audio information and the target background audio information at the current moment to obtain the target audio information at the current moment.

[0082] The audio signal processing device provided in the embodiment of the present invention can execute the audio signal processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not described in detail in this embodiment, reference can be made to the description in any method embodiment of the present invention.

[0083] An embodiment of the present invention also provides a computer program product.

[0084] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer program products, which can include one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor, which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, referring to Figure 4 , Figure 4 The electronic device 12 shown is only an example and should not limit the functions and scope of use of the embodiments of the present application. Figure 4 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0086] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0087] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0088] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0089] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0090] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0091] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing an audio signal processing method provided in an embodiment of the present invention: when receiving a start instruction sent by a user, obtaining at least one human voice audio information at the current moment in response to the start instruction; obtaining the target human voice audio information at the current moment based on the pre-acquired sound effect guidance text and the human voice audio information at the current moment; determining the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment; generating and playing the target audio based on the target audio information.

[0092] The embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, an audio signal processing method as provided in all the embodiments of the present invention is implemented: when a start instruction sent by a user is received, at least one human voice audio information at the current moment is obtained in response to the start instruction; based on the pre-acquired sound effect guidance text and each human voice audio information at the current moment, the target human voice audio information at the current moment is obtained; the target audio information at the current moment is determined according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment; the target audio is generated and played based on the target audio information. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic device, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction-executing electronic device, apparatus, or device.

[0093] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction-executing electronic device, apparatus, or device.

[0094] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0095] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An audio signal processing method, characterized in that: The method comprises: When receiving a start instruction sent by a user, acquiring at least one human voice audio information at the current moment in response to the start instruction; Based on the pre-acquired sound effect guidance text and the audio information of each human voice at the current moment, obtaining the target human voice audio information at the current moment; Determining the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment; Generate and play target audio based on the target audio information.

2. The method according to claim 1, characterized in that The sound effect guidance text includes scene contents of each time period. Before receiving a start instruction sent by a user, the method further includes: For the scene contents in each time period, text analysis is performed on the current scene content in the current time period to obtain the current scene type corresponding to the current scene content; Searching for all background audio signals corresponding to the current scene type in a predetermined background sound effect library; The background audio information at the current moment is determined according to all background audio signals.

3. The method according to claim 1, characterized in that Based on the pre-acquired sound effect guidance text and the audio information of each human voice at the current moment, the target human voice audio information at the current moment is obtained, including: Determine the target scene content in the sound effect guidance text at the current moment, and perform role analysis on the target scene content to obtain the characteristics of each role of the target scene content; Based on the characteristics of each character in the target scene content and the audio information of each human voice at the current moment, the target human voice audio information at the current moment is obtained.

4. The method according to claim 3, characterized in that Based on the characteristics of each character in the target scene content and the audio information of each human voice at the current moment, the target human voice audio information at the current moment is obtained, including: Performing sentiment analysis on the current human voice audio information according to the characteristics of each character and a preset sentiment analysis model to obtain a sentiment analysis result of the current human voice audio information; wherein the sentiment analysis result includes a corresponding sentiment type and sentiment depth; The current human voice audio information is adjusted based on the emotion analysis result to obtain the target human voice audio information at the current moment.

5. The method according to claim 4, characterized in that The current human voice audio information is adjusted based on the emotion analysis result to obtain the target human voice audio information at the current moment, including: Determining a target audio volume corresponding to an emotion analysis result of the current human voice audio information according to a predetermined emotion audio information rule; The audio volume of the current human voice audio is adjusted based on the target audio volume to obtain the target human voice audio information at the current moment.

6. The method according to claim 1, characterized in that Determining the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment includes: Adjusting the background audio information at the current moment based on the target human voice audio information at the current moment to obtain the target background audio information at the current moment; Audio fusion is performed on the target human voice audio information and the target background audio information at the current moment to obtain the target audio information at the current moment.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements an audio signal processing method according to any one of claims 1 to 6.

8. An audio signal processing device, characterized in that: include: An information acquisition module, configured to, upon receiving a start instruction sent by a user, acquire at least one human voice audio information at a current moment in response to the start instruction; A first determination module, configured to obtain the target human voice audio information at the current moment based on the pre-acquired sound effect guidance text and the human voice audio information at the current moment; A second determination module is used to determine the target audio information at the current moment according to the target human voice audio information at the current moment and the predetermined background audio information at the current moment; The audio playing module is used to generate and play the target audio based on the target audio information.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the audio signal processing method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the audio signal processing method according to any one of claims 1 to 6 is implemented.

Images