AI voice changing method and related equipment based on drama culture

Abstract

The present application provides an AI voice-changing method based on drama culture and related equipment, the method comprising: obtaining target audio data and target drama information of a target user; the target drama information includes a target drama style, a target role type and a target emotional state; extracting features from the target audio data to obtain target features; obtaining a sample set corresponding to a preset AI sound effect model; obtaining the similarity between the sample features of each sound effect sample in a plurality of sound effect samples and the target features to obtain a plurality of similarities; determining the sound effect sample corresponding to the maximum similarity among the plurality of similarities as a reference sound effect; adjusting the reference sound effect according to the target drama style, the target role type and the target emotional state through the AI ​​sound effect model to obtain a target sound effect; performing voice-changing processing on the target audio data according to the target sound effect to obtain a target voice-changing file. The accuracy and flexibility of AI voice-changing can be improved.

Description

Technical Field

[0001] The present application relates to the field of AI voice changing technology, and in particular to an AI voice changing method based on drama culture and related equipment. Background Art

[0002] In the field of digital audio processing, the acquisition and feature extraction of audio data are important steps in building personalized sound effects. However, traditional audio processing technology usually relies on preset parameters and rules, lacks personalization and intelligence, and is difficult to meet users' needs for AI voice changing for dramatic sound effects. At the same time, traditional sound effects are often based on fixed templates or preset parameters, lack dynamic adjustment and personalized settings, and are difficult to accurately restore and convert various timbres in dramas.

[0003] Therefore, how to improve the accuracy and personalization of AI voice changing for drama needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide an AI voice changing method and related equipment based on theatrical culture. By comprehensively analyzing the user's audio data and drama information, combined with the drama style, character type and emotional state, a sound effect that meets the user's personalized needs is generated, and the user's audio data is subjected to AI voice changing based on the sound effect, thereby improving the accuracy and flexibility of AI voice changing.

[0005] In a first aspect, an embodiment of the present application provides an AI voice changing method based on drama culture, the method comprising:

[0006] Acquire target audio data and target drama information of a target user; the target drama information includes a target drama genre, a target role type, and a target emotional state;

[0007] Extracting features from the target audio data to obtain target features;

[0008] Obtain a sample set corresponding to a preset AI sound effect model; the sample set includes multiple sound effect samples;

[0009] Obtaining a similarity between a sample feature of each sound effect sample in the plurality of sound effect samples and the target feature to obtain a plurality of similarities;

[0010] Determine the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect;

[0011] The reference sound effect is adjusted by the AI ​​sound effect model according to the target drama style, the target character type and the target emotional state to obtain a target sound effect;

[0012] The target audio data is voice-changed according to the target sound effect to obtain a target voice-changed file.

[0013] In a second aspect, an embodiment of the present application provides an AI voice changing device based on drama culture, the device comprising an acquisition module, a processing module, a determination module, an adjustment module and a voice changing module, wherein:

[0014] The acquisition module is used to acquire the target audio data and target drama information of the target user; the target drama information includes the target drama style, target role type and target emotional state;

[0015] The processing module is used to extract features from the target audio data to obtain target features;

[0016] The acquisition module is further used to acquire a sample set corresponding to a preset AI sound effect model; the sample set includes a plurality of sound effect samples;

[0017] The acquisition module is further used to acquire the similarity between the sample feature of each sound effect sample in the multiple sound effect samples and the target feature to obtain multiple similarities;

[0018] The determination module is used to determine the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect;

[0019] The adjustment module is used to adjust the reference sound effect according to the target drama style, the target character type and the target emotional state through the AI ​​sound effect model to obtain the target sound effect;

[0020] The voice changing module is used to perform voice changing processing on the target audio data according to the target sound effect to obtain a target voice changing file.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method in the first aspect of the embodiment of the present application.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0024] By implementing the embodiments of the present application, a comprehensive analysis of the user's audio data and drama information can be performed, combining the style of the drama, character type and emotional state to generate sound effects that meet the user's personalized needs, and based on the sound effects, the user's audio data can be AI voice-changed, thereby improving the accuracy and flexibility of AI voice-changing. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 This is a system architecture diagram of an AI voice changing system provided in an embodiment of the present application;

[0027] Figure 2 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0028] Figure 3 It is a flowchart of an AI voice changing method based on drama culture provided in an embodiment of the present application;

[0029] Figure 4 This is a block diagram of the functional modules of an AI voice changing device based on drama culture provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0032] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article indicates that the associated objects before and after are in an "or" relationship. The "plurality" appearing in the embodiments of the present application refers to two or more.

[0033] In the embodiments of the present application, "at least one item" or similar expressions refer to any combination of these items, including any combination of single items or plural items, and refer to one or more, and multiple refers to two or more. For example, at least one item of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Among them, each of a, b, and c can be an element or a set containing one or more elements.

[0034] The "connection" that appears in the embodiments of the present application refers to various connection methods such as direct connection or indirect connection to achieve communication between devices, and the embodiments of the present application do not impose any limitations on this.

[0035] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] In the field of digital audio processing, the acquisition and feature extraction of audio data are important steps in building personalized sound effects. However, traditional audio processing technology usually relies on preset parameters and rules, lacks personalization and intelligence, and is difficult to meet users' needs for AI voice changing for dramatic sound effects. At the same time, traditional sound effects are often based on fixed templates or preset parameters, lack dynamic adjustment and personalized settings, and are difficult to accurately restore and convert various timbres in dramas. Therefore, how to improve the accuracy and personalization of AI voice changing for dramas needs to be solved urgently.

[0037] To solve the above problems, the embodiment of the present application provides an AI voice changing method and related equipment based on drama culture, obtaining the target audio data and target drama information of the target user; the target drama information includes the target drama style, target role type and target emotional state; extracting features from the target audio data to obtain target features; obtaining a sample set corresponding to a preset AI sound effect model; the sample set includes multiple sound effect samples; obtaining the similarity between the sample feature of each sound effect sample in the multiple sound effect samples and the target feature to obtain multiple similarities; determining the sound effect sample corresponding to the maximum similarity among the multiple similarities as a reference sound effect; adjusting the reference sound effect according to the target drama style, the target role type and the target emotional state through the AI ​​sound effect model to obtain the target sound effect; performing voice changing processing on the target audio data according to the target sound effect to obtain the target voice changing file. It can be seen that by comprehensively analyzing the user's audio data and drama information, combining the drama style, role type and emotional state to generate sound effects that meet the user's personalized needs, and performing AI voice changing on the user's audio data based on the sound effect, the accuracy and flexibility of AI voice changing are improved.

[0038] Combine the following Figure 1 The system architecture of an AI voice changing system for executing an AI voice changing method based on drama culture in an embodiment of the present application is described. Figure 1 This is a system architecture diagram of an AI voice changing system provided in an embodiment of the present application. The AI ​​voice changing system includes a user input module, a sound effect generation module and a voice changing processing module.

[0039] Among them, the user input module can receive audio data uploaded or specified by the user. The audio data can be in a variety of common audio formats, such as MP3, WAV, etc., to provide raw materials for subsequent AI voice changing processing. The user input module can also receive drama information provided by the user, which includes drama style (such as Gan Opera, Peking Opera, Yue Opera, Huangmei Opera, etc.), role type (such as Sheng, Dan, Jing, Chou, etc.) and emotional state (such as happy, sad, angry, etc.).

[0040] Among them, the sound effect generation module can use professional audio signal processing technology and algorithms to deeply analyze the multi-dimensional features of the audio for the audio data uploaded by the user. For example, in the time domain, it extracts features such as rhythm, pitch, and duration, and in the frequency domain, it obtains information such as spectrum distribution, frequency center of gravity, and harmonic structure. At the same time, it also considers acoustic features related to timbre, such as resonance peak features, etc., and integrates these rich features to form a feature representation that can fully describe the audio. The sound effect generation module manages and maintains a sample set containing different sound effects, which contains various sound effect samples covering many drama styles, different character types, and various emotional states. For each sound effect sample, the feature extraction technology is also used to obtain its corresponding feature representation. Then, through a suitable similarity calculation algorithm (such as cosine similarity, an algorithm based on distance measurement, etc.), the similarity between the features of each sound effect sample and the features of the user's audio data is calculated. Then, the sound effect sample with the highest similarity is selected and determined as the reference sound effect. The reference sound effect is closest to the user's audio data in terms of basic features, and can be used as the basis for subsequent adjustment to generate the target sound effect. The sound effect generation module can call the preset AI sound effect model, and input the determined reference sound effects and user-specified drama style, role type, emotional state and other information into the model. Then, through the learning and generation capabilities of the AI ​​sound effect model, the reference sound effects are adjusted and optimized based on the input drama information. For example, if the user specifies the sad emotional state of a female role in Gan Opera, the model will adjust the timbre of the reference sound effect to make it more in line with the soft timbre characteristics of the female role, and at the same time incorporate audio elements that can reflect sad emotions, such as appropriately lowering the pitch, adding vibrato, etc., and finally generate a target sound effect that meets the user's personalized needs.

[0041] It should be noted that the AI ​​sound effect model can be a deep convolutional generative adversarial network sound effect model. By using a deep convolutional neural network to improve the model's ability to extract and generate audio features, and then through operations such as convolutional layers and pooling layers, the model can automatically learn features such as the spectrum of the audio, and generate corresponding AI sound-changing sound effects based on specific theatrical cultural information. The AI ​​sound effect model can also be a hybrid model that integrates traditional audio processing technology and AI technology. It has a built-in traditional audio analysis module, which can use fast Fourier transform to obtain the spectrum information of the audio, which is used to analyze the basic features such as the frequency and energy distribution of the reference sound effect, and then use the fundamental frequency estimation method to grasp the pitch of the audio. Then, the traditional audio features are passed to the AI ​​part for further analysis and processing. The AI ​​part can use machine learning algorithms such as decision trees, support vector machines, etc. that can be used for classification and regression to decide how to adjust the audio parameters based on the received traditional audio features and the theatrical information input by the user, or deep learning algorithms to generate specific adjustment strategies, such as enhancing or weakening specific frequency components of the reference sound effect to change the timbre, and raising or lowering the pitch to an appropriate degree, etc., to generate the final AI sound-changing sound effect.

[0042] The voice-changing processing module uses advanced audio synthesis, mixing and other voice-changing algorithms and technologies to fuse the target sound effect with the original audio data. For example, according to the timbre, pitch, rhythm and other characteristics of the target sound effect, the corresponding audio parameters of the original audio data are adjusted and modified, so that the sound effect of the original audio data gradually approaches the target sound effect, achieving the voice-changing effect, and finally obtaining the target voice-changing file.

[0043] It can be seen that through the above system architecture, corresponding target sound effects can be generated according to the user's personalized drama needs, so that the AI ​​voice changing operation can be performed according to the target sound effects to improve the accuracy and personalization of the AI ​​voice changing.

[0044] Combine the following Figure 2 The electronic device in the embodiment of the present application is described. Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 2 As shown, the electronic device includes one or more processors, a memory, a communication interface and one or more programs, and the processor is communicatively connected with the memory and the communication interface via an internal communication bus.

[0045] Among them, the processor is mainly used for:

[0046] Acquire target audio data and target drama information of a target user; the target drama information includes a target drama genre, a target role type, and a target emotional state;

[0047] Extracting features from the target audio data to obtain target features;

[0048] Obtain a sample set corresponding to a preset AI sound effect model; the sample set includes multiple sound effect samples;

[0049] Obtaining a similarity between a sample feature of each sound effect sample in the plurality of sound effect samples and the target feature to obtain a plurality of similarities;

[0050] Determine the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect;

[0051] The reference sound effect is adjusted by the AI ​​sound effect model according to the target drama style, the target character type and the target emotional state to obtain a target sound effect;

[0052] The target audio data is voice-changed according to the target sound effect to obtain a target voice-changed file.

[0053] The one or more programs are stored in the above-mentioned memory and are configured to be executed by the above-mentioned processor, and the one or more programs include instructions for executing any step in the above-mentioned method embodiment.

[0054] Among them, the processor can be, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, units and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like. The communication unit can be a communication interface, a transceiver, a transceiver circuit, and the like, and the storage unit can be a memory.

[0055] The memory may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM).

[0056] It is understandable that the electronic device may include more or fewer structural elements than those in the above structural block diagram, for example, including a power module, physical buttons, Wi-Fi module, speaker, Bluetooth module, sensor, display module, etc., which are not limited here. It is understandable that the electronic device may be equipped with Figure 1 The system architecture described.

[0057] After understanding the software and hardware architecture of this application, Figure 3 An AI voice changing method based on drama culture in an embodiment of the present application is described. Figure 3 : is a flowchart of an AI voice changing method based on drama culture provided in an embodiment of the present application, which specifically includes the following steps:

[0058] Step S301, obtaining target audio data and target drama information of a target user.

[0059] The target drama information includes target drama genre, target role type and target emotional state.

[0060] Specifically, the target audio data of the target user can be received, and the target audio data can be pre-recorded audio data, for example, lines recorded with a recording device, reading clips, etc., or voice clips recorded on site, which are not specifically limited here. Among them, when the target audio data is obtained, the audio data will be preliminarily checked. If the audio data is damaged or the audio parameters do not meet the processing requirements (such as the sampling rate is too low, affecting the subsequent voice change effect), etc., the prompt information will be fed back to the target user in time to prompt the user to resubmit the audio data that meets the processing requirements. In addition, the target drama information personalized by the target user can also be received, and the target drama information includes the target drama style, target role type and target emotional state, wherein the target drama style includes but is not limited to Gan Opera, Peking Opera, Yue Opera, Huangmei Opera, and Henan Opera, the target role type includes but is not limited to Shengjiao, Danjiao, Jingjiao, and Choujiao, and the target emotional state includes but is not limited to cheerfulness, sadness, anger, and calmness, which are not specifically limited here.

[0061] Step S302: extract features from the target audio data to obtain target features.

[0062] The step of extracting features from the target audio data to obtain target features specifically includes:

[0063] The target audio data is framed according to a preset time interval to obtain a plurality of audio frames; the fundamental frequency of each audio frame in the plurality of audio frames is estimated to obtain a plurality of fundamental frequencies; the maximum fundamental frequency and the minimum fundamental frequency in the plurality of fundamental frequencies are obtained; the average value of the plurality of fundamental frequencies is determined as the average fundamental frequency; the target audio data is fast Fourier transformed to obtain a target spectrum; the target spectrum includes a plurality of frequency points and a plurality of energy values; the center frequency is determined according to the plurality of frequency points and the plurality of energy values; the target feature is determined according to the average fundamental frequency, the maximum fundamental frequency, the minimum fundamental frequency and the center frequency.

[0064] Specifically, first, the target audio signal corresponding to the target audio data is obtained. Since the audio signal changes continuously over time, in order to facilitate analysis and processing, it needs to be divided into multiple shorter segments, that is, it is framed to obtain multiple audio frames. Then, the fundamental frequency of each audio frame can be estimated by the autocorrelation method or the cepstrum method to obtain multiple fundamental frequencies. Among them, the fundamental frequency refers to the lowest frequency component in a periodic signal, which is closely related to the pitch of the audio. Then, the maximum and minimum values ​​of the multiple fundamental frequencies, that is, the maximum fundamental frequency and the minimum fundamental frequency, are obtained, and the average value of the multiple fundamental frequencies is calculated to obtain the average fundamental frequency. Then, the target audio signal can be fast Fourier transformed to convert it from the time domain to the frequency domain, so as to obtain the target spectrum. Among them, the target spectrum represents the distribution of the target audio signal in the frequency domain, that is, the energy value corresponding to different frequency points, the frequency point represents the frequency component, and the corresponding energy value represents the intensity of the frequency component. Then, multiple frequency points and multiple energy values ​​are calculated according to the preset center of gravity frequency calculation formula to obtain the center of gravity frequency. Among them, the center of gravity frequency calculation formula is as follows:

[0065]

[0066] in, represents the center of gravity frequency; represents the i-th frequency point among multiple frequency points; Represents the i-th energy value among multiple energy values.

[0067] Finally, the target features of the target audio data are determined according to the average fundamental frequency, the maximum fundamental frequency, the minimum fundamental frequency and the center of gravity frequency.

[0068] Step S303, obtaining a sample set corresponding to a preset AI sound effect model.

[0069] The sample set includes multiple sound effect samples, and the AI ​​sound effect model is used to generate target sound effects that meet the personalized needs of the target user. The sample set corresponding to the AI ​​sound effect model contains a rich variety of sound effect samples, which is convenient for screening out the sound effect samples that are most similar to the target audio data, so as to retain the audio characteristics of the target user to the greatest extent.

[0070] Step S304: obtaining the similarity between the sample feature of each sound effect sample in the multiple sound effect samples and the target feature to obtain multiple similarities.

[0071] The obtaining of the similarity between the sample feature of each sound effect sample in the plurality of sound effect samples and the target feature to obtain a plurality of similarities specifically comprises the following steps:

[0072] Obtain a first sample feature of a first sound effect sample; the first sound effect sample is any one of the multiple sound effect samples; the first sample feature includes a first average fundamental frequency, a first maximum fundamental frequency, a first minimum fundamental frequency and a first centroid frequency; calculate the similarity between the target feature and the first sample feature according to a preset cosine similarity calculation formula to obtain a reference similarity; determine the absolute value of the reference similarity as the first similarity; the first similarity is the similarity corresponding to the first sound effect sample among the multiple similarities.

[0073] Specifically, the similarity between the target feature and the sample feature of each sound effect sample is calculated according to the cosine similarity calculation formula, and the absolute values ​​are taken to obtain multiple similarities. The calculation result of the cosine similarity ranges from -1 to 1. After taking the absolute value, it can be converted to a value between 0 and 1, which is more convenient for expressing the similarity between different features, where 0 represents complete dissimilarity and 1 represents complete similarity.

[0074] Step S305 : determining the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect.

[0075] Specifically, the similarity reflects the degree of proximity between the sample features of the sound effect sample and the target features. The sample features of the sound effect sample corresponding to the maximum similarity are closest to the target features. This sound effect sample can be selected as a reference sound effect and provide a benchmark for subsequent sound effect adjustments.

[0076] Step S306, adjusting the reference sound effect according to the target drama style, the target character type and the target emotional state through the AI ​​sound effect model to obtain the target sound effect.

[0077] The step of adjusting the reference sound effect according to the target drama style, the target role type and the target emotional state through the AI ​​sound effect model to obtain the target sound effect specifically includes:

[0078] Obtain a reference rhythm, reference timbre and reference pitch of the reference sound effect; determine a first rhythm corresponding to the target drama style according to a preset mapping relationship between the drama style and the rhythm; compare the first rhythm with the reference rhythm to obtain a first difference; if the first difference is less than or equal to a preset rhythm difference threshold, determine that the reference rhythm is the target rhythm; if the first difference is greater than the rhythm difference threshold, adjust the reference rhythm according to the first rhythm to obtain the target rhythm; determine a fullness range and a clarity range of the first timbre corresponding to the target character type; adjust the reference timbre according to the fullness range and the clarity range to obtain the target timbre; obtain a pitch range of the first tone corresponding to the target emotional state; the pitch range includes a lowest pitch and a highest pitch; adjust the reference tone according to the lowest pitch and the highest pitch to obtain the target tone; determine the target sound effect according to the target rhythm, the target timbre and the target pitch.

[0079] Specifically, basic features of the reference sound effect are obtained, such as reference rhythm, reference timbre and reference pitch. According to the mapping relationship between the preset drama style and rhythm, the first rhythm corresponding to the target drama style is determined. Then, the first rhythm is compared with the reference rhythm to obtain a first difference. If the first difference is less than or equal to the preset rhythm difference threshold, it means that the reference rhythm is close to the first rhythm required by the target drama style, and the reference rhythm can be determined to be the target rhythm. If the first difference is greater than the rhythm difference threshold, the reference rhythm needs to be adjusted according to the first rhythm. For example, if the rhythm speed of the first rhythm is faster than the rhythm speed of the reference rhythm, it is necessary to speed up the rhythm speed of the reference rhythm until a target rhythm that meets the requirements is obtained.

[0080] Next, the fullness range and clarity range of the first timbre corresponding to the target role type are determined. Different role types, such as Sheng, Dan, Jing, and Chou, have different requirements for the fullness and clarity of the timbre. The reference timbre is then adjusted according to the fullness range and clarity range to obtain the target timbre. For example, if the target role type requires a fuller timbre, and the reference timbre is relatively thin, the reference timbre needs to be processed so that its fullness reaches the range required by the target role type.

[0081] Finally, the pitch range of the first tone corresponding to the target emotional state is obtained, including the lowest pitch and the highest pitch. Different emotional states, such as happiness, sadness, and anger, have different expressions in pitch. For example, a happy emotion may correspond to a higher pitch range, while a sad emotion may correspond to a lower pitch range. The target sound effect is then determined based on the target rhythm, target timbre, and target pitch.

[0082] The reference rhythm includes a reference rhythm speed and a reference beat rule, and the reference rhythm is adjusted according to the first rhythm to obtain the target rhythm, and the specific steps include:

[0083] Obtain a first rhythm speed and a first beat regularity of the first rhythm; if the reference rhythm speed is less than the first rhythm speed, obtain the difference between the first rhythm speed and the reference rhythm speed to obtain a first speed difference; determine a first adjustment factor corresponding to the first speed difference; adjust the reference rhythm speed according to the first adjustment factor to obtain a target rhythm speed; if the reference rhythm speed is greater than the first rhythm speed, obtain the difference between the reference rhythm speed and the first rhythm speed to obtain a second speed difference; determine a second adjustment factor corresponding to the second speed difference; adjust the reference rhythm speed according to the second adjustment factor to obtain the target rhythm speed; if the reference rhythm speed is equal to the first rhythm speed, determine the reference rhythm speed as the target rhythm speed; adjust the reference beat regularity according to the first beat regularity to obtain a target beat regularity; the difference between the target beat regularity and the first beat regularity is less than a preset beat difference threshold; determine the target rhythm according to the target rhythm speed and the target beat regularity.

[0084] Specifically, first, the first rhythm speed and the first beat law of the first rhythm are obtained. The rhythm speed usually indicates the speed of the rhythm, and the beat law describes the characteristics such as the arrangement order of strong beats and weak beats. When the reference rhythm speed is less than the first rhythm speed, the difference between the first rhythm speed and the reference rhythm speed is calculated to obtain the first speed difference. The corresponding first adjustment factor is determined according to the first speed difference. The larger the first speed difference, the larger the first adjustment factor. Then the reference rhythm speed is adjusted according to the first adjustment factor to obtain the target rhythm speed. When the reference rhythm speed is greater than the first rhythm speed, the difference between the reference rhythm speed and the first rhythm speed is calculated to obtain the second speed difference. The corresponding second adjustment factor is determined according to the second speed difference. The larger the second speed difference, the larger the second adjustment factor. Then the reference rhythm speed is adjusted according to the second adjustment factor to obtain the target rhythm speed. When the reference rhythm speed is equal to the first rhythm speed, the reference rhythm speed is determined to be the target rhythm speed, and no adjustment is required. Then, the reference rhythm law is adjusted according to the first beat law to obtain the target beat law, and the difference between the target beat law and the first beat law is less than the preset beat difference threshold. Among them, if the first beat pattern is the first strong beat, the first weak beat, and the second weak beat, and the reference beat pattern is the first weak beat, the first strong beat, and the second weak beat, then the corresponding adjustment method can be used, for example, exchanging the positions of the first strong beat and the first weak beat in the reference beat pattern to make the reference beat pattern closer to the first beat pattern until the difference between the two meets the requirements and the target beat pattern is obtained, without specific limitation here.

[0085] The step of adjusting the reference timbre according to the fullness range and the clarity range to obtain a target timbre specifically includes:

[0086] Obtain a reference fullness and a reference clarity of the reference timbre; if the reference fullness is outside the fullness range, adjust the reference fullness to within the fullness range to obtain a target fullness; otherwise, determine the reference fullness to be the target fullness; if the reference clarity is outside the clarity range, adjust the reference clarity to within the clarity range to obtain a target clarity; otherwise, determine the reference clarity to be the target clarity; determine the target timbre according to the target fullness and the target clarity.

[0087] Specifically, obtain the reference fullness and reference clarity of the reference timbre. The fullness usually describes the thickness of the timbre, while the clarity involves the sharpness or roundness, easy to distinguish and other characteristics of the timbre. If the reference fullness is outside the fullness range corresponding to the target role type, it means that the reference fullness of the reference timbre does not meet the requirements of the target role type, and it is necessary to adjust the reference fullness to within the fullness range to obtain the target fullness. If the reference fullness is within the fullness range, the reference fullness is determined to be the target fullness. If the reference clarity is outside the clarity range, it is adjusted to within the clarity range to obtain the target clarity. If the reference clarity is within the clarity range, the reference clarity is directly determined to be the target clarity, and no further adjustment is required. Finally, the target timbre is determined based on the target fullness and target clarity.

[0088] It can be seen that by accurately adjusting the fullness and clarity of the timbre, a timbre that suits a specific character type can be accurately generated, thereby facilitating improving the reliability and accuracy of the voice changing sound effect.

[0089] The step of adjusting the reference pitch according to the lowest pitch and the highest pitch to obtain the target pitch specifically includes:

[0090] If the reference tone is less than the lowest tone, the tone difference between the lowest tone and the reference tone is obtained to obtain a first tone difference; a first adjustment parameter corresponding to the first tone difference is determined; if the reference tone is greater than the highest tone, the tone difference between the reference tone and the highest tone is obtained to obtain a second tone difference; a second adjustment parameter corresponding to the second tone difference is determined; the reference tone is adjusted according to the first adjustment parameter and / or the second adjustment parameter to obtain the target tone; if the reference tone is greater than or equal to the lowest tone, and less than or equal to the highest tone, the reference tone is determined to be the target tone.

[0091] Specifically, when the reference tone is less than the lowest tone corresponding to the target emotional state, it is necessary to calculate the difference between the lowest tone and the reference tone to obtain the first tone difference. According to the mapping relationship between the preset tone difference and the adjustment parameter, the first adjustment parameter corresponding to the first tone difference is determined. The larger the first tone difference, the larger the corresponding first adjustment parameter. When the reference tone is greater than the highest tone corresponding to the target emotional state, the difference between the reference tone and the highest tone is calculated to obtain the second tone difference. According to the mapping relationship between the preset tone difference and the adjustment parameter, the second adjustment parameter corresponding to the second tone difference is determined. The larger the second tone difference, the larger the corresponding second adjustment parameter. Then, the reference tone is adjusted according to the first adjustment parameter and / or the second adjustment parameter to obtain the target tone. If the reference tone is greater than or equal to the lowest tone corresponding to the target emotional state, and less than or equal to the highest tone, it means that the reference tone is within the tone range required by the target emotional state, and no adjustment is required, and the reference tone can be determined as the target tone.

[0092] It can be seen that by judging and adjusting the reference pitch, the pitch can be accurately made to meet the requirements of the target emotional state, and then when generating the target sound effect, the desired emotion can be better conveyed through the pitch, thereby enhancing the accuracy and appeal of the entire sound effect in terms of emotional expression.

[0093] Step S307, performing voice change processing on the target audio data according to the target sound effect to obtain a target voice change file.

[0094] Specifically, the target sound effect includes target timbre, target pitch, and target rhythm. When performing voice change processing, it is necessary to analyze the difference between the target timbre and the original timbre of the target audio data, and then use the preset audio processing algorithm to change the timbre of the target audio data to make it closer to the target timbre. For example, if the target timbre is richer, the timbre of the target audio data may be made thicker and fuller by increasing the harmonic components in the audio signal, adjusting the resonance peak, etc.; if the target timbre is clearer, the transient part of the audio signal of the target audio data may be enhanced to improve the recognizability of the sound details, etc. Then, the target pitch is compared with the original pitch of the target audio data and adjusted accordingly. For example, if the target pitch is higher than the original pitch of the target audio data, the preset audio pitch change algorithm will be used to increase the original pitch of the target audio data, such as time stretching based on the time domain, frequency shifting based on the frequency domain, etc.; on the contrary, if the target pitch is lower, the pitch is lowered by the corresponding algorithm to ensure that the pitch of the target audio data meets the requirements of the target sound effect, so as to accurately convey the corresponding emotional or style characteristics. Finally, the original rhythm of the target audio data is changed according to the target rhythm. For example, when the target rhythm is faster, the rhythm of the target audio data can be accelerated by shortening the duration of the notes in the target audio data, speeding up the beat interval, etc. If the target rhythm has a specific beat pattern, the distribution of strong beats and weak beats in the target audio data will be rearranged to make its rhythm pattern consistent with the target rhythm, enhance the overall sense of rhythm, and match the rhythm style of the target sound effect. The audio data after voice change processing is integrated to generate a new audio file to obtain the target voice change file. The format of the target voice change file can be determined according to actual needs and system settings. Common audio formats such as MP3, WAV, FLAC, etc. can be used as output formats, so that subsequent users can easily play, edit or share the file on different devices and platforms to meet the needs of diverse usage scenarios.

[0095] It can be seen that through the above method, the user's audio data and drama information can be comprehensively analyzed, combined with the style of the drama, role type and emotional state to generate sound effects that meet the user's personalized needs, and based on the sound effects, the user's audio data can be AI voice-changed, thereby improving the accuracy and flexibility of AI voice-changing.

[0096] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0097] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0098] In the case of dividing each functional module into corresponding functional modules, Figure 4 4 is a functional module composition block diagram of an AI voice changing device based on drama culture provided in an embodiment of the present application. The AI ​​voice changing device based on drama culture 400 includes an acquisition module 410, a processing module 420, a determination module 430, an adjustment module 440 and a voice changing module 450, wherein:

[0099] The acquisition module 410 is used to acquire the target audio data and target drama information of the target user; the target drama information includes the target drama style, target role type and target emotional state;

[0100] The processing module 420 is used to extract features from the target audio data to obtain target features;

[0101] The acquisition module 410 is further used to acquire a sample set corresponding to a preset AI sound effect model; the sample set includes a plurality of sound effect samples;

[0102] The acquisition module 410 is further configured to acquire a similarity between a sample feature of each sound effect sample in the plurality of sound effect samples and the target feature, to obtain a plurality of similarities;

[0103] The determination module 430 is used to determine the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect;

[0104] The adjustment module 440 is used to adjust the reference sound effect according to the target drama style, the target character type and the target emotional state through the AI ​​sound effect model to obtain the target sound effect;

[0105] The voice changing module 450 is used to perform voice changing processing on the target audio data according to the target sound effect to obtain a target voice changing file.

[0106] Optionally, in the aspect of extracting features from the target audio data to obtain target features, the processing module 420 is specifically used to:

[0107] Performing frame processing on the target audio data according to a preset time interval to obtain a plurality of audio frames;

[0108] Performing fundamental frequency estimation on each of the multiple audio frames to obtain multiple fundamental frequencies;

[0109] Obtaining a maximum fundamental frequency and a minimum fundamental frequency among the multiple fundamental frequencies;

[0110] Determine an average value of the plurality of fundamental frequencies as an average fundamental frequency;

[0111] Performing a fast Fourier transform on the target audio data to obtain a target spectrum; the target spectrum includes a plurality of frequency points and a plurality of energy values;

[0112] Determine a center of gravity frequency according to the multiple frequency points and the multiple energy values;

[0113] The target feature is determined according to the average fundamental frequency, the maximum fundamental frequency, the minimum fundamental frequency and the center of gravity frequency.

[0114] Optionally, in the aspect of obtaining the similarity between the sample feature of each sound effect sample in the plurality of sound effect samples and the target feature to obtain a plurality of similarities, the processing module 420 is further specifically configured to:

[0115] Acquire a first sample feature of a first sound effect sample; the first sound effect sample is any one of the multiple sound effect samples; the first sample feature includes a first average fundamental frequency, a first maximum fundamental frequency, a first minimum fundamental frequency and a first center of gravity frequency;

[0116] Calculating the similarity between the target feature and the first sample feature according to a preset cosine similarity calculation formula to obtain a reference similarity;

[0117] An absolute value of the reference similarity is determined as a first similarity; the first similarity is a similarity corresponding to the first sound effect sample among the multiple similarities.

[0118] Optionally, in the aspect of adjusting the reference sound effect according to the target drama style, the target character type and the target emotional state by the AI ​​sound effect model to obtain the target sound effect, the adjustment module 440 is specifically used to:

[0119] Obtaining a reference rhythm, a reference timbre, and a reference pitch of the reference sound effect;

[0120] Determine the first rhythm corresponding to the target drama style according to a preset mapping relationship between the drama style and the rhythm;

[0121] Comparing the first rhythm with the reference rhythm to obtain a first difference;

[0122] If the first difference is less than or equal to a preset rhythm difference threshold, determining the reference rhythm as the target rhythm;

[0123] If the first difference is greater than the rhythm difference threshold, adjusting the reference rhythm according to the first rhythm to obtain the target rhythm;

[0124] Determining a fullness range and a clarity range of a first timbre corresponding to the target character type;

[0125] Adjusting the reference timbre according to the fullness range and the clarity range to obtain a target timbre;

[0126] Acquire a pitch range of a first tone corresponding to the target emotional state; the pitch range includes a lowest pitch and a highest pitch;

[0127] Adjusting the reference pitch according to the lowest pitch and the highest pitch to obtain a target pitch;

[0128] The target sound effect is determined according to the target rhythm, the target timbre and the target pitch.

[0129] Optionally, the reference rhythm includes a reference rhythm speed and a reference beat rule. In adjusting the reference rhythm according to the first rhythm to obtain the target rhythm, the adjustment module 440 is further specifically used to:

[0130] Acquire a first rhythm speed and a first beat regularity of the first rhythm;

[0131] If the reference rhythm speed is less than the first rhythm speed, obtaining a difference between the first rhythm speed and the reference rhythm speed to obtain a first speed difference;

[0132] determining a first adjustment factor corresponding to the first speed difference;

[0133] Adjusting the reference rhythm speed according to the first adjustment factor to obtain a target rhythm speed;

[0134] If the reference rhythm speed is greater than the first rhythm speed, obtaining a difference between the reference rhythm speed and the first rhythm speed to obtain a second speed difference;

[0135] determining a second adjustment factor corresponding to the second speed difference;

[0136] Adjusting the reference rhythm speed according to the second adjustment factor to obtain the target rhythm speed;

[0137] If the reference rhythm speed is equal to the first rhythm speed, determining the reference rhythm speed as the target rhythm speed;

[0138] The reference beat law is adjusted according to the first beat law to obtain a target beat law; the difference between the target beat law and the first beat law is less than a preset beat difference threshold;

[0139] The target rhythm is determined according to the target rhythm speed and the target beat rule.

[0140] Optionally, in the aspect of adjusting the reference timbre according to the fullness range and the clarity range to obtain a target timbre, the adjustment module 440 is further specifically configured to:

[0141] Obtaining a reference fullness and a reference clarity of the reference timbre;

[0142] If the reference fullness is outside the fullness range, the reference fullness is adjusted to be within the fullness range to obtain a target fullness; otherwise, the reference fullness is determined to be the target fullness;

[0143] If the reference definition is outside the definition range, the reference definition is adjusted to be within the definition range to obtain the target definition; otherwise, the reference definition is determined to be the target definition;

[0144] The target timbre is determined according to the target fullness and the target clarity.

[0145] Optionally, in the aspect of adjusting the reference pitch according to the lowest pitch and the highest pitch to obtain the target pitch, the adjustment module 440 is further specifically configured to:

[0146] If the reference tone is less than the lowest tone, obtaining a tone difference between the lowest tone and the reference tone to obtain a first tone difference;

[0147] Determining a first adjustment parameter corresponding to the first tone difference;

[0148] If the reference tone is greater than the highest tone, obtaining a tone difference between the reference tone and the highest tone to obtain a second tone difference;

[0149] Determining a second adjustment parameter corresponding to the second tone difference;

[0150] Adjust the reference tone according to the first adjustment parameter and / or the second adjustment parameter to obtain the target tone;

[0151] If the reference tone is greater than or equal to the lowest tone and less than or equal to the highest tone, the reference tone is determined to be the target tone.

[0152] It can be seen that by comprehensively analyzing the user's audio data and drama information, combining the style of the drama, role type and emotional state, to generate sound effects that meet the user's personalized needs, and based on the sound effects, the user's audio data is AI voice-changed, thereby improving the accuracy and flexibility of AI voice-changing.

[0153] It should be noted that the specific implementation of each operation can adopt the corresponding description of the method embodiment shown above. The AI ​​voice changing device 400 based on drama culture can be used to execute the above method embodiment of the present application, which will not be repeated here.

[0154] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments, and the above computer includes an electronic device.

[0155] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an electronic device.

[0156] It should be noted that, for the above-mentioned various embodiments, for the sake of simple description, they are all expressed as a series of action combinations. Those skilled in the art should be aware that the present application is not limited by the described order of actions, because some steps in the embodiments of the present application can be performed in other orders or simultaneously. In addition, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions, steps, modules or units involved are not necessarily required by the embodiments of the present application.

[0157] In the above embodiments, the embodiments of the present application have different focuses on the description of each embodiment. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0158] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0159] The steps of the method or algorithm described in the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (electrically EPROM, EEPROM), registers, hard disks, mobile hard disks, read-only compact disks (CD-ROMs) or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a terminal device or a management device. Of course, the processor and the storage medium can also exist in a terminal device or a management device as discrete components.

[0160] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0161] The modules / units included in the devices and products described in the above embodiments may be software modules / units or hardware modules / units, or may be partially software modules / units and partially hardware modules / units. For example, for the devices and products applied to or integrated in the chip, the modules / units included therein may all be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on the processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits; for the devices and products applied to or integrated in the chip module, the modules / units included therein may all be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of software programs. It is implemented in the form of a software program that runs on a processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in hardware such as circuits; for various devices and products applied to or integrated in the terminal equipment, the various modules / units contained therein can be implemented in hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or different components in the terminal equipment, or at least some of the modules / units can be implemented in the form of a software program that runs on a processor integrated inside the terminal equipment, and the remaining (if any) modules / units can be implemented in hardware such as circuits.

[0162] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above description is only the specific implementation method of the embodiments of the present application and is not intended to limit the protection scope of the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. An AI voice changing method based on drama culture, characterized in that: The method comprises: Acquire target audio data and target drama information of a target user; the target drama information includes a target drama genre, a target role type, and a target emotional state; Extracting features from the target audio data to obtain target features; Obtain a sample set corresponding to a preset AI sound effect model; the sample set includes multiple sound effect samples; Obtaining a similarity between a sample feature of each sound effect sample in the plurality of sound effect samples and the target feature to obtain a plurality of similarities; Determine the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect; The reference sound effect is adjusted by the AI ​​sound effect model according to the target drama style, the target character type and the target emotional state to obtain a target sound effect; The target audio data is voice-changed according to the target sound effect to obtain a target voice-changed file.

2. The method according to claim 1, characterized in that The step of extracting features from the target audio data to obtain target features includes: Performing frame processing on the target audio data according to a preset time interval to obtain a plurality of audio frames; Performing fundamental frequency estimation on each of the multiple audio frames to obtain multiple fundamental frequencies; Obtaining a maximum fundamental frequency and a minimum fundamental frequency among the multiple fundamental frequencies; Determine an average value of the plurality of fundamental frequencies as an average fundamental frequency; Performing a fast Fourier transform on the target audio data to obtain a target spectrum; the target spectrum includes a plurality of frequency points and a plurality of energy values; Determine a center of gravity frequency according to the multiple frequency points and the multiple energy values; The target feature is determined according to the average fundamental frequency, the maximum fundamental frequency, the minimum fundamental frequency and the center of gravity frequency.

3. The method according to claim 2, characterized in that The obtaining of the similarity between the sample feature of each sound effect sample in the plurality of sound effect samples and the target feature to obtain a plurality of similarities includes: Acquire a first sample feature of a first sound effect sample; the first sound effect sample is any one of the multiple sound effect samples; the first sample feature includes a first average fundamental frequency, a first maximum fundamental frequency, a first minimum fundamental frequency and a first center of gravity frequency; Calculating the similarity between the target feature and the first sample feature according to a preset cosine similarity calculation formula to obtain a reference similarity; An absolute value of the reference similarity is determined as a first similarity; the first similarity is a similarity corresponding to the first sound effect sample among the multiple similarities.

4. The method according to any one of claims 1 to 3, characterized in that: The step of adjusting the reference sound effect according to the target drama style, the target character type and the target emotional state by the AI ​​sound effect model to obtain the target sound effect includes: Obtaining a reference rhythm, a reference timbre, and a reference pitch of the reference sound effect; Determine the first rhythm corresponding to the target drama style according to a preset mapping relationship between the drama style and the rhythm; Comparing the first rhythm with the reference rhythm to obtain a first difference; If the first difference is less than or equal to a preset rhythm difference threshold, determining the reference rhythm as the target rhythm; If the first difference is greater than the rhythm difference threshold, adjusting the reference rhythm according to the first rhythm to obtain the target rhythm; Determining a fullness range and a clarity range of a first timbre corresponding to the target character type; Adjusting the reference timbre according to the fullness range and the clarity range to obtain a target timbre; Acquire a pitch range of a first tone corresponding to the target emotional state; the pitch range includes a lowest pitch and a highest pitch; Adjusting the reference pitch according to the lowest pitch and the highest pitch to obtain a target pitch; The target sound effect is determined according to the target rhythm, the target timbre and the target pitch.

5. The method according to claim 4, characterized in that The reference rhythm includes a reference rhythm speed and a reference beat rule, and the step of adjusting the reference rhythm according to the first rhythm to obtain the target rhythm includes: Acquire a first rhythm speed and a first beat regularity of the first rhythm; If the reference rhythm speed is less than the first rhythm speed, obtaining a difference between the first rhythm speed and the reference rhythm speed to obtain a first speed difference; determining a first adjustment factor corresponding to the first speed difference; Adjusting the reference rhythm speed according to the first adjustment factor to obtain a target rhythm speed; If the reference rhythm speed is greater than the first rhythm speed, obtaining a difference between the reference rhythm speed and the first rhythm speed to obtain a second speed difference; determining a second adjustment factor corresponding to the second speed difference; Adjusting the reference rhythm speed according to the second adjustment factor to obtain the target rhythm speed; If the reference rhythm speed is equal to the first rhythm speed, determining the reference rhythm speed as the target rhythm speed; The reference beat law is adjusted according to the first beat law to obtain a target beat law; the difference between the target beat law and the first beat law is less than a preset beat difference threshold; The target rhythm is determined according to the target rhythm speed and the target beat rule.

6. The method according to claim 4, characterized in that The step of adjusting the reference timbre according to the fullness range and the clarity range to obtain a target timbre includes: Obtaining a reference fullness and a reference clarity of the reference timbre; If the reference fullness is outside the fullness range, the reference fullness is adjusted to be within the fullness range to obtain a target fullness; otherwise, the reference fullness is determined to be the target fullness; If the reference definition is outside the definition range, the reference definition is adjusted to be within the definition range to obtain the target definition; otherwise, the reference definition is determined to be the target definition; The target timbre is determined according to the target fullness and the target clarity.

7. The method according to claim 4, characterized in that The step of adjusting the reference pitch according to the lowest pitch and the highest pitch to obtain a target pitch includes: If the reference tone is less than the lowest tone, obtaining a tone difference between the lowest tone and the reference tone to obtain a first tone difference; Determining a first adjustment parameter corresponding to the first tone difference; If the reference tone is greater than the highest tone, obtaining a tone difference between the reference tone and the highest tone to obtain a second tone difference; Determining a second adjustment parameter corresponding to the second tone difference; Adjust the reference tone according to the first adjustment parameter and / or the second adjustment parameter to obtain the target tone; If the reference tone is greater than or equal to the lowest tone and less than or equal to the highest tone, the reference tone is determined to be the target tone.

8. An AI voice changing device based on drama culture, characterized in that: The device includes an acquisition module, a processing module, a determination module, an adjustment module and a voice changing module, wherein: The acquisition module is used to acquire the target audio data and target drama information of the target user; the target drama information includes the target drama style, target role type and target emotional state; The processing module is used to extract features from the target audio data to obtain target features; The acquisition module is further used to acquire a sample set corresponding to a preset AI sound effect model; the sample set includes a plurality of sound effect samples; The acquisition module is further used to acquire the similarity between the sample feature of each sound effect sample in the multiple sound effect samples and the target feature to obtain multiple similarities; The determination module is used to determine the sound effect sample corresponding to the maximum similarity among the multiple similarities as the reference sound effect; The adjustment module is used to adjust the reference sound effect according to the target drama style, the target character type and the target emotional state through the AI ​​sound effect model to obtain the target sound effect; The voice changing module is used to perform voice changing processing on the target audio data according to the target sound effect to obtain a target voice changing file.

9. An electronic device, characterized in that: include: A processor, a memory, a communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for executing the steps in the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images