An audio processing method, an electronic device, and a readable storage medium

By extracting feature parameters from user audio data and combining them with an AI composition model, the problem of generating personalized music in existing technologies has been solved, enabling personalized music creation without the need for manual parameter settings.

CN119920231BActive Publication Date: 2026-11-10HUAWEI TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202311444262.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-31
Publication Date
2026-11-10
Estimated Expiration
2043-10-31

AI Technical Summary

Technical Problem

Existing music creation technology cannot provide users with personalized music works. Users need to manually set multiple parameters to generate songs, and it cannot take into account the user's timbre characteristics.

Method used

By acquiring audio data of user speech, user feature parameters such as timbre, pitch, and speech rate are extracted. Pre-trained algorithms are used to determine music feature parameters such as musical emotion, style, and rhythm. Composition is then carried out using an AI composition model, and music simulating the user's timbre is synthesized through speech synthesis technology.

Benefits of technology

It enables personalized music creation, eliminating the need for users to manually set parameters. The generated music matches the user's vocal characteristics, satisfying personalized creative needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119920231B_ABST
    Figure CN119920231B_ABST
Patent Text Reader

Abstract

This application relates to the field of speech recognition, and particularly to an audio processing method, an electronic device, and a readable storage medium. The audio processing method includes: processing audio data input by a user in the form of reading aloud to obtain user feature parameters such as speech rate, pitch, and timbre; and based on multiple user feature parameters, using a pre-trained algorithm to obtain corresponding musical feature parameters of the audio data, such as musical emotion, musical style, musical rhythm, and musical tonality. Then, the aforementioned musical feature parameters and the corresponding audio text are input into an AI composition model for music composition, resulting in musical data corresponding to the audio data. Finally, speech synthesis technology is used to synthesize the user feature parameters such as timbre and pitch with the musical data obtained based on the AI ​​composition model to obtain a musical work simulating the user's timbre. In this way, music simulating the user's singing can be obtained based on the user's audio, providing a way for users to create personalized music.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of speech recognition, and in particular to an audio processing method, an electronic device, and a readable storage medium. Background Technology

[0002] Artificial intelligence (AI) has functions such as speech recognition, image recognition, natural language processing, and music creation. For example, AI's music creation function can include music recognition, algorithmic composition, and automatic music arrangement.

[0003] Specifically, in some technical solutions, users can input a fixed melody into an application with a music model. The application can then convert the melody's timbre into a specific instrument timbre, such as selecting a violin timbre, thus creating a musical piece played on a violin. In other technical solutions, users can input both melody and text into an application with a music model, resulting in a song synthesized from the melody and text. It can be seen that these music creation functions can only convert user-input melodies into specific instrument timbres, or perform simple, preset combinations of user-input melodies and text to create songs; they do not offer personalized creation options for users. Summary of the Invention

[0004] To address the aforementioned issues, embodiments of this application provide an audio processing method, an electronic device, and a readable storage medium.

[0005] In a first aspect, embodiments of this application provide an audio processing method applied to an electronic device. The method includes: acquiring audio data including user speech; determining user feature parameters of the audio data; determining music feature parameters corresponding to the audio data based on the user feature parameters of the audio data; and obtaining music data corresponding to the audio data based on the music feature parameters.

[0006] It is understood that user feature parameters may be timbre, pitch, speech rate, volume, etc. mentioned in the embodiments of this application, and music feature parameters may be music style, music emotion, music rhythm, and music tonality, etc. mentioned in the embodiments of this application.

[0007] Based on the above solution, the system can acquire arrangement features such as musical emotion, style, rhythm, and tonality—i.e., musical characteristic parameters—from user-recorded or uploaded audio data. These features, combined with the user's vocal characteristics (e.g., timbre), enable personalized music creation to meet individual user needs. Furthermore, it eliminates the need for users to manually set various parameters for song generation, such as emotion, rhythm, style, instruments, chords, and lyrics, thus facilitating personalized music creation.

[0008] In one possible implementation of the first aspect above, acquiring audio data including user speech includes: acquiring audio data including user speech obtained by the user through reading aloud or humming.

[0009] In one possible implementation of the first aspect above, determining the user feature parameters of the audio data includes: performing frame segmentation, windowing, and Mel-frequency cepstral coefficient feature extraction on the audio data including the user's speech to obtain the user feature parameters corresponding to the audio data, wherein the user feature parameters include the user's timbre, pitch, volume, or speech rate.

[0010] In one possible implementation of the first aspect above, the music feature parameters include music style; based on the user feature parameters of the audio data, the music feature parameters corresponding to the audio data are determined, including: performing semantic analysis on the audio text corresponding to the audio data through a pre-trained semantic recognition algorithm to determine the music style corresponding to the audio data.

[0011] In one possible implementation of the first aspect above, the music feature parameters include music emotion; determining the music feature parameters corresponding to the audio data based on the user feature parameters of the audio data also includes: classifying the audio data using a pre-trained emotion recognition algorithm to determine the music emotion corresponding to the audio data; the pre-trained emotion recognition algorithm is an emotion analysis method based on an emotion dictionary, which extracts emotion words from the audio text corresponding to the audio data, calculates the total weight of the audio text based on the weight of the emotion tendency corresponding to the extracted emotion words, and determines the music emotion corresponding to the audio data through the total weight of the audio text.

[0012] In one possible implementation of the first aspect above, the music feature parameters include music rhythm; based on the user feature parameters of the audio data, the music feature parameters corresponding to the audio data are determined, and the method further includes: inputting speech rate and music style into a first mapping function, wherein the first mapping function is a mapping function of music rhythm, music style and speech rate, and then determining the music rhythm through the first mapping function.

[0013] In one possible implementation of the first aspect above, determining the music feature parameters corresponding to the audio data based on the user feature parameters of the audio data further includes: the music feature parameters include the tonality; inputting the pitch and music style into a second mapping function, the second mapping function being a mapping function of music tonality, music style and pitch, then determining the music tonality through the second mapping function.

[0014] In one possible implementation of the first aspect above, obtaining music data corresponding to the audio data based on music feature parameters includes: inputting the music style, music emotion, music rhythm, music tonality and the audio text corresponding to the audio data into a first composition model, and the first composition model obtaining first music data corresponding to the audio data based on the music style, music emotion, music rhythm, music tonality and the audio text corresponding to the audio data.

[0015] It is understood that the first composition model can be the AI ​​composition model mentioned in the embodiments of this application, and the first music data can be the arranged music obtained by the AI ​​composition model mentioned in the embodiments of this application.

[0016] In one possible implementation of the first aspect above, the music data obtained based on music feature parameters and corresponding audio data further includes: synthesizing the first music data and user feature parameters through speech synthesis technology to obtain second music data with user voice features.

[0017] It is understood that the second music data may be the music data of simulated user timbre, pitch, etc., as mentioned in the embodiments of this application.

[0018] In one possible implementation of the first aspect above, the method further includes: acquiring music data including user voice; determining user feature parameters corresponding to the music data; determining audio feature parameters corresponding to the music data based on the music data; and obtaining audio data corresponding to the music data based on the audio feature parameters.

[0019] Based on the above scheme, it is possible to obtain the corresponding audio data (reading audio) based on the music data recorded or uploaded by the user, and the audio data is an audio that simulates the user's voice reading aloud.

[0020] In other words, the audio processing method provided in this application embodiment can obtain music data corresponding to the audio data based on the audio data recorded or uploaded by the user, and the music data is music sung in a simulated user voice. It can also realize bidirectional conversion between song creation and user reading to meet the user's personalized creation needs.

[0021] In one possible implementation of the first aspect above, the audio feature parameters corresponding to the music data are determined based on the music data, including: the audio feature parameters corresponding to the music data include audio emotion; the audio text corresponding to the music data is obtained through a pre-trained speech processing algorithm; based on the audio text, the audio emotion corresponding to the music data is determined through a pre-trained emotion recognition algorithm; the audio data corresponding to the music data is obtained based on the audio feature parameters, including: synthesizing the audio text, audio emotion, and user feature parameters through speech synthesis technology to obtain audio data with user voice characteristics, wherein the audio data is an audio simulating a user reading the audio text.

[0022] In a second aspect, embodiments of this application provide an electronic device, including: a memory for storing instructions; and a processor for executing the instructions to cause the electronic device to implement the first aspect and any possible implementation of the provided audio processing method.

[0023] Thirdly, embodiments of this application provide a readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the first aspect and any possible implementation of the provided audio processing method. Attached Figure Description

[0024] Figure 1 According to some embodiments provided in this application, a schematic diagram of the interface of an online program is shown;

[0025] Figure 2 According to some embodiments provided in this application, a schematic diagram of the framework of an audio processing system is shown;

[0026] Figure 3 According to some embodiments provided in this application, a schematic diagram of a sentiment analysis method based on a sentiment dictionary is shown;

[0027] Figure 4 According to some embodiments provided in this application, a flowchart of an audio processing method is shown;

[0028] Figure 5 According to some embodiments provided in this application, a simplified schematic diagram of determining music feature parameters based on audio data is shown;

[0029] Figure 6 According to some embodiments provided in this application, a flowchart of another audio processing method is shown;

[0030] Figure 7 A simplified design flow diagram of an audio processing method is shown based on some embodiments provided in this application.

[0031] Figure 8 According to some embodiments provided in this application, a schematic diagram of the pre-training process of an emotion recognition model is shown;

[0032] Figure 9 According to some embodiments provided in this application, a schematic diagram of the pre-training process of a semantic analysis model is shown;

[0033] Figure 10 According to some embodiments provided in this application, a schematic diagram of the pre-training process of an AI speech synthesis model is shown. Detailed Implementation

[0034] The illustrative embodiments of this application include, but are not limited to, an audio processing method, an electronic device, and a readable storage medium.

[0035] It is understood that the audio processing method provided in this application embodiment can be applied to any electronic device, including but not limited to mobile phones, tablets, laptops, desktop computers, smart wearable devices and any other electronic devices that support recording functions, and is not limited here.

[0036] The technical solution of this application is described below with reference to the accompanying drawings.

[0037] As mentioned earlier, in some technical solutions, users can input a fixed melody into an application with a music model, and the application can then convert the melody's timbre into the timbre of a specific instrument. For example, an audio file can be converted into music using a ToneTransfer program. ToneTransfer, an online program developed by Google, can convert the timbre of any audio file into the timbre of four instruments: flute, saxophone, trumpet, and violin. Figure 1 A schematic diagram of a Tone Transfer interface is shown. In Tone Transfer interface 200, users can upload audio files to be processed by clicking the "Record or Upload" control, or hum aloud using a microphone and upload it, i.e., inputting the melody of the audio to be processed into Tone Transfer. Further, in Tone Transfer interface 201, users can select the desired instrument timbre by clicking the instrument timbre selection controls: "Flute," "Saxophone," "Trumpet," and "Violin." Tone Transfer can then convert the aforementioned audio files or recordings into music using any of the four timbre options and play it.

[0038] However, Tone Transfer transmits pitch and amplitude information from audio files at a 1:1 ratio. If the pronunciation of the input audio sample cannot be expressed by the four preset instrument timbres, the Tone Transfer algorithm will guess its pronunciation and play unnatural music; for example, playing a high-frequency sound from an audio sample on a violin will sound harsh. Furthermore, Tone Transfer can only simply replace the audio timbres with the timbres of the four preset instruments, and the input song melody is fixed, without involving arrangement.

[0039] As mentioned earlier, in other technical solutions, users can also input melody and text into an application with a music model to obtain a song synthesized from the melody and text. For example, an audio file can be converted into music using an audio generation model (MusicLM). MusicLM has been trained on a 280,000-hour music dataset and is able to generate corresponding music based on the input text description; users can input text and melody (in the form of humming, singing, whistling, or playing an instrument) into MusicLM to obtain music that follows the melody and text description.

[0040] However, MusicLM, which generates music based on the input text description and melody, lacks the user's own vocal characteristics and uses the user-provided melody, failing to offer personalized musical compositions. If a user wants to create something unique, they need to manually adjust various parameters used to generate the song, such as musical emotion, rhythm, and style, before the song can be generated. Manually setting these parameters is difficult for users without professional music knowledge.

[0041] To address the aforementioned issues, this application provides an audio processing method. In this method, user-input audio data, such as speech rate, pitch, and timbre, are processed to obtain user feature parameters (i.e., audio feature parameters). Based on these multiple user feature parameters, a pre-trained algorithm is used to obtain corresponding music feature parameters, such as musical emotion, musical style, musical rhythm, and musical tonality. Then, the aforementioned music feature parameters and the corresponding audio text are input into an AI composition model for music arrangement, resulting in music data (i.e., musical melody) corresponding to the audio data. Finally, speech synthesis technology is used to synthesize the user feature parameters, such as timbre and pitch, with the music data obtained from the AI ​​composition model to produce a musical work simulating the user's timbre.

[0042] Based on the above solution, the arrangement features corresponding to the audio data recorded or uploaded by the user, such as musical emotion, musical style, musical rhythm, and musical tonality, can be obtained, that is, musical feature parameters. Combined with the user's voice characteristics (such as timbre), personalized music can be created. Users do not need to manually set various parameters used to generate songs, such as song emotion, rhythm, musical style, instruments, chords, lyrics, etc.

[0043] Specifically, in some embodiments, users record audio data by reading aloud, humming, etc., and then process the audio data using methods such as frame segmentation, windowing, and mel frequency cepstrum coefficient (MFCC) feature extraction to obtain user feature parameters such as speech rate, volume, timbre, and pitch. Based on these user feature parameters, the corresponding music feature parameters can be obtained. First, the audio data is classified using a pre-trained semantic recognition algorithm. Based on the semantics of the corresponding audio text, the music style parameter is determined. For example, the music style parameter M1 is determined based on the semantic recognition algorithm, and the music style parameter M1 corresponds to Chinese style music. Next, the audio data is classified using a pre-trained emotion recognition algorithm to determine the corresponding music emotion parameter. For example, the music emotion parameter N1 is determined based on the emotion recognition algorithm, and N1 corresponds to the music emotion of pleasure. Further, based on the mapping function of speech rate, music style, and music rhythm, the music rhythm corresponding to the audio data is determined. For example, the music rhythm can be 60 beats per minute. Then, based on the mapping function of pitch, musical style, and tonality, the tonality corresponding to the audio data is determined. For example, the tonality is F major. Finally, the above musical feature parameters and the corresponding audio text of the audio data are input into the AI ​​composition model to compose music, resulting in music corresponding to the audio data. Then, speech synthesis technology is used to synthesize the user's feature parameters (such as timbre and pitch) with the music obtained based on the AI ​​composition model to obtain a musical work simulating the user's singing.

[0044] For example, Figure 2 A schematic diagram of the framework of an audio processing system is shown. (Reference) Figure 2 Users can record audio data through music or recording apps, and then input the recorded audio data into the recording analysis module for processing to obtain user characteristic parameters. The recording analysis module can include audio sentiment analysis, audio speech rate analysis, audio semantic analysis, and pitch analysis. Specifically, music sentiment analysis maps to musical emotions; audio speech rate analysis maps to musical rhythm; audio semantic analysis maps to musical style; and pitch analysis maps to musical tonality.

[0045] First, the recording analysis module needs to perform frame segmentation, windowing, and MFCC feature extraction on the audio data to obtain user feature parameters, including timbre, pitch, volume, and speech rate. Frame segmentation divides the audio data into multiple frames, each ranging from 10ms to 50ms. Windowing uses a window function to apply a rectangular window to the multi-frame audio data, truncating the frames in the time domain to ensure they meet the periodicity requirements of Fourier transform processing. MFCC feature extraction involves performing a Fast Fourier Transform on the segmented and windowed audio data to convert it from a time-domain signal to a frequency-domain signal. Then, a Mel spectrogram is obtained based on the Mel filter bank, with time on the horizontal axis and frequency on the vertical axis. Based on the Mel spectrogram, logarithmic operations on the audio data's spectrum yield the envelope and detail; the envelope represents timbre, and the detail represents pitch. Furthermore, an inverse Fourier transform (ceptomography) is performed on the spectrum after logarithmic operation to obtain the cepstral plot, from which the timbre corresponding to the audio data can be directly obtained.

[0046] Audio sentiment analysis can utilize sentiment analysis methods based on sentiment dictionaries to determine the musical sentiment corresponding to a user's audio. Sentiment analysis methods based on sentiment dictionaries refer to extracting sentiment words from the audio text corresponding to the audio data to determine the musical sentiment. For example... Figure 3 , Figure 3 A schematic diagram of a sentiment analysis method based on a sentiment dictionary is shown. (Reference) Figure 3 First, the audio text corresponding to the audio data undergoes preprocessing, specifically word segmentation and extraction of sentiment words. Then, these sentiment words are matched against words in a pre-trained sentiment lexicon, where each word has a corresponding weight indicating its sentiment tendency. For example, a positive sentiment word has a weight of "1," while a negative sentiment word has a weight of "-1." Finally, based on pre-defined judgment rules, the overall weight of the audio text determines the musical sentiment corresponding to the audio data.

[0047] It is understandable that sentiment analysis methods based on sentiment vocabularies include positive sentiment vocabularies, negative sentiment vocabularies, and negation sentiment vocabularies. Therefore, the sentiment vocabulary includes positive sentiment words, negative sentiment words, and negation sentiment words. For example, negation sentiment words may include "no," positive sentiment words may include "like" and "love," and negative sentiment words may include "dislike" and "hate," etc., without further limitation.

[0048] It is understood that in other embodiments, machine learning-based sentiment analysis methods can also be used to determine the musical sentiment corresponding to the audio data. Machine learning-based sentiment analysis methods refer to selecting sentiment words in the audio data as feature words, matrixing the text, and then using methods such as logistic regression, Naive Bayes, and support vector machines (SVM) to classify the audio text to determine the musical sentiment. No further limitations are specified here.

[0049] Audio semantic analysis utilizes pre-trained semantic recognition algorithms to analyze the semantics of audio text corresponding to audio data, thereby determining the corresponding musical style. In essence, semantic recognition algorithms enable computers to understand the meaning of human language and classify the semantics of audio text, determining the corresponding arrangement style, or musical genre.

[0050] It is understandable that audio semantic analysis can directly obtain the audio text (i.e. lyrics) corresponding to the audio data.

[0051] It is understandable that audio semantic analysis can also be combined with the musical styles of top-tier songs that users frequently listen to in music applications, and this is not a limitation here.

[0052] It is understandable that musical styles can include traditional Chinese music, pop music, hip-hop music, jazz music, etc., and no specific restrictions are imposed here.

[0053] Audio speech rate analysis and pitch analysis can be obtained based on the above-mentioned frame segmentation, windowing, and MFCC feature extraction.

[0054] Since the speech rate and musical style are known, the musical rhythm corresponding to the audio data can be determined through a mapping function of musical rhythm, musical style, and speech rate. In some embodiments, the mapping function of musical rhythm is assumed to be y1 = Ax1 + Bx2 + C, where y1 represents musical rhythm, x1 represents speech rate, x2 represents musical style, and A, B, and C are preset mapping coefficients. Therefore, given the speech rate and musical style, the musical rhythm can be obtained based on the mapping function of musical rhythm. For example, the musical rhythm obtained from the mapping function of musical rhythm is y1 = 60, which means the musical rhythm is 60 beats per minute.

[0055] It is understood that in other embodiments, the music rhythm can also be determined based on a recommendation algorithm by combining the music rhythm of songs that the user has previously liked and the music rhythm of songs that the user has created, and this is not limited here.

[0056] Since the pitch and musical style are known, the musical tonality corresponding to the audio data can be determined using a mapping function of musical tonality, musical style, and pitch. In some embodiments, the mapping function of musical rhythm is assumed to be y2 = Ex3 + Fy1 + G, where y2 represents musical tonality, ex3 represents pitch, y1 represents musical style, and E, F, and G are preset mapping coefficients. Therefore, given the pitch and musical style, the musical tonality can be obtained based on the mapping function of musical rhythm. For example, the musical tonality obtained based on the mapping function of musical tonality is F major.

[0057] Audio detail features refer to the user-specific parameters corresponding to the audio data, such as pitch, timbre, phonemes, loudness, and Mel-ray spectrum. Phonemes can be divided into vowels and consonants; for example, a Chinese syllable (ā) has only one phoneme, a syllable (ài) has two phonemes, and a syllable (dài) has three phonemes. Loudness represents volume.

[0058] After analyzing the audio data using the aforementioned recording analysis module, the corresponding musical feature parameters can be obtained. Furthermore, musical emotion, rhythm, style, tonality, and lyrics (audio text) can be input into the AI ​​composition model to obtain the corresponding music (i.e., song). The AI ​​composition model refers to an algorithm that takes musical feature parameters such as lyrics, style, tonality, and rhythm as input and outputs a song corresponding to those parameters. For example, the AI ​​composition model could be an AI Singer model; this is not a limitation. Finally, speech synthesis technology is used to synthesize the user's characteristic parameters (such as timbre and pitch) corresponding to the audio data with the music obtained from the AI ​​composition model, thus synthesizing the user's timbre and other voice features with the song to obtain a musical work simulating the user's voice.

[0059] To further understand the audio processing method provided in the embodiments of this application, Figure 4 A flowchart illustrating an audio processing method is shown. (Reference) Figure 4 The specific process includes:

[0060] S101: Acquire audio data.

[0061] In some embodiments, users can record audio data using a music or recording app, such as by reading aloud or humming. Alternatively, users can upload pre-recorded audio data that requires processing to the music app. The music app can then process the audio data recorded or uploaded by the user.

[0062] S102: Perform preprocessing on the audio data, such as frame segmentation, windowing, and noise reduction, to obtain user characteristic parameters such as speech rate, volume, timbre, and pitch corresponding to the audio data.

[0063] In some embodiments, the audio data obtained above is preprocessed, which may include framing, windowing, noise reduction, and MFCC feature extraction. First, the audio data is segmented into multiple frames of fixed size, with each frame typically ranging from 10ms to 30ms in length. That is, the audio data is divided into multiple frames of 10ms to 50ms each. Then, each frame is windowed, using a window function to apply a rectangular window to each frame, truncating each frame in the time domain to ensure that each frame meets the periodicity requirement of Fourier transform processing. Furthermore, noise reduction is performed to filter out steady-state and non-steady-state noise in the audio data. Finally, MFCC feature extraction is performed on the audio data. MFCC refers to performing a Fast Fourier Transform (FFT) on the audio data after frame segmentation, windowing, and noise reduction to convert the audio data from the time domain to the frequency domain. Then, a Mel spectrogram is obtained based on the Mel filter bank, where the horizontal axis of the Mel spectrogram is time and the vertical axis is frequency. Based on the Mel spectrogram, a logarithmic operation is performed on the spectrum of the audio data to obtain the envelope and detail, where the envelope represents timbre and the detail represents pitch. Then, an inverse Fourier transform (i.e., cepstrum) is performed on the logarithmically processed spectrum to obtain the cepstrum. The pitch, timbre, and loudness (i.e., volume) of the audio data can be directly obtained from the cepstrum.

[0064] S103: Classify audio data using a pre-trained semantic recognition algorithm, and determine the music style corresponding to the audio data based on the semantics of the audio text corresponding to the audio data.

[0065] In some embodiments, semantic analysis is performed on the audio text corresponding to the audio data. Audio semantic analysis refers to using a pre-trained semantic recognition algorithm to analyze the semantics of the audio text corresponding to the audio data to determine the corresponding musical style. Semantic recognition algorithms can be used to enable computers to understand the meaning of human language and classify the semantics of text to determine the arrangement style (i.e., musical style) corresponding to the text semantics. Therefore, the musical style corresponding to the audio data can be determined based on the semantic recognition algorithm. For example, if the musical style parameter M1 corresponding to the audio data is determined based on the semantic recognition algorithm, and the musical style corresponding to the musical style parameter M1 in the semantic recognition algorithm is Chinese style music, then the musical style corresponding to the audio data can be determined to be Chinese style music. It can be understood that musical style refers to musical genre. Based on audio semantic analysis, detailed audio text (i.e., lyrics) corresponding to the audio data can be obtained.

[0066] It is understandable that musical styles can include traditional Chinese music, pop music, hip-hop music, jazz music, etc., and no specific restrictions are imposed here.

[0067] S104: Classify audio data using a pre-trained emotion recognition algorithm to determine the corresponding musical emotion.

[0068] In some embodiments, sentiment analysis methods based on sentiment dictionaries can be used to determine the musical sentiment corresponding to audio data. Sentiment analysis methods based on sentiment dictionaries refer to determining the sentiment tendency of an audio text by extracting sentiment words from the audio text corresponding to the audio data, based on a pre-constructed sentiment dictionary, thereby determining the musical sentiment corresponding to the audio data.

[0069] For example, the above Figure 3 This diagram illustrates a sentiment analysis method based on a sentiment dictionary. First, the sentences in the audio text corresponding to the audio data are preprocessed, specifically by segmenting the audio data and extracting sentiment words. Then, these sentiment words are matched against words in the sentiment dictionary, a pre-trained lexicon where each word has a corresponding weight indicating sentiment tendency. For example, a positive sentiment word has a weight of "1", while a negative sentiment word has a weight of "-1". Finally, the musical sentiment corresponding to the audio data is determined based on a pre-defined judgment rule, specifically the positive or negative weight of the total weight corresponding to the audio text.

[0070] It is understandable that sentiment analysis methods based on sentiment lexicons include positive sentiment lexicons, negative sentiment lexicons, and negation sentiment lexicons. The sentiment lexicons include positive sentiment words such as "like" and "love"; negative sentiment words such as "dislike" and "hate"; and negation sentiment words such as "no," without further elaboration here.

[0071] S105: Determine the music rhythm corresponding to the audio data through a mapping function of music rhythm, speech rate, and music style.

[0072] In some embodiments, based on a known speech rate and musical style, the musical rhythm corresponding to the audio data can be determined using a mapping function between musical rhythm and musical style / speech rate. Assuming the musical rhythm mapping function is y1 = Ax1 + Bx2 + C, where y1 represents musical rhythm, x1 represents speech rate, x2 represents musical style, and A, B, and C are preset mapping coefficients, the musical rhythm can be obtained based on the musical rhythm mapping function when the speech rate and musical style are known. For example, if the musical rhythm y1 obtained through the musical rhythm mapping function is 60, then the musical rhythm is 60 beats per minute.

[0073] It is understood that in other embodiments, the music rhythm can also be determined based on a recommendation algorithm by combining the music rhythm of songs that the user has previously liked and the music rhythm of songs that the user has created, and this is not limited here.

[0074] S106: Determine the musical tonality corresponding to the audio data through a mapping function of pitch, tone, and musical style.

[0075] In some embodiments, based on a known pitch and musical style, the key of the user's audio can be determined using a mapping function between musical tonality and musical style / pitch. Assuming the mapping function for musical rhythm is y2 = Ex3 + Fy1 + G, where y2 represents musical tonality, x3 represents pitch, y1 represents musical style, and E, F, and G are preset mapping coefficients, then the musical tonality can be obtained based on the mapping function for musical rhythm, given the known pitch and musical style. For example, the musical tonality obtained through the mapping function for musical tonality is F major.

[0076] S107: Create music based on musical style, musical emotion, musical rhythm, musical tonality and audio text.

[0077] In some embodiments, musical emotion, rhythm, style, tonality, and lyrics (audio text) can be input into an AI composition model to obtain music (i.e., a song) corresponding to the audio data. For example, if the musical style is Chinese folk music, the musical emotion is pleasant, the tonality is F major, and the rhythm is 60 beats per minute, then the AI ​​composition model can create music (i.e., a song) corresponding to the audio data based on the above musical feature parameters.

[0078] In essence, an AI composition model refers to an algorithmic model that takes musical feature parameters such as lyrics, musical style, key, and rhythm as input and outputs a song corresponding to those parameters. For example, an AI composition model could be the AISinger model, but this is not a limitation.

[0079] S108: Through speech synthesis technology, the user feature parameters corresponding to the audio data are synthesized with music created based on music feature parameters to obtain a musical work that simulates the user's singing.

[0080] In some embodiments, music (i.e., songs) corresponding to audio data is obtained through an AI composition model. In order to obtain musical works that can simulate user singing, speech synthesis technology can be further used to synthesize user feature parameters (such as timbre and pitch) corresponding to audio data with music obtained based on AI composition model, so as to synthesize user voice features such as timbre and pitch with music and obtain musical works that simulate user singing.

[0081] It can be understood that the processing procedure of the speech synthesis technology may include a language analysis part and an acoustic system part. The language analysis part mainly performs analysis based on the audio text to generate a corresponding linguistic specification, that is, how to sing the audio text; the acoustic system part mainly generates corresponding music according to the phonetic specification provided by the speech analysis part, so as to realize the vocalization function. Specifically, the language analysis part mainly performs text structure and language type judgment, text normalization, text-to-phoneme conversion, and prosody prediction of pauses on the basis of the audio text. The text structure and language type judgment refers to determining the language type of the audio text and splitting the audio text into individual sentences according to the grammatical rules of the corresponding language. The text normalization refers to converting Arabic numerals or letters in the audio text into characters according to preset rules, so as to normalize the synthesized text. For example, "8787" is converted into the Chinese characters "八七八七". The text-to-phoneme conversion refers to converting the characters in the audio text into corresponding pinyin by labeling the characters with pinyin, and determining the pronunciation and tone for polyphonic characters through word segmentation and part-of-speech syntactic analysis. For example, "希望在前方" is represented as "xi1wang4zai4qian2fang1". The prosody prediction of pauses refers to performing prosody prediction on the audio text to determine pause positions and pause durations, as well as the stress and unstressed pronunciation of words, so as to realize the cadence of music.

[0082] The acoustic system part uses a vocoder to synthesize the acoustic features obtained based on the language analysis part with the user's voice features such as timbre and pitch, and convert the result into a waveform, which is the music simulating the user's singing.

[0083] Through the above solution, the music feature parameters corresponding to the audio data can be acquired based on the audio data recorded or uploaded by the user for music arrangement creation, and then the user feature parameters such as the user's timbre are synthesized with the created music to obtain the music data corresponding to the audio data, which is the music sung in the simulated user timbre. The solution meets the personalized music creation demands of users, and users can obtain personalized music works with their own voice characteristics.

[0084] For example, Figure 5This diagram illustrates a simplified approach to determining music feature parameters based on audio data. Specifically, it involves feature analysis and feature mapping of user-recorded or uploaded audio data (audio files). Semantic analysis of the audio text corresponding to the user's audio data can determine the corresponding musical style, which can then be further determined through feature mapping. Emotion recognition of the audio data can determine the corresponding musical emotion, which can then be further determined through feature mapping. Pitch recognition of the audio data can determine the pitch, which can then be combined with musical style to determine the musical tonality. Speech rate analysis of the audio data can determine the speech rate, which can then be combined with musical style to determine the musical rhythm. Feature analysis also includes text conversion of the audio data to obtain the corresponding audio text, which can then determine the lyrics. Based on this, a series of music feature parameters are determined based on the audio data.

[0085] It is understood that in some other embodiments, users can manually change instruments, as different instruments have different timbres, and instrument timbres can also be incorporated into the music arrangement to enrich the music arrangement.

[0086] In other embodiments provided in this application, audio data simulating user reading aloud can also be obtained based on music data sung or uploaded by the user. For example... Figure 6 , Figure 6 A flowchart illustrating another audio processing method is shown. (Reference) Figure 6 The specific process includes:

[0087] S201: Obtain music data.

[0088] In some embodiments, users can use a music app or recording app to sing and record a piece of music, such as by humming. Alternatively, users can upload pre-recorded music data that needs processing to the music app. The music app can then process the music data.

[0089] S202: Perform preprocessing on the music data, such as frame segmentation and windowing, to obtain user characteristic parameters such as timbre and pitch.

[0090] In some embodiments, the music data sung by the user undergoes framing, windowing, and MFCC feature extraction. First, the music data is segmented into multiple fixed-size frames, each typically 10ms to 30ms in length. Then, each frame is windowed using a window function, applying a rectangular window to each frame to truncate it in the time domain, ensuring that each frame meets the periodicity requirement of Fourier transform processing. Finally, MFCC feature extraction is performed on the music data to obtain user characteristic parameters such as timbre and pitch.

[0091] S203: Convert music data into audio text (read-aloud text) using a pre-trained speech processing algorithm.

[0092] In some embodiments, a pre-trained speech processing algorithm can process the user's singing music data to convert it into text that can be read aloud. It can be understood that the pre-trained speech processing algorithm's music data-to-text conversion process mainly consists of two stages: a signal processing stage and a signal recognition stage. In the signal processing stage, the music data is converted from analog to digital form that a computer can process, and preprocessing such as noise reduction is performed to improve the accuracy of music data recognition. In the signal recognition stage, the processed music data can be input into a speech recognition model, thereby directly obtaining the corresponding audio text, i.e., the text to be read aloud, based on the speech recognition model.

[0093] It is understood that speech recognition models can be based on Hidden Markov Models (HMMs), Recurrent Neural Networks (RNNs) based on deep learning, Long Short-Term Memory Networks (LSTMs), and Transcription Attention Models, etc., and no specific limitation is made here.

[0094] S204: Determine the audio emotion corresponding to the music data through a pre-trained emotion recognition algorithm.

[0095] In some embodiments, a pre-trained emotion recognition algorithm can be used to determine the audio emotion corresponding to music data. The pre-trained speech processing algorithm can determine the audio emotion corresponding to the music data based on keywords (e.g., emotion words) in the corresponding music text. For example, if the music text contains a large number of positive words such as "happiness" and "hope," then "happiness" and "hope" can be used as keywords. Based on the positive emotion tendency of "happiness" and "hope," the emotion of the read text can be determined to be pleasant.

[0096] S205: By using speech synthesis technology, audio text, audio emotion, and user feature parameters are synthesized to obtain audio that simulates a user reading aloud.

[0097] In some embodiments, based on the determined audio text and audio emotion, the user's timbre, pitch and other user feature parameters can be synthesized with the audio text and audio emotion through speech synthesis technology. That is, based on the audio emotion, the audio text is read aloud with the user's timbre and other user feature parameters to obtain audio that simulates the user's timbre reading.

[0098] Based on the above solutions, it is possible to obtain music data corresponding to the audio data recorded or uploaded by the user, and this music data is music sung in a simulated user voice. Alternatively, it is possible to obtain audio data (reading audio) corresponding to the music data recorded or uploaded by the user, and this audio data is audio read aloud in a simulated user voice. In other words, the audio processing method provided in this application embodiment can achieve bidirectional conversion between song creation and user reading aloud, to meet the user's personalized creation needs.

[0099] The following is based on Figure 7 The schematic diagram shown illustrates a simplified design flow of the audio processing method provided in this application embodiment.

[0100] First, the main idea of ​​the audio processing method provided in this application is to create musical works with the user's voice characteristics from the user's audio data. Therefore, based on certain technical prerequisites, this application mainly uses various pre-trained models when processing audio, such as emotion recognition (algorithm) models, semantic recognition (algorithm) models, AI onomatopoeia / vocalization techniques, AI music composition techniques, etc. Furthermore, various training sets are used when training the models, such as music preferred by the user, music collections previously created by the user, and audio files of different instruments. The pre-training process of each model will be briefly described below, and will not be elaborated upon here.

[0101] Secondly, after obtaining user feature parameters (such as timbre, pitch, etc.), audio text (lyrics) corresponding to audio data, and music feature parameters (such as emotion, style, etc.) corresponding to audio data through pre-trained models, these feature parameters are input into the AI ​​composition model for music arrangement and creation. During the arrangement and creation process, the instruments in the arrangement can be changed, and the timbres of different types of instruments can be incorporated into the arrangement, making the final music arrangement richer.

[0102] Finally, speech synthesis technology can be used to synthesize the user's voice with music created by an AI composition model to obtain music data that simulates the user's singing.

[0103] The pre-training process of the emotion recognition (algorithm) model mentioned in the embodiments of this application is briefly described below.

[0104] The process of emotion recognition from audio data (i.e., user speech) can be divided into two main parts: acoustic feature extraction and sentiment analysis. Acoustic feature extraction refers to extracting emotion-related sound features from audio data (speech signals). Emotion-related sound features can include the fundamental frequency, pitch, speech rate, volume, energy, and phonemes of the speech. Then, digital signal processing techniques are used to extract the short-time energy, short-time zero-crossing rate, linear predictive coding, and Mel-frequency cepstral coefficients of these emotion-related sound features. Sentiment analysis refers to using machine learning algorithms to analyze the acoustic features to obtain the user's emotional state. It can be understood that sentiment analysis typically involves classification, such as classifying emotions into longing, lyricism, lightness, joy, pleasure, enthusiasm, vitality, sacredness, and sadness. It can be understood that classification can be trained using supervised learning algorithms, such as support vector machines, random forests, neural networks, and deep learning algorithms, which are not limited here.

[0105] The following is combined Figure 8 The flowchart of the pre-training process of the emotion recognition model is briefly explained below. The pre-training process of the emotion recognition model may specifically include: S301, preprocessing the training set, which may include noise reduction processing of the audio data in the training set to filter out noise; S302, extracting emotion features from the audio data, that is, extracting emotion words from the audio data and using the extracted emotion words as emotion features. For example, if "scenery," "very," and "spectacular" are extracted, the sentiment of sentences containing these sentiment words will be classified as "positive." S303: Perform feature dimensionality reduction on the extracted sentiment features. Feature dimensionality reduction refers to reducing the number of sentiment features. By performing feature dimensionality reduction on sentiment features, the algorithm for determining sentiment can be accelerated, and the noise caused by rich sentiment features can be reduced, effectively improving the accuracy of sentiment feature classification. S304: Input the above sentiment features into the classification model for classification. S305: Predict the sentiment corresponding to the audio data based on the classification results of the above classification model. S306: Determine whether the sentiment prediction result corresponding to the above-obtained audio data is accurate. If accurate, directly use the predicted sentiment as the sentiment corresponding to the audio data. If inaccurate, optimize the parameters of the classification model to improve the accuracy of the classification model in sentiment classification based on sentiment features. S308: Retrain the processed training set based on the optimized classification model to improve the accuracy of the classification results obtained by the classification model based on sentiment features.

[0106] The pre-training process of the semantic analysis (algorithm) model mentioned in the embodiments of this application is briefly described below.

[0107] Semantic analysis of audio data (i.e., user speech) requires converting the audio data into text and then classifying it using a text classification model. In this embodiment, the semantic analysis model (i.e., the text classification model) needs to convert the audio data (i.e., user speech) into text and then determine the music genre (i.e., musical style) that matches the audio data based on the meaning (semantics) of the text. For example, the text classification model can be TextCNN, TextRCNN, BERT, etc., and is not limited here.

[0108] The following is combined Figure 9 The diagram illustrating the pre-training process of the semantic analysis model (i.e., the text classification model) is briefly explained below. The pre-trained language model BERT (bidirectional encoder representations from transformers) is used as an example.

[0109] The pre-training process of the BERT language model can be specifically divided into the following steps: First, the training set is preprocessed. This preprocessing stage may include recognizing the speech (audio data) in the training set and converting it into corresponding text. The language of the audio data is then determined to identify the corresponding grammatical rules for the audio text, and the audio text is segmented into individual sentences according to the grammatical rules of the corresponding language. Next, the segmented sentences are input into the BERT language model. The BERT language model can obtain text vectors from the input text sequence. These text vectors can be understood as feature vectors from which features and key information have been extracted from the text; they are intermediate products of the language model. Then, the text vectors are used as input to the classifier. Finally, the classifier can be used to determine the genre (i.e., the classification result) corresponding to the text vectors.

[0110] It is understandable that the classifier can be a single-layer or multi-layer neural network. If it is necessary to increase the number of labels for different music styles (such as pop, traditional Chinese style, classical, etc.), the number of neurons can be increased by adjusting the classifier network. There is no limitation here.

[0111] The following is a brief description of the process of pre-training music data with user voice characteristics using AI composition model and speech synthesis technology mentioned in the embodiments of this application, that is, using an AI speech synthesis model to obtain music data with user voice characteristics.

[0112] The process of acquiring music data based on an AI speech synthesis model can specifically include the following steps: First, based on the relevant information of the music arrangement (score) obtained through an AI composition model, it is treated as a phoneme sequence and used as input to the AI ​​speech synthesis model. A phoneme is the smallest unit of speech, which can be divided into vowels and consonants. For example, a Chinese syllable (ā) has only one phoneme, while a syllable (ài) has two. Then, by analyzing the score, features such as pronunciation, duration, pauses, pitch, style, and singing techniques of each phoneme in the lyrics and music are extracted and predicted from multiple dimensions, including text, melody, and rhythm. Finally, a deep neural network acoustic model and a vocoder model are used to synthesize music data (i.e., vocal audio) that simulates the user's singing, similar in timbre and pitch.

[0113] The following is combined Figure 10 The diagram illustrating the pre-training process of the AI ​​speech synthesis model is briefly explained.

[0114] like Figure 10 In this example, assuming a line of sheet music is "The scenery here is truly magnificent," sheet music analysis yields the sheet music duration, lyrics, and pitch. The sheet music duration refers to the relative duration between notes, and the pitch refers to the high or low note. Further, by predicting the vocal duration based on the sheet music duration, the user's vocal duration can be obtained; by predicting pronunciation pauses in the lyrics, textual features corresponding to the lyrics can be obtained, such as the pronunciation and pauses of words during singing; and by predicting pitch techniques based on the sheet music pitch, the user's vocal pitch can be obtained. Then, the obtained vocal duration, textual features, and vocal pitch are input into a vocal acoustic feature prediction system to determine the pronunciation, duration, pauses, pitch, style, and singing techniques of each phoneme in the lyrics. Finally, a deep neural network acoustic model and a vocoder model are used to synthesize music data (i.e., vocal audio) simulating the user's timbre and pitch.

[0115] It is understandable that the vocoder model can be obtained by end-to-end acoustic modeling based on an autoregressive model, and no limitation is made here.

[0116] The embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0117] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0118] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0119] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other propagation signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0120] In the accompanying drawings, certain structural or methodological features are shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0121] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0122] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. While this application has been illustrated and described with reference to certain preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the scope of this application.

Claims

1. An audio processing method, characterized in that, Applied to electronic devices, the method includes: Acquire audio data, including user voice; Determine the user characteristic parameters of the audio data; Based on the user feature parameters of the audio data, the music feature parameters corresponding to the audio data are determined; Music data corresponding to the audio data is obtained based on the music feature parameters; The musical characteristic parameters include musical style and musical tonality; The process of determining the music feature parameters corresponding to the audio data based on the user feature parameters of the audio data includes: The audio text corresponding to the audio data is semantically analyzed by a pre-trained semantic recognition algorithm to determine the music style corresponding to the audio data. The pitch and the musical style are input into a preset second mapping function, which is a mapping function between musical tonality, musical style and pitch, and the musical tonality is determined by the second mapping function.

2. The method according to claim 1, characterized in that, The acquisition of audio data, including user voice, includes: The audio data, including the user's voice, is obtained by the user through reading aloud or humming.

3. The method according to claim 1, characterized in that, The user characteristic parameters for determining the audio data include: Mel frequency cepstral coefficient features are extracted from the audio data including the user's speech, and user feature parameters corresponding to the audio data are determined based on the results of the Mel frequency cepstral coefficient feature extraction. The user feature parameters include the user's timbre, pitch, volume, or speech rate.

4. The method according to claim 1, characterized in that, The musical characteristic parameters include musical emotion; The method of determining the music feature parameters corresponding to the audio data based on the user feature parameters of the audio data further includes: The audio data is classified using a pre-trained emotion recognition algorithm to determine the corresponding musical emotion. The pre-trained emotion recognition algorithm is a sentiment analysis method based on an emotion dictionary. It extracts emotion words from the audio text corresponding to the audio data, calculates the total weight of the audio text based on the weight of the emotion tendency corresponding to the extracted emotion words, and determines the musical emotion corresponding to the audio data through the total weight of the audio text.

5. The method according to claim 1, characterized in that, The music feature parameters include music rhythm; The method of determining the music feature parameters corresponding to the audio data based on the user feature parameters of the audio data further includes: The speech rate and the music style are input into a first mapping function, which is a mapping function between music rhythm, music style and speech rate, and the music rhythm is determined through the first mapping function.

6. The method according to claim 1, characterized in that, The process of obtaining music data corresponding to the audio data based on the music feature parameters includes: The music style, music emotion, music rhythm, music tonality, and the audio text corresponding to the audio data are input into the first composition model. The first composition model obtains the first music data corresponding to the audio data based on the music style, music emotion, music rhythm, music tonality, and the audio text corresponding to the audio data.

7. The method according to claim 3 or 6, characterized in that, The process of obtaining the music data corresponding to the audio data based on the music feature parameters further includes: The first music data and the user feature parameters are synthesized using speech synthesis technology to obtain second music data with the user's voice characteristics.

8. The method according to claim 1, characterized in that, The method further includes: Acquire music data, including user voice recordings; Determine the user characteristic parameters corresponding to the music data; Based on the music data, the audio feature parameters corresponding to the music data are determined; The audio data corresponding to the music data is obtained based on the audio feature parameters.

9. The method according to claim 8, characterized in that, The step of determining the audio feature parameters corresponding to the music data based on the music data includes: The audio feature parameters corresponding to the music data include audio emotion. The audio text corresponding to the music data is obtained through a pre-trained speech processing algorithm. Based on the audio text, the audio emotion corresponding to the music data is determined through a pre-trained emotion recognition algorithm. The process of obtaining the audio data corresponding to the music data based on the audio feature parameters includes: The audio text, the audio emotion, and the user feature parameters are synthesized using speech synthesis technology to obtain audio data with user voice characteristics. The audio data is an audio recording simulating a user reading the audio text.

10. An electronic device, characterized in that, include: Memory, used to store instructions; A processor for executing the instructions to implement the audio processing method according to any one of claims 1-9.

11. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the audio processing method according to any one of claims 1-9.

Citation Information

Patent Citations

  • Attention-based audio and lyric multi-mode music style classification method

    CN113177136A

  • Method and system for realizing emotion judgment through voice recognition

    CN114550696A

  • Vocal music audio generation method and device, equipment and storage medium

    CN116013227A