Information processing method and system for voice conversion

Abstract

The invention belongs to the field of information processing, and provides an information processing method and system for voice conversion, and the method comprises the steps: obtaining all voice signal data needing voice conversion based on all interfaces for collecting voice data, carrying out the noise reduction processing, and obtaining an initial noise reduction voice signal; acquiring fundamental frequency and formant positions of the voice signals, analyzing and processing the fundamental frequency and formant positions to obtain gender characteristic values, and dividing the gender characteristic values; based on the voice information subjected to gender classification, waveform data of the voice information is acquired and subjected to comprehensive analysis processing to obtain emotion feature values, and classification processing is carried out; on the basis of the voice information after noise reduction processing, official text information corresponding to the voice information is obtained through conversion, and gender and emotion information carried by the voice information is added to the text information; according to the technical scheme, official language characters obtained after voice conversion are converted into official voice information, gender and emotion information carried by the voice information is attached to the converted voice information, the man-machine interaction quality is improved, and the human daily life and work experience is improved.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular to an information processing method and system for voice conversion. Background Art

[0002] With the deepening of globalization and the popularization of Internet technology, the demand for language communication is increasing. However, due to the huge differences between different languages ​​and dialects, traditional speech conversion systems are often limited to conversion between a single language or a few languages, and it is difficult to meet diverse needs. Especially in countries with vast territories and many dialects such as China, the demand for conversion between dialects and Mandarin, and between dialects is particularly urgent.

[0003] A Chinese patent application with publication number CN107068165 discloses a speech conversion method, including: obtaining a conversion function required for speech conversion by training a parallel corpus with an adaptive Gaussian mixture model and bilinear frequency bending plus amplitude adjustment, and then using the conversion function to perform high-quality speech conversion.

[0004] In the above-mentioned prior art, speech conversion is achieved, but in practice, in the process of converting multiple speech sounds (such as Japanese to Chinese, Chinese to English), users have a great demand for information processing of speech conversion. Recognizing more information in addition to text information in speech information can achieve more feedback, and analyzing the changes in emotions in speech and reflecting them is easier to understand and analyze than text or other forms of information, which can better improve people's daily life and work experience.

[0005] To this end, the present invention provides an information processing method and system for voice conversion. Summary of the invention

[0006] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0007] The technical solution adopted by the present invention to solve the technical problem is: an information processing method for voice conversion, comprising:

[0008] Based on all interfaces for collecting voice data, all voice signal data requiring voice conversion are obtained, wherein the signal data includes the waveform of the voice signal, and the waveform of the voice signal is subjected to noise reduction processing to obtain a noise reduction waveform;

[0009] Based on the noise-reduced speech signal data, the fundamental frequency and the resonance peak position of the speech signal are obtained, and the fundamental frequency ratio and the resonance peak position ratio of the speech signal are obtained by analysis and processing. Based on the fundamental frequency ratio and the resonance peak position ratio obtained by analysis and processing, the gender characteristic value is obtained by a formula, and the speech signal is divided based on the gender characteristic value obtained by analysis and processing;

[0010] Based on the spectrum of normal human voice, the noise spectrum outside the normal human voice spectrum is subtracted. Similarly, based on the minimum mean square error criterion, the statistical characteristics of noise and speech signals are used to design a filter for noise reduction to obtain a noise-reduced speech signal.

[0011] Based on the noise reduction speech signal obtained by analysis and processing, according to the difference in fundamental frequency range and formant peak of male voice and female voice, the fundamental frequency proportion and formant peak position proportion of the speech signal are obtained by analysis and processing;

[0012] The fundamental frequency proportion of the male voice is processed by the difference with the fundamental frequency proportion of the female voice to obtain the fundamental frequency proportion difference JP. Similarly, the formant position proportion of the male voice is processed by the difference with the formant position proportion of the female voice to obtain the formant position proportion difference GZ.

[0013] The obtained fundamental frequency proportion difference JP and resonance peak proportion difference GZ are processed by data, and the formula is: XB = a1*JP + a2*GZ 3 Obtain the gender characteristic value XB, where a1 and a2 are preset equilibrium values;

[0014] Based on the calculated gender feature value, the voice information to be processed is classified into male voice information and female voice information;

[0015] Compare the gender characteristic value with the gender characteristic threshold. The specific comparison process is:

[0016] If the gender feature value is greater than the gender feature threshold, the voice information is marked as male voice information and classified in the male voice information area;

[0017] If the gender feature value is less than the gender feature threshold, the voice information is marked as female voice information and classified in the female voice information area;

[0018] Based on the speech information after gender classification, waveform data of the speech information is obtained, wherein the waveform data includes the zero-crossing rate and short-time energy of the speech signal, and the speech information is divided into different areas according to the different energy distributions of normal human emotions. There may be multiple emotions in the speech information, so the speech information can be divided into multiple parts according to the coherence of the sentences, and the zero-crossing rate of each part is normalized to obtain the normalized zero-crossing rate, and the short-time energy average value of each part is obtained. According to the proportion of all short-time energies of the speech signal in different emotional energy areas, the short-time energy characteristic value of each part is obtained by analysis and processing, and the obtained normalized zero-crossing rate and short-time energy characteristic value are comprehensively analyzed and processed to obtain the emotional characteristic value;

[0019] The speech information is divided into multiple parts according to the coherence of the sentences, and the speech signal is divided into several overlapping short time frames. The length of each frame is set to 50 milliseconds, and the frame shift is set to half of the frame length, that is, 25 milliseconds. This is done to capture the local characteristics of the speech signal, obtain the average zero-crossing rate of each part of the speech signal per frame, and analyze and calculate the normalized zero-crossing rate PL of each part;

[0020] For each frame of each divided part, the square sum of the signal sample values ​​in each frame is calculated as the short-time energy. The specific calculation formula is:

[0021] Where E(n) represents the short-time energy of the nth frame, x[n+m] represents the sample value of the speech signal at the n+mth moment, and M is the length of the frame, i.e. the number of samples;

[0022] Obtain the short-time energy of all frames of each part, sum the short-time energy of all frames, and average the short-time energy of each part to obtain the average short-time energy PD;

[0023] The average zero-crossing rate PL and the average short-time energy PD are processed by the formula: QX = a1*PL 2 +a2*PD to obtain the emotional characteristic value QX, where a1 and a2 are preset correlation coefficients;

[0024] Based on the calculated emotion feature values, the voice information to be converted is divided into voice information to be converted with different emotions;

[0025] According to the various language information samples obtained from big data, the distribution law of the characteristic values ​​corresponding to the emotions is obtained, and the emotional characteristic values ​​corresponding to the emotions of the language to be converted and the target conversion language are obtained. According to the calculated emotional characteristic values, the emotions can be divided into seven types: calm, happy, sad, angry, fearful, surprised, and disgusted. The emotional characteristic values ​​are compared with the emotional characteristic thresholds. Different emotional characteristic values ​​correspond to different emotions. The specific classification process is as follows:

[0026] If the emotional feature value is greater than the emotional feature threshold b1, the emotion of the voice information is marked as anger, and the voice information is divided into the anger area;

[0027] If the emotional feature value is less than the emotional feature threshold b1 and greater than the emotional feature threshold b2, the emotion of the voice information is marked as surprise, and the voice information is divided into the surprise area;

[0028] If the emotional feature value is less than the emotional feature threshold b2 and greater than the emotional feature threshold b3, the emotion of the voice information is marked as happy, and the voice information is divided into the happy area;

[0029] If the emotion feature value is less than the emotion feature threshold b3 and greater than the emotion feature threshold b4, the emotion of the voice information is marked as flat, and the voice information is divided into a flat area;

[0030] If the emotional feature value is less than the emotional feature threshold b4 and greater than the emotional feature threshold b5, the emotion of the voice information is marked as disgust, and the voice information is divided into the disgust area;

[0031] If the emotional feature value is less than the emotional feature threshold b5 and greater than the emotional feature threshold b6, the emotion of the voice information is marked as fear, and the voice information is divided into the fear area;

[0032] If the emotional feature value is less than the emotional feature threshold b6, the emotion of the voice information is marked as sad, and the voice information is divided into the sad area;

[0033] A large number of speech samples of the language corresponding to the speech information to be converted are collected through big data, and the pre-processed and classified speech signals are converted into official language and text through the speech-to-text recognition conversion terminal, and the emotions carried by the speech information and the gender of the inputter are marked. When the official text is converted into official speech, the corresponding gender and emotion are added to the speech and output;

[0034] Collect speech samples of the languages ​​corresponding to all the speech to be converted based on big data, learn and summarize the correspondence between the language to be converted and the official language through the speech-to-text recognition conversion terminal, and recognize and convert the official text information corresponding to the speech information through the speech-to-text recognition conversion terminal based on the speech information after noise reduction processing and store it. At the same time, the gender and emotion information carried by the speech information is attached to the text information;

[0035] The official language and characters after speech conversion are converted into official speech information through a speech-to-text recognition conversion terminal, a large number of speech samples of the official language are collected through big data, and signal characteristic values ​​of gender and emotion corresponding to the language are obtained, wherein the signal characteristic values ​​include: fundamental frequency, resonance peak, zero crossing rate and short-time energy value, and according to the divided official language and characters, the converted characters are assigned gender and emotion through a speech-to-text recognition conversion terminal to convert them into speech signals of the official language;

[0036] A speech conversion information processing system, comprising:

[0037] Voice signal receiving module: receives the voice signal to be converted and stores it;

[0038] Speech signal denoising module: obtains all speech signal data that require speech conversion, wherein the signal data includes the waveform of the speech signal, performs denoising on the waveform of the speech signal, and obtains a denoised waveform;

[0039] Gender differentiation module: Based on the noise-reduced speech signal data, the fundamental frequency and formant position of the speech signal are obtained, and the fundamental frequency ratio and formant position ratio of the speech signal are obtained by analysis and processing. Based on the fundamental frequency ratio and formant position ratio obtained by analysis and processing, the gender characteristic value is obtained through the formula, and the speech signal is divided based on the gender characteristic value obtained by analysis and processing;

[0040] Emotional differentiation module: based on the speech information after gender classification, the waveform data of the speech information is obtained, wherein the waveform data includes the zero-crossing rate and short-time energy of the speech signal, the zero-crossing rate of each part is normalized to obtain the normalized zero-crossing rate, the short-time energy average value of each part is obtained, and the short-time energy characteristic value of each part is obtained by analyzing and processing according to the proportion of all short-time energy of the speech signal in different emotional energy areas, and the obtained normalized zero-crossing rate and short-time energy characteristic value are comprehensively analyzed and processed to obtain the emotional characteristic value;

[0041] Speech signal to text module: collects speech samples of the languages ​​corresponding to all the speech to be converted based on big data, learns and summarizes the correspondence between the language to be converted and the official language through the speech to text recognition conversion terminal, and recognizes and converts the official text information corresponding to the speech information through the speech to text recognition conversion terminal based on the speech information after noise reduction processing and stores it. At the same time, the gender and emotion information carried by the speech information is attached to the text information;

[0042] Text-to-speech signal module: The official language text after speech conversion is converted into official voice information through the speech-to-text recognition conversion terminal, and a large number of official language speech samples are collected through big data to obtain the signal characteristic values ​​of gender and emotion corresponding to the language, where the signal characteristic values ​​include: fundamental frequency, resonance peak, zero-crossing rate and short-time energy value. According to the divided official language text, the converted text is assigned gender and emotion through the speech-to-text recognition conversion terminal to convert it into the official language voice signal;

[0043] Information export module: outputs the converted text or voice signal with added gender and emotion.

[0044] The beneficial effects of the present invention are as follows:

[0045] 1. The noise-reduced voice signal can be more accurately divided into noise-reduced male voice signals and noise-reduced female voice signals. Gender classification of voice information can help better understand user needs and provide more personalized services. The traditional method based on simple statistical features of audio signals can no longer meet the growing accuracy requirements, so it is necessary to develop more advanced technical means to perform more accurate gender classification. This technical solution combines multi-dimensional features with advanced algorithms to achieve efficient and accurate classification of gender attributes in voice information, which has high practical value and development prospects.

[0046] 2. Obtain the normalized zero-crossing rate and average short-time energy of voice information, analyze and process the data to obtain the emotional feature value, and compare it with the emotional feature threshold. Divide the voice information into various areas, representing voice information of different emotions, convey and understand emotions through sound, and recognize emotions through voice, which can imitate the natural way of human communication and make the interaction more natural and smooth. The characteristics of tone, rhythm and intensity in voice can directly reflect the emotional state of the speaker, which is easier to understand and analyze than text or other forms of information. Voice emotion recognition can achieve real-time feedback and instantly analyze the speaker's emotional changes, which is very important for applications that require instant response. Automatically identify the customer's emotional state, adjust the response strategy in time, and improve customer satisfaction. Monitor the patient's voice characteristics, assist in the diagnosis of mental illness, track the patient's emotional changes, analyze the students' emotional reactions in class, adjust the teaching strategy in time, and improve the teaching quality. In games or virtual reality environments, the user's emotional state can be recognized through voice to provide a personalized interactive experience. Using voice information for emotion recognition can not only improve the quality of human-computer interaction, but also bring practical application value in many fields. With the continuous advancement of technology, voice emotion recognition will play an important role in more scenarios and improve people's daily life and work experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings.

[0048] Figure 1 It is a flowchart of the steps of an information processing method for voice conversion according to an embodiment of the present invention;

[0049] Figure 2 It is a flowchart of an information processing system for voice conversion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0051] Example 1

[0052] like Figure 1 As shown, a method for processing information of speech conversion according to an embodiment of the present invention includes:

[0053] Step 1: Based on all interfaces for collecting voice data, all voice signal data that need voice conversion are obtained, wherein the signal data includes the waveform of the voice signal, and the waveform of the voice signal is subjected to noise reduction processing to obtain a noise reduction waveform;

[0054] Based on the noise-reduced speech signal data, the fundamental frequency and the resonance peak position of the speech signal are obtained, and the fundamental frequency ratio and the resonance peak position ratio of the speech signal are obtained by analysis and processing. Based on the fundamental frequency ratio and the resonance peak position ratio obtained by analysis and processing, the gender characteristic value is obtained by a formula, and the speech signal is divided based on the gender characteristic value obtained by analysis and processing;

[0055] It should be noted that the speech signal represents the signal converted from the speech information, the fundamental frequency of the speech signal represents the basic vibration frequency of the speech signal waveform, and it should also be noted that the resonance peak position represents the specific frequency value of the waveform in some areas where the instantaneous energy of the speech signal is relatively concentrated in the spectrum, where the instantaneous energy of the speech signal represents the square of the signal amplitude;

[0056] In some embodiments, based on the spectrum of normal human voice, the noise spectrum outside the normal human voice spectrum range is subtracted. Similarly, based on the minimum mean square error criterion, the statistical characteristics of noise and speech signals are used to design a filter for noise reduction to obtain a noise-reduced speech signal.

[0057] It should be noted that the spectrum of normal human voice is divided into the normal female voice spectrum and the normal male voice spectrum, wherein the normal female voice spectrum ranges from 165 to 255 Hz, and the normal male voice spectrum ranges from 60 to 125 Hz;

[0058] Based on the noise reduction speech signal obtained by analysis and processing, according to the difference in fundamental frequency range and formant peak of male voice and female voice, the fundamental frequency proportion and formant peak position proportion of the speech signal are obtained by analysis and processing;

[0059] The fundamental frequency proportion of the male voice is processed by the difference with the fundamental frequency proportion of the female voice to obtain the fundamental frequency proportion difference JP. Similarly, the formant position proportion of the male voice is processed by the difference with the formant position proportion of the female voice to obtain the formant position proportion difference GZ.

[0060] The obtained fundamental frequency proportion difference JP and resonance peak proportion difference GZ are processed by data, and the formula is: XB = a1*JP + a2*GZ 3 Obtain the gender characteristic value XB, where a1 and a2 are preset equilibrium values;

[0061] It should be noted that the meaning of the gender characteristic value XB is: the gender characteristic value XB is calculated by the fundamental frequency proportion difference JP and the formant position proportion difference GZ, wherein the fundamental frequency proportion difference is positive and the larger it is, the more voice information in the voice information is in the normal male voice spectrum, which means that the voice information is closer to the male voice. Similarly, the formant position proportion difference is positive and the larger it is, the more voice information in the voice information is in the formant position of the normal male voice, which means that the voice information is closer to the male voice. The fundamental frequency proportion difference JP and the formant position proportion difference GZ reflect the criticality of the voice information as a whole, that is, the gender characteristic value is positive and the larger it is, the voice information is closer to the male voice and can be classified as male voice information. Conversely, the gender characteristic value is negative and the smaller it is, the voice information is closer to the female voice and can be classified as female voice information.

[0062] Step 2: Classify the voice information to be processed based on the calculated gender feature value, and divide it into male voice information and female voice information;

[0063] In some embodiments, the gender characteristic value is compared with the gender characteristic threshold, and the specific comparison process is:

[0064] If the gender feature value is greater than the gender feature threshold, the voice information is marked as male voice information and classified in the male voice information area;

[0065] If the gender feature value is less than the gender feature threshold, the voice information is marked as female voice information and classified in the female voice information area;

[0066] The technical solution of the embodiment of the present invention is: to more accurately divide the noise-reduced voice signal into the noise-reduced male voice signal and the noise-reduced female voice signal, and to classify the voice information by gender, which can help better understand user needs and provide more personalized services. The traditional method based on simple statistical features of audio signals can no longer meet the growing accuracy requirements, so it is necessary to develop more advanced technical means to perform more accurate gender classification. This technical solution combines multi-dimensional features with advanced algorithms to achieve efficient and accurate classification of gender attributes in voice information, and has high practical value and development prospects.

[0067] Example 2

[0068] Based on Example 1, the present invention further classifies and processes the classified voice information. Usually, the voice information carries various emotions of the recorder when it is recorded, but after voice conversion, it cannot clearly express the original emotions of the recorder, resulting in certain obstacles in communication and even misunderstandings. Therefore, the voice information is optimized and analyzed to distinguish the different emotions of the voice information and classify it, such as Figure 1As shown, the information processing method for voice conversion described in the embodiment of the present invention includes:

[0069] Step 3: Based on the speech information after gender classification, waveform data of the speech information is obtained, wherein the waveform data includes the zero-crossing rate and short-time energy of the speech signal, and the speech information is divided into different regions according to the different energy distributions of normal human emotions. There may be multiple emotions in the speech information, so the speech information can be divided into multiple parts according to the coherence of the sentences, and the zero-crossing rate of each part is normalized to obtain the normalized zero-crossing rate, and the short-time energy average value of each part is obtained. According to the proportion of all short-time energies of the speech signal in different emotional energy regions, the short-time energy characteristic value of each part is obtained by analysis and processing, and the obtained normalized zero-crossing rate and short-time energy characteristic value are comprehensively analyzed and processed to obtain the emotional characteristic value;

[0070] It should be noted that short-time energy refers to the characteristic parameter obtained by squaring and summing the sample values ​​of the speech signal in a short time window, and zero-crossing rate normalization refers to dividing the number of zero-crossing events by the length of the time window to obtain the zero-crossing rate per unit time.

[0071] In some embodiments, the speech information is divided into multiple parts according to the coherence of the sentences, and the speech signal is divided into a number of overlapping short-time frames, the length of each frame is set to 50 milliseconds, and the frame shift is set to half of the frame length, that is, 25 milliseconds. This is done to capture the local characteristics of the speech signal, obtain the average zero-crossing rate of each frame of each part of the speech signal, and analyze and calculate the normalized zero-crossing rate PL of each part;

[0072] For each frame of each divided part, the square sum of the signal sample values ​​in each frame is calculated as the short-time energy. The specific calculation formula is:

[0073] Where E(n) represents the short-time energy of the nth frame, x[n+m] represents the sample value of the speech signal at the n+mth moment, and M is the length of the frame, i.e. the number of samples;

[0074] It should be noted that the frame shift refers to the overlap between frames, and the sample value refers to the discrete data points converted from the continuous analog voice signal by sampling during the digitization process, reflecting the amplitude or strength of the signal at a specific time point;

[0075] Obtain the short-time energy of all frames of each part, sum the short-time energy of all frames, and average the short-time energy of each part to obtain the average short-time energy PD;

[0076] The average zero-crossing rate PL and the average short-time energy PD are processed and the formula is: QX = a1*PL 2+a2*PD to obtain the emotional characteristic value QX, where a1 and a2 are preset correlation coefficients;

[0077] It should be noted that the meaning of the emotional characteristic value QX is: the emotional characteristic value is calculated by normalizing the zero-crossing rate and the average short-time energy value, wherein the larger the normalized zero-crossing rate and the average short-time energy value, the larger the emotional characteristic value, the more excited the corresponding emotion, and different emotional characteristic values ​​correspond to different emotions carried by the speech information;

[0078] Step 4: Based on the calculated emotional feature values, the voice information to be converted is divided into voice information to be converted with different emotions;

[0079] In some embodiments, based on various language information samples obtained from big data, the characteristic value distribution law corresponding to the emotion is obtained, and the emotion characteristic values ​​corresponding to the emotions of the language to be converted and the target conversion language are obtained. According to the calculated emotion characteristic values, the emotions can be divided into seven types: calm, happy, sad, angry, fearful, surprised, and disgusted. The emotion characteristic values ​​are compared with the emotion characteristic thresholds. Different emotion characteristic values ​​correspond to different emotions. The specific classification process is as follows:

[0080] If the emotional feature value is greater than the emotional feature threshold b1, the emotion of the voice information is marked as anger, and the voice information is divided into the anger area;

[0081] If the emotional feature value is less than the emotional feature threshold b1 and greater than the emotional feature threshold b2, the emotion of the voice information is marked as surprise, and the voice information is divided into the surprise area;

[0082] If the emotional feature value is less than the emotional feature threshold b2 and greater than the emotional feature threshold b3, the emotion of the voice information is marked as happy, and the voice information is divided into the happy area;

[0083] If the emotion feature value is less than the emotion feature threshold b3 and greater than the emotion feature threshold b4, the emotion of the voice information is marked as flat, and the voice information is divided into a flat area;

[0084] If the emotional feature value is less than the emotional feature threshold b4 and greater than the emotional feature threshold b5, the emotion of the voice information is marked as disgust, and the voice information is divided into the disgust area;

[0085] If the emotional feature value is less than the emotional feature threshold b5 and greater than the emotional feature threshold b6, the emotion of the voice information is marked as fear, and the voice information is divided into the fear area;

[0086] If the emotional feature value is less than the emotional feature threshold b6, the emotion of the voice information is marked as sad, and the voice information is divided into the sad area;

[0087] The technical solution of the embodiment of the present invention is: obtaining the normalized zero-crossing rate and average short-time energy of the speech information, analyzing and processing the data to obtain the emotional feature value, and comparing it with the emotional feature threshold to divide the speech information into various areas, representing speech information of different emotions, and conveying and understanding emotions through sound.

[0088] Example 3

[0089] Based on Embodiment 1 and Embodiment 2, the information processing method for voice conversion described in the embodiment of the present invention includes:

[0090] Step 5: Collect a large number of speech samples in the language corresponding to the speech information to be converted through big data, convert the pre-processed and classified speech signals into official language and text through the speech-to-text recognition conversion terminal, and mark the emotions carried by the speech information and the gender of the inputter. When the official text is converted to official speech, the corresponding gender and emotion are added to the speech and output;

[0091] In some embodiments, voice samples of the languages ​​corresponding to all the voices to be converted based on big data are collected, and the correspondence between the language to be converted and the official language is learned and summarized through the voice-to-text recognition conversion terminal. Based on the voice information after noise reduction processing, the official text information corresponding to the voice information is recognized and converted through the voice-to-text recognition conversion terminal and stored, and at the same time, the gender and emotion information carried by the voice information is attached to the text information;

[0092] The official language and characters after speech conversion are converted into official speech information through a speech-to-text recognition conversion terminal, a large number of speech samples of the official language are collected through big data, and signal characteristic values ​​of gender and emotion corresponding to the language are obtained, wherein the signal characteristic values ​​include: fundamental frequency, resonance peak, zero crossing rate and short-time energy value, and according to the divided official language and characters, the converted characters are assigned gender and emotion through a speech-to-text recognition conversion terminal to convert them into speech signals of the official language;

[0093] The technical solution of the embodiment of the present invention is: through voice recognition of emotions, the natural way of human communication can be imitated to make the interaction more natural and smooth. The characteristics of the tone, rhythm and intensity in the voice can directly reflect the emotional state of the speaker, which is easier to understand and analyze than text or other forms of information. Voice emotion recognition can achieve real-time feedback and instantly analyze the speaker's emotional changes, which is very important for applications that require instant response. It can automatically identify the customer's emotional state, adjust the response strategy in time, improve customer satisfaction, monitor the patient's voice characteristics, assist in the diagnosis of mental illness, track the patient's emotional changes, analyze the students' emotional reactions in class, adjust the teaching strategy in time, and improve the teaching quality. In games or virtual reality environments, the emotional state of the user can be recognized through voice to provide a personalized interactive experience. Using voice information for emotion recognition can not only improve the quality of human-computer interaction, but also bring practical application value in multiple fields. With the continuous advancement of technology, voice emotion recognition will play an important role in more scenarios and improve people's daily life and work experience.

[0094] Example 4

[0095] like Figure 2 As shown, a voice conversion information processing system according to an embodiment of the present invention includes:

[0096] Voice signal receiving module: receives the voice signal to be converted and stores it;

[0097] Speech signal denoising module: obtains all speech signal data that require speech conversion, wherein the signal data includes the waveform of the speech signal, performs denoising on the waveform of the speech signal, and obtains a denoised waveform;

[0098] Gender differentiation module: Based on the noise-reduced speech signal data, the fundamental frequency and formant position of the speech signal are obtained, and the fundamental frequency ratio and formant position ratio of the speech signal are obtained by analysis and processing. Based on the fundamental frequency ratio and formant position ratio obtained by analysis and processing, the gender characteristic value is obtained through the formula, and the speech signal is divided based on the gender characteristic value obtained by analysis and processing;

[0099] Emotional differentiation module: based on the speech information after gender classification, the waveform data of the speech information is obtained, wherein the waveform data includes the zero-crossing rate and short-time energy of the speech signal, the zero-crossing rate of each part is normalized to obtain the normalized zero-crossing rate, the short-time energy average value of each part is obtained, and the short-time energy characteristic value of each part is obtained by analyzing and processing according to the proportion of all short-time energy of the speech signal in different emotional energy areas, and the obtained normalized zero-crossing rate and short-time energy characteristic value are comprehensively analyzed and processed to obtain the emotional characteristic value;

[0100] Speech signal to text module: collects speech samples of the languages ​​corresponding to all the speech to be converted based on big data, learns and summarizes the correspondence between the language to be converted and the official language through the speech to text recognition conversion terminal, and recognizes and converts the official text information corresponding to the speech information through the speech to text recognition conversion terminal based on the speech information after noise reduction processing and stores it. At the same time, the gender and emotion information carried by the speech information is attached to the text information;

[0101] Text-to-speech signal module: The official language text after speech conversion is converted into official voice information through the speech-to-text recognition conversion terminal, and a large number of official language speech samples are collected through big data to obtain the signal characteristic values ​​of gender and emotion corresponding to the language, where the signal characteristic values ​​include: fundamental frequency, resonance peak, zero-crossing rate and short-time energy value. According to the divided official language text, the converted text is assigned gender and emotion through the speech-to-text recognition conversion terminal to convert it into the official language voice signal;

[0102] Information export module: outputs the converted text or voice signal with added gender and emotion.

[0103] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method and system for information processing of speech conversion, characterized in that: include: Based on all interfaces for collecting voice data, all voice signal data requiring voice conversion are obtained, wherein the signal data includes the waveform of the voice signal, and the waveform of the voice signal is subjected to noise reduction processing to obtain a noise reduction waveform; Based on the noise-reduced speech signal data, the fundamental frequency and the resonance peak position of the speech signal are obtained, and the fundamental frequency ratio and the resonance peak position ratio of the speech signal are obtained by analysis and processing. Based on the fundamental frequency ratio and the resonance peak position ratio obtained by analysis and processing, the gender characteristic value is obtained by a formula, and the speech signal is divided based on the gender characteristic value obtained by analysis and processing; Based on the calculated gender feature value, the voice information to be processed is classified into male voice information and female voice information; Based on the speech information after gender classification, waveform data of the speech information is obtained, wherein the waveform data includes the zero-crossing rate and short-time energy of the speech signal. The speech information is divided into multiple parts according to the coherence of the sentence, the zero-crossing rate of each part is normalized to obtain the normalized zero-crossing rate, and the short-time energy average value of each part is obtained. According to the proportion of all short-time energies of the speech signal in different emotional energy areas, the short-time energy characteristic value of each part is obtained through analysis and processing, and the obtained normalized zero-crossing rate and short-time energy characteristic value are comprehensively analyzed and processed to obtain the emotional characteristic value.

2. The information processing method for voice conversion according to claim 1, characterized in that: The specific process of obtaining the gender characteristic value is as follows: Based on the spectrum of normal human voice, the noise spectrum outside the normal human voice spectrum is subtracted. Similarly, based on the minimum mean square error criterion, the statistical characteristics of noise and speech signals are used to design a filter for noise reduction to obtain a noise-reduced speech signal. Based on the noise reduction speech signal obtained by analysis and processing, according to the difference in fundamental frequency range and formant peak of male voice and female voice, the fundamental frequency proportion and formant peak position proportion of the speech signal are obtained by analysis and processing; The fundamental frequency proportion of the male voice is processed by the difference with the fundamental frequency proportion of the female voice to obtain the fundamental frequency proportion difference JP. Similarly, the formant position proportion of the male voice is processed by the difference with the formant position proportion of the female voice to obtain the formant position proportion difference GZ. The obtained fundamental frequency proportion difference JP and resonance peak proportion difference GZ are processed by data, and the formula is: XB = a1*JP + a2*GZ 3 The gender characteristic value XB is obtained, where a1 and a2 are preset equilibrium values.

3. The information processing method for voice conversion according to claim 1, characterized in that: The specific process of gender classification of voice information is as follows: If the gender feature value is greater than the gender feature threshold, the voice information is marked as male voice information and classified in the male voice information area; If the gender feature value is less than the gender feature threshold, the voice information is marked as female voice information and classified in the female voice information area.

4. The information processing method for voice conversion according to claim 1, characterized in that: The specific acquisition process of the normalized zero-crossing rate is: The speech information is divided into multiple parts according to the coherence of the sentences, and the speech signal is divided into several overlapping short-time frames. The length of each frame is set to 50 milliseconds, and the frame shift is set to half of the frame length, that is, 25 milliseconds. This is done to capture the local characteristics of the speech signal, obtain the average zero-crossing rate of each part of the speech signal per frame, and analyze and calculate the normalized zero-crossing rate PL of each part.

5. The information processing method for voice conversion according to claim 1, characterized in that: The specific process of obtaining the average short-time energy is as follows: For each frame of each divided part, the square sum of the signal sample values ​​in each frame is calculated as the short-time energy. The specific calculation formula is: Where E(n) represents the short-time energy of the nth frame, x[n+m] represents the sample value of the speech signal at the n+mth moment, and M is the length of the frame, i.e. the number of samples; It should be noted that the frame shift refers to the overlap between frames, and the sample value refers to the discrete data points converted from the continuous analog voice signal by sampling during the digitization process, reflecting the amplitude or strength of the signal at a specific time point; The short-time energy of all frames of each part is obtained by summing up the short-time energy of all frames, and averaging the short-time energy of each part to obtain the average short-time energy PD.

6. The information processing method for voice conversion according to claim 5, characterized in that: The specific process of obtaining the emotional feature value is as follows: The average zero-crossing rate PL and the average short-time energy PD are processed and the formula is: QX = a1*PL 2 +a2*PD to obtain the emotion feature value QX, where a1 and a2 are preset correlation coefficients.

7. The information processing method for voice conversion according to claim 5, characterized in that: The specific division process of the emotional feature value is as follows: According to the various language information samples obtained from big data, the distribution law of the characteristic values ​​corresponding to their emotions is obtained, and the emotional characteristic values ​​corresponding to the emotions of the language to be converted and the target conversion language are obtained. According to the calculated emotional characteristic values, the emotions can be divided into seven types: calm, happy, sad, angry, fearful, surprised, and disgusted. The emotional characteristic values ​​are compared with the emotional characteristic thresholds. Different emotional characteristic values ​​correspond to different emotions.

8. The information processing method for voice conversion according to claim 7, characterized in that: The specific process of voice-to-text conversion is as follows: Collect speech samples of the languages ​​corresponding to all the speech to be converted based on big data, learn and summarize the correspondence between the language to be converted and the official language through the speech-to-text recognition conversion terminal, and based on the speech information after noise reduction processing, recognize and convert the official text information corresponding to the speech information through the speech-to-text recognition conversion terminal and store it. At the same time, the gender and emotion information carried by the voice information is attached to the text information.

9. The information processing method for voice conversion according to claim 7, characterized in that: The specific process of the text-to-speech conversion is as follows: The official language and text after speech conversion are converted into official voice information through a speech-to-text recognition conversion terminal. A large number of speech samples of the official language are collected through big data to obtain the signal characteristic values ​​of the gender and emotion corresponding to the language. According to the divided official language and text, the converted text is assigned gender and emotion and converted into a speech signal of the official language through a speech-to-text recognition conversion terminal.

10. A voice conversion information processing system, the system being used to implement the voice conversion information processing method according to any one of claims 1 to 9, characterized in that: include: Voice signal receiving module: receives the voice signal to be converted and stores it; Speech signal denoising module: obtains all speech signal data that require speech conversion, wherein the signal data includes the waveform of the speech signal, performs denoising on the waveform of the speech signal, and obtains a denoised waveform; Gender differentiation module: Based on the noise-reduced speech signal data, the fundamental frequency and formant position of the speech signal are obtained, and the fundamental frequency ratio and formant position ratio of the speech signal are obtained by analysis and processing. Based on the fundamental frequency ratio and formant position ratio obtained by analysis and processing, the gender characteristic value is obtained through the formula, and the speech signal is divided based on the gender characteristic value obtained by analysis and processing; Emotional differentiation module: based on the speech information after gender classification, the waveform data of the speech information is obtained, wherein the waveform data includes the zero-crossing rate and short-time energy of the speech signal, the zero-crossing rate of each part is normalized to obtain the normalized zero-crossing rate, the short-time energy average value of each part is obtained, and the short-time energy characteristic value of each part is obtained by analyzing and processing according to the proportion of all short-time energy of the speech signal in different emotional energy areas, and the obtained normalized zero-crossing rate and short-time energy characteristic value are comprehensively analyzed and processed to obtain the emotional characteristic value; Speech signal to text module: collects speech samples of the languages ​​corresponding to all the speech to be converted based on big data, learns and summarizes the correspondence between the language to be converted and the official language through the speech to text recognition conversion terminal, and recognizes and converts the official text information corresponding to the speech information through the speech to text recognition conversion terminal based on the speech information after noise reduction processing and stores it. At the same time, the gender and emotion information carried by the speech information is attached to the text information; Text-to-speech signal module: The official language text after speech conversion is converted into official voice information through the speech-to-text recognition conversion terminal, and a large number of official language speech samples are collected through big data to obtain the signal characteristic values ​​of gender and emotion corresponding to the language, where the signal characteristic values ​​include: fundamental frequency, resonance peak, zero-crossing rate and short-time energy value. According to the divided official language text, the converted text is assigned gender and emotion through the speech-to-text recognition conversion terminal to convert it into the official language voice signal; Information export module: outputs the converted text or voice signal with added gender and emotion.

Images