Voice category acquisition method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device and storage medium for obtaining a voice category, and belongs to the field of machine learning technology. The method includes: obtaining reference voice data, obtaining the original voice data of the target user, performing voiceprint feature extraction on the reference voice data to obtain a preset feature vector, and performing voiceprint feature extraction processing on the original voice data to obtain a voiceprint feature vector, performing similarity calculation on the voiceprint feature vector and the preset feature vector to obtain a similarity value, filtering out a target feature vector from the preset feature vector according to the similarity value, and filtering out a target voice category from at least two voice categories through the target feature vector. The voice category acquisition method of the present application can obtain the voice category that is most similar to the target user. In the subsequent use process, personalized speech synthesis can be performed on the input text, without the user having to record a large amount of voice training data to train the speech synthesis model, and the use process is simpler and more convenient.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device, and storage medium for acquiring a speech category. Background Art

[0002] Personalized speech synthesis is widely used in current voice interaction scenarios. Users can generate voice information with different voice categories (timbre, pitch, intensity, etc.) by inputting text information. Related art personalized speech generation methods generally require collecting user-recorded voice information to train a speech synthesis model, so that the speech synthesis model can output voice information similar to the user's voice category. However, this method requires users to record a large amount of voice training data based on specified text, which is not only cumbersome but also time-consuming. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to provide a method, device, electronic device and storage medium for obtaining a speech category, which can improve the accuracy and efficiency of obtaining the corresponding speech category.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present application provides a method for obtaining a speech category, the method comprising:

[0005] Acquire reference voice data; wherein the reference voice data includes at least two voice categories, and the reference voice data is generated by recording at least two real speakers based on the first text information;

[0006] Acquire original voice data of a target user; wherein the original voice data is recorded by the target user based on the first text information;

[0007] Performing voiceprint feature extraction on the reference voice data to obtain a preset feature vector, and performing voiceprint feature extraction processing on the original voice data to obtain a voiceprint feature vector;

[0008] Filtering the preset feature vector according to the voiceprint feature vector to obtain a target feature vector;

[0009] A target speech category is selected from the at least two speech categories using the target feature vector.

[0010] In some embodiments, obtaining reference voice data includes:

[0011] Inputting the second text information into a preset multi-speech synthesis model to obtain virtual audio information; wherein the virtual audio information includes the at least two speech categories, each of the speech categories corresponding to one of the real speakers;

[0012] The reference voice data generated by the real speaker through recording based on the first text information is obtained according to the virtual audio information.

[0013] In some embodiments, extracting voiceprint features from the reference voice data to obtain a preset feature vector includes:

[0014] Extracting Mel-frequency cepstral coefficient features from the reference speech data to obtain a speech feature vector;

[0015] The speech feature vector is input into a trained generalized end-to-end feature extraction model to extract voiceprint features to obtain the preset feature vector.

[0016] In some embodiments, extracting Mel-frequency cepstral coefficient features from the reference speech data to obtain a speech feature vector includes:

[0017] performing pre-emphasis processing on the reference speech data to obtain high-frequency information;

[0018] performing frame processing on the high-frequency information to obtain frame information;

[0019] Performing windowing processing on the frame information to obtain optimized information;

[0020] Performing a fast Fourier transform on the optimization information to obtain frequency domain information;

[0021] Filtering the frequency domain information through a Mel filter bank to obtain filtered information;

[0022] Performing logarithmic processing on the filtered information to obtain local information;

[0023] performing discrete cosine transform on the local information to obtain compressed information;

[0024] Dynamic differential parameter extraction is performed on the compressed information to obtain the speech feature vector.

[0025] In some embodiments, the filtering of the preset feature vector according to the voiceprint feature vector to obtain a target feature vector includes:

[0026] Calculating the similarity between the voiceprint feature vector and the preset feature vector to obtain a similarity value;

[0027] The target feature vector is selected from the preset feature vectors according to the similarity value.

[0028] In some embodiments, the filtering of the preset feature vector according to the voiceprint feature vector to obtain a target feature vector further includes:

[0029] Calculating the cosine similarity between the voiceprint feature vector and the preset feature vector to obtain the similarity value;

[0030] Performing maximum value screening processing on the similarity values ​​to obtain a maximum similarity value;

[0031] The target feature vector is selected from the preset feature vectors according to the maximum similarity value.

[0032] In some embodiments, filtering out a target speech category from the at least two speech categories using the target feature vector includes:

[0033] Performing matching processing on the reference speech data using the target feature vector to obtain a target reference speech;

[0034] The target speech category is obtained by performing matching processing on the at least two speech categories using the target reference speech.

[0035] To achieve the above-mentioned purpose, the second aspect of the present application proposes a device for obtaining a speech category, the device comprising:

[0036] A first speech acquisition module, configured to acquire reference speech data; wherein the reference speech data includes at least two speech categories and is generated by recording at least two real speakers based on the first text information;

[0037] a second voice acquisition module, configured to acquire original voice data of a target user; wherein the original voice data is recorded by the target user based on the first text information;

[0038] A feature extraction module is configured to perform voiceprint feature extraction on the reference speech data to obtain a preset feature vector, and perform voiceprint feature extraction processing on the original speech data to obtain a voiceprint feature vector;

[0039] a first screening module, configured to screen the preset feature vector according to the voiceprint feature vector to obtain a target feature vector;

[0040] A second screening module is configured to screen out a target speech category from the at least two speech categories using the target feature vector.

[0041] To achieve the above-mentioned objectives, the third aspect of the present application provides an electronic device, comprising:

[0042] at least one memory;

[0043] at least one processor;

[0044] at least one program;

[0045] The program is stored in the memory, and the processor executes at least one program to implement the method described in the first aspect of the present application.

[0046] To achieve the above-mentioned object, the fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute:

[0047] As described in the first aspect above.

[0048] The voice category acquisition method, voice category acquisition device, electronic device and storage medium proposed in the embodiments of the present application first obtain the original voice data recorded by the target user and the reference voice data recorded by a real speaker, and then perform voiceprint feature extraction processing on the original voice data and the reference voice data respectively to obtain the corresponding voiceprint feature vector and the preset feature vector, and filter out the feature vector that is most similar to the voiceprint feature vector from the preset feature vector as the target feature vector. Since the target feature vector corresponds to the reference voice data, and the reference voice data corresponds to the voice category, the voice category corresponding to the target feature vector can be obtained through screening, and the obtained voice category is most similar to the voice features of the original voice data input by the target user. In the subsequent use process, by selecting the obtained voice category to perform personalized voice synthesis on the input text of the target user, voice data similar to the voice features of the target user can be obtained. There is no need for the user to record a large amount of voice training data to train the voice synthesis model, and the use process is simpler and more convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of a method for obtaining a voice category according to an embodiment of the present application;

[0050] Figure 2 yes Figure 1 Flowchart of step S100 in the embodiment;

[0051] Figure 3 yes Figure 1 Flowchart of step S300 in the embodiment;

[0052] Figure 4 yes Figure 3 Flowchart of step S310 in the embodiment;

[0053] Figure 5 yes Figure 1 Flowchart of step S500 in the embodiment;

[0054] Figure 6 yes Figure 1 Flowchart of step S500 in another embodiment;

[0055] Figure 7 yes Figure 1 Flowchart of step S600;

[0056] Figure 8 Schematic diagram of a method for obtaining a voice category according to an embodiment of the present application;

[0057] Figure 9 This is a module diagram of a device for acquiring a speech category provided in an embodiment of the present application;

[0058] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0062] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0063] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0064] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0065] First, let’s analyze some of the terms used in this application:

[0066] Speech synthesis: also known as Text To Speech (TTS), is a technology that can convert any input text into corresponding speech. Traditional speech synthesis systems usually include two modules: front-end and back-end. The front-end module mainly analyzes the input text and extracts the linguistic information required by the back-end module. For Chinese synthesis systems, the front-end module generally includes sub-modules such as text regularization, word segmentation, part-of-speech prediction, polyphone disambiguation, and prosody prediction. The back-end module generates speech waveforms using certain methods based on the front-end analysis results. Back-end systems are generally divided into speech synthesis based on statistical parameter modeling (or parametric synthesis) and speech synthesis based on unit selection and waveform splicing (or splicing synthesis).

[0067] For parameter synthesis in the backend system, this method performs context-sensitive modeling of speech acoustic features and duration information during the training phase. During the synthesis phase, acoustic feature parameters are predicted using the duration and acoustic models. These acoustic feature parameters are then post-processed, and the speech waveform is finally restored using a vocoder. This method can achieve relatively stable synthesis results even with a relatively small speech corpus. However, its drawbacks include the over-smoothing of acoustic feature parameters caused by statistical modeling and the degradation of sound quality caused by the vocoder.

[0068] For concatenative synthesis in the backend system, the training phase is essentially the same as parameter synthesis. During the synthesis phase, model calculations guide unit selection, a dynamic programming algorithm is used to select the optimal unit sequence, and the selected units are then subjected to energy regularization and waveform concatenation. Concatenative synthesis directly uses real speech clips, preserving speech quality to the greatest extent possible. However, the disadvantage is that the required sound library is generally large, and synthesis performance cannot be guaranteed for out-of-domain text.

[0069] Multi-speech synthesis model: This model collects speech data from multiple different speakers as training data, and then trains the speech synthesis model based on the training data. The trained multi-speech synthesis model can process a piece of text information input by the user through speech synthesis and output speech information of multiple different speech categories, thereby achieving the effect of personalized speech synthesis.

[0070] Mel Frequency Cepstral Coefficents (MFCCs): These are the coefficients that make up the Mel frequency cepstral. The frequency bands of the Mel frequency cepstral are equally spaced on the Mel scale. They better approximate the human auditory system than the linearly spaced frequency bands used in the normal logarithmic cepstrum and are widely used in speech recognition, deep learning and other fields.

[0071] Generalized end-to-end (GE2E): Based on TE2E, the loss function is improved. GE2E loss includes two specific forms: softmax and contrast. The goal of each form is not only to increase the cosine score between the sample and the speaker center to which it belongs, but also to reduce the cosine score between the sample and the speaker center to which it belongs. In addition, when GE2E calculates the consine score, it uses a similarity matrix calculation form to calculate all consine scores at once, which can significantly speed up the calculation process compared to TE2E. The GE2E network model uses a multi-layer LSTMP form to extract the speaker expression of the input sample, then calculates the cosine score of the verification sample and all speaker centers, and finally performs loss statistics based on the similarity score through softmax or contrast, and backpropagates to update the parameters.

[0072] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI): is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, speech recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0074] Related art methods for personalized speech generation require collecting large amounts of user-recorded speech data based on specified text to train a speech synthesis model, enabling it to output speech information similar to the user's speech category. However, using this method for personalized speech generation requires training a separate model for each user, resulting in high model deployment costs and a cumbersome and time-consuming model training process.

[0075] Based on this, the embodiments of the present application provide a speech category acquisition method, a speech category acquisition device, an electronic device and a storage medium, which can conveniently obtain speech categories similar to the user's language features without the user having to record a large amount of speech training data, thereby meeting the needs of personalized speech synthesis.

[0076] The embodiments of the present application provide a method for obtaining a voice category, a device for obtaining a voice category, an electronic device, and a storage medium, which are specifically described through the following embodiments. First, the method for obtaining a voice category in the embodiments of the present application is described.

[0077] The voice category acquisition method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The voice category acquisition method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the voice category acquisition method, etc., but is not limited to the above forms.

[0078] Figure 1 This is an optional flowchart of the method for obtaining the voice category provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S100 to S600.

[0079] S100, obtaining reference speech data; wherein the reference speech data includes at least two speech categories, and the reference speech data is generated by recording at least two real speakers based on first text information;

[0080] S200, obtaining original voice data of a target user; wherein the original voice data is recorded by the target user according to the first text information;

[0081] S300, extracting voiceprint features from the reference speech data to obtain a preset feature vector;

[0082] S400, performing voiceprint feature extraction processing on the original speech data to obtain a voiceprint feature vector;

[0083] S500, filtering a preset feature vector according to the voiceprint feature vector to obtain a target feature vector;

[0084] S600: Filter out a target speech category from at least two speech categories using a target feature vector.

[0085] The voice category acquisition method disclosed in the embodiment of the present application first obtains the original voice data recorded by the target user and the reference voice data recorded by a real speaker, and then performs voiceprint feature extraction processing on the original voice data and the reference voice data respectively to obtain the corresponding voiceprint feature vector and the preset feature vector, and filters out the feature vector that is most similar to the voiceprint feature vector from the preset feature vector as the target feature vector. Since the target feature vector corresponds to the reference voice data, and the reference voice data corresponds to the voice category, the voice category corresponding to the target feature vector can be obtained through screening, and the obtained voice category is most similar to the voice features of the original voice data input by the target user. In the subsequent use process, by selecting the obtained voice category to perform personalized voice synthesis on the input text of the target user, voice data similar to the voice features of the target user can be obtained, without the user having to record a large amount of voice training data to train the voice synthesis model, and the use process is simpler and more convenient.

[0086] In some embodiments, the reference speech data obtained in step S100 includes speech data of at least two speech categories. The speech categories in this application are used to characterize the speech characteristics of different speakers. Speech data recorded by different speakers may have different sound characteristics such as timbre, pitch, and intensity. By distinguishing the sound characteristics of the speech data, the corresponding speech category can be determined.

[0087] The voice categories in the embodiments of the present application correspond one-to-one to real speakers, that is, the voice data recorded by each real speaker corresponds to a voice category. For example, when the voice categories are divided according to the gender of the speaker, the voice category can be male or female, and the corresponding real speaker is a male speaker or a female speaker; when the voice categories are divided according to the age of the speaker, the voice categories can be young, middle-aged, and old, and the corresponding real speakers are young speakers, middle-aged speakers, and old speakers. In this case, when selecting real speakers, there is no need to consider gender differences. It is understandable that since the voice characteristics of different speakers are different, when dividing the voice categories, it is also possible to divide them separately according to the voice characteristics of each real speaker.

[0088] When recording the reference voice data, the real speaker needs to record the voice according to the content of the first text information. That is, in the reference voice data, the content of the voice data is the same piece of text information, and the only difference is the sound characteristics of the voice data.

[0089] In some embodiments, reference Figure 2 In step S100, obtaining reference speech data specifically includes:

[0090] S110, inputting the second text information into a preset multi-speech synthesis model to obtain virtual audio information; wherein the virtual audio information includes at least two speech categories, each speech category corresponding to a real speaker;

[0091] S120: Obtain reference voice data generated by a real speaker recording based on the first text information according to the virtual audio information.

[0092] First, a brief description of the multi-speech synthesis model of the present application is given. In an embodiment of the present application, the multi-speech synthesis model can be constructed using a multi-speech synthesis network such as Tacotron, Tacotron2, etc. When constructing the multi-speech synthesis model, the model is trained by using a training data set of multiple speakers. The training data set contains the training data of each speaker, and each person's training data can contain the speaker's speech data and corresponding text, as well as linguistic features and acoustic features extracted from the speech data and text. Each speaker has a corresponding speaker number in this model. When using the multi-speech synthesis model to output audio information, by inputting text information and selecting the corresponding speaker number, the multi-speech synthesis model can output audio information with the corresponding speaker's voice characteristics.

[0093] When acquiring reference voice data, the embodiment of the present application first uses a multi-speech synthesis model to output virtual audio information through step S110. The virtual audio information includes at least two voice categories, and each voice category corresponds to a real speaker. It is understandable that by changing the speaker number in the multi-speech synthesis model and inputting the second text information, audio information of different voice categories can be obtained, thereby obtaining virtual audio information. It is understandable that the virtual audio information in the embodiment of the present application is only used to select real speakers with similar voice categories. Therefore, the second text information input can be any text content and can be different from the first text information.

[0094] Through step S120, reference speech data generated by a real speaker recording based on the first text information is obtained based on the virtual audio information. The specific process is as follows: first, the speech categories corresponding to the multiple audio information in the virtual audio information are determined, that is, the sound features corresponding to the audio information. Then, speakers with similar sound features are selected, and the multiple speakers are regarded as real speakers. After the selection of real speakers is completed, the real speaker performs a speech recording based on the first text information, thereby obtaining reference speech data including multiple speech data. It can be understood that the obtained reference speech data corresponds one-to-one with the real speaker, the real speaker corresponds one-to-one with the virtual audio information, and the virtual audio information corresponds one-to-one with the speaker number in the multi-speech synthesis model.

[0095] Since the virtual audio information is obtained through speech synthesis using a multi-speech synthesis model, the sound features of the virtual audio information are not as natural as the sound features of a real speaker, which is not conducive to the subsequent voiceprint feature extraction process. In addition, in the subsequent comparison and screening process, the similarity comparison is performed with the original voice data recorded by the target user. If the virtual audio information is directly compared with the original voice data, the problem of inaccurate sound feature matching may occur. Therefore, the embodiment of the present application introduces a real speaker with the same voice category as the virtual audio information, and collects the real speaker's voice data as reference voice data to perform a similarity comparison with the original voice data of the target user, which can improve the accuracy in the voiceprint feature extraction and similarity matching process. This makes the target voice category finally screened out more consistent with the real voice features of the target user.

[0096] It is understood that in the embodiments of the present application, after the target voice category is obtained through screening and matching based on the user's original voice data, a specific speaker number in the multi-speech synthesis model corresponding to the target voice category can be obtained. In subsequent use, after the speaker number of the multi-speech synthesis model is set to the specific speaker number, the sound characteristics of the audio information obtained by the multi-speech synthesis model are the same as or similar to the voice characteristics of the target user, thereby achieving the purpose of personalized speech synthesis without using the target user's voice data to train the speech synthesis model.

[0097] In some other embodiments, virtual audio information can be obtained without going through multiple speech synthesis models. For example, the corresponding virtual audio information can be output by an independently trained speech synthesis model. In the subsequent matching and screening process, the speech synthesis model whose sound characteristics of the output speech information are most similar to those of the target user can be screened out; or when obtaining reference speech data, the speech data recorded by a real speaker based on preset text when training the speech synthesis model can be directly obtained, without having to reselect the real speaker for speech recording; or the speech categories correspond to different speech synthesis databases, and the speech synthesis databases include text information and speech information corresponding to the text information. By matching the input text information, the corresponding speech information can be obtained. The target user can also achieve the purpose of personalized speech synthesis by matching the corresponding speech synthesis database.

[0098] In some embodiments, the speech categories include: young male category, middle-aged male category, elderly male category, young female category, middle-aged female category, and elderly female category. The speech categories of the embodiment of the present application include the speech features of six real people of different ages or genders. It can be understood that when the multi-speech synthesis model in this embodiment outputs the corresponding virtual audio information, by setting the speaker number in the multi-speech synthesis model to: young male speaker number, middle-aged male speaker number, elderly male speaker number, young female speaker number, middle-aged female speaker number, and elderly female speaker number, the speech categories of the virtual audio information output after speech synthesis processing by the multi-speech synthesis model include: young male category, middle-aged male category, elderly male category, young female category, middle-aged female category, and elderly female category. Correspondingly, when selecting a real speaker by speech category, the speech features of the real speaker correspond to young male, middle-aged male, elderly male, young female, middle-aged female, and elderly female. After voice recording, reference speech data of the corresponding speech category is obtained.

[0099] After the target user compares and screens the original voice data with the reference voice data, the corresponding target voice category can be obtained, thereby completing personalized voice synthesis. In the embodiment of the present application, by setting six voice categories, the classification of the voice characteristics of multiple target users can be completed, which can meet the needs of most users, and the synthesized voice obtained is relatively similar to the user's own voice characteristics, which can meet the user's personalized voice synthesis requirements. In some other embodiments, the voice category can also be divided by gender, and the voice category can be male or female, or the voice category can be divided by age, and the voice category can be young, middle-aged, or elderly, or the voice category can be further subdivided into multiple types with unique voice characteristics.

[0100] In some embodiments, in step S200, the original voice data of the target user can be obtained in the following manner. When the voice category acquisition method of the embodiment of the present application is applied to an intelligent terminal, such as a smart phone, tablet computer, laptop computer, desktop computer or smart watch, etc., a first text message can be displayed on the intelligent terminal, and then the target user reads out the specific text content according to the displayed first text message, and the intelligent terminal performs voice collection through the voice collection device set by itself, thereby obtaining the original voice data of the target user. It can be understood that the original voice data collected by the embodiment of the present application is only used for the subsequent similarity comparison process, rather than for training the speech synthesis model. Therefore, the first text message can be a short sentence, and the duration of the original voice data collected can also be relatively short. For example, a period of 2 seconds of original voice data can be collected to meet the needs of the subsequent similarity comparison step.

[0101] When recording the original voice data, the text content of the first text information referenced must be the same as the text content referenced by the real speaker when recording the voice when obtaining the reference voice data in step S100, to facilitate the subsequent similarity comparison process. The specific text content of the first text information can be set as needed and is not specifically limited here.

[0102] In some embodiments, reference Figure 3 In step S300, voiceprint feature extraction is performed on the reference speech data to obtain a preset feature vector, which specifically includes:

[0103] S310, extracting Mel-frequency cepstral coefficient features from the reference speech data to obtain a speech feature vector;

[0104] S320: Input the speech feature vector into the trained generalized end-to-end feature extraction model to extract voiceprint features and obtain a preset feature vector.

[0105] The voiceprint feature extraction of the reference voice data in the embodiment of the present application mainly includes the following two steps. First, in step S310, feature extraction is performed on the reference voice data to extract the Mel-frequency cepstral coefficient features in the reference voice data, and the Mel-frequency cepstral coefficient features are used as voice feature vectors. When using a smart terminal to obtain reference voice data, the smart terminal can perform feature extraction on the obtained reference voice data through a built-in processor, or the obtained reference voice data can be uploaded to a cloud server, and feature extraction is performed on the reference voice data through a processing program in the cloud server to obtain a voice feature vector.

[0106] The speech feature vectors of the present application are obtained by processing reference speech data using a speech feature extraction algorithm based on Mel-frequency cepstral coefficients. In some other embodiments, the speech feature extraction algorithm may also be one of linear prediction coefficients, linear prediction cepstral coefficients, spectral line frequencies, discrete wavelet transform, and perceptual linear prediction algorithms. The aforementioned speech feature extraction algorithm can be used to process the reference speech data into low-dimensional, numerically represented vector features. The extracted speech feature vectors preserve sound state information while reducing the amount of computation required in subsequent processing.

[0107] After the speech feature vector is extracted, the speech feature vector is input into a trained generalized end-to-end feature extraction model to perform voiceprint feature extraction in step S320 to obtain a preset feature vector. The generalized end-to-end feature extraction model in the embodiment of the present application is a GE2E model. After being trained with a large amount of speech data, the generalized end-to-end feature extraction model can better detect the similarity between the verification sample and the central speaker. It is understandable that the reference speech data includes multiple speech data, and the corresponding speech feature vector also includes multiple feature vectors. The preset feature vector after voiceprint feature extraction also includes multiple feature vectors.

[0108] In the embodiments of the present application, voiceprint features are extracted by inputting speech feature vectors into a generalized end-to-end feature extraction model. The resulting feature vectors in the preset feature vectors can more accurately distinguish the voice characteristics of different speakers, thereby improving the accuracy of vector matching. In some other embodiments, only the speech feature extraction algorithm can be used to extract feature vectors of the speech data in the reference speech data. The corresponding original speech data is also subjected to feature extraction using the same algorithm for subsequent similarity calculation.

[0109] In some embodiments, reference Figure 4 In step S310, the Mel-frequency cepstral coefficient features in the reference speech data are extracted to obtain a speech feature vector, which specifically includes:

[0110] S311, performing pre-emphasis processing on the reference speech data to obtain high-frequency information;

[0111] S312, performing frame processing on the high-frequency information to obtain frame information;

[0112] S313, performing windowing processing on the frame information to obtain optimized information;

[0113] S314, performing fast Fourier transform on the optimized information to obtain frequency domain information;

[0114] S315, filtering the frequency domain information through a Mel filter bank to obtain filtered information;

[0115] S316, performing logarithmic processing on the filtered information to obtain local information;

[0116] S317, performing discrete cosine transform on the local information to obtain compressed information;

[0117] S318: Perform dynamic differential parameter extraction on the compressed information to obtain a speech feature vector.

[0118] The present embodiment uses Mel-frequency cepstral coefficients to extract features from the reference speech data. The specific process is as follows: The reference speech data is first pre-emphasized, multiplying it in the frequency domain by a coefficient that is positively correlated with frequency to extract the high-frequency information contained in the sound information. In some other embodiments, a high-pass filter can also be used to achieve this.

[0119] Then, the high-frequency information is framed and divided into multiple fixed segments of sound signals to obtain frame information, which is convenient for subsequent processing. The frame information is windowed to eliminate spectral leakage caused by the ends of each segment of the sound signal. Commonly used window functions include square window, Hamming window and Hanning window, which can be selected according to the specific processing process. After windowing, optimized information is obtained.

[0120] The optimized information is then subjected to a fast Fourier transform, and the time domain signal of the optimized information is converted into a frequency domain signal to obtain frequency domain information; since there is a lot of redundancy in the frequency domain information, the frequency domain information is input into the Mel filter group, and the frequency domain information is filtered, and each frequency band is represented by a value to obtain the filtered information after filtering; since the human ear's perception of sound is not linear, the filtered information is logarithmically processed to obtain local information; then the local information is subjected to a discrete cosine transform, and the local information is reduced in dimension to obtain compressed information; finally, the compressed information is subjected to a discrete cosine transform to obtain a one-dimensional speech feature vector, which is convenient for the subsequent vector comparison process.

[0121] In some embodiments, in step S400, voiceprint feature extraction is performed on the original speech data to obtain a voiceprint feature vector. The specific method for performing voiceprint feature extraction on the original speech data is the same as the specific method for performing voiceprint feature extraction on the reference speech data, to ensure comparability between the extracted voiceprint feature vector and the preset feature vector. The specific process is the same as the method for performing voiceprint feature extraction on the reference speech in the above embodiment and will not be further described here.

[0122] In some embodiments, reference Figure 5 In step S500, the preset feature vector is screened according to the voiceprint feature vector to obtain the target feature vector, including:

[0123] S510, performing similarity calculation on the voiceprint feature vector and the preset feature vector to obtain a similarity value;

[0124] S520: Filter out a target feature vector from the preset feature vectors according to the similarity value.

[0125] When obtaining the target feature vector in the embodiment of the present application, a similarity calculation method is adopted. First, through step S510, the similarity between the voiceprint feature vector and the preset feature vector is calculated. The number of feature vectors in the voiceprint feature vector is only one, and the number of feature vectors in the preset feature vector is multiple. Therefore, through step S510, when calculating the similarity between the voiceprint feature vector and the preset feature vector, the similarity between the voiceprint feature vector and the multiple feature vectors in the preset feature vector is calculated respectively, thereby obtaining a similarity value including multiple similarity data. Then, through step S520, the target feature vector is filtered out from the preset feature vector according to the similarity value. The target feature vector of the present application is the feature vector in the preset feature vector that is most similar to the voiceprint feature vector.

[0126] In some embodiments, reference Figure 6 In step S500, the preset feature vector is screened according to the voiceprint feature vector to obtain the target feature vector, and the method further includes:

[0127] S530, calculating the cosine similarity between the voiceprint feature vector and the preset feature vector to obtain a similarity value;

[0128] S540, performing maximum value screening processing on the similarity values ​​to obtain the maximum similarity value;

[0129] S550: Filter out a target feature vector from the preset feature vectors according to the maximum similarity value.

[0130] The similarity calculation in the present embodiment specifically employs cosine similarity. First, in step S530, the cosine similarity between the voiceprint feature vector and a preset feature vector is calculated. When comparing the cosine similarity between two feature vectors, the feature vectors must have the same dimensions. The resulting cosine similarity data is then used as the similarity value. When the similarity value is cosine similarity data, the larger the cosine similarity value, the higher the similarity between the two feature vectors. In this case, in step S540, the similarity values ​​are filtered for maximum value to obtain the maximum similarity value. Finally, in step S550, based on the correspondence between the maximum similarity values, a target feature vector is filtered from the preset feature vectors, thereby obtaining the feature vector that is most similar to the voiceprint feature vector among the preset feature vectors. In other embodiments, Euclidean similarity, edit distance, or other methods may also be used to calculate the similarity between the voiceprint feature vector and the preset feature vector. It will be appreciated that the specific process for filtering the target feature vector based on the similarity value is related to the specific method for calculating similarity; it is sufficient as long as the target feature vector obtained by filtering is the feature vector that is most similar to the voiceprint feature vector among the preset feature vectors.

[0131] In some embodiments, reference Figure 7 In step S600, the target speech category is selected from at least two speech categories using the target feature vector, including:

[0132] S610, performing matching processing on the reference speech data using the target feature vector to obtain a target reference speech;

[0133] S620: Match at least two speech categories using the target reference speech to obtain a target speech category.

[0134] Since the target feature vector is obtained by screening from the preset feature vector, and the preset feature vector is obtained by extracting the voiceprint feature of the reference voice data, according to the corresponding relationship of the target feature vector, through step S610, the corresponding voice data can be matched from the reference voice data to serve as the target reference voice. Since the reference voice data is recorded and generated by a real speaker according to the voice category, the target reference voice can obtain the corresponding voice category after matching processing and serve as the target voice category. The obtained target voice category is similar to the voice characteristics of the target user himself. When the reference voice is obtained from a multi-speech synthesis model, the target voice category will correspond to the speaker number in the multi-speech synthesis model. Subsequently, by selecting the speaker number in the multi-speech synthesis model, the multi-speech synthesis model will output a synthesized voice with voice characteristics that are the same or similar to those of the target user, thereby achieving the function of personalized voice selection.

[0135] Reference Figure 8The following describes the speech category acquisition method of the present application in detail using a specific embodiment. In this embodiment, corresponding reference speech data is obtained using a multi-speech synthesis model. First, by selecting the speaker numbers in the multi-speech synthesis model, the speaker numbers are set to young male speaker number, middle-aged male speaker number, elderly male speaker number, young female speaker number, middle-aged female speaker number, and elderly female speaker number, respectively. The corresponding speech categories are young male category, middle-aged male category, elderly male category, young female category, middle-aged female category, and elderly female category, and the corresponding virtual audio is young male audio, middle-aged male audio, elderly male audio, young female audio, middle-aged female audio, and elderly female audio. After obtaining the virtual audio information, the corresponding real speaker is selected based on the virtual audio information. In this embodiment, it is young male, middle-aged male, elderly male, young female, middle-aged female, and elderly female. Then, the real speaker records the speech based on the first text information, generating recordings of young male, middle-aged male, elderly male, young female, middle-aged female, and elderly female, respectively, as reference speech data.

[0136] When the target user uses the voice category acquisition method of the present application, the target user first records the original voice data on one end according to the first text information, and then performs voiceprint feature extraction, similarity calculation, target feature vector screening and other steps on the original voice data and the reference voice data, so as to obtain the target voice category corresponding to the target user. For example, if the target user is a middle-aged male, the target voice category obtained by the voice category acquisition method of the present application is a middle-aged male. Through the corresponding relationship, the speaker number of the multi-speech synthesis model can be set to the middle-aged male speaker number. In the subsequent process of using the multi-speech synthesis model for speech synthesis, the speech category of the synthesized speech is the middle-aged male category, and the speech features of the synthesized speech are relatively similar to the speech features of the target user, thereby achieving the effect of personalized speech synthesis.

[0137] In some embodiments, reference Figure 9 The present application also proposes a voice category acquisition device, which includes:

[0138] A first speech acquisition module, the first speech acquisition module is used to acquire reference speech data; wherein the reference speech data includes at least two speech categories, and the reference speech data is generated by recording at least two real speakers based on the first text information;

[0139] A second voice acquisition module, the second voice acquisition module is used to acquire original voice data of the target user; wherein the original voice data is recorded by the target user according to the first text information;

[0140] The feature extraction module is used to extract voiceprint features from the reference speech data to obtain a preset feature vector, and to extract voiceprint features from the original speech data to obtain a voiceprint feature vector;

[0141] A first screening module is used to screen the preset feature vector according to the voiceprint feature vector to obtain a target feature vector;

[0142] The second screening module is used to screen out a target speech category from at least two speech categories using a target feature vector.

[0143] The voice category acquisition device of the embodiment of the present application first obtains the original voice data recorded by the target user and the reference voice data recorded by a real speaker, and then performs voiceprint feature extraction processing on the original voice data and the reference voice data respectively to obtain the corresponding voiceprint feature vector and the preset feature vector, and filters out the feature vector that is most similar to the voiceprint feature vector from the preset feature vector as the target feature vector. Since the target feature vector corresponds to the reference voice data, and the reference voice data corresponds to the voice category, the voice category corresponding to the target feature vector can be obtained through screening, and the obtained voice category is most similar to the voice features of the original voice data input by the target user. In the subsequent use process, by selecting the obtained voice category to perform personalized voice synthesis on the input text of the target user, voice data similar to the voice features of the target user can be obtained. There is no need for the user to record a large amount of voice training data to train the voice synthesis model, and the use process is simpler and more convenient.

[0144] In some embodiments, the first voice acquisition module obtains reference voice data, specifically including: inputting the second text information into a preset multi-voice synthesis model to obtain virtual audio information; wherein the virtual audio information includes at least two voice categories, each voice category corresponds to a real speaker; based on the virtual audio information, obtaining reference voice data generated by the real speaker recording based on the first text information.

[0145] In some embodiments, the voice categories include: young male category, middle-aged male category, old male category, young female category, middle-aged female category, old female category.

[0146] In some embodiments, the feature extraction module performs voiceprint feature extraction on the reference speech data to obtain a preset feature vector, specifically including: extracting the Mel-frequency cepstral coefficient features in the reference speech data to obtain a speech feature vector; inputting the speech feature vector into a trained generalized end-to-end feature extraction model to perform voiceprint feature extraction to obtain a preset feature vector.

[0147] In some embodiments, the feature extraction module extracts the Mel-frequency cepstral coefficient features in the reference speech data to obtain a speech feature vector, specifically including: pre-emphasis processing on the reference speech data to obtain high-frequency information; framing processing on the high-frequency information to obtain framing information; windowing processing on the framing information to obtain optimization information; performing fast Fourier transform on the optimization information to obtain frequency domain information; filtering the frequency domain information through a Mel filter group to obtain filtered information; taking the logarithm of the filtered information to obtain local information; performing discrete cosine transform on the local information to obtain compressed information; and performing dynamic differential parameter extraction on the compressed information to obtain a speech feature vector.

[0148] In some embodiments, the first screening module is used to screen the preset feature vector according to the voiceprint feature vector to obtain the target feature vector, specifically including: calculating the similarity between the voiceprint feature vector and the preset feature vector to obtain a similarity value; and screening the target feature vector from the preset feature vector according to the similarity value.

[0149] In some embodiments, the first screening module is used to screen the preset feature vector based on the voiceprint feature vector to obtain the target feature vector, and also includes: calculating the cosine similarity between the voiceprint feature vector and the preset feature vector to obtain a similarity value; performing maximum value screening on the similarity value to obtain a maximum similarity value; and screening the target feature vector from the preset feature vector based on the maximum similarity value.

[0150] In some embodiments, the second screening module filters out the target speech category from at least two speech categories through the target feature vector, specifically including: matching the reference speech data through the target feature vector to obtain the target reference speech; matching at least two speech categories through the target reference speech to obtain the target speech category.

[0151] An embodiment of the present application further provides an electronic device, including:

[0152] at least one memory;

[0153] at least one processor;

[0154] at least one program;

[0155] The program is stored in the memory, and the processor executes the at least one program to implement the above-mentioned voice category acquisition method implemented in this application. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0156] See also Figure 10 , Figure 10The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0157] The processor may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0158] The memory can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called by the processor to execute the voice category acquisition method of the embodiments of this application;

[0159] Input / output interface, used to realize information input and output;

[0160] Input / communication interface, used to enable communication between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0161] A bus that transfers information between the various components of a device (e.g., processor, memory, input / output interfaces, and input / communication interfaces);

[0162] The processor, memory, input / output interface and input / communication interface are connected to each other through a bus within the device.

[0163] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned voice category acquisition method.

[0164] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The electronic device and storage medium disclosed in the embodiment of the present application first obtain the original voice data recorded by the target user and the reference voice data recorded by a real speaker, and then perform voiceprint feature extraction processing on the original voice data and the reference voice data respectively to obtain the corresponding voiceprint feature vector and the preset feature vector, and filter out the feature vector that is most similar to the voiceprint feature vector from the preset feature vector as the target feature vector. Since the target feature vector corresponds to the reference voice data, and the reference voice data corresponds to the voice category, the voice category corresponding to the target feature vector can be obtained through screening, and the obtained voice category is most similar to the voice features of the original voice data input by the target user. In the subsequent use process, by selecting the obtained voice category to perform personalized voice synthesis on the input text of the target user, voice data similar to the voice features of the target user can be obtained, without the user having to record a large amount of voice training data to train the voice synthesis model, and the use process is simpler and more convenient.

[0166] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0167] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0168] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0169] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0170] It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in sequences other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0171] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0173] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0174] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0176] The embodiments of the present application have been described in detail above with reference to the accompanying drawings. However, the present application is not limited to the above embodiments. Various modifications can be made within the scope of knowledge possessed by ordinary technicians in the relevant technical field without departing from the purpose of the present application. In addition, the embodiments of the present application and the features of the embodiments can be combined with each other unless there is a conflict.

Claims

1. A method for acquiring a speech category, characterized in that: The method comprises: Acquire reference voice data; wherein the reference voice data includes at least two voice categories, and the reference voice data is generated by recording at least two real speakers based on the first text information; Acquire original voice data of a target user; wherein the original voice data is recorded by the target user based on the first text information; Performing voiceprint feature extraction on the reference voice data to obtain a preset feature vector, and performing voiceprint feature extraction processing on the original voice data to obtain a voiceprint feature vector; Filtering the preset feature vector according to the voiceprint feature vector to obtain a target feature vector; Filtering a target speech category from the at least two speech categories using the target feature vector; The obtaining of reference voice data includes: By changing the speaker number in a preset multi-speech synthesis model and inputting the second text information, virtual audio information is obtained; wherein the virtual audio information includes the at least two speech categories, each speech category corresponds to one real speaker, each real speaker has a speaker number, any two speech categories are different, and any two real speakers are different; According to the virtual audio information, reference voice data generated by the real speaker through recording based on the first text information is obtained.

2. The method according to claim 1, characterized in that The extracting voiceprint features from the reference voice data to obtain a preset feature vector includes: Extracting Mel-frequency cepstral coefficient features from the reference speech data to obtain a speech feature vector; The speech feature vector is input into a trained generalized end-to-end feature extraction model to extract voiceprint features to obtain the preset feature vector.

3. The method according to claim 2, characterized in that The step of extracting Mel-frequency cepstral coefficient features from the reference speech data to obtain a speech feature vector includes: performing pre-emphasis processing on the reference speech data to obtain high-frequency information; performing frame processing on the high-frequency information to obtain frame information; Performing windowing processing on the frame information to obtain optimized information; Performing a fast Fourier transform on the optimization information to obtain frequency domain information; Filtering the frequency domain information through a Mel filter bank to obtain filtered information; Performing logarithmic processing on the filtered information to obtain local information; performing discrete cosine transform on the local information to obtain compressed information; Dynamic differential parameter extraction is performed on the compressed information to obtain the speech feature vector.

4. The method according to any one of claims 1 to 3, characterized in that The filtering process of the preset feature vector according to the voiceprint feature vector to obtain a target feature vector includes: Calculating the similarity between the voiceprint feature vector and the preset feature vector to obtain a similarity value; The target feature vector is selected from the preset feature vectors according to the similarity value.

5. The method according to claim 4, characterized in that The filtering and processing of the preset feature vector according to the voiceprint feature vector to obtain a target feature vector further includes: Calculating the cosine similarity between the voiceprint feature vector and the preset feature vector to obtain the similarity value; Performing maximum value screening processing on the similarity values ​​to obtain a maximum similarity value; The target feature vector is selected from the preset feature vectors according to the maximum similarity value.

6. The method according to claim 4, characterized in that The step of filtering out a target speech category from the at least two speech categories by using the target feature vector includes: Performing matching processing on the reference speech data using the target feature vector to obtain a target reference speech; Matching processing is performed on the at least two speech categories using the target reference speech to obtain the target speech category.

7. A device for acquiring a speech category, characterized in that: The device comprises: A first speech acquisition module, configured to acquire reference speech data; wherein the reference speech data includes at least two speech categories and is generated by recording at least two real speakers based on the first text information; a second voice acquisition module, configured to acquire original voice data of a target user; wherein the original voice data is recorded by the target user based on the first text information; a feature extraction module configured to perform voiceprint feature extraction on the reference speech data to obtain a preset feature vector, and perform voiceprint feature extraction processing on the original speech data to obtain a voiceprint feature vector; a first screening module, configured to screen the preset feature vector according to the voiceprint feature vector to obtain a target feature vector; a second screening module, configured to screen out a target speech category from the at least two speech categories using the target feature vector; The first voice acquisition module is used to acquire reference voice data, including: By changing the speaker number in a preset multi-speech synthesis model and inputting the second text information, virtual audio information is obtained; wherein the virtual audio information includes the at least two speech categories, each speech category corresponds to one real speaker, each real speaker has a speaker number, any two speech categories are different, and any two real speakers are different; According to the virtual audio information, reference voice data generated by the real speaker through recording based on the first text information is obtained.

8. An electronic device, characterized in that: include: at least one memory; at least one processor; at least one program; The program is stored in the memory, and the processor executes at least one of the programs to implement: The method according to any one of claims 1 to 6.

9. A storage medium, wherein the storage medium is a computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute: The method according to any one of claims 1 to 6.

Images