Voice model adjusting method and device

Abstract

The invention discloses a voice model adjusting method and device, and the method comprises the steps: obtaining a first text and user voice, and enabling a user to read the voice of the first text; determining a plurality of unmatched audio clips which are not matched with the first text in the user voice; multiple target audio clips are determined from the multiple unmatched audio clips, the multiple target audio clips have a corresponding relation, and the number of the target audio clips meets the number requirement; determining a target audio feature according to the target audio clip; and performing parameter adjustment on the first voice model based on the target audio feature to obtain a second voice model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for adjusting a speech model. Background Art

[0002] Currently, the user's voice can be cloned through the voice model to obtain a voice that matches the user's voice. Since the user may not pronounce some words standardly, the voice output by the voice model cannot accurately match the user's pronunciation characteristics, and the output voice has low accuracy. Summary of the invention

[0003] An embodiment of the present application provides a speech model adjustment method, including: obtaining a first text and a user voice, wherein the user voice is the voice of a user reading the first text; determining a plurality of unmatched audio segments in the user voice that do not match the first text; determining a plurality of target audio segments from the plurality of unmatched audio segments, wherein the plurality of target audio segments have a corresponding relationship with each other, and the number of the target audio segments meets a quantity requirement; determining a target audio feature according to the target audio segment; and adjusting parameters of the first speech model based on the target audio feature to obtain a second speech model.

[0004] In some embodiments, determining multiple unmatched audio segments in the user voice that do not match the first text includes: performing voice recognition on the user voice to obtain a second text; comparing the second text with the first text to determine text information in the second text that does not match the first text; and determining the unmatched audio segments from the user voice based on the text information.

[0005] In some embodiments, determining multiple unmatched audio segments in the user voice that do not match the first text includes: performing speech synthesis on the first text to obtain a first audio; and comparing the first audio with the user voice to obtain the unmatched audio segments.

[0006] In some embodiments, performing speech synthesis on the first text to obtain the first audio includes: determining the pronunciation style parameters of the user's voice; and performing speech synthesis on the first text using a speech synthesis model that conforms to the pronunciation style parameters to obtain the first audio.

[0007] In some embodiments, determining multiple target audio segments from the multiple unmatched audio segments includes: clustering the unmatched audio segments to obtain multiple cluster sets; determining a target cluster set from the multiple cluster sets, the number of unmatched audio segments in the target cluster set meets the number requirement, and the unmatched audio segments in the target cluster set are the target audio segments.

[0008] In some embodiments, clustering the unmatched audio segments includes at least one of the following: clustering the unmatched audio segments according to text corresponding to the unmatched audio segments; clustering the unmatched audio segments according to phonemes corresponding to the unmatched audio segments.

[0009] In some embodiments, the characters corresponding to the target cluster set are different from any characters in a preset character library.

[0010] In some embodiments, the method of adjusting parameters of the first speech model based on the target audio feature to obtain the second speech model includes: generating a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature; and adjusting parameters of the first speech model based on the phoneme audio feature pair to obtain the second speech model.

[0011] In some embodiments, the first speech model includes a first speech cloning sub-model and a first speech adjustment sub-model, and the parameters of the first speech model are adjusted based on the target audio feature to obtain the second speech model, including: generating a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature; based on the phoneme audio feature pair, adjusting the parameters of the first speech adjustment sub-model to obtain the second speech model, the first speech cloning sub-model can generate the first speech after the third text is input, and the adjusted first speech adjustment sub-model can generate the target speech after the first speech is input.

[0012] An embodiment of the present application also proposes a speech model adjustment device, including: an acquisition module, used to obtain a first text and a user voice, wherein the user voice is the voice of the user reading the first text; a first determination module, used to determine a plurality of non-matching audio segments in the user voice that do not match the first text; a second determination module, used to determine a plurality of target audio segments from the plurality of non-matching audio segments, wherein there is a corresponding relationship between the plurality of target audio segments, and the number of the target audio segments meets the quantity requirement; a third determination module, used to determine a target audio feature according to the target audio segment; and an adjustment module, used to adjust the parameters of the first speech model based on the target audio feature to obtain a second speech model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 A flowchart of a method for adjusting a speech model according to an embodiment of the present application;

[0015] Figure 2 The process of determining multiple unmatched audio segments according to an embodiment of the present application Figure 1 ;

[0016] Figure 3 The process of determining multiple unmatched audio segments according to an embodiment of the present application Figure 2 ;

[0017] Figure 4 This is a flowchart of performing speech synthesis on a first text according to an embodiment of the present application;

[0018] Figure 5 A flowchart of determining multiple target audio segments according to an embodiment of the present application;

[0019] Figure 6 The process of adjusting the parameters of the first speech model based on the target audio features in the embodiment of the present application is as follows: Figure 1 ;

[0020] Figure 7 The process of adjusting the parameters of the first speech model based on the target audio features in the embodiment of the present application Figure 2 ;

[0021] Figure 8 This is a structural block diagram of a speech model adjustment device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0023] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of the embodiments. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0024] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0025] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0026] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0027] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0028] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0029] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0030] A speech model adjustment method according to an embodiment of the present application determines multiple unmatched audio segments in a user's speech that do not match a first text, and determines multiple target audio segments from the multiple unmatched audio segments, and then adjusts parameters of a first speech model according to target audio features determined by the target audio segments to obtain a second speech model. Since there is a corresponding relationship between the multiple target audio segments and the number of target audio segments meets the quantity requirement, it is possible to extract audio features that are more consistent with the user's pronunciation features, so that the adjusted speech model can output speech that is more consistent with the user's pronunciation features, thereby improving the accuracy of the speech output by the speech model.

[0031] like Figure 1 As shown, the following steps are included:

[0032] Step S101 , obtaining a first text and a user voice, wherein the user voice is the voice of a user reading out the first text.

[0033] In this embodiment, the second voice model is obtained by adjusting the parameters of the first voice model. The first voice model can specifically be a first voice cloning model. Specifically, the first voice model can perform voice synthesis on the text after the text is input to obtain a voice that simulates the user. The first voice model can adopt any one of the voice cloning models including ChatTTS, Seed-TTS, VALL-E2, Coqui TTS, etc.

[0034] The first text may be a text that meets the target conditions, which may include one or more of the following conditions: reaching a target number of words, conforming to a target language, and corresponding phonemes conforming to target phoneme types, so that the user's voice can more comprehensively and accurately reflect the user's pronunciation characteristics.

[0035] The user voice can be obtained by recording the voice of the user reading the first text. In some embodiments, the recorded voice may contain non-human voice data interference due to factors such as the user's own activities or the recording environment. The denoising model can be used to perform differentiated noise reduction processing on the recorded voice to obtain the user voice, so as to improve the accuracy of the user voice.

[0036] Step S102: determining a plurality of unmatched audio segments in the user voice that do not match the first text.

[0037] In this embodiment, when the user reads the first text, the pronunciation may not meet the target standard, resulting in a mismatch between the pronunciation and the content in the first text. Multiple audio segments in the user's voice that do not match the first text are determined as multiple mismatched audio segments. The mismatched audio segments may correspond to one or more characters in the first text, one or more words in the first text, or one or more sentences in the first text.

[0038] Step S103: determining a plurality of target audio segments from the plurality of unmatched audio segments, wherein the plurality of target audio segments have a corresponding relationship with each other and the number of the target audio segments meets the quantity requirement.

[0039] In this embodiment, after determining multiple unmatched audio segments, multiple target audio segments are determined from the multiple unmatched audio segments, and there is a corresponding relationship between the multiple target audio segments. For example, the multiple target audio segments may all correspond to the same character or word in the first text, and the multiple target audio segments may also correspond to multiple characters or words in the first text. The number of target audio segments meets the quantity requirement, so that the target audio segments can accurately characterize the pronunciation characteristics of the user. For example, if the multiple target audio segments all correspond to the same character or word in the first text, the number of target audio segments reaches the target number. If the multiple target audio segments correspond to multiple characters or words in the first text, the number of target audio segments corresponding to the same character or word reaches the target number.

[0040] Step S104: determining a target audio feature according to the target audio segment.

[0041] In this embodiment, the target audio features are determined by extracting features from the target audio segment. The target audio features may include one or more of time domain features (such as duration, energy, short-time energy), frequency domain features (such as spectrum, power spectrum, Mel spectrum), transform domain features (such as Mel-frequency cepstral coefficients, linear predictive coding), statistical features (such as zero-crossing rate, short-time autocorrelation function), perceptual features (such as loudness, pitch, timbre), and other features (such as fundamental frequency, resonance rate, speaking rate).

[0042] Step S105: adjusting parameters of the first speech model based on the target audio features to obtain a second speech model.

[0043] In this embodiment, sample text and target audio features can be input into the first speech model for model training. During the training process, the parameters of the first speech model are adjusted so that the first speech model can learn the target audio features and output speech that meets the target audio features. After the training is completed, the second speech model is obtained.

[0044] The speech model adjustment method of the embodiment of the present application obtains a first text and a user speech, wherein the user speech is the speech of the user reading the first text; determines a plurality of mismatched audio segments in the user speech that do not match the first text; determines a plurality of target audio segments from the plurality of mismatched audio segments, wherein the plurality of target audio segments have a corresponding relationship, and the number of the target audio segments meets the quantity requirement; determines the target audio features according to the target audio segments; and adjusts the parameters of the first speech model based on the target audio features to obtain a second speech model. This allows the extraction of audio features that are more consistent with the user's pronunciation features, and further enables the adjusted speech model to output speech that is more consistent with the user's pronunciation features, thereby improving the accuracy of the speech output by the speech model.

[0045] In some embodiments of the present application, the determining of multiple unmatched audio segments in the user speech that do not match the first text is as follows: Figure 2 As shown, the following steps are included:

[0046] Step S1021: Perform voice recognition on the user voice to obtain a second text.

[0047] In this embodiment, the user's voice is recognized by speech recognition technology, and the user's voice is converted into text to obtain a second text. Among them, the speech recognition technology can adopt any one of dynamic time warping, hidden Markov model, vector quantization, Gaussian mixture model, support vector machine, artificial neural network and deep neural network. In some embodiments of the present application, the pronunciation style parameters of the user can be obtained, such as the pronunciation style parameters of the user's voice can be determined according to the user's gender, the dialect to which the user's voice belongs, the region or country to which the user's voice belongs, and the pronunciation style parameters of the user's voice are used for speech recognition according to the pronunciation style parameters to obtain the second text.

[0048] Step S1022: compare the second text with the first text to determine text information in the second text that does not match the first text.

[0049] In this embodiment, the second text is compared with the first text, and the characters, words, or sentences in the second text that do not match the first text are determined as unmatched text information. Specifically, the second text and the first text can be compared by any of the methods including the longest common subsequence algorithm, a comparison algorithm based on a hash value, a Levenshtein distance, an algorithm based on an edit distance, a longest common substring algorithm, a bag-of-words model, a method based on deep learning, etc., or the first text and the second text are directly compared word by word.

[0050] Step S1023: determining the unmatched audio segment from the user voice according to the text information.

[0051] The corresponding audio segment is determined from the user's voice according to the text information, and the corresponding audio segment is determined as an unmatched audio segment, thereby achieving efficient determination of the unmatched audio segment.

[0052] In some embodiments of the present application, determining the unmatched audio segment from the user voice based on the text information includes: determining the voice time slice corresponding to the text information, and determining the unmatched audio segment from the user voice based on the voice time slice, thereby achieving more accurate determination of the unmatched audio segment.

[0053] In some embodiments of the present application, as an alternative, the determining of multiple unmatched audio segments in the user speech that do not match the first text, such as Figure 3 As shown, the following steps are included:

[0054] Step S1024: perform speech synthesis on the first text to obtain a first audio.

[0055] In this embodiment, a speech synthesis model is used to perform speech synthesis on the first text to obtain a first audio. The speech synthesis model can adopt any one of the models including Tacotron2, GST, Glow-TTS, Flow-TTS, BLSTM-RNN, WaveNet, Deep Voice, etc.

[0056] Step S1025: compare the first audio with the user voice to obtain the unmatched audio segment.

[0057] The first audio is compared with the user's voice to determine the audio segments in the user's voice that do not match the first audio, and the unmatched audio segments are determined as unmatched audio segments, thereby achieving efficient determination of unmatched audio segments.

[0058] In some embodiments of the present application, speech synthesis is performed on the first text to obtain a first audio, such as Figure 4 As shown, the following steps are included:

[0059] Step S1241, determining the pronunciation style parameters of the user's voice.

[0060] In this embodiment, the pronunciation style parameter of the user's voice may be determined according to one or more of the user's gender, the dialect to which the user's voice belongs, and the region or country to which the user's voice belongs.

[0061] Step S1242: Perform speech synthesis on the first text using a speech synthesis model that conforms to the pronunciation style parameters to obtain the first audio.

[0062] In this embodiment, the first text is speech synthesized using a speech synthesis model that conforms to the pronunciation style parameters, so that the first audio can conform to the pronunciation style parameters of the user audio. After comparing the two, the unmatched audio in the user voice can be determined more accurately.

[0063] For example, if the pronunciation style parameters are the user's gender and the dialect of the user's voice, and the user is male and the dialect is the target dialect, the first text is speech synthesized using the corresponding speech synthesis model to obtain a first audio that conforms to the target dialect of male pronunciation.

[0064] In some embodiments of the present application, the step of determining a plurality of target audio segments from the plurality of unmatched audio segments is as follows: Figure 5 As shown, the following steps are included:

[0065] Step S1031: cluster the unmatched audio segments to obtain a plurality of cluster sets.

[0066] In this embodiment, after determining the unmatched audio segments, the matching audio segments are clustered by a clustering algorithm to obtain multiple cluster sets. The clustering algorithm may be, for example, any one of the K-Means algorithm, GMM (Gaussian mixture model), DBSCAN (density-based spatial clustering noise reduction) and other algorithms.

[0067] Step S1032: determining a target cluster set from the multiple cluster sets, the number of the unmatched audio segments in the target cluster set meeting the number requirement, and the unmatched audio segments in the target cluster set being the target audio segments.

[0068] In this embodiment, after determining multiple cluster sets, the number of unmatched audio clips in each cluster set is determined. If the number of unmatched audio clips meets the quantity requirement, the corresponding cluster set is determined as the target cluster set, and each unmatched audio clip in the target cluster set is determined as the target audio clip, thereby achieving efficient determination of the target audio clip. Meeting the quantity requirement may include at least one of the following: the number of unmatched audio clips reaches a quantity threshold, and the number of unmatched audio clips ranks in the top N among multiple cluster sets.

[0069] In some embodiments of the present application, clustering the mismatched audio segments includes at least one of the following:

[0070] performing clustering processing on the unmatched audio segments according to the texts corresponding to the unmatched audio segments;

[0071] The unmatched audio segments are clustered according to the phonemes corresponding to the unmatched audio segments.

[0072] In some embodiments, the non-matching audio segments are clustered according to the text corresponding to the non-matching audio segments, including at least one of the following: clustering the non-matching audio segments corresponding to the same text into the same category; clustering the non-matching audio segments corresponding to the same pronunciation text into the same category.

[0073] In this embodiment, phonemes are the smallest speech units divided according to the natural properties of speech. Unmatched audio segments may correspond to corresponding texts or phonemes. The texts corresponding to the unmatched audio segments may be determined first, and the unmatched audio segments may be clustered based on the texts corresponding to the unmatched audio segments, so as to divide the unmatched audio segments belonging to the same text into a cluster set. It is also possible to first determine the phonemes corresponding to the unmatched audio segments, and cluster the unmatched audio segments based on the phonemes corresponding to the unmatched audio segments, so as to divide the unmatched audio segments belonging to the same phoneme into a cluster set. It is also possible to cluster a part of the unmatched audio segments according to the texts, and cluster the remaining part of the unmatched audio segments according to the phonemes. This allows for more flexible determination of multiple cluster sets.

[0074] In some embodiments of the present application, the text corresponding to the target cluster set is different from any text in a preset text library.

[0075] In this embodiment, when the user is reading the first text, the user may mispronounce the corresponding text due to not recognizing a certain character or word. The mispronounced text cannot represent the personal characteristics of the user. Therefore, a preset text library can be set, which includes multiple characters whose mispronunciation probability reaches the target probability. By making the characters corresponding to the target cluster set different from any characters in the preset text library, the target cluster set does not include characters that are easily mispronounced, thereby further improving the accuracy of the target audio segment.

[0076] In some embodiments of the present application, the first speech model is parameter-adjusted based on the target audio feature to obtain a second speech model, such as Figure 6 As shown, the following steps are included:

[0077] Step S1051: Generate a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature.

[0078] In this embodiment, the first speech model is a one-stage model, which determines the phonemes corresponding to the target audio features and generates phoneme audio feature pairs according to the target audio features and the corresponding phonemes.

[0079] Step S1052: Based on the phoneme audio feature pair, adjust the parameters of the first speech model to obtain the second speech model.

[0080] In this embodiment, one input of the first speech model is sample text, and the other input is a phoneme audio feature pair. The first speech model is trained by the sample text and the phoneme audio feature pair, and the parameters are adjusted according to the speech output by the first speech model. When the target number of iterations is reached or the accuracy of the output speech reaches the target accuracy, the second speech model is obtained, thereby achieving efficient acquisition of the second speech model.

[0081] In some embodiments of the present application, as an alternative, the first speech model includes a first speech cloning sub-model and a first speech adjustment sub-model, and the first speech model is parameter-adjusted based on the target audio feature to obtain a second speech model, such as Figure 7 As shown, the following steps are included:

[0082] Step S1053: Generate a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature.

[0083] In this embodiment, the first speech model is a two-stage model, including a first speech cloning sub-model in the first stage and a first speech adjustment sub-model in the second stage, and the first speech adjustment sub-model can adjust the speech output by the first speech cloning sub-model. Determine the phoneme corresponding to the target audio feature, and generate a phoneme audio feature pair according to the target audio feature and the corresponding phoneme.

[0084] Step S1054, based on the phoneme audio feature pair, the parameters of the first speech adjustment sub-model are adjusted to obtain the second speech model, the first speech cloning sub-model can generate the first speech after the third text is input, and the adjusted first speech adjustment sub-model can generate the target speech after the first speech is input.

[0085] In this embodiment, the first voice cloning sub-model outputs voice after inputting the sample text, one input of the first voice adjustment sub-model is the voice output by the first voice cloning sub-model, and the other input is the phoneme audio feature pair, the first voice adjustment sub-model is trained by the voice and phoneme audio features output by the first voice cloning sub-model, the parameters of the voice adjustment sub-model are adjusted according to the voice output by the first voice adjustment sub-model, and the training ends when the target number of iterations is reached or the accuracy of the output voice reaches the target accuracy, and the second voice model is obtained. The second voice model includes the first voice cloning sub-model and the adjusted first voice adjustment sub-model, the first voice cloning sub-model can generate the first voice after the third text is input, and the adjusted first voice adjustment sub-model can generate the target voice after the first voice is input.

[0086] The phoneme audio features are used to adjust the parameters of the first speech adjustment sub-model in the two-stage model, so that the second speech model can be obtained efficiently.

[0087] The present application also proposes a speech model adjustment device, such as Figure 8As shown, it includes: an acquisition module, used to obtain a first text and a user voice, wherein the user voice is the voice of a user reading the first text; a first determination module, used to determine a plurality of unmatched audio segments in the user voice that do not match the first text; a second determination module, used to determine a plurality of target audio segments from the plurality of unmatched audio segments, wherein there is a corresponding relationship between the plurality of target audio segments, and the number of the target audio segments meets the quantity requirement; a third determination module, used to determine a target audio feature according to the target audio segment; and an adjustment module, used to adjust the parameters of the first voice model based on the target audio feature to obtain a second voice model.

[0088] The speech model adjustment device of the embodiment of the present application determines multiple unmatched audio segments in the user's speech that do not match the first text through a second determination module, and determines multiple target audio segments from the multiple unmatched audio segments through a third determination module, and then uses the adjustment module to adjust the parameters of the first speech model according to the target audio features determined by the target audio segments to obtain a second speech model. Since there is a corresponding relationship between the multiple target audio segments and the number of target audio segments meets the quantity requirement, it is possible to extract audio features that are more in line with the user's pronunciation features, and then the adjusted speech model can output speech that is more in line with the user's pronunciation features, thereby improving the accuracy of the speech output by the speech model.

[0089] In a specific application scenario, the first determination module is specifically used to: perform speech recognition on the user voice to obtain a second text; compare the second text with the first text to determine the text information in the second text that does not match the first text; and determine the unmatched audio segment from the user voice based on the text information.

[0090] In a specific application scenario, as an alternative, the first determination module is specifically used to: perform speech synthesis on the first text to obtain a first audio; and compare the first audio with the user voice to obtain the unmatched audio segment.

[0091] In a specific application scenario, the first determination module is further specifically used to: determine the pronunciation style parameters of the user's voice; and perform speech synthesis on the first text using a speech synthesis model that conforms to the pronunciation style parameters to obtain the first audio.

[0092] In a specific application scenario, the second determination module is specifically used to: cluster the unmatched audio segments to obtain multiple cluster sets; determine a target cluster set from the multiple cluster sets, the number of unmatched audio segments in the target cluster set meets the quantity requirement, and the unmatched audio segments in the target cluster set are the target audio segments.

[0093] In a specific application scenario, the second determination module is also specifically used to execute at least one of the following: clustering the unmatched audio segments according to the text corresponding to the unmatched audio segments; clustering the unmatched audio segments according to the phonemes corresponding to the unmatched audio segments.

[0094] In a specific application scenario, the text corresponding to the target cluster set is different from any text in a preset text library.

[0095] In a specific application scenario, the adjustment module is specifically used to: generate a phoneme audio feature pair based on the target audio feature and the phoneme corresponding to the target audio feature; and adjust the parameters of the first speech model based on the phoneme audio feature pair to obtain the second speech model.

[0096] In a specific application scenario, as an alternative, the first speech model includes a first speech cloning sub-model and a first speech adjustment sub-model, and the adjustment module is specifically used to: generate a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature; based on the phoneme audio feature pair, adjust the parameters of the first speech adjustment sub-model to obtain the second speech model, the first speech cloning sub-model can generate the first speech after the third text is input, and the adjusted first speech adjustment sub-model can generate the target speech after the first speech is input.

[0097] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0098] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for adjusting a speech model, comprising: Obtaining a first text and a user voice, wherein the user voice is a voice of a user reading the first text; determining a plurality of unmatched audio segments in the user speech that do not match the first text; Determine a plurality of target audio segments from the plurality of unmatched audio segments, wherein the plurality of target audio segments have a corresponding relationship with each other, and the number of the target audio segments meets a quantity requirement; Determining a target audio feature according to the target audio segment; The parameters of the first speech model are adjusted based on the target audio features to obtain a second speech model.

2. The method for adjusting the speech model according to claim 1, wherein determining a plurality of unmatched audio segments in the user speech that do not match the first text comprises: Performing speech recognition on the user's speech to obtain a second text; Comparing the second text with the first text to determine text information in the second text that does not match the first text; The unmatched audio segment is determined from the user voice according to the text information.

3. The method for adjusting the speech model according to claim 1, wherein the step of determining a plurality of unmatched audio segments in the user speech that do not match the first text comprises: Performing speech synthesis on the first text to obtain a first audio; The first audio is compared with the user voice to obtain the unmatched audio segment.

4. The method for adjusting the speech model according to claim 3, wherein the first text is subjected to speech synthesis to obtain the first audio, comprising: Determining pronunciation style parameters of the user's voice; The first text is speech synthesized using a speech synthesis model that conforms to the pronunciation style parameters to obtain the first audio.

5. The method for adjusting the speech model according to claim 1, wherein determining a plurality of target audio segments from the plurality of unmatched audio segments comprises: Performing clustering processing on the unmatched audio segments to obtain a plurality of cluster sets; A target cluster set is determined from the multiple cluster sets, the number of the unmatched audio segments in the target cluster set meets the number requirement, and the unmatched audio segments in the target cluster set are the target audio segments.

6. The method for adjusting the speech model according to claim 5, wherein clustering the mismatched audio segments comprises at least one of the following: performing clustering processing on the unmatched audio segments according to the texts corresponding to the unmatched audio segments; The unmatched audio segments are clustered according to the phonemes corresponding to the unmatched audio segments.

7. The speech model adjustment method as described in claim 6, wherein the text corresponding to the target cluster set is different from any text in the preset text library.

8. The method for adjusting a speech model according to claim 1, wherein adjusting parameters of the first speech model based on the target audio feature to obtain the second speech model comprises: Generate a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature; Based on the phoneme audio feature pair, the parameters of the first speech model are adjusted to obtain the second speech model.

9. The method for adjusting a speech model according to claim 1, wherein the first speech model comprises a first speech cloning sub-model and a first speech adjustment sub-model, and adjusting parameters of the first speech model based on the target audio feature to obtain a second speech model comprises: Generate a phoneme audio feature pair according to the target audio feature and the phoneme corresponding to the target audio feature; Based on the phoneme audio feature pair, the parameters of the first speech adjustment submodel are adjusted to obtain the second speech model, the first speech cloning submodel can generate the first speech after the third text is input, and the adjusted first speech adjustment submodel can generate the target speech after the first speech is input.

10. A speech model adjustment device, comprising: An acquisition module, used for acquiring a first text and a user voice, wherein the user voice is a voice of a user reading out the first text; A first determination module, configured to determine a plurality of unmatched audio segments in the user voice that do not match the first text; A second determining module is used to determine a plurality of target audio segments from the plurality of unmatched audio segments, wherein the plurality of target audio segments have a corresponding relationship with each other and the number of the target audio segments meets a quantity requirement; A third determination module, configured to determine a target audio feature according to the target audio segment; An adjustment module is used to adjust parameters of the first speech model based on the target audio feature to obtain a second speech model.

Images