Accent data generation method and device, equipment and storage medium

Abstract

The invention discloses an accent data generation method and apparatus, a device and a storage medium. The method comprises the steps of obtaining original voice data; generating first voice data according to a preset target accent; slicing the original voice data to obtain each original voice slice, and slicing the first voice data to obtain each first voice slice; determining each spliced voice slice according to the original voice slice and the first voice slice; inputting each spliced voice slice into a pre-trained accent recognition model to obtain an accent tag of each spliced voice slice; and generating target accent data according to each original voice slice and based on the accent tag of each spliced voice slice and the target accent. Compared with an existing generation mode, the method is finer, the process of directly generating the accent data according to the original voice data may be affected by complex and changeable global features, and the generation effect is poor, but the slicing mode of the method can well avoid the problem.

Description

Technical Field

[0001] The present application relates to the technical field of accent data generation, and in particular to a method, device, equipment and storage medium for generating accent data. Background Art

[0002] With the acceleration of globalization and the continuous expansion of voice interaction application scenarios, such as voice assistants, intelligent customer service, language learning software, etc., the demand for voice data with diverse accents has increased dramatically. People from different regions and cultural backgrounds have their own unique accents. In order to make the use of voice technology more extensive, the industry has begun to focus on generating multiple voice data with different accents for the same voice data.

[0003] However, existing speech generation technologies often simply generate speech data with specific accent requirements from original speech data. This method is relatively rough and the generated speech data is not accurate. Summary of the invention

[0004] In view of this, the present application provides a method, apparatus, device and storage medium for generating accent data, which is used to solve the problem that the existing speech generation technology often just directly generates speech data with specific accent requirements from the original speech data, which is relatively rough and the generated speech data is not accurate.

[0005] To achieve the above objectives, the proposed solution is as follows:

[0006] In a first aspect, a method for generating accent data comprises:

[0007] Get the original voice data;

[0008] generating first voice data corresponding to the original voice data according to a preset target accent;

[0009] Slicing the original voice data to obtain original voice slices, and slicing the first voice data to obtain first voice slices;

[0010] Determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices;

[0011] Input each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain an accent label of each of the spliced ​​speech slices; the accent recognition model is trained using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label;

[0012] Target accent data is generated according to each of the original speech slices and based on the accent tags of each of the concatenated speech slices and the target accent.

[0013] Preferably, the determining of each spliced ​​voice slice from each of the original voice slices and each of the first voice slices comprises:

[0014] Combining the original speech slices into an original speech sequence, and determining the sequence position of each original speech slice as the original sequence position;

[0015] Grouping the first speech slices into a first speech sequence, and determining a sequence position of each of the first speech slices as a first sequence position;

[0016] Based on the original sequence position and the first sequence position, corresponding each of the original voice slices and each of the first voice slices;

[0017] The corresponding original voice slices are aligned and spliced ​​with the first voice slices to obtain spliced ​​voice slices.

[0018] Preferably, the corresponding each of the original voice slices and each of the first voice slices based on the original sequence position and the first sequence position includes:

[0019] For each of the original speech slices, determining a first speech slice whose first sequence position is the same as the original sequence position of the original speech slice;

[0020] Acquire the text data of the first voice slice, and simultaneously acquire the text data of the original voice slice;

[0021] Determine whether the text data of the first voice slice is completely consistent with the text data of the original voice slice;

[0022] If so, the first voice slice is used as the first voice slice corresponding to the original voice slice.

[0023] Preferably, the process of establishing the voice sample set includes:

[0024] Extracting a plurality of speech sample data from a predetermined target accent database as first sample data;

[0025] Extracting a plurality of speech sample data from other predetermined accent databases as second sample data;

[0026] Slicing each of the first sample data respectively to obtain each first sample slice, and slicing each of the second sample data respectively to obtain each second sample slice;

[0027] For each of the first sample slices, generating a first target accent sample corresponding to the first sample slice according to the target accent;

[0028] Aligning and splicing each of the first sample slices with its corresponding first target accent sample to obtain first spliced ​​samples;

[0029] For each of the second sample slices, generating a second target accent sample corresponding to the second sample slice according to the target accent;

[0030] Aligning and splicing each of the second sample slices with its corresponding second target accent sample to obtain respective second spliced ​​samples;

[0031] Each of the first spliced ​​samples and each of the second spliced ​​samples are sliced ​​and aggregated as spliced ​​samples to obtain a speech sample set.

[0032] Preferably, generating target accent data according to each of the original speech slices and based on the accent tags of each of the spliced ​​speech slices and the target accent comprises:

[0033] Using each concatenated speech slice whose accent label is a preset first label as each first accent concatenated slice;

[0034] Determine an original speech slice in each of the first accent spliced ​​slices;

[0035] Generating a target accent slice corresponding to each original speech slice in the first accent spliced ​​slice according to the target accent;

[0036] Using each concatenated speech slice whose accent label is a preset second label as each second accent concatenated slice;

[0037] determining an original speech slice in each of the second accented concatenated slices;

[0038] The target accent slice corresponding to the original speech slice in each of the first accent splicing slices is aggregated with the original speech slice in each of the second accent splicing slices to generate target accent data.

[0039] Preferably, slicing the original voice data to obtain individual original voice slices includes:

[0040] Identifying each pause position in the original speech data;

[0041] Dividing the original speech data into first original slices according to the pause positions;

[0042] Determining text data corresponding to the original voice data;

[0043] Identify each punctuation mark in the text data;

[0044] Matching each of the punctuation marks with the original voice data to determine the position of each of the punctuation marks in the original voice data as each punctuation mark position;

[0045] Determining whether each of the punctuation positions in the original speech data coincides with the pause position;

[0046] The non-overlapping punctuation positions are used as the positions to be segmented;

[0047] Each to-be-segmented position in each of the first original slices is segmented to obtain each original speech slice.

[0048] Preferably, the generating the first voice data corresponding to the original voice data according to the preset target accent comprises:

[0049] Extracting the timbre characteristics, rhythm characteristics and emotion characteristics of the original speech data;

[0050] Determining text data corresponding to the original voice data;

[0051] According to the text data, first voice data corresponding to the original voice data is generated based on the target accent, timbre characteristics, prosody characteristics and emotional characteristics.

[0052] In a second aspect, a device for generating accent data includes:

[0053] An original voice data acquisition module, used to acquire original voice data;

[0054] A first voice data generating module, configured to generate first voice data corresponding to the original voice data according to a preset target accent;

[0055] A slicing module, used for slicing the original voice data to obtain various original voice slices, and slicing the first voice data to obtain various first voice slices;

[0056] A spliced ​​voice slice determining module, configured to determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices;

[0057] An accent label obtaining module is used to input each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain the accent label of each of the spliced ​​speech slices; the accent recognition model is trained using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label;

[0058] The target accent data generating module is used to generate target accent data according to each of the original voice slices, based on the accent tags of each of the spliced ​​voice slices and the target accent.

[0059] In a third aspect, a device for generating accent data includes a memory and a processor;

[0060] The memory is used to store programs;

[0061] The processor is used to execute the program to implement each step of the method for generating accent data as described in any one of the first aspects.

[0062] In a fourth aspect, a storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the method for generating accent data as described in any one of the first aspects are implemented.

[0063] It can be seen from the above technical scheme that the present application obtains original voice data; generates first voice data corresponding to the original voice data according to a preset target accent; slices the original voice data to obtain individual original voice slices, and slices the first voice data to obtain individual first voice slices; determines individual spliced ​​voice slices from the individual original voice slices and the individual first voice slices; inputs each spliced ​​voice slice into a pre-trained accent recognition model to obtain the accent label of each spliced ​​voice slice; the accent recognition model is trained with a voice sample set as a training sample and the accent label of each spliced ​​sample slice in the voice sample set as a sample label; generates target accent data according to each original voice slice, based on the accent label of each spliced ​​voice slice and the target accent. The present application first generates first voice data corresponding to the original voice data according to a preset target accent. This can be regarded as standard voice data under the target accent and is directly generated voice data. Then, the original voice data and the first voice data are sliced ​​respectively to divide the original voice data and the first voice data into multiple small parts respectively. This is to reduce the volume of the original voice data. Generating accent data from small parts will be more detailed and accurate. Among them, the slices are spliced ​​to form each spliced ​​voice slice, and the present application pre-trains an accent recognition model. The spliced ​​voice slices are input into the accent recognition model, which can process the spliced ​​voice slices and predict the accent labels of the spliced ​​voice slices, so as to determine the accent information corresponding to the spliced ​​voice slices. Thus, the accent of each original voice slice can be determined, and then the target accent data can be generated according to each original voice slice, the accent label and the target accent. Compared with the generation method of the prior art, the present application is more sophisticated. The process of directly generating accent data according to the original voice data may be affected by the complex and changeable global features in the original voice data, resulting in poor generation effect. The slicing form of the present application can well avoid this problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0065] Figure 1 An optional flow chart of a method for generating accent data provided in an embodiment of the present application;

[0066] Figure 2 A schematic diagram of the structure of spliced ​​voice slices provided in an embodiment of the present application;

[0067] Figure 3 A schematic diagram of the structure of a device for generating accent data provided in an embodiment of the present application;

[0068] Figure 4 A schematic diagram of the structure of a device for generating accent data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] In recent years, on the one hand, big model technology has made great progress. Big prediction models represented by ChatGPT have continuously broken through the AI ​​ceiling in terms of dialogue and text generation. At the same time, some big voice models have achieved outstanding results in the fields of speech synthesis and speech translation, and can realize real acoustic features in speech. On the other hand, more and more companies now have the need to go overseas, and cultural exports are also in full swing. Nowadays, non-English speaking countries are more favored in the world, but at the same time, it has also increased the difficulty of communication. Therefore, the mastery of foreign languages ​​is not limited to daily communication and personal learning. Good voice technology support is needed in film and television dramas, e-commerce, cultural media and other aspects.

[0071] In particular, attention is paid to aspects such as generating corresponding speech data of other accents from a piece of speech data. However, existing speech generation technologies often simply directly generate speech data with specific accent requirements from original speech data. This method is relatively rough and the generated speech data is not accurate.

[0072] In order to solve the above problems of the prior art, an embodiment of the present invention provides a method for generating accent data. The present invention can be used in many general or special computing device environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or devices, etc. It can also be applied to various computer terminals or smart terminals, and its execution subject can be a processor or server of a computer terminal or smart terminal. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0073] S1: Get the original voice data.

[0074] The original voice data obtained in the present application may be sound data directly uttered by a person, voice data emitted by voice synthesis software, or voice data emitted from a broadcaster.

[0075] This application has no restrictions on the accent, rhythm, timbre, and length of the original voice data, which can be any voice data that can generate corresponding text data content.

[0076] S2: Generate first voice data corresponding to the original voice data according to a preset target accent.

[0077] The target accent preset in this application can be the accent of any dialect in Chinese, such as Sichuan accent, Cantonese accent, and Mandarin accent (this application assumes that Mandarin is also considered an accent); it can also be any accent in a foreign language, because not only Chinese has a variety of different accents, but foreign languages ​​also have various accents due to the influence of geographical, cultural, historical and other factors. For example, English has British accent, American accent, Australian accent, Indian accent, etc., and this embodiment does not limit this.

[0078] The purpose of this solution is to generate speech data of another accent based on original speech data of one accent, which is referred to as target accent data in this application. The other accent is the target accent. For example, this can be achieved through a large speech model, that is, the original speech data is input into a pre-acquired large speech model, and the target accent of the large speech model is set. The large speech model will output the first speech data under the target accent corresponding to the original speech data. A large speech model with functions such as zero-shot speech cloning, speech editing, generation of standard pronunciation of the target language, and control of the duration of generated speech can be selected. In addition, when cloning speech, the timbre, emotion, rhythm and other characteristics of the original speech data can be accurately transferred.

[0079] However, before the desired voice data is generated, this step first directly generates voice data corresponding to the original voice according to the target accent as the first voice data. Therefore, this step first generates the first voice data in order to assist the subsequent generation process of the target accent data.

[0080] S3: Slicing the original voice data to obtain original voice slices, and slicing the first voice data to obtain first voice slices.

[0081] The method for slicing the original voice data and the method for slicing the first voice data may be the same or different; the slicing may be performed simultaneously, or the original voice data may be sliced ​​first and then the first voice data, or the first voice data may be sliced ​​first and then the original voice data. The slicing may be performed by using a voice activity detection method (VAD) combined with semantic information corresponding to the voice.

[0082] Taking the original voice data as an example, the slicing method may also include:

[0083] 1) Based on voice endpoints.

[0084] It is understandable that the energy of speech data is significantly different between the parts with speech and those without speech, so the original speech signal can be divided into frames, the short-time energy of each frame can be calculated, and an energy threshold can be set. The short-time energy of each frame can be compared with the energy threshold. When the short-time energy of a frame exceeds the energy threshold, it is considered that the speech has started. When the short-time energy of a frame is lower than the energy threshold and lasts for a period of time (such as a few seconds), it is considered that the speech has ended. In this way, one or more speech endpoints in the original speech data can be determined. By dividing the original speech data into sentences at each speech endpoint, each sentence can be regarded as a slice.

[0085] 2) Based on the prosodic features of speech.

[0086] Generally speaking, the prosodic features of speech include intonation and stress. The prosody between sentences or phrases usually changes. For example, the intonation may drop at the end of a declarative sentence, while the intonation may rise at the end of an interrogative sentence. Therefore, by analyzing the intonation and stress of the original speech data, the prosodic boundaries can be determined for slicing.

[0087] 3) Based on speech and text data.

[0088] In addition to slicing the original voice data directly, you can also slice the corresponding text data. First, convert the original voice data into text data, then identify the words and sentences in the text data, and segment them based on the word and sentence segmentation points.

[0089] The present application abandons the existing method of directly generating other accent data from a long voice segment, and slices the original voice data and the first voice data respectively, so that the volume of the original voice data can be reduced. Starting from the small part, slices of other accents can be generated for each slice separately in the future, so the generated data will be more accurate and detailed.

[0090] S4: Determine each spliced ​​voice slice based on each of the original voice slices and each of the first voice slices.

[0091] The original speech slice and the first speech slice are concatenated to enable the accent recognition model to better recognize the accent. The first speech slice is equivalent to providing a comparison, so that the model can obtain richer accent comparison information. Since the first speech segment is generated according to the target accent and is the same in all other aspects except the accent, the original speech slice and the first speech slice will only form a sharp contrast in accent, which can highlight the characteristics of the original speech slice. The subsequent model can more keenly identify the accent characteristics of the original speech slice during processing.

[0092] There are many ways of splicing, such as horizontal splicing, splicing the original voice slice behind the first voice slice, or splicing the first voice slice behind the original voice slice, and vertical splicing, such as using the connectionist temporal classification method (CTC) for vertical alignment splicing.

[0093] S5: Input each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain the accent label of each of the spliced ​​speech slices; the accent recognition model is trained using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label.

[0094] The model can use a binary classification model, which is a supervised learning model, mainly used to divide data into two categories. This is because the original speech slice and the first speech slice may be speech slices with the same accent, or they may be slices with two different accents. The use of a binary classification model can quickly and accurately identify the spliced ​​speech slices. However, although the accent recognition model pre-trained in this application processes the input spliced ​​speech slices and outputs the accent label corresponding to the spliced ​​speech slices, in fact, the accent label output by the model can be regarded as the accent label of the original speech slice, that is, whether the original speech slice has a different accent or the same accent compared to the first accent slice. The binary classification model can specifically use a logistic regression model, a support vector machine model, a decision tree model, etc., and this application does not limit this.

[0095] The label can be in the form of numbers or text:

[0096] In digital form, if the output of the accent recognition model is set in digital form, then there are two types of accent labels: 1 and 0. "1" means that the two slices in the spliced ​​speech slices recognized by the model belong to two accents respectively, and "0" means that the two slices belong to the same accent. In an example, taking the first speech slice as the target accent, if the accent label of one of the spliced ​​speech slices output by the accent recognition model is 0, it means that the accent of the original speech slice in the spliced ​​speech slice belongs to the target accent. If the accent label of the spliced ​​speech slice is 1, it means that the accent of the original speech slice in the spliced ​​speech slice does not belong to the target accent, but belongs to other accents.

[0097] In text form, if the output of the accent recognition model is set in text form, then the accent labels also include two types: yes and no. "Yes" means that the two slices in the concatenated speech slices recognized by the model belong to two different accents respectively, and "no" means that the two slices belong to the same accent.

[0098] The speech sample set contains countless spliced ​​sample slices, and an excellent accent recognition model is trained using the speech sample set. Among them, an accent recognition model can be trained for a certain accent. For example, the accent recognition model of the present application is trained for a target accent, so its output is also related to the target accent.

[0099] S6: Generate target accent data according to each of the original speech slices, based on the accent tags of each of the spliced ​​speech slices and the target accent.

[0100] In the above steps, the accent labels of the spliced ​​voice slices have been clearly identified, that is, it is clear whether each original voice slice belongs to the target accent. Then, the original voice slices that belong to the target accent or do not belong to the target accent can be converted. For example, corresponding slices belonging to the target accent are generated for one or more original voice slices that do not belong to the target accent, and then replaced, so as to achieve the purpose of replacing and converting the original voice data. All the obtained slices belong to the voice data of the target accent and constitute the target accent data.

[0101] It can be seen from the above technical scheme that the present application first generates first voice data corresponding to the original voice data according to a preset target accent. This can be regarded as standard voice data under the target accent and is directly generated voice data. Then, the original voice data and the first voice data are sliced ​​respectively to divide the original voice data and the first voice data into multiple small parts. This is to reduce the volume of the original voice data. Generating accent data based on small parts will be more detailed and accurate. The slices are spliced ​​to form each spliced ​​voice slice, and the present application pre-trains an accent recognition model to splice the spliced ​​voice slices. The slice is input into the accent recognition model, which can process the spliced ​​speech slice and predict the accent label of the spliced ​​speech slice, so as to determine the accent information corresponding to the spliced ​​speech slice, thereby determining the accent of each original speech slice, and then generating the target accent data according to each original speech slice, the accent label and the target accent. The present application is more sophisticated than the generation method of the prior art. The process of directly generating accent data according to the original speech data may be affected by the complex and changeable global features in the original speech data, resulting in poor generation effect. The slicing form of the present application can avoid this problem well.

[0102] On the other hand, the present application can also be applied to the field of speech correction, such as correcting original speech data into speech data with a target accent.

[0103] Optionally, the step of generating the first voice data corresponding to the original voice data according to the preset target accent in the present application is specifically as follows:

[0104] Extracting the timbre characteristics, rhythm characteristics and emotion characteristics of the original speech data;

[0105] Determining text data corresponding to the original voice data;

[0106] According to the text data, first voice data corresponding to the original voice data is generated based on the target accent, timbre characteristics, prosody characteristics and emotional characteristics.

[0107] Specifically, in order to ensure that the subsequent generation process of the target accent data is more efficient, when the first voice data corresponding to the original voice data is generated according to the target accent at the beginning, only the accent is considered, and some other voice features remain unchanged. Therefore, the voice features of the original voice data can be extracted first, including but not limited to timbre features, prosodic features and emotional features. In addition, the corresponding text data needs to be determined. The text can be recognized using a large language model or through automatic speech recognition technology (ASR, Automatic Speech Recognition). Then, according to the text data, the first voice data corresponding to the original voice data can be generated with the target accent, timbre features, prosodic features and emotional features as the standard. For example, the existing large voice model is called, the text data, target accent, timbre features, prosodic features and emotional features are input into it, and the first voice data is output.

[0108] Optionally, the above process is to make the original voice data and the target accent data have no difference except the accent, so the acoustic characteristics of the original voice data such as timbre, rhythm, emotion, etc. are retained, and only the accent is changed. For example, a user records his own original voice data with a Sichuan accent, and wants to generate target accent data with a Mandarin accent based on the original voice data, and all aspects except the accent remain unchanged. In addition, he also wants to generate target accent data with a Northeastern accent based on the original voice data. This can meet the multiple accent generation needs of the same user.

[0109] In addition, you can also arbitrarily change or set acoustic features such as timbre, rhythm, and emotion to match different voice application scenarios, which can meet the needs of multiple users in multiple different scenarios, making the original voice data more usable / applicable and improving the user experience.

[0110] After obtaining the first voice data, it is necessary to slice the original voice data to obtain various original voice slices, and at the same time slice the first voice data to obtain various first voice slices. The same method can be used to slice the original voice data and the first voice data. The following is a detailed description taking the original voice data as an example.

[0111] Identifying each pause position in the original speech data;

[0112] Dividing the original speech data into first original slices according to the pause positions;

[0113] Determining text data corresponding to the original voice data;

[0114] Identify each punctuation mark in the text data;

[0115] Matching each of the punctuation marks with the original voice data to determine the position of each of the punctuation marks in the original voice data as each punctuation mark position;

[0116] Determining whether each of the punctuation positions in the original speech data coincides with the pause position;

[0117] The non-overlapping punctuation positions are used as the positions to be segmented;

[0118] Each to-be-segmented position in each of the first original slices is segmented to obtain each original speech slice.

[0119] Specifically, in order to ensure that the segmentation is sufficient and the original speech slices obtained after segmentation are not abrupt, and to maintain the authenticity of the original speech data, segmentation can be carried out based on two reference points, namely the pause positions at the speech level and the punctuation marks at the text data level, to prevent missing the positions that could have been segmented.

[0120] First, the pause position at the voice level is used as the reference point. It is understandable that there are pause points in the real voice data. Therefore, starting from the pause position as the reference point, each pause position in the original voice data is identified, and then the original voice data is divided into each first original slice according to the pause position; then the text data corresponding to the original voice data is determined, and the punctuation marks in the text data are identified. The punctuation marks are used to separate the sentences in the text, and also play a role in connecting the previous and the next. Therefore, using punctuation marks to assist in segmentation can also effectively segment the slices. It should be noted that the pause position and punctuation position determined from the two reference points may overlap, so segmentation from the two reference points separately may result in repeated segmentation, making the operation cumbersome and wasting time. Therefore, after segmentation according to the pause position, it is determined whether there is an overlapping position between the pause position and the punctuation position. Then only the non-overlapping punctuation positions can be segmented, thereby improving efficiency.

[0121] In order to ensure the quality and accuracy of the original speech slices obtained, they can be tested after the segmentation is completed, which further ensures the quality and accuracy of the target accent data finally generated.

[0122] In the method provided in the embodiment of the present invention, the process of determining each spliced ​​voice slice by each of the original voice slices and each of the first voice slices is specifically described as follows:

[0123] Combining the original speech slices into an original speech sequence, and determining the sequence position of each original speech slice as the original sequence position;

[0124] Grouping the first speech slices into a first speech sequence, and determining a sequence position of each of the first speech slices as a first sequence position;

[0125] Based on the original sequence position and the first sequence position, corresponding each of the original voice slices and each of the first voice slices;

[0126] The corresponding original voice slices are aligned and spliced ​​with the first voice slices to obtain spliced ​​voice slices.

[0127] Specifically, since each first voice slice only replaces the accent compared to each original voice slice, and the rest remains unchanged, for each original voice slice, there must be a corresponding first voice slice, and their positions in the original voice data and the first voice data are the same. In addition, because the slice processing has been performed, whether it is the original voice slice or the first voice slice, they are each independent, so they can be recombined to form a sequence, and their original position in the voice data is the current sequence position. The original voice slice and the first voice slice with the same sequence position can be considered to correspond, and the number of corresponding text data is also the same, and then alignment and splicing can be performed, such as Figure 2 As shown, Figure 2 That is, the original voice slice and the first voice slice are spliced ​​together, which is top-bottom splicing. The first voice slice can also be placed on the top and the original voice slice on the bottom. This embodiment does not limit this. Such splicing can make the form of the spliced ​​voice slices neater, which is convenient for the accent recognition model to process.

[0128] The process of performing corresponding operations on each of the original voice slices and each of the first voice slices based on the original sequence position and the first sequence position may include:

[0129] For each of the original speech slices, determining a first speech slice whose first sequence position is the same as the original sequence position of the original speech slice;

[0130] Acquire the text data of the first voice slice, and simultaneously acquire the text data of the original voice slice;

[0131] Determine whether the text data of the first voice slice is completely consistent with the text data of the original voice slice;

[0132] If so, the first voice slice is used as the first voice slice corresponding to the original voice slice.

[0133] Specifically, continue to refer to Figure 2, determine that the original sequence position of the original voice slice "A1B1C1D1F1" in the original voice data is the same as the first sequence position of the first voice slice "A2B2C2D2F2" in the first voice data. In this example, A1 and A2 are set to be the same text, just to distinguish the original voice slice from the first voice slice. It can be seen that the two corresponding text characters within a dotted line, such as A1 and A2, are the same, so the text data of the first voice slice is completely consistent with the text data of the original voice slice, so the two are corresponding and can be spliced. Furthermore, in order to enhance accuracy, the number of characters can also be determined at the same time during the text data comparison process. For example, Figure 2 The number of characters in the original voice slice is 5, and the number of the first voice slice is also 5. At the same time, each character is consistent, so these two slices correspond.

[0134] In order to enable the pre-trained accent recognition model to process the spliced ​​speech data, a large number of slice-type samples should be input into the model when training the model so that the model can learn the similarities and differences between the two slices in the slice-type samples. The following is an optimal way to construct a speech sample set:

[0135] Extracting a plurality of speech sample data from a predetermined target accent database as first sample data;

[0136] Extracting a plurality of speech sample data from other predetermined accent databases as second sample data;

[0137] Slicing each of the first sample data respectively to obtain each first sample slice, and slicing each of the second sample data respectively to obtain each second sample slice;

[0138] For each of the first sample slices, generating a first target accent sample corresponding to the first sample slice according to the target accent;

[0139] Aligning and splicing each of the first sample slices with its corresponding first target accent sample to obtain first spliced ​​samples;

[0140] For each of the second sample slices, generating a second target accent sample corresponding to the second sample slice according to the target accent;

[0141] Aligning and splicing each of the second sample slices with its corresponding second target accent sample to obtain respective second spliced ​​samples;

[0142] Each of the first spliced ​​samples and each of the second spliced ​​samples are sliced ​​and aggregated as spliced ​​samples to obtain a speech sample set.

[0143] Specifically, the function of the accent recognition model of the present application is to enable the accent recognition model to identify whether the accent of the original voice slice in the input spliced ​​voice slice is the same as the accent of the first voice slice. Therefore, the sample data used in training the accent recognition model needs to include sample data with the same accent as the first voice slice, and also needs to include sample data with a different accent from the first voice slice. Therefore, sample data are collected for the accent of the first voice slice (i.e., the target accent) and other accents respectively, and a target accent database is determined. All the voice data contained in this database are under the target accent. At the same time, other accent databases are determined. The voice data contained in this database are under various accents except the target accent. Then, multiple sample data are extracted from these two databases as the first sample data and the second sample data, and then sliced. In order to enrich the sample data, the sample data can be spliced ​​separately when splicing.

[0144] First is the first sample slice. Although the accent of the first sample slice is the target accent, it is real data obtained from the target accent database and may not be the standard target accent. Therefore, it is necessary to use the data corresponding to the standard target accent for splicing. Therefore, a first target accent sample corresponding to the first sample slice is generated according to the target accent, and then the first sample slice is spliced ​​with the first target accent sample.

[0145] Next is the second sample slice, which may be in any accent, so it is more necessary to generate a second target accent sample corresponding to the second sample slice according to the target accent, and then splice it.

[0146] In summary, if the target accent is represented as a, accents other than the target accent are represented as b, and the accent of the first sample slice is represented as a', then according to the above-mentioned method for constructing a speech sample set, the concatenated sample slices in the constructed speech sample set include two types, represented by accents, namely "a'-a" and "ba".

[0147] After the accent recognition model identifies the accent labels of the original speech slices in the concatenated speech slices, the target accent data can be generated according to the original speech slices, the accent labels of the concatenated speech slices, and the target accent. The specific process is as follows:

[0148] Using each concatenated speech slice whose accent label is a preset first label as each first accent concatenated slice;

[0149] Determine an original speech slice in each of the first accent spliced ​​slices;

[0150] Generating a target accent slice corresponding to each original speech slice in the first accent spliced ​​slice according to the target accent;

[0151] Using each concatenated speech slice whose accent label is a preset second label as each second accent concatenated slice;

[0152] determining an original speech slice in each of the second accented concatenated slices;

[0153] The target accent slice corresponding to the original speech slice in each of the first accent splicing slices is aggregated with the original speech slice in each of the second accent splicing slices to generate target accent data.

[0154] Specifically, following the above example, the first label and the second label can be set to "1, 0", or "yes, no". In one example, the first label is "1" and the second label is "0", which means that the original speech slices in the spliced ​​speech slices with the accent label "1" do not belong to the same accent as the first speech slices, and the original speech slices in the spliced ​​speech slices with the accent label "0" belong to the same accent as the first speech slices. If this example wants to generate speech data under the target accent, then the original speech slices in the spliced ​​speech slices with the accent label "1" need to be converted into slices under the target accent, that is, the target accent slices corresponding to the original speech slices are generated according to the target accent. In this way, it can be ensured that all speech slices corresponding to the original speech data are under the target accent. The spliced ​​speech slices with the label "0" are used as the second accent spliced ​​slices, and the original speech slices in the second accent spliced ​​slices are aggregated with the target accent slices that have been converted to the target accent, so as to generate the target accent data.

[0155] In one example, the process of generating a target accent slice corresponding to the original speech slice according to the target accent may include: establishing a conversion mechanism, determining the accent of the original speech data, analyzing the difference between the accent and the target accent, such as focusing on phonemes and intonation. In terms of phonemes, performing phoneme conversion, making a phoneme mapping table, and mapping the phonemes of the accent to the corresponding phonemes of the target accent; in terms of intonation, determining the intonation patterns of multiple different sentence patterns such as declarative sentences, interrogative sentences, and rhetorical questions in the target accent, such as the changing rules between rising tones, falling tones, or inflected tones; and finally generating a target accent slice corresponding to the original speech slice according to the factor mapping table and the intonation pattern.

[0156] Optionally, the present application can also achieve the mixing of accents. Since it is clear whether the accent of each original voice slice is the same as the target accent, the original voice slices can be converted to any accent as needed, such as converting a part of the original voice slices with a non-target accent to the target accent, or converting all / part of the original voice slices with the target accent to other accents, or setting a gradual accent conversion to create a unique voice effect. For example, in the field of film and television dubbing and audio novels, this method can produce voice data with a special style to add artistic color to the work.

[0157] Since each original speech slice is relatively independent, when the target accent slice generated by a certain original speech slice does not meet expectations, the original speech slice can be directly reprocessed without reprocessing the entire segment of target accent data; in addition, generating target accent data in the form of slices can achieve error location and correction. For example, if an error occurs in the final target accent data generation process, the slice format adopted in this application can more easily locate the specific location where the error occurs.

[0158] Preferably, the target accent slice generated from each original speech slice can be evaluated individually to control the quality of accent conversion more carefully. For example, the quality indicators such as authenticity and naturalness of each original speech slice before and after conversion can be compared, so as to optimize one or more target accent slices in a targeted manner, which helps to improve the overall quality of accent conversion and make the generated target accent data more detailed, natural and accurate.

[0159] and Figure 1 Corresponding to the method described above, the embodiment of the present invention further provides a device for generating accent data, which is used for Figure 1 In a specific implementation of the method, the device for generating accent data provided in the embodiment of the present invention can be used in a computer terminal or various mobile devices, combined with Figure 3 , the generation device of accent data is introduced, such as Figure 3 As shown, the device may include:

[0160] The original voice data acquisition module 10 is used to acquire the original voice data;

[0161] A first voice data generating module 20, configured to generate first voice data corresponding to the original voice data according to a preset target accent;

[0162] A slicing module 30, configured to slice the original voice data to obtain various original voice slices, and to slice the first voice data to obtain various first voice slices;

[0163] A spliced ​​voice slice determining module 40, configured to determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices;

[0164] The accent label obtaining module 50 is used to input each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain the accent label of each of the spliced ​​speech slices; the accent recognition model is trained using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label;

[0165] The target accent data generating module 60 is used to generate target accent data according to each of the original speech slices, based on the accent tags of each of the spliced ​​speech slices and the target accent.

[0166] It can be seen from the above technical scheme that the present application first generates first voice data corresponding to the original voice data according to a preset target accent. This can be regarded as standard voice data under the target accent and is directly generated voice data. Then, the original voice data and the first voice data are sliced ​​respectively to divide the original voice data and the first voice data into multiple small parts. This is to reduce the volume of the original voice data. Generating accent data based on small parts will be more detailed and accurate. The slices are spliced ​​to form each spliced ​​voice slice, and the present application pre-trains an accent recognition model to splice the spliced ​​voice slices. The slice is input into the accent recognition model, which can process the spliced ​​speech slice and predict the accent label of the spliced ​​speech slice, so as to determine the accent information corresponding to the spliced ​​speech slice, thereby determining the accent of each original speech slice, and then generating the target accent data according to each original speech slice, the accent label and the target accent. The present application is more sophisticated than the generation method of the prior art. The process of directly generating accent data according to the original speech data may be affected by the complex and changeable global features in the original speech data, resulting in poor generation effect. The slicing form of the present application can avoid this problem well.

[0167] Furthermore, the embodiment of the present application provides a device for generating accent data. Optionally, Figure 4 The hardware structure block diagram of the device for generating accent data is shown in FIG. Figure 4 The hardware structure of the accent data generating device may include: at least one processor 01 , at least one communication interface 02 , at least one memory 03 and at least one communication bus 04 .

[0168] In the embodiment of the present application, the number of the processor 01 , the communication interface 02 , the memory 03 , and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 , and the memory 03 communicate with each other through the communication bus 04 .

[0169] The processor 01 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0170] The memory 03 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory.

[0171] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to execute the following method for generating accent data, including:

[0172] Get the original voice data;

[0173] generating first voice data corresponding to the original voice data according to a preset target accent;

[0174] Slicing the original voice data to obtain original voice slices, and slicing the first voice data to obtain first voice slices;

[0175] Determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices;

[0176] Input each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain an accent label of each of the spliced ​​speech slices; the accent recognition model is trained using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label;

[0177] Target accent data is generated according to each of the original speech slices and based on the accent tags of each of the concatenated speech slices and the target accent.

[0178] Optionally, the detailed functions and extended functions of the program may refer to the description of the method for generating accent data in the method embodiment.

[0179] The embodiment of the present application further provides a storage medium, which may store a program suitable for execution by a processor, and when the program is executed, controls the device where the storage medium is located to execute the following method for generating accent data, including:

[0180] Get the original voice data;

[0181] generating first voice data corresponding to the original voice data according to a preset target accent;

[0182] Slicing the original voice data to obtain original voice slices, and slicing the first voice data to obtain first voice slices;

[0183] Determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices;

[0184] Input each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain an accent label of each of the spliced ​​speech slices; the accent recognition model is trained using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label;

[0185] Target accent data is generated according to each of the original speech slices and based on the accent tags of each of the concatenated speech slices and the target accent.

[0186] Specifically, the storage medium may be a computer-readable storage medium, and the computer-readable storage medium may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk or a ROM.

[0187] Optionally, the detailed functions and extended functions of the program may refer to the description of the method for generating accent data in the method embodiment.

[0188] In addition, the functional modules in the various embodiments of the present disclosure can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a live broadcast device, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present disclosure.

[0189] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0190] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0191] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating accent data, characterized in that: include: Get the original voice data; generating first voice data corresponding to the original voice data according to a preset target accent; Slicing the original voice data to obtain original voice slices, and slicing the first voice data to obtain first voice slices; Determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices; Inputting each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain an accent label of each of the spliced ​​speech slices; The accent recognition model is trained by using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label; Target accent data is generated according to each of the original speech slices and based on the accent tags of each of the concatenated speech slices and the target accent.

2. The method according to claim 1, characterized in that The determining of each spliced ​​voice slice from each of the original voice slices and each of the first voice slices includes: Combining the original speech slices into an original speech sequence, and determining the sequence position of each original speech slice as the original sequence position; Grouping the first speech slices into a first speech sequence, and determining a sequence position of each of the first speech slices as a first sequence position; Based on the original sequence position and the first sequence position, corresponding each of the original voice slices and each of the first voice slices; The corresponding original voice slices are aligned and spliced ​​with the first voice slices to obtain spliced ​​voice slices.

3. The method according to claim 2, characterized in that The corresponding each of the original voice slices and each of the first voice slices based on the original sequence position and the first sequence position includes: For each of the original speech slices, determining a first speech slice whose first sequence position is the same as the original sequence position of the original speech slice; Acquire the text data of the first voice slice, and simultaneously acquire the text data of the original voice slice; Determine whether the text data of the first voice slice is completely consistent with the text data of the original voice slice; If so, the first voice slice is used as the first voice slice corresponding to the original voice slice.

4. The method according to claim 1, characterized in that: The process of establishing the speech sample set includes: Extracting a plurality of speech sample data from a predetermined target accent database as first sample data; Extracting a plurality of speech sample data from other predetermined accent databases as second sample data; Slicing each of the first sample data respectively to obtain each first sample slice, and slicing each of the second sample data respectively to obtain each second sample slice; For each of the first sample slices, generating a first target accent sample corresponding to the first sample slice according to the target accent; Aligning and splicing each of the first sample slices with its corresponding first target accent sample to obtain first spliced ​​samples; For each of the second sample slices, generating a second target accent sample corresponding to the second sample slice according to the target accent; Aligning and splicing each of the second sample slices with its corresponding second target accent sample to obtain respective second spliced ​​samples; Each of the first spliced ​​samples and each of the second spliced ​​samples are sliced ​​and aggregated as spliced ​​samples to obtain a speech sample set.

5. The method according to claim 1, characterized in that The generating target accent data according to each of the original voice slices and based on the accent tags of each of the spliced ​​voice slices and the target accent includes: Using each concatenated speech slice whose accent label is a preset first label as each first accent concatenated slice; Determine an original speech slice in each of the first accent spliced ​​slices; Generating a target accent slice corresponding to each original speech slice in the first accent spliced ​​slice according to the target accent; Using each concatenated speech slice whose accent label is a preset second label as each second accent concatenated slice; determining an original speech slice in each of the second accented concatenated slices; The target accent slice corresponding to the original speech slice in each of the first accent splicing slices is aggregated with the original speech slice in each of the second accent splicing slices to generate target accent data.

6. The method according to any one of claims 1 to 5, characterized in that The slicing of the original voice data to obtain individual original voice slices includes: Identifying each pause position in the original speech data; Dividing the original speech data into first original slices according to the pause positions; Determining text data corresponding to the original voice data; Identify each punctuation mark in the text data; Matching each of the punctuation marks with the original voice data to determine the position of each of the punctuation marks in the original voice data as each punctuation mark position; Determining whether each of the punctuation positions in the original speech data coincides with the pause position; The non-overlapping punctuation positions are used as the positions to be segmented; Each to-be-segmented position in each of the first original slices is segmented to obtain each original speech slice.

7. The method according to any one of claims 1 to 5, characterized in that The step of generating first voice data corresponding to the original voice data according to a preset target accent includes: Extracting the timbre characteristics, rhythm characteristics and emotion characteristics of the original speech data; Determining text data corresponding to the original voice data; According to the text data, first voice data corresponding to the original voice data is generated based on the target accent, timbre characteristics, prosody characteristics and emotional characteristics.

8. A device for generating accent data, characterized in that: include: An original voice data acquisition module, used to acquire original voice data; A first voice data generating module, configured to generate first voice data corresponding to the original voice data according to a preset target accent; A slicing module, used for slicing the original voice data to obtain various original voice slices, and slicing the first voice data to obtain various first voice slices; A spliced ​​voice slice determining module, configured to determine each spliced ​​voice slice from each of the original voice slices and each of the first voice slices; An accent label obtaining module, used for inputting each of the spliced ​​speech slices into a pre-trained accent recognition model to obtain an accent label of each of the spliced ​​speech slices; The accent recognition model is trained by using a speech sample set as a training sample and using the accent label of each spliced ​​sample slice in the speech sample set as a sample label; The target accent data generating module is used to generate target accent data according to each of the original voice slices, based on the accent tags of each of the spliced ​​voice slices and the target accent.

9. A device for generating accent data, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the method for generating accent data as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the method for generating accent data according to any one of claims 1 to 7 is implemented.

Images