Pinyin broadcasting method and device, electronic equipment and storage medium
By establishing a pinyin-to-voice rule library and dynamically generating voice waveforms, the problem of too many audio files in the existing broadcasting methods is solved, efficient pinyin broadcasting is achieved and maintenance difficulty is reduced.
Patent Information
- Application Number
- CN202411778944.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-04-18
AI Technical Summary
The existing broadcasting method requires pre-storing audio and storing it locally on the cloud server and terminal, resulting in too many audio files and increasing and maintaining difficulties.
By analyzing the pronunciation rules and characteristics of Chinese pinyin, establishing a rule library from pinyin to pronunciation, dynamically generate voice waveforms and perform pinyin broadcasts, avoiding the need to pre-storage audio.
It improves the working efficiency of pinyin broadcasting, reduces the number of audio files stored locally in the terminal, and reduces the difficulty of subsequent increase and maintenance.
Smart Images
Figure CN119851650A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to a pinyin broadcasting method, device, electronic equipment and storage medium. Background Art
[0002] With the development of artificial intelligence and the continuous updating and upgrading of the Internet of Things, users are interacting more and more frequently with smart terminal devices in their daily lives. There are many voice broadcast products on the market, and the competition is also fierce. Under limited cost conditions, enriching product functions and improving user experience have become important manifestations of product competitiveness. The voice broadcast method of the prior art generally requires pre-storing audio before broadcasting, and then calls the network path through the mobile terminal to transmit it to the cloud server, and then the cloud server forwards it to the local terminal. After the local terminal receives the voice data, it broadcasts the voice information. The local terminal stores too many audio files, which leads to the problem of poor maintenance of subsequent audio.
[0003] Therefore, the existing broadcasting method has the problem that the audio needs to be stored in advance before broadcasting, stored in the cloud server, and downloaded to the terminal for local playback. The terminal stores too many audio files locally, which leads to the problem of poor maintenance in the future. Summary of the invention
[0004] The embodiment of the present invention provides a pinyin broadcasting method, aiming to provide an efficient and intelligent pinyin broadcasting method to improve the working efficiency of pinyin broadcasting. By analyzing the pronunciation rules and characteristics of Chinese pinyin, a rule library for pinyin to speech is established. When a certain pinyin needs to be broadcasted, a speech waveform is dynamically generated according to the rule library, and the pinyin is broadcasted, thereby solving the problem that the existing broadcasting method needs to store audio in advance before broadcasting, save it in a cloud environment, and download it to a terminal for local playback, and the terminal stores too many audio files locally, resulting in poor maintenance for subsequent additions.
[0005] In a first aspect, an embodiment of the present invention provides a pinyin broadcasting method, the method comprising the following steps:
[0006] Get the pinyin text to be broadcast;
[0007] Extracting the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and synthesizing the feature value groups corresponding to the various pinyin components into target feature values in a preset order;
[0008] Generate and process the target feature through a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast;
[0009] The pinyin text to be broadcast is broadcasted in pinyin based on the target speech waveform.
[0010] Optionally, extracting the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and combining the feature values corresponding to the various pinyin components into a target feature value in a preset order, including:
[0011] Splitting the pinyin text to be broadcast according to the pinyin composition rule to obtain multiple pinyin components of the pinyin text to be broadcast;
[0012] Performing feature extraction on each of the pinyin components respectively to obtain feature values corresponding to each of the pinyin components;
[0013] The characteristic values corresponding to the various pinyin components are combined in a preset order to form the target characteristics of the pinyin text to be broadcast.
[0014] Optionally, the to-be-broadcasted pinyin text is split according to a pinyin composition rule to obtain a plurality of pinyin components of the to-be-broadcasted pinyin text, including:
[0015] Determine the pinyin type of the pinyin text to be broadcast, the pinyin type includes a combined pronunciation type, a single vowel occurrence type and an overall reading pronunciation type, and different pinyin types correspond to different pinyin composition rules;
[0016] Determine the pinyin composition rule corresponding to the pinyin text to be broadcast according to the pinyin type of the pinyin text to be broadcast;
[0017] Based on the pinyin composition rule corresponding to the pinyin text to be broadcasted, the pinyin text to be broadcasted is split and processed to obtain multiple pinyin components of the pinyin text to be broadcasted.
[0018] Optionally, the pinyin type is the combined pronunciation type, and the pinyin text to be broadcast is split based on the pinyin composition rule corresponding to the pinyin text to be broadcast to obtain multiple pinyin components of the pinyin text to be broadcast, including:
[0019] The pinyin text to be broadcast is split and processed according to the pinyin composition rules corresponding to the combined pronunciation type to obtain the initial consonant part, the final consonant part, the tone part, the first transition part between the initial consonant and the final consonant, and the second transition part between the final consonant and the final consonant of the pinyin text to be broadcast.
[0020] Optionally, the extracting features of each of the pinyin components to obtain feature values corresponding to each of the pinyin components includes:
[0021] For each of the pinyin components, the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin component are extracted.
[0022] Optionally, the extracting of the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin components includes:
[0023] If the pinyin component is a vowel part, and the vowel part is a vowel type, then the mouth opening feature and tongue position feature of the vowel part are also extracted.
[0024] Optionally, before the target feature is generated by the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast, the method further includes:
[0025] Acquire a model to be trained and a training data set, wherein the training data set includes sample feature values and actual speech waveforms corresponding to the sample features;
[0026] Inputting the sample feature value into the model to be trained to perform speech waveform generation processing to obtain an output speech waveform;
[0027] The error loss between the output speech waveform and the actual speech waveform is calculated, and the model parameters of the model to be trained are adjusted with the goal of minimizing the error loss until the training is completed to obtain a trained speech generation model.
[0028] In a second aspect, an embodiment of the present invention further provides a pinyin broadcasting device, the pinyin broadcasting device comprising:
[0029] The acquisition module is used to obtain the pinyin text to be broadcast;
[0030] The first processing module is used to extract the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and synthesize the feature values corresponding to the various pinyin components into a target feature value in a preset order;
[0031] The second processing module is used to generate the target feature through the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast;
[0032] The broadcasting module is used to broadcast the pinyin text to be broadcast based on the target speech waveform.
[0033] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the pinyin broadcasting method provided in the embodiment of the present invention when executing the computer program.
[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the pinyin broadcasting method provided in the embodiment of the invention are implemented.
[0035] In an embodiment of the present invention, a pinyin text to be broadcast is obtained; the characteristic values corresponding to each pinyin component in the pinyin text to be broadcast are extracted, and the characteristic value groups corresponding to each pinyin component are synthesized into target characteristic values in a preset order; the target characteristics are generated and processed by a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast; and the pinyin text to be broadcast is broadcasted in pinyin based on the target speech waveform. By obtaining and extracting the characteristic values corresponding to each pinyin component in the pinyin text to be broadcast, and synthesizing the target characteristic values by a preset order of the characteristic value groups corresponding to each pinyin component, and generating and processing the target characteristics by a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast, the pinyin text to be broadcast is broadcasted in pinyin using the target speech waveform, which solves the problem that the existing broadcasting method needs to store the audio in advance before broadcasting, store it in a cloud environment, and download it to the terminal for local playback, resulting in too many stored audio files and subsequent increase in maintenance. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 is a flow chart of a pinyin broadcasting method provided by an embodiment of the present invention;
[0038] Figure 2 It is a structural schematic diagram of a pinyin broadcasting device provided by an embodiment of the present invention;
[0039] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] like Figure 1 As shown, Figure 1 1 is a flow chart of a pinyin broadcasting method provided by an embodiment of the present invention, the pinyin broadcasting method comprising the steps of:
[0042] 101. Obtain the pinyin text to be broadcast.
[0043] In an embodiment of the present invention, the above-mentioned pinyin broadcasting method can be applied to a voice management platform, and the above-mentioned voice management platform can be constructed by a server or a distributed server. The above-mentioned voice management platform includes a database, a data processing application, a knowledge database, a large language model or a calling interface of a large language model, and the voice text to be broadcast can be obtained through the data interface, and the voice text to be broadcast can be obtained through user upload.
[0044] The above-mentioned pinyin text to be broadcast includes the pinyin information to be broadcast. The pinyin information can be a syllable composed of initial consonants and finals, or a whole-word syllable. The pinyin information also includes tone information. The above-mentioned initial consonants are consonants used before finals, which can form syllables with finals, including initial consonants such as b, p, m, and f. The above-mentioned finals include simple vowels, complex vowels, and nasal vowels. Simple vowels include a, o, e, etc., complex vowels include ai, ei, ui, etc., and nasal vowels include an, in, ang, ong, etc. The whole-word syllable includes zhi, chi, yin, etc. The tones include the first tone "ˉ", the second tone "ˊ", the third tone "ˇ", and the fourth tone "ˋ".
[0045] The pinyin text to be broadcasted may be the pinyin of a single Chinese character, or the pinyin of a word composed of multiple Chinese characters, or the pinyin combination of all Chinese characters in a sentence, etc. For example, the pinyin text to be broadcasted may be lá, tōng, shān xī, bā, yì, etc.
[0046] 102. Extract feature values corresponding to each pinyin component in the pinyin text to be broadcast, and synthesize the feature value groups corresponding to each pinyin component into target feature values in a preset order.
[0047] In an embodiment of the present invention, the pinyin text to be broadcast can be split according to the pinyin composition rules to obtain multiple pinyin components of the pinyin text to be broadcast, and feature extraction is performed on each pinyin component to obtain the feature value corresponding to each pinyin component, so that the feature values corresponding to each pinyin component are combined into the target features of the pinyin text to be broadcast in a preset order.
[0048] The above-mentioned pinyin composition rules can be understood as the composition rules of Mandarin syllables, including the combination of initial consonants and finals, single vowels, whole-word syllables, and syllable composition rules of tones.
[0049] The above feature extraction can be understood as a process of extracting the pronunciation features of initials, finals, and tones. The pronunciation features include pronunciation part features, pronunciation method features, vowel features, tone features, etc. The pronunciation method of initials includes pronunciation part and pronunciation method. The pronunciation part includes bilabial sounds, tip-median sounds, etc. The pronunciation method includes stops, fricatives, etc. For example, the initial consonant "b" is a bilabial unaspirated voiceless stop.
[0050] The above preset order can be an initial consonant, a final consonant and a tone rule strategy. The initial consonant rule is used to describe the pronunciation of each initial consonant, including the pronunciation position and pronunciation method. The pronunciation position includes bilabial sounds, tip-median sounds, etc.; the pronunciation method includes stops, fricatives, etc. For example, the initial consonant "b" is a bilabial unaspirated voiceless stop. The final consonant rule is used to describe the pronunciation of each final consonant. For finals, the different pronunciation characteristics of simple finals, complex finals and nasal finals need to be considered. Simple finals have specific oral shapes and pronunciation positions, complex finals need to consider the transition of vowels, and nasal finals involve nasal resonance. For example, the final "a" is a central vowel with a larger opening. The above tone rules describe the change pattern of pitch, and there are four tones, namely, yinping, yangping, shangsheng and qusheng. For example, yinping is a high level tone, and the pitch is kept at a high level and basically unchanged when it is pronounced.
[0051] Furthermore, the pronunciation feature values of the initial consonants, final consonants, and tones of the pinyin may be combined according to the combination rule strategy of the initial consonants, final consonants, and tones to form the features of the target pinyin text.
[0052] 103. Generate and process the target features through the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast.
[0053] In an embodiment of the present invention, the trained speech generation model is obtained by training a pre-trained model according to a training data set, wherein the training data set includes sample feature values and actual speech waveforms corresponding to sample features. The sample feature values include initial consonant pronunciation features, final consonant pronunciation features, initial consonant and final consonant combination pronunciation features, and tone change features.
[0054] The above-mentioned pre-trained model can be a model built based on deep learning or machine learning, which can be a convolutional neural network (CNN), which can effectively extract local features, or a model that is combined with RNN to form a CNN-RNN architecture. In this CNN-RNN architecture, CNN is mainly used to extract local features of images, while the RNN part can be used to process sequence data, such as time series data or text data. This combination of CNN-RNN can make full use of the advantages of both and achieve more complex tasks.
[0055] Specifically, the sample features are used as input and the speech waveform is used as output to train the pre-trained model. During the training process, a loss function can be used to calculate the error loss between the output speech waveform and the actual speech waveform. With the goal of minimizing the error loss, the model parameters of the model to be trained are adjusted to minimize the difference between the predicted speech waveform and the actual speech waveform. The above loss function can be mean square error (MSE) or the like.
[0056] Furthermore, the trained speech generation model can accurately analyze the input pinyin features and output the speech waveform corresponding to the pinyin features.
[0057] 104. Perform pinyin broadcasting on the pinyin text to be broadcasted based on the target speech waveform.
[0058] In the embodiment of the present invention, after the speech waveform corresponding to the target feature is generated, the pinyin is broadcasted.
[0059] Voice playback can be performed according to the target voice waveform through the voice management platform or the media API on the user side.
[0060] In this embodiment, the present invention analyzes the pinyin features of the pinyin text to be broadcast through a trained speech generation model, accurately outputs the corresponding speech waveform, and broadcasts it without relying on pre-stored audio files, thereby improving the working efficiency of pinyin broadcasting.
[0061] In an embodiment of the present invention, a pinyin text to be broadcast is obtained; the characteristic values corresponding to each pinyin component in the pinyin text to be broadcast are extracted, and the characteristic value groups corresponding to each pinyin component are synthesized into target characteristic values in a preset order; the target characteristics are generated and processed by a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast; and the pinyin text to be broadcast is broadcasted in pinyin based on the target speech waveform. By obtaining and extracting the characteristic values corresponding to each pinyin component in the pinyin text to be broadcast, and synthesizing the target characteristic values by a preset order of the characteristic value groups corresponding to each pinyin component, and generating and processing the target characteristics by a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast, the pinyin text to be broadcast is broadcasted in pinyin using the target speech waveform, which solves the problem that the existing broadcasting method needs to store the audio in advance before broadcasting, save it in a cloud environment, and download it to the terminal for local playback, and the terminal stores too many audio files locally, resulting in subsequent increase and poor maintenance.
[0062] It can be understood that in the specific implementation of this application, voice data, text data, knowledge data and other related data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data, as well as the training, deployment and calling of word segmentation tools and large language models, must comply with the relevant laws, regulations and standards of relevant countries and regions.
[0063] Optionally, in the step of extracting the feature values corresponding to each pinyin component in the pinyin text to be broadcast, and combining the feature values corresponding to each pinyin component into a target feature value in a preset order, the pinyin text to be broadcast can be split according to the pinyin composition rules to obtain multiple pinyin components of the pinyin text to be broadcast; feature extraction is performed on each pinyin component to obtain the feature value corresponding to each pinyin component; and the feature values corresponding to each pinyin component are combined in a preset order into the target feature of the pinyin text to be broadcast.
[0064] In the embodiment of the present invention, the above-mentioned pinyin composition rules refer to the composition rules of Mandarin syllables, including the combination of initial consonants and finals, single finals, whole-recognition syllables and syllable composition rules of tones. Initial consonants include b, p, m, etc., finals include a, o, ai, ei, etc., whole-recognition syllables include zhi, ci, ri, yin, etc. Tones include four tones, namely, yinping, yangping, shangsheng and qusheng.
[0065] The pinyin text to be broadcasted can be split according to the combination of initial consonants and finals, finals, whole syllables and syllable composition rules of tones to obtain the components of the initial consonants, finals, whole syllables and tones of the pinyin text to be broadcasted. For example, the pinyin "zào" can be split into three parts: "z", "ao" and "ˋ"; the pinyin "yù" can be split into two parts: "yu" and "ˋ"; the syllable "huà" can be split into "h", "u", "a", "ˋ", etc.
[0066] The above feature extraction can be understood as a process of extracting the pronunciation features of initials, finals, and tones. The pronunciation features include pronunciation part features, pronunciation method features, vowel features, tone features, etc. The pronunciation method of initials includes pronunciation part and pronunciation method. The pronunciation part includes bilabial sounds, tip-median sounds, etc. The pronunciation method includes stops, fricatives, etc. For example, the initial consonant "b" is a bilabial unaspirated voiceless stop.
[0067] It should be noted that a series of articulation part categories can be defined in the system. For example, different articulation parts can be represented by vectors, such as bilabial consonants can be represented by [1,0,0,0,...]. A series of articulation method categories can be redefined in the system. For example, different articulation methods can be represented by vectors, such as stop consonants can be represented by [1,1,0,0,...]. Tones can be represented by arrays, such as yinping can be represented by 1, yangping can be represented by 2, shangsheng can be represented by 3, and qusheng can be represented by 4.
[0068] The above preset order can be understood as the initial consonant, final vowel and tone rule strategy.
[0069] Furthermore, the pronunciation feature values of the initial consonants, final consonants, and tones of the pinyin may be combined according to the combination rule strategy of the initial consonants, final consonants, and tones to form the features of the target pinyin text.
[0070] Optionally, in the step of splitting the Pinyin text to be broadcast according to the Pinyin composition rules to obtain multiple Pinyin components of the Pinyin text to be broadcast, the Pinyin type of the Pinyin text to be broadcast can be determined; according to the Pinyin type of the Pinyin text to be broadcast, the Pinyin composition rules corresponding to the Pinyin text to be broadcast can be determined; based on the Pinyin composition rules corresponding to the Pinyin text to be broadcast, the Pinyin text to be broadcast is split to obtain multiple Pinyin components of the Pinyin text to be broadcast.
[0071] In the embodiment of the present invention, the above-mentioned pinyin types include a combined pronunciation type, a single vowel pronunciation type and an overall reading pronunciation type, and different pinyin types correspond to different pinyin composition rules.
[0072] The above-mentioned pinyin composition rules include the combination of initial consonants and finals, single vowels, whole syllables and syllable composition rules of tones.
[0073] It can be understood that when the pinyin type of the pinyin text to be broadcast is a combined pronunciation type, the pinyin composition rule corresponding to the pinyin text to be broadcast is determined to be a combination composition rule of initial consonants, finals and tones, and the initial consonants, finals and tones are split to obtain the initial consonant part, final vowel part and tone part of the pinyin text to be broadcast.
[0074] When the pinyin type of the pinyin text to be broadcast is a single vowel pronunciation type, the pinyin composition rule corresponding to the pinyin text to be broadcast is determined to be a single vowel and a tone composition rule, and the vowels and tones are split to obtain the vowel part and the tone part of the pinyin text to be broadcast.
[0075] When the pinyin type of the pinyin text to be broadcast is the overall recognition pronunciation type, the pinyin composition rules corresponding to the pinyin text to be broadcast are determined to be the composition rules of the overall recognition syllables and tones, and the overall recognition syllables and tones are split to obtain the overall recognition part and the tone part of the pinyin text to be broadcast.
[0076] Optionally, the Pinyin type is a combined pronunciation type. In the step of splitting the Pinyin text to be broadcast based on the Pinyin composition rules corresponding to the Pinyin text to be broadcast to obtain multiple Pinyin components of the Pinyin text to be broadcast, the Pinyin text to be broadcast can be split according to the Pinyin composition rules corresponding to the combined pronunciation type to obtain the initial consonant part, the final consonant part, the tone part, the first transition part between the initial consonant and the final consonant, and the second transition part between the final consonant and the final consonant of the Pinyin text to be broadcast.
[0077] In the embodiment of the present invention, the initial consonant part refers to the part of the pinyin representing the consonant, such as "b", "p", "m" and the like.
[0078] The above-mentioned final part refers to the part of pinyin that represents the final, such as "a", "o", "ai", "an", etc.
[0079] The above-mentioned tone part refers to the tone symbols in pinyin, such as “ˉ”, “ˊ”, “ˇ” and “ˋ”.
[0080] For example, the pinyin "zào" can be divided into three parts: "z", "ao", and "ˋ"; the pinyin "yù" can be divided into two parts: "yu", and "ˋ"; the syllable "huà" can be divided into "h", "u", "a", and "ˋ", etc.
[0081] In one possible embodiment, for example, in the syllable "ba", there is a short transition process from the pronunciation of the initial consonant "b" to the pronunciation of the final consonant "a". For syllables with medials, such as "jia", the effect of the medial "i" on the pronunciation of the initial consonant and the final consonant and the transition between them should be considered. The tone should be correctly attached to the entire syllable to affect the pitch trend of the syllable.
[0082] Optionally, in the step of extracting features for each pinyin component and obtaining feature values corresponding to each pinyin component, the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin component can be extracted for each pinyin component.
[0083] In the embodiment of the present invention, the above-mentioned pronunciation part feature value can be understood as the feature parameter of the position and shape of the pronunciation organ (such as tongue, lips, teeth, etc.). The pronunciation part feature can use vectors to represent different pronunciation parts. It is assumed that a series of pronunciation part categories are defined, such as bilabial sounds can be represented by [1,0,0,0,...].
[0084] The above-mentioned pronunciation method characteristic values can be understood as characteristic parameters of the movement mode and strength of the pronunciation organs (such as vocal cords, tongue, lips, etc.). The pronunciation method category can be redefined in the system. The pronunciation method characteristics can be represented by vectors, such as stops can be represented by [1,1,0,0,...]. For example, the initial consonant "b" is a characteristic of a bilabial unaspirated voiceless stop. For finals, the different pronunciation characteristics of simple finals, complex finals and nasal finals should be considered. Simple finals have specific oral shapes and pronunciation positions, complex finals should consider the transition of vowels, and nasal finals involve nasal resonance. For example, the final "a" is a central vowel with a larger opening.
[0085] The above tone feature values can be understood as parameters of characteristics such as pitch, strength, etc. Tone features can be represented by numerical values, for example, 1 for yinping, 2 for yangping, 3 for shangsheng, and 4 for qusheng.
[0086] Optionally, in the step of extracting the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin component, if the pinyin component is a final part and the final part is a vowel type, the opening degree feature and tongue position feature of the final part are also extracted.
[0087] In an embodiment of the present invention, for the final part, if the final includes a vowel type, it is also necessary to extract the vowel feature value. The vowel feature includes extracting the opening degree, tongue position and other features of the vowel. The opening degree can be represented by a numerical value, and the tongue position can be represented by a two-dimensional coordinate, for example, front and back position, high and low position, etc.
[0088] Optionally, before the step of generating and processing the target features through the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast, the model to be trained and the training data set can also be obtained; the sample feature values are input into the model to be trained for speech waveform generation processing to obtain an output speech waveform; the error loss between the output speech waveform and the actual speech waveform is calculated, and the model parameters of the model to be trained are adjusted with the goal of minimizing the error loss until the training is completed to obtain a trained speech generation model.
[0089] In the embodiment of the present invention, the training data set includes sample feature values and actual speech waveforms corresponding to the sample features. The sample feature values include initial consonant pronunciation features, final consonant pronunciation features, initial consonant and final consonant combination pronunciation features, and tone change features.
[0090] The above-mentioned model to be trained can be a model built based on deep learning or machine learning, and can be a convolutional neural network (CNN). The convolutional neural network can effectively extract local features and can be used in combination with RNN to form a CNN-RNN architecture. In this CNN-RNN architecture, CNN is mainly used to extract local features of images, while the RNN part can be used to process sequence data, such as time series data or text data. This combined use method can make full use of the advantages of both and achieve more complex tasks.
[0091] During the training process, a loss function can be used to calculate the error loss between the output speech waveform and the actual speech waveform. With the goal of minimizing the error loss, the model parameters of the training model are adjusted to minimize the difference between the predicted speech waveform and the actual speech waveform. The above loss function can be mean square error (MSE) or the like.
[0092] It should be noted that by collecting a large number of initial consonant pronunciation features, final consonant pronunciation features, initial consonant and final consonant combination pronunciation features, and tone change features, these features are used as input, and the speech waveform is used as output to train the model. During the training process, the error loss between the output speech waveform and the actual speech waveform is calculated by using the loss function, and the model parameters are adjusted with the minimization of the loss value as the optimization goal. The model parameters of the model are adjusted by back propagation, and the parameter adjustment process is iterated until the model training is completed to obtain a trained speech generation model. The above back propagation is an efficient parameter update method, which updates the model parameters according to the gradient of the loss function.
[0093] like Figure 2 As shown, an embodiment of the present invention provides a pinyin broadcasting device, the pinyin broadcasting device comprising:
[0094] The acquisition module 201 is used to acquire the pinyin text to be broadcasted;
[0095] The first processing module 202 is used to extract the feature values corresponding to each pinyin component in the pinyin text to be broadcast, and synthesize the feature value groups corresponding to each pinyin component into a target feature value in a preset order;
[0096] The second processing module 203 is used to generate the target feature through the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast;
[0097] The broadcasting module 204 is used to broadcast the pinyin text to be broadcast based on the target speech waveform.
[0098] Optionally, the first processing module 202 is also used to split the pinyin text to be broadcast according to the pinyin composition rules to obtain multiple pinyin components of the pinyin text to be broadcast; perform feature extraction on each of the pinyin components to obtain the feature values corresponding to each of the pinyin components; and combine the feature values corresponding to each of the pinyin components into the target features of the pinyin text to be broadcast in a preset order.
[0099] Optionally, the first processing module 202 is also used to determine the pinyin type of the pinyin text to be broadcast, the pinyin type including the combined pronunciation type, the single vowel occurrence type and the overall recognition pronunciation type, and different pinyin types correspond to different pinyin composition rules; according to the pinyin type of the pinyin text to be broadcast, determine the pinyin composition rule corresponding to the pinyin text to be broadcast; based on the pinyin composition rule corresponding to the pinyin text to be broadcast, split the pinyin text to be broadcast to obtain multiple pinyin components of the pinyin text to be broadcast.
[0100] Optionally, the first processing module 202 is also used to split the pinyin text to be broadcast according to the pinyin composition rules corresponding to the combined pronunciation type to obtain the initial consonant part, the final consonant part, the tone part, the first transition part between the initial consonant and the final consonant, and the second transition part between the final consonant and the final consonant of the pinyin text to be broadcast.
[0101] Optionally, the first processing module 202 is further configured to extract, for each of the pinyin components, a pronunciation part feature value, a pronunciation method feature value, and a tone feature value corresponding to the pinyin component.
[0102] Optionally, the first processing module 202 is also used to extract the mouth opening feature and tongue position feature of the final part if the pinyin component is a vowel part and the final part is a vowel type. Optionally, the device is also used to obtain a model to be trained and a training data set, wherein the training data set includes sample feature values and actual speech waveforms corresponding to the sample features; input the sample feature values into the model to be trained for speech waveform generation processing to obtain an output speech waveform; calculate the error loss between the output speech waveform and the actual speech waveform, and adjust the model parameters of the model to be trained with the goal of minimizing the error loss until the training is completed to obtain a trained speech generation model.
[0103] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, including a processor, and the processor can execute any one of the above-mentioned pinyin broadcasting methods.
[0104] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing the pinyin broadcasting method which is stored in the memory 302 and can be run on the processor 301, wherein:
[0105] The processor 301 runs the computer program of the pinyin broadcasting method stored in the memory 302, and performs the following steps:
[0106] Get the pinyin text to be broadcast;
[0107] Extracting the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and synthesizing the feature value groups corresponding to the various pinyin components into target feature values in a preset order;
[0108] Generate and process the target feature through a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast;
[0109] The pinyin text to be broadcast is broadcasted in pinyin based on the target speech waveform.
[0110] Optionally, the processor 301 extracts the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and combines the feature values corresponding to the various pinyin components into a target feature value in a preset order, including:
[0111] Splitting the pinyin text to be broadcast according to the pinyin composition rule to obtain multiple pinyin components of the pinyin text to be broadcast;
[0112] Performing feature extraction on each of the pinyin components respectively to obtain feature values corresponding to each of the pinyin components;
[0113] The characteristic values corresponding to the various pinyin components are combined in a preset order to form the target characteristics of the pinyin text to be broadcast.
[0114] Optionally, the processor 301 performs the splitting process on the pinyin text to be broadcast according to the pinyin composition rule to obtain multiple pinyin components of the pinyin text to be broadcast, including:
[0115] Determine the pinyin type of the pinyin text to be broadcast, the pinyin type includes a combined pronunciation type, a single vowel occurrence type and an overall reading pronunciation type, and different pinyin types correspond to different pinyin composition rules;
[0116] Determine the pinyin composition rule corresponding to the pinyin text to be broadcast according to the pinyin type of the pinyin text to be broadcast;
[0117] Based on the pinyin composition rule corresponding to the pinyin text to be broadcasted, the pinyin text to be broadcasted is split and processed to obtain multiple pinyin components of the pinyin text to be broadcasted.
[0118] Optionally, the pinyin type is the combined pronunciation type, and the processor 301 executes the pinyin composition rule corresponding to the pinyin text to be broadcast to split the pinyin text to be broadcast to obtain multiple pinyin components of the pinyin text to be broadcast, including:
[0119] The pinyin text to be broadcast is split and processed according to the pinyin composition rules corresponding to the combined pronunciation type to obtain the initial consonant part, the final consonant part, the tone part, the first transition part between the initial consonant and the final consonant, and the second transition part between the final consonant and the final consonant of the pinyin text to be broadcast.
[0120] Optionally, the processor 301 performs feature extraction on each of the pinyin components to obtain feature values corresponding to each of the pinyin components, including:
[0121] For each of the pinyin components, the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin component are extracted.
[0122] Optionally, the extracting of the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin component performed by the processor 301 includes:
[0123] If the pinyin component is a vowel part, and the vowel part is a vowel type, then the mouth opening feature and tongue position feature of the vowel part are also extracted.
[0124] Optionally, before the target feature is generated by the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast, the method executed by the processor 301 further includes:
[0125] Acquire a model to be trained and a training data set, wherein the training data set includes sample feature values and actual speech waveforms corresponding to the sample features;
[0126] Inputting the sample feature value into the model to be trained to perform speech waveform generation processing to obtain an output speech waveform;
[0127] The error loss between the output speech waveform and the actual speech waveform is calculated, and the model parameters of the model to be trained are adjusted with the goal of minimizing the error loss until the training is completed to obtain a trained speech generation model.
[0128] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the pinyin broadcasting method or the application-side pinyin broadcasting method provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0129] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0130] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. A pinyin broadcasting method, characterized in that: The method comprises the following steps: Get the pinyin text to be broadcast; Extracting the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and synthesizing the feature value groups corresponding to the various pinyin components into target feature values in a preset order; The target feature is generated and processed by a trained speech generation model to obtain a target speech waveform of the pinyin text to be broadcast; The pinyin text to be broadcast is broadcasted in pinyin based on the target speech waveform.
2. The pinyin broadcasting method as claimed in claim 1, characterized in that: Extracting the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and combining the feature values corresponding to the various pinyin components into target feature values in a preset order, including: Splitting the pinyin text to be broadcast according to the pinyin composition rule to obtain multiple pinyin components of the pinyin text to be broadcast; Performing feature extraction on each of the pinyin components respectively to obtain feature values corresponding to each of the pinyin components; The characteristic values corresponding to the various pinyin components are combined in a preset order to form the target characteristics of the pinyin text to be broadcast.
3. The pinyin broadcasting method as claimed in claim 2, characterized in that: The step of splitting the pinyin text to be broadcast according to the pinyin composition rule to obtain multiple pinyin components of the pinyin text to be broadcast includes: Determine the pinyin type of the pinyin text to be broadcast, the pinyin type includes a combined pronunciation type, a single vowel pronunciation type and an overall reading pronunciation type, and different pinyin types correspond to different pinyin composition rules; Determine the pinyin composition rule corresponding to the pinyin text to be broadcast according to the pinyin type of the pinyin text to be broadcast; Based on the pinyin composition rule corresponding to the pinyin text to be broadcasted, the pinyin text to be broadcasted is split and processed to obtain multiple pinyin components of the pinyin text to be broadcasted.
4. The pinyin broadcasting method as claimed in claim 3, characterized in that: The pinyin type is the combined pronunciation type, and the pinyin composition rule corresponding to the pinyin text to be broadcast is used to split the pinyin text to be broadcast to obtain multiple pinyin components of the pinyin text to be broadcast, including: The pinyin text to be broadcast is split and processed according to the pinyin composition rules corresponding to the combined pronunciation type to obtain the initial consonant part, the final consonant part, the tone part, the first transition part between the initial consonant and the final consonant, and the second transition part between the final consonant and the final consonant of the pinyin text to be broadcast.
5. The pinyin broadcasting method as claimed in claim 4, characterized in that: The extracting features of each of the pinyin components to obtain feature values corresponding to each of the pinyin components includes: For each of the pinyin components, the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin component are extracted.
6. The pinyin broadcasting method as claimed in claim 5, characterized in that: The extracting of the pronunciation part feature value, pronunciation method feature value and tone feature value corresponding to the pinyin components includes: If the pinyin component is a vowel part, and the vowel part is a vowel type, then the mouth opening feature and tongue position feature of the vowel part are also extracted.
7. The pinyin broadcasting method according to any one of claims 1 to 5, characterized in that: Before the target feature is generated by the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast, the method further includes: Acquire a model to be trained and a training data set, wherein the training data set includes sample feature values and actual speech waveforms corresponding to the sample features; Inputting the sample feature value into the model to be trained to perform speech waveform generation processing to obtain an output speech waveform; The error loss between the output speech waveform and the actual speech waveform is calculated, and the model parameters of the model to be trained are adjusted with the goal of minimizing the error loss until the training is completed to obtain a trained speech generation model.
8. A pinyin broadcasting device, characterized in that: The pinyin broadcasting device comprises: The acquisition module is used to obtain the pinyin text to be broadcast; The first processing module is used to extract the feature values corresponding to the various pinyin components in the pinyin text to be broadcast, and synthesize the feature values corresponding to the various pinyin components into a target feature value in a preset order; The second processing module is used to generate the target feature through the trained speech generation model to obtain the target speech waveform of the pinyin text to be broadcast; The broadcasting module is used to broadcast the speech text to be broadcast in pinyin based on the target speech waveform.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the pinyin broadcasting method as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the pinyin broadcasting method as described in any one of claims 1 to 7 are implemented.