A volume control method and device for synthesized speech

Abstract

This specification discloses a method and device for controlling the volume of synthesized speech. The method first obtains a to-be-processed text containing a target text annotated in a speech synthesis markup language. The to-be-processed text is then input into a preset speech synthesis model. A first sampling point sequence for the synthesized speech corresponding to the to-be-processed text is determined, and a second sampling point sequence corresponding to the target text is determined from the first sampling point sequence. Finally, a gain corresponding to the second sampling point sequence is determined based on a preset automatic gain control algorithm, and the volume of the second sampling point sequence is controlled. By nonlinearly adjusting the volume of each frame of the sampling point sequence corresponding to the target text in the synthesized speech, the volume is significantly enhanced, thereby reducing the distortion of the synthesized speech obtained after volume control and improving the playback quality of the synthesized speech.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for controlling the volume of synthesized speech. Background Art

[0002] With the development of computer technology, speech synthesis technology is also constantly improving, and the application of synthesized speech is also on the rise. For example, delivery applications can use synthesized speech to play order information and provide navigation services for delivery drivers.

[0003] A common speech synthesis method is to first perform text analysis based on the text content to determine the text features, then use acoustic model building to determine the corresponding mel spectrum based on the text features, and then use a vocoder to generate a speech sampling point sequence based on the mel spectrum. Finally, through post-processing, the sampling rate, speaking rate, volume, and pitch of the speech sampling point sequence are adjusted to obtain synthesized speech data.

[0004] In the prior art, in volume adjustment during post-processing, a linear adjustment is usually performed on the synthesized speech based on a fixed gain value, which easily leads to speech distortion and degradation of speech quality. Summary of the Invention

[0005] The embodiments of this specification provide a method and device for controlling the volume of synthesized speech, which are used to at least partially solve the problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] Obtaining a text to be processed including a target text annotated by a speech synthesis markup language;

[0008] Inputting the text to be processed into a preset speech synthesis model, determining a first sampling point sequence of synthesized speech corresponding to the text to be processed, and determining a second sampling point sequence corresponding to the target text from the first sampling point sequence;

[0009] According to a preset automatic gain control algorithm, the gain corresponding to each frame of the second sampling point sequence is determined, and the volume of each frame of the second sampling point sequence is controlled.

[0010] Optionally, obtaining the text to be processed including the target text annotated by the speech synthesis markup language specifically includes:

[0011] Obtaining a text to be processed, and performing named entity recognition on the text to be processed;

[0012] The target text requiring volume adjustment is determined according to the recognition result, and the target text is marked using a speech synthesis markup language.

[0013] Optionally, determining a gain corresponding to each frame of the second sampling point sequence and controlling the volume of each frame of the second sampling point sequence specifically includes:

[0014] determining a valid voice sampling point sequence from the second sampling point sequence according to a preset voice activity detection algorithm;

[0015] The gain corresponding to each frame of the valid speech sampling point sequence is determined, and the volume of each frame of the valid speech sampling point sequence is controlled.

[0016] Optionally, determining a second sampling point sequence corresponding to the target text from the first sampling point sequence specifically includes:

[0017] According to the position of the target text in the text to be processed, determining the position of the text feature corresponding to the target text from the text features corresponding to the text to be processed;

[0018] Determining, from the mel-spectrogram corresponding to the text to be processed according to the position of the text feature corresponding to the target text, the time period in which the mel-spectrogram corresponding to the target text is located;

[0019] According to the time period of the mel spectrum corresponding to the target text, a second sampling point sequence corresponding to the target text is determined from the first sampling point sequence.

[0020] Optionally, determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and controlling the volume of each frame of the second sampling point sequence specifically includes:

[0021] Determining whether a volume increase control request is received;

[0022] If yes, determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and performing nonlinear enhancement control on the volume of each frame of the second sampling point sequence;

[0023] If not, the volume of the second sampling point sequence is linearly enhanced according to a preset gain.

[0024] Optionally, the method further includes:

[0025] Determining at least one character adjacent to the target text from the text to be processed according to the position of the target text in the text to be processed;

[0026] Determining, from the first sampling point sequence, a sequence of adjacent sampling points corresponding to the character according to the position of the character in the text to be processed;

[0027] Controlling the volume of the adjacent sampling point sequence according to a preset intermediate gain;

[0028] The intermediate gain is smaller than a gain corresponding to a second sampling point sequence adjacent to the adjacent sampling point sequence.

[0029] Optionally, determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and controlling the volume of each frame of the second sampling point sequence specifically includes:

[0030] Determining the type of each character in the target text and the subsampling point sequence corresponding to each character;

[0031] For each character type, the gain corresponding to each frame of the subsampling point sequence corresponding to the character type is determined according to the preset automatic gain control algorithm corresponding to the character type, and the volume of each frame of the subsampling point sequence corresponding to the character type is controlled.

[0032] This specification provides a volume control device for synthesized speech, comprising:

[0033] An acquisition module, configured to acquire a text to be processed including a target text annotated by a speech synthesis markup language;

[0034] a sampling point determination module, configured to input the to-be-processed text into a preset speech synthesis model, determine a first sampling point sequence of synthesized speech corresponding to the to-be-processed text, and determine a second sampling point sequence corresponding to the target text from the first sampling point sequence;

[0035] The volume control module is configured to determine the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and to control the volume of each frame of the second sampling point sequence.

[0036] This specification provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for controlling the volume of synthesized speech is implemented.

[0037] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned method for controlling the volume of synthesized speech is implemented.

[0038] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0039] The volume control method for synthesized speech provided in this specification first obtains a to-be-processed text containing a target text annotated in a speech synthesis markup language. The to-be-processed text is then input into a preset speech synthesis model to determine a first sampling point sequence for the synthesized speech corresponding to the to-be-processed text, and a second sampling point sequence corresponding to the target text is determined from the first sampling point sequence. Finally, based on a preset automatic gain control algorithm, the gain corresponding to the second sampling point sequence is determined, and the volume of the second sampling point sequence is controlled. By nonlinearly adjusting the volume of each frame of the sampling point sequence corresponding to the target text in the synthesized speech, the distortion of the synthesized speech obtained after volume control is reduced, thereby improving the playback quality of the synthesized speech. In particular, when the volume is significantly enhanced, the distortion of the synthesized speech after volume enhancement is reduced, thereby improving the playback quality of the synthesized speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 A flow chart of volume control for synthesized speech provided in this manual;

[0042] Figure 2 A diagram showing the correspondence between text features and synthesized speech provided in this specification;

[0043] Figure 3 This is a schematic diagram of an interactive process provided in this manual;

[0044] Figure 4 This is a volume transition diagram provided in this manual;

[0045] Figure 5 A schematic diagram of a volume control device for synthesized speech provided in this specification;

[0046] Figure 6 This is a schematic diagram of an electronic device that implements a volume control method for synthesized speech provided in this specification. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] In the prior art, when adjusting the volume of synthesized speech, a fixed gain value is usually applied to the volume of the synthesized speech, that is, the amplitude value of each sampling point of the synthesized speech is multiplied by a fixed value to linearly adjust the volume of the synthesized speech.

[0049] However, the volume of synthesized speech through linear adjustment can only be applied to a certain volume adjustment range. For example, when the amplitude value of the sampling point of synthesized speech is represented by a 16-bit binary number, excluding the sign bit that indicates the positive or negative amplitude value, the maximum absolute value of the amplitude value is 2 15 -1 = 32767. Therefore, in the above case, when the amplitude value of each sampling point of the synthesized speech multiplied by the preset fixed gain is not greater than 32767, the linear adjustment method can be used to adjust the volume of the synthesized speech. However, when the amplitude value of a sampling point of the synthesized speech multiplied by the preset fixed gain is greater than 32767, popping sounds may occur, seriously reducing the playback quality of the synthesized speech.

[0050] In the field of instant delivery, when using synthesized speech during delivery, especially in noisy outdoor environments, it's often necessary to significantly increase the volume of the synthesized speech to make it easier for the delivery person to discern the content. However, when using linear enhancement to significantly increase the volume of synthesized speech, the fixed gain used is often large, making the amplitude of each sampling point of the synthesized speech multiplied by the preset fixed gain likely to exceed the maximum absolute amplitude value, which can easily cause popping sounds. This results in distortion of the synthesized speech after volume enhancement, reducing playback quality and providing a poor user experience.

[0051] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0052] Figure 1 The flow chart of a method for controlling the volume of synthesized speech in this specification specifically includes the following steps:

[0053] S100: Acquire a text to be processed including a target text annotated by a speech synthesis markup language.

[0054] Currently, when performing speech synthesis, the text to be converted into speech is typically first obtained. Next, the text is processed to determine text features. Then, the mel-spectrogram corresponding to these text features is determined using an acoustic model. Finally, a vocoder can generate a sequence of sampling points for synthesized speech based on the mel-spectrogram. Based on this, in one or more embodiments of this specification, a service platform server may first obtain a text to be processed that includes target text annotated in a speech synthesis markup language, and then, during the processing of the text to be processed, determine the sequence of sampling points corresponding to the target text.

[0055] The text to be processed can be represented by Speech Synthesis Markup Language (SSML). The target text can be annotated by the control characters in the SSML tag. The target text is the text for which volume adjustment is required and can include at least one text segment. This specification does not limit the specific content of the target text. For example, the text corresponding to the geographic location, time, amount, order number, etc. in the text to be processed can be used as the target text.

[0056] For example, the text to be processed is "You have a new order, the originating address is X address, and the destination address is Y address", and the target text is "X address" and "Y address". This can be expressed in SSML as:

[0057] <speak>

[0058] You have a new order, the originating address is<prosody volume=98 use_agc="1"> X address, the end address is<prosody use_agc="1"> Y address

[0059] < / speak>

[0060] In the example, <speak>as well as <prosody>Both are tags in SSML. volume is a volume control symbol set according to SSML language rules. The value "98" of this control symbol can be used to determine the volume adjustment of the speech segment corresponding to the target text in the synthesized speech. use_agc is an annotation control symbol set according to SSML language rules. It is used to mark the target text for volume adjustment. That is, in the text to be processed represented by SSML, the text enclosed by tags carrying the preset annotation control symbol is the target text.

[0061] Of course, the server can also send the obtained text to be processed to the user's terminal, and the terminal will perform subsequent steps to determine the sampling point sequence corresponding to the target text in the synthesized speech and implement nonlinear gain on the volume of the sampling point sequence corresponding to the target text.

[0062] The server mentioned in this specification can be a server installed on a business platform, or a device such as a desktop computer or laptop computer capable of executing the solutions described in this specification. The terminal can be a user's terminal, such as a mobile phone or tablet computer. For ease of explanation, the following description only uses the server as the execution entity.

[0063] S102: Inputting the text to be processed into a preset speech synthesis model, determining a first sampling point sequence of synthesized speech corresponding to the text to be processed, and determining a second sampling point sequence corresponding to the target text from the first sampling point sequence.

[0064] After determining the text to be processed and the target text as described above, in one or more embodiments of the present specification, the server may input the text to be processed into a preset speech synthesis model to determine a first sampling point sequence of synthesized speech corresponding to the text to be processed, and determine a second sampling point sequence corresponding to the target text from the first sampling point sequence.

[0065] The speech synthesis model mentioned above is a relatively mature technology. The specific speech synthesis model to be used can be determined as needed and is not limited in this specification. The sampling point at least includes the position information (timing) and amplitude value (the amplitude value determines the volume of the synthesized speech) in the sampling point sequence. The sampling rate of the sampling sequence can be determined as needed and is not limited in this specification.

[0066] In one or more embodiments of this specification, a server may use a speech synthesis model to perform text normalization, word segmentation, phonetic notation, and prosody prediction on input SSML text to determine text features corresponding to the text to be processed. During this process, the server may determine the location of the text features corresponding to the target text based on the start and end positions of the target text in the text to be processed.

[0067] For example, taking Chinese as an example, the pronunciation of a Chinese character is one syllable, and one syllable usually contains one or more phonemes. After determining the phoneme sequence corresponding to the text to be processed, the phoneme sequence corresponding to the target text can be determined according to the start and end positions of the target text in the text to be processed. Of course, the phonemes mentioned here are only an example of text features, and a similar method can be adopted for other text features.

[0068] After the server determines the text features corresponding to the text to be processed through the speech synthesis model, it can further determine the Mel spectrogram corresponding to the text to be processed according to the text features through the speech synthesis model, and then determine the sampling point sequence of the synthesized speech corresponding to the text to be processed according to the Mel spectrogram. Of course, in this process, the server can determine the time period in which the Mel spectrogram corresponding to the target text is located from the Mel spectrogram according to the position of the text features corresponding to the target text, and determine the second sampling point sequence corresponding to the target text from the first sampling point sequence according to the time period in which the Mel spectrogram corresponding to the target text is located. Obviously, the first sampling point sequence contains the second sampling point sequence, that is, the second sampling point sequence is a partial sampling point sequence of the first sampling point sequence.

[0069] Figure 2 This is a schematic diagram of the correspondence between text features and synthesized speech provided in this specification. Figure 2 In [the figure], "Chinese investment" is the text to be processed, "China" is the target text, and the phoneme sequence corresponding to the text to be processed is below "Chinese investment". Then the phoneme sequence corresponding to the text to be processed is "zh ong1 g uo2 t ou2 z i1", and the phoneme sequence corresponding to the target text is "zh ong' g uo2". The number after the vowel represents the tone. Below the phoneme sequence is the Mel spectrogram corresponding to this phoneme sequence. The abscissa of this Mel spectrogram represents time, and the ordinate represents frequency. Of course, for the convenience of description, this Mel spectrogram is only a simple exemplary representation, and some details are not specifically given. Below the Mel spectrogram is the waveform diagram of the synthesized speech corresponding to this Mel spectrogram. The abscissa of this waveform diagram represents time, and the ordinate represents amplitude. It can be understood that the speech synthesis model can determine the sampling point sequence of the corresponding synthesized speech according to this Mel spectrogram. Since the sampling rate of the sampling sequence is usually high, it is difficult to clearly show the sampling point sequence of the synthesized speech in the figure. Therefore, for the convenience of description, this waveform diagram is used as an example for description. Then this waveform diagram can be understood as the waveform diagram of the synthesized speech formed after connecting the sampling points of the synthesized speech.

[0070] When determining the Mel spectrogram corresponding to the text to be processed through the speech synthesis model, the time period in which the Mel spectrogram corresponding to the target text is located can be determined according to the correspondence between the phoneme position and the time period of each frame of the Mel spectrogram, so as to Figure 2 Taking the mel spectrum below the phoneme sequence as an example, each vertical small rectangle in the mel spectrum is a frame, and the mel spectrum corresponding to the target text is the first 17 frames. Similarly, when determining the sampling point sequence of the synthesized speech corresponding to the text to be processed based on the mel spectrum, the sampling point sequence of the synthesized speech corresponding to the target text can be determined based on the time period of the mel spectrum corresponding to the target text. Figure 2 In the waveform diagram below the mel spectrum, the sampling points contained in the synthesized speech waveform diagram segment before the fourth thick vertical solid line are the second sampling point sequence corresponding to the target text.

[0071] How to determine the second sampling point sequence corresponding to the target text from the first sampling point sequence can be determined as needed, and this specification does not impose any limitation on this.

[0072] S104: Determine the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and control the volume of each frame of the second sampling point sequence.

[0073] After determining the second sampling point sequence corresponding to the target text as described above, in one or more embodiments of this specification, the server may determine the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and control the volume of each frame of the second sampling point sequence.

[0074] Automatic gain control algorithms are relatively mature technologies. The specific automatic gain control algorithm to be used can be determined based on needs, and this manual does not impose any restrictions on this. Regarding the gain being greater than zero, for a frame of synthesized speech, if the determined gain is greater than 1, the volume of that frame of synthesized speech is increased; if the determined gain is less than 1, the volume of that frame of synthesized speech is decreased. The specific degree of increase or decrease can be determined based on the gain value. For speech, the volume is primarily determined by the amplitude value. Therefore, for each sampling point corresponding to that frame of synthesized speech, the determined gain can be multiplied by the amplitude value of each sampling point to achieve volume control.

[0075] based on Figure 1 The volume control method for synthesized speech illustrated in the figure first obtains a text to be processed containing a target text annotated in a speech synthesis markup language. The text to be processed is then input into a preset speech synthesis model. A first sampling point sequence for the synthesized speech corresponding to the text to be processed is determined, and a second sampling point sequence corresponding to the target text is determined from the first sampling point sequence. Finally, a gain corresponding to the second sampling point sequence is determined based on a preset automatic gain control algorithm, and the volume of the second sampling point sequence is controlled. By nonlinearly adjusting the volume of each frame of the sampling point sequence corresponding to the target text in the synthesized speech, the distortion of the synthesized speech obtained after volume control is reduced, thereby improving the playback quality of the synthesized speech. In particular, when the volume is significantly enhanced, the distortion of the synthesized speech after volume enhancement is reduced, thereby improving the playback quality of the synthesized speech.

[0076] In addition, in one or more embodiments of the present specification, in step S100, when the server obtains the text to be processed that includes the target text marked by the speech synthesis markup language, the server may first obtain the text to be processed, and then perform named entity recognition (NER) on the text to be processed to determine the target text that needs to be volume adjusted, and mark the target text obtained by the recognition through the speech synthesis markup language. Among them, named entity recognition has a relatively mature technology, and the specific method to be adopted can be determined according to needs, and this specification does not limit this. Usually, the target text that needs to be volume adjusted is the text in the text to be processed that corresponds to the geographical location, time, amount, order number and other information. The content of the target text obtained by the recognition marked by the speech synthesis markup language can refer to the corresponding description in step S100, which will not be repeated here.

[0077] Furthermore, in one or more embodiments of this specification, after performing named entity recognition on the processed text, a large number of named entities may be obtained, but not all of these named entities need to be used as target text for volume control. Therefore, as needed, the server can also preset a blacklist of named entities that are recognized by common named entity recognition algorithms but are not required as target text. The server then uses the preset blacklist to filter the named entities recognized by the named entity recognition algorithm, and finally determines the target text for volume adjustment based on the filtered named entities.

[0078] In addition, in one or more embodiments of the present specification, before step S104, the server may also perform voice activity detection (VAD) on the determined second sampling point sequence, and only control the volume of the sampling point sequence corresponding to the detected valid speech portion, thereby avoiding amplifying the volume of background noise. Specifically, the server may determine the valid speech sampling point sequence from the second sampling point sequence based on a preset voice activity detection algorithm. Then, the server may further determine the gain corresponding to each frame of the valid speech sampling point sequence based on a preset automatic gain control algorithm, and control the volume of each frame of the valid speech sampling point sequence. Among them, the voice activity detection algorithm is a relatively mature technology, and the specific method to be used can be determined as needed, and this specification does not impose any restrictions on this. The valid speech mentioned here refers to the speech corresponding to the text in the synthesized speech, that is, the remaining part of the synthesized speech excluding background noise.

[0079] Furthermore, in one or more embodiments of this specification, in step S104, when the server controls the volume of the synthesized speech, the server may also determine whether a volume enhancement control request has been received. If so, the server determines the gain corresponding to each frame of the second sampling point sequence based on a preset automatic gain control algorithm and performs nonlinear enhancement control on the volume of each frame of the second sampling point sequence. If not, the server performs linear enhancement control on the volume of the second sampling point sequence based on the preset gain. The volume enhancement control request may be a volume enhancement control request initiated by the terminal to the server in response to a user operation. The technology for performing linear enhancement control on the volume of synthesized speech is relatively mature and will not be further described in this specification.

[0080] When the server receives a volume boost control request, it can be understood that the user has selected the ultra-high volume mode and sent a volume boost control request to the server via the terminal. At this point, the server can use an automatic gain control algorithm to perform nonlinear volume boost control on each frame of the second sampling point sequence. This significantly boosts the volume of each frame of the second sampling point sequence while preventing popping sounds. If the server does not receive a volume boost control request, meaning the user does not use the ultra-high volume mode, the server can simply perform linear volume boost control on the second sampling point sequence based on a preset gain. Of course, this preset gain does not cause the amplitude of each sampling point of the synthesized speech multiplied by the preset gain to exceed the maximum absolute value of the amplitude, thus preventing popping sounds. The specific preset gain can be determined as needed and is not limited in this specification. It is understood that linear adjustment has a narrower range of synthesized speech volume adjustment than nonlinear adjustment, while nonlinear adjustment has a wider range of synthesized speech volume adjustment.

[0081] For example, the terminal may prompt the user to select "Do you want to use ultra-high volume mode?" If the user selects "yes," the terminal may initiate a volume increase control request to the server in response to the user's operation. When the server detects the request, it performs nonlinear volume control on the volume of each frame of the second sampling point sequence in the synthesized speech according to a preset automatic gain control algorithm. If the user selects "no," the terminal will not initiate a volume increase control request to the server in response to the user's operation. If the server does not detect the request, it performs linear volume control on the volume of each frame of the second sampling point sequence in the synthesized speech according to a preset gain.

[0082] Of course, in one or more embodiments of this specification, the terminal may also prompt the user to select a mode, and in response to the user's operation, directly send the user selection result to the server, such as Figure 3 shown.

[0083] Figure 3 This is a schematic diagram of an interactive process provided in this specification. As can be seen from the figure, the terminal can first prompt the user to select whether to use the ultra-high volume mode, and then send the user's selection result to the server in response to the user's operation. Secondly, the server can determine whether to use the automatic gain control algorithm to control the volume of each frame of the second sampling point sequence in the synthesized speech based on the received user selection result. If the user chooses to use the ultra-high volume mode, the volume of each frame of the second sampling point sequence in the synthesized speech is nonlinearly controlled according to the preset automatic gain control algorithm. If the user chooses not to use the ultra-high volume mode, the volume of each frame of the second sampling point sequence in the synthesized speech is linearly controlled according to the preset gain. Finally, the sampling point sequence of the synthesized speech obtained after volume control can be returned to the terminal, which plays it to provide the user with corresponding voice service.

[0084] Furthermore, in one or more embodiments of the present specification, when the terminal prompts the user to select whether to use the ultra-high volume mode, if the user chooses to use the ultra-high volume mode, the user may be prompted to determine the volume of the target text in the ultra-high volume mode, and then the user's selection result and the volume determined by the user may be sent to the server.

[0085] Correspondingly, after obtaining the text to be processed, the server performs named entity recognition on the text to be processed. When determining the target text for volume adjustment, the server can simultaneously determine the volume of the target text based on the volume determined by the user, and annotate the target text using the speech synthesis markup language while also annotating the volume of the target text. The specific annotation method can be referred to the example in step S100 and will not be further described here.

[0086] For example, if the user selects the ultra-high volume mode and determines that the volume of the target text in ultra-high volume mode is 98% of the maximum volume, then for the text to be processed in step S100, "You have a new order, the originating address is X address, and the destination address is Y address", the target text obtained through named entity recognition is "X address" and "Y address". This can be represented in SSML as:

[0087] <speak>

[0088] You have a new order, the originating address is<prosody volume=98 use_agc="1"> X address< / speak> < / prosody> , the end address is<prosody use_agc="1"> Y address

[0089] < / speak>

[0090] Here, use_agc is a markup control character set according to the SSML language rules, which is used to mark the target text for volume adjustment. Volume = 98 is the markup of the target text volume based on the volume determined by the user. The subsequent automatic gain control algorithm can adjust the volume of the second sampling point sequence corresponding to the target text in the synthesized speech based on this markup.

[0091] Furthermore, in one or more embodiments of the present specification, step S104 controls the volume of each frame of the second sampling point sequence corresponding to the target text in the synthesized speech. Therefore, when the synthesized speech is played from non-target text to target text, or vice versa, a sudden change in volume occurs, reducing the user experience. Based on this, the server can also determine at least one character adjacent to the target text in the to-be-processed text based on the position of the target text in the to-be-processed text. Then, based on the position of the character in the to-be-processed text, determine the adjacent sampling point sequence corresponding to the character from the first sampling point sequence. Finally, the volume of the adjacent sampling point sequence is controlled based on a preset intermediate gain.

[0092] Among them, when determining at least one character adjacent to the target text from the text to be processed, the character should be a readable character such as Chinese characters, English or numbers, that is, the character is not a punctuation mark or other characters that will be omitted in normal speech playback. The determined character is then used as a transition character for volume transition, and the so-called adjacent sampling point sequence is the sampling point sequence corresponding to the determined character in the synthesized speech. Finally, the intermediate gain can be determined based on the volume change trend of the text before and after the determined transition character, and the volume of the adjacent sampling point sequence can be controlled. The intermediate gain should be determined corresponding to the volume change trend of the text from the front to the back of the transition character, so that when the synthesized speech is played, there is a volume transition between the target text and the non-target text, thereby improving the user experience, such as Figure 4 shown.

[0093] Figure 4 This is a volume transition diagram for volume control provided in this manual. Figure 4 In the figure, the dotted line represents the first sampling point sequence of the synthesized speech, and each point is a sampling point. When controlling the volume of the synthesized speech, it is usually done frame by frame, corresponding to Figure 4 In the example, the sampling point sequence between each two vertical lines is regarded as the sampling point sequence corresponding to one frame of synthesized speech. Figure 4 In the example, the last four frames are the second sampling point sequence corresponding to the target text, and the two frames before the second sampling point sequence are the determined adjacent sampling point sequence. The horizontal line between each two vertical lines represents the gain corresponding to the sampling point sequence of the synthesized speech in that frame.

[0094] Assume that the volume of each frame of synthesized speech corresponding to the target text is enhanced, and the gain determined by the sub-sampling point sequence of the first frame of synthesized speech in the second sampling point sequence is 4. Then the sampling point sequence corresponding to the non-target text can be regarded as having a gain of 1. Therefore, the intermediate gain corresponding to the adjacent sampling point sequence can be any value greater than 1 and less than 4, that is, the intermediate gain is less than the gain corresponding to the sub-sampling point sequence of the first frame of synthesized speech in the second sampling point sequence adjacent to the adjacent sampling point sequence. The specific value of the intermediate gain can be determined as needed, and this specification does not impose any limitation on this. Figure 4 For example, if the adjacent sampling point sequence contains two frames of synthesized speech, the gain corresponding to the first frame can be 2, and the gain corresponding to the second frame can be 3. This allows for a volume transition between the target text and the non-target text, improving the user experience.

[0095] Furthermore, in one or more embodiments of the present specification, considering that the user's auditory experience is related to the type of character, the user's auditory experience for different types of characters at the same volume often differs. Therefore, in step S104, when the server determines the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and controls the volume of each frame of the second sampling point sequence, the server can determine the type of each character in the target text and the subsampling point sequence corresponding to each character. Then, for each character type, the server determines the gain corresponding to each frame of the subsampling point sequence corresponding to that character type according to the preset automatic gain control algorithm corresponding to that character type, and controls the volume of each frame of the subsampling point sequence corresponding to that character type. The different automatic gain control algorithms mentioned above can simply be different parameters in the algorithm, or different automatic gain control algorithms can be used. This improves the user's auditory experience and makes the synthesized speech more natural.

[0096] In addition, in one or more embodiments of the present specification, in step S102, when determining the second sampling point sequence corresponding to the target text from the first sampling point sequence, the server may further determine the first sampling point sequence of the synthesized speech based on the text to be processed using a speech synthesis model, then determine the second sampling point sequence of the synthesized speech based on the target text using the speech synthesis model, and then match the second sampling point sequence with the first sampling point sequence using a matching algorithm to determine the position of the second sampling point sequence in the first sampling point sequence.

[0097] The synthesized speech volume control method provided in this specification can be applied to delivery services, such as in scenarios where delivery personnel are delivering express or takeout food. In the delivery field, the synthesized speech volume control method provided in this specification can be used to significantly and nonlinearly enhance the volume of the synthesized speech in a noisy delivery environment, thereby reducing the distortion of the synthesized speech obtained after volume enhancement and improving the playback quality of the synthesized speech. In particular, when the volume is significantly enhanced, the distortion of the synthesized speech after volume enhancement is reduced and the playback quality of the synthesized speech is improved. This facilitates the provision of corresponding services to delivery personnel through synthesized speech.

[0098] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0099] The above is a method for controlling the volume of synthesized speech provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for controlling the volume of synthesized speech, such as Figure 5 shown.

[0100] Figure 5 A schematic diagram of a volume control device for synthesized speech provided in this specification includes:

[0101] An acquisition module 200 is configured to acquire a text to be processed including a target text annotated by a speech synthesis markup language;

[0102] The sampling point determination module 202 is configured to input the text to be processed into a preset speech synthesis model, determine a first sampling point sequence of synthesized speech corresponding to the text to be processed, and determine a second sampling point sequence corresponding to the target text from the first sampling point sequence;

[0103] The volume control module 204 is configured to determine the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and control the volume of each frame of the second sampling point sequence.

[0104] Optionally, the acquisition module 200 acquires the text to be processed, performs named entity recognition on the text to be processed, determines the target text that needs volume adjustment according to the recognition result, and marks the target text using a speech synthesis markup language.

[0105] Optionally, the volume control module 204 is configured to determine a valid voice sampling point sequence from the second sampling point sequence according to a preset voice activity detection algorithm, determine a gain corresponding to each frame of the valid voice sampling point sequence, and control the volume of each frame of the valid voice sampling point sequence.

[0106] Optionally, the sampling point determination module 202 determines, based on the position of the target text in the to-be-processed text, a position of a text feature corresponding to the target text from the text features corresponding to the to-be-processed text; determines, based on the position of the text feature corresponding to the target text, a time period of the mel-spectrogram corresponding to the target text from the mel-spectrogram corresponding to the to-be-processed text; and determines, based on the time period of the mel-spectrogram corresponding to the target text, a second sampling point sequence corresponding to the target text from the first sampling point sequence.

[0107] Optionally, the volume control module 204 determines whether a volume enhancement control request is received. If so, determines a gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and performs nonlinear enhancement control on the volume of each frame of the second sampling point sequence. If not, performs linear enhancement control on the volume of the second sampling point sequence according to a preset gain.

[0108] Optionally, the apparatus further includes: a volume gradient control module 206, configured to determine, from the text to be processed, a plurality of characters adjacent to the target text based on the position of the target text in the text to be processed; determine, from the first sampling point sequence, an adjacent sampling point sequence corresponding to the character based on the position of the character in the text to be processed; and control the volume of the adjacent sampling point sequence based on a preset intermediate gain, wherein the intermediate gain is smaller than a gain corresponding to a second sampling point sequence adjacent to the adjacent sampling point sequence.

[0109] Optionally, the volume control module 204 determines the type of each character in the target text and the sub-sampling point sequence corresponding to each character, and for each character type, determines the gain corresponding to each frame of the sub-sampling point sequence corresponding to the character type according to a preset automatic gain control algorithm corresponding to the character type, and controls the volume of each frame of the sub-sampling point sequence corresponding to the character type.

[0110] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 Provides volume control methods for synthesized speech.

[0111] This manual also provides Figure 6 The structural diagram of the electronic device shown in FIG. Figure 6 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The volume control method of synthesized speech.

[0112] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as the combination of logic device XOR software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic device.

[0113] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0114] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0115] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0116] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0117] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0122] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0123] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0125] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0127] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0128] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for controlling the volume of a synthesized speech, characterized in that: include: Obtaining a text to be processed including a target text annotated by a speech synthesis markup language; Inputting the text to be processed into a preset speech synthesis model, determining a first sampling point sequence of synthesized speech corresponding to the text to be processed, and determining a second sampling point sequence corresponding to the target text from the first sampling point sequence; determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and controlling the volume of each frame of the second sampling point sequence; Determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and controlling the volume of each frame of the second sampling point sequence specifically includes: Determining whether a volume increase control request is received; If yes, determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and performing nonlinear enhancement control on the volume of each frame of the second sampling point sequence; If not, the volume of the second sampling point sequence is linearly enhanced according to a preset gain.

2. The method according to claim 1, wherein Obtain the text to be processed, which includes the target text annotated by the speech synthesis markup language, including: Obtaining a text to be processed, and performing named entity recognition on the text to be processed; The target text requiring volume adjustment is determined according to the recognition result, and the target text is marked using a speech synthesis markup language.

3. The method according to claim 1, wherein Determining a gain corresponding to each frame of the second sampling point sequence and controlling the volume of each frame of the second sampling point sequence specifically includes: determining a valid voice sampling point sequence from the second sampling point sequence according to a preset voice activity detection algorithm; The gain corresponding to each frame of the valid speech sampling point sequence is determined, and the volume of each frame of the valid speech sampling point sequence is controlled.

4. The method according to claim 1, wherein Determining a second sampling point sequence corresponding to the target text from the first sampling point sequence specifically includes: According to the position of the target text in the text to be processed, determining the position of the text feature corresponding to the target text from the text features corresponding to the text to be processed; Determining, from the mel-spectrogram corresponding to the text to be processed according to the position of the text feature corresponding to the target text, the time period in which the mel-spectrogram corresponding to the target text is located; According to the time period of the mel spectrum corresponding to the target text, a second sampling point sequence corresponding to the target text is determined from the first sampling point sequence.

5. The method according to claim 1, wherein The method further comprises: Determining a plurality of characters adjacent to the target text from the text to be processed according to the position of the target text in the text to be processed; Determining, from the first sampling point sequence, a sequence of adjacent sampling points corresponding to the character according to the position of the character in the text to be processed; Controlling the volume of the adjacent sampling point sequence according to a preset intermediate gain; The intermediate gain is smaller than a gain corresponding to a second sampling point sequence adjacent to the adjacent sampling point sequence.

6. The method according to claim 1, wherein Determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and controlling the volume of each frame of the second sampling point sequence specifically includes: Determining the type of each character in the target text and the subsampling point sequence corresponding to each character; For each character type, the gain corresponding to each frame of the subsampling point sequence corresponding to the character type is determined according to the preset automatic gain control algorithm corresponding to the character type, and the volume of each frame of the subsampling point sequence corresponding to the character type is controlled.

7. A volume control device for synthesized speech, characterized in that: include: An acquisition module, configured to acquire a text to be processed including a target text annotated by a speech synthesis markup language; a sampling point determination module, configured to input the to-be-processed text into a preset speech synthesis model, determine a first sampling point sequence of synthesized speech corresponding to the to-be-processed text, and determine a second sampling point sequence corresponding to the target text from the first sampling point sequence; A volume control module is configured to determine the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and control the volume of each frame of the second sampling point sequence. The volume control module is configured to determine the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm and control the volume of each frame of the second sampling point sequence, specifically comprising: Determining whether a volume increase control request is received; If yes, determining the gain corresponding to each frame of the second sampling point sequence according to a preset automatic gain control algorithm, and performing nonlinear enhancement control on the volume of each frame of the second sampling point sequence; If not, the volume of the second sampling point sequence is linearly enhanced according to a preset gain.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images