Chinese speech signal segmentation method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a Chinese speech signal segmentation method and device, equipment and a storage medium, and the method comprises the steps: carrying out the sampling of a target audio signal containing the speech corresponding to a target Chinese text, and obtaining signal amplitudes corresponding to a plurality of sampling points; performing voice endpoint detection on the target audio data based on the signal amplitudes corresponding to the plurality of sampling points to obtain a plurality of voice segments in the target audio data; determining a vowel position sequence corresponding to the target voice segment based on the formant energy of the voice signal in the target voice segment; determining syllable segmentation points and initial and final segmentation points of the target speech segment based on signal amplitudes of sampling points between two adjacent vowel positions in the vowel position sequence and initial and final consonants of the Chinese text segment corresponding to the target speech segment; and based on the syllable segmentation points and the initial and final segmentation points of the target voice segment, segmenting the target voice segment to obtain segmented voice elements. According to the embodiment of the invention, the accuracy of voice signal segmentation can be improved.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, speech recognition technology, speech synthesis technology, and in particular, a method, apparatus, device, and storage medium for segmenting Chinese speech signals. Background Technology

[0002] In speech recognition and speech synthesis technologies, large-scale training corpora are frequently required. Currently, many corpus resources are available, but these corpora need further processing. Both speech recognition and speech synthesis technologies process speech at the unit level of speech molecules. Therefore, accurate segmentation of initials and finals in Chinese speech to obtain accurate speech molecules is essential for building Chinese corpora. Achieving automatic and accurate initial and final segmentation is crucial for speech recognition and speech synthesis, as it directly affects the accuracy of speech recognition and the naturalness of speech synthesis.

[0003] Currently, various existing Chinese initial and final segmentation algorithms are based on multiple acoustic features for segmentation. However, continuous speech signals contain a lot of noise, and the methods used in related technologies are easily affected by noise when segmenting continuous speech, resulting in low accuracy of initial and final segmentation. Manual correction is required, and when the corpus is large, the workload of manual correction is very large, making it impossible to efficiently segment the initials and finals of continuous speech. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and storage medium for segmenting Chinese speech signals, in order to at least partially solve the problems in related technologies.

[0005] One aspect of this disclosure provides a method for segmenting Chinese speech signals, comprising:

[0006] The target audio signal containing the speech corresponding to the target Chinese text is sampled to obtain the signal amplitude corresponding to multiple sampling points;

[0007] Based on the signal amplitude corresponding to the multiple sampling points, speech endpoint detection is performed on the target audio signal to obtain multiple speech segments in the target audio signal, wherein different speech segments correspond to different Chinese text segments, and the Chinese text segments belong to the target Chinese text;

[0008] Each of the plurality of speech segments is taken as the target speech segment. Based on the formant energy of the speech signal in the target speech segment, the vowel position sequence corresponding to the target speech segment is determined. The vowel position sequence contains the vowel position of each vowel in the target speech segment.

[0009] determine a syllable segmentation point and a phonetic tone and vowel segmentation point of the target speech segment based on the signal amplitude of the sampling points between two adjacent vowel positions in the vowel position sequence and the phonetic tone and vowel of the Chinese text segment corresponding to the target speech segment;

[0010] segment the target speech segment based on the syllable segmentation point and the phonetic tone and vowel segmentation point of the target speech segment to obtain a segmented speech primitive, the speech primitive being a basic unit of the target audio signal.

[0011] In an exemplary embodiment, the speech endpoint detection of the target audio signal based on the signal amplitudes of the sampling points includes:

[0012] calculate the average value of the signal amplitudes of the sampling points in each frame of audio signal to obtain the frame average amplitude of each frame of audio signal;

[0013] if the frame average amplitudes of K consecutive frames of audio signal are less than an amplitude threshold value, traverse the audio frames contained in the target audio signal from the end of the K consecutive frames of audio signal as a starting point, wherein K is a positive integer and K is a preset value, and the amplitude threshold value is calculated based on the average value of the signal amplitudes of the sampling points;

[0014] In the traversal process, if the noise energy of each frame of audio signal in M consecutive frames of audio signal is greater than an energy threshold value, and the sign preserving rate of each frame of audio signal in the M consecutive frames of audio signal is greater than a sign preserving rate threshold value, determine the M consecutive frames of audio signal as the speech segment, wherein M is a positive integer and M is a preset value, the energy threshold value is calculated based on the average noise energy of the K consecutive frames of audio signal, and the sign preserving rate threshold value is calculated based on the average sign preserving rate of the K consecutive frames of audio signal.

[0015] In an exemplary embodiment, the determination of the vowel position sequence corresponding to the target speech segment based on the formant energy of the speech signal in the target speech segment includes:

[0016] window and frame the target speech segment to obtain a short-time signal;

[0017] perform Fourier transform on the short-time signal to obtain a frequency domain signal;

[0018] calculate the formant energy of each frame of audio signal in the target speech segment based on the frequency domain signal;

[0019] perform traversal search on the target speech segment based on the formant energy of each frame of audio signal;

[0020] In the traversal search process, in a case that a resonance peak energy of continuous S frames of audio signals is greater than an energy threshold, sampling points in the continuous S frames of audio signals are determined as vowel positions, where S is a preset positive integer.

[0021] In one exemplary embodiment, the determining of the syllable segmentation point and the initial-final segmentation point of the target speech segment based on the signal amplitude of the sampling points between the adjacent two vowel positions in the vowel position sequence and the initial-final of the Chinese text segment corresponding to the target speech segment comprises:

[0022] The initial type of a Pth syllable in which a Pth vowel position point in the vowel position sequence is based on the initial-final of the Chinese text segment corresponding to the target speech segment, and P is a positive integer;

[0023] The second-order difference value of the continuous frames between the P-1th vowel position point and the Pth vowel position point is calculated based on the signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence;

[0024] The initial-final segmentation point and the syllable segmentation point between the P-1th vowel position point and the Pth vowel position point are determined based on the initial-final type of the Pth syllable and the second-order difference value.

[0025] In one exemplary embodiment, the calculating of the second-order difference value of the continuous frames between the P-1th vowel position point and the Pth vowel position point based on the signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence comprises:

[0026] The speech segment in the P-1th vowel position point and the Pth vowel position point is segmented into a plurality of speech frames based on the signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence;

[0027] The correlation coefficients of adjacent frames are obtained by performing correlation operation on the adjacent frames in the plurality of speech frames;

[0028] The second-order difference of the correlation coefficients corresponding to the continuous three speech frames is performed to obtain the second-order difference values of the continuous three speech frames.

[0029] In one exemplary embodiment, the segmenting of the speech segment in the P-1th vowel position point and the Pth vowel position point into a plurality of speech frames based on the amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence comprises:

[0030] searching a target zero-crossing point from the P-1 vowel position point, the target zero-crossing point being a sample point at which the signal amplitude crosses zero and the future signal amplitude trend changes to positive;

[0031] In the searching process, the i-th speech frame starting from the i-th target zero-crossing point is determined, and the i+1-th target zero-crossing point is searched starting from the end point of the i-th speech frame, where i is a positive integer.

[0032] In an exemplary embodiment, the determination of the syllable division point and the consonant-vowel division point between the P-1 vowel position point and the P-th vowel position point based on the initial type of the P-th syllable and the second-order difference value comprises:

[0033] In the case where the initial type of the P-th syllable is zero initial, the middle frame of the three continuous speech frames corresponding to the maximum second-order difference value is determined as the syllable division point;

[0034] In the case where the initial type of the P-th syllable is clear initial, the middle frame of the three continuous speech frames corresponding to the maximum second-order difference value is determined as the first division point, and the middle frame of the three continuous speech frames corresponding to the second largest second-order difference value is determined as the second division point; the earlier division point of the first division point and the second division point is determined as the syllable division point, and the later division point of the first division point and the second division point is determined as the consonant-vowel division point;

[0035] In the case where the initial type of the P-th syllable is turbid initial, the first T maximum points in the second-order difference value are obtained, and the middle frames between the first T maximum points are combined to obtain a plurality of combined frame segments; the syllable division point and the consonant-vowel division point are determined based on the average zero-crossing rate of the plurality of combined frame segments, where T is a predetermined positive integer.

[0036] In an exemplary embodiment, the plurality of combined frame segments include the minimum speech segments between every two adjacent points in the first T maximum points and the combined segments obtained by combining the minimum speech segments;

[0037] The determination of the syllable division point and the consonant-vowel division point based on the average zero-crossing rate of the plurality of combined frame segments comprises:

[0038] The average zero-crossing rate of the combined segments and the average zero-crossing rate of each minimum speech segment are calculated;

[0039] In the case where the deviation between the average zero-crossing rate of the combined segments and the average zero-crossing rate of the minimum speech segments contained in the combined segments is greater than a preset deviation threshold, the combined segment in the combined frame segment is filtered;

[0040] determining a maximum zero-crossing rate average value of average values of zero-crossing rates in the remaining combined frame segment, and determining the combined frame segment corresponding to the maximum zero-crossing rate average value as the cut segment;

[0041] determining a left end point of the cut segment as the syllable cut point, and determining a right end point of the cut segment as the initial-final cut point.

[0042] Another aspect of the embodiment provides a cutting device for Chinese speech signals, comprising:

[0043] a sampling module configured to sample a target audio signal containing speech corresponding to a target Chinese text to obtain signal amplitudes corresponding to a plurality of sampling points;

[0044] an end point detection module configured to perform speech end point detection on the target audio signal based on the signal amplitudes corresponding to the plurality of sampling points to obtain a plurality of speech segments in the target audio signal, wherein different speech segments correspond to different Chinese text segments, and the Chinese text segments belong to the target Chinese text;

[0045] a vowel detection module configured to take each speech segment in the plurality of speech segments as a target speech segment, and determine a vowel position sequence corresponding to the target speech segment based on formant energies of speech signals in the target speech segment, wherein the vowel position sequence contains vowel positions of each vowel in the target speech segment;

[0046] a cut point determination module configured to determine a syllable cut point and an initial-final cut point of the target speech segment based on signal amplitudes of sampling points between adjacent two vowel positions in the vowel position sequence and an initial-final of a Chinese text segment corresponding to the target speech segment;

[0047] a cutting module configured to cut the target speech segment based on the syllable cut point and the initial-final cut point of the target speech segment to obtain a cut speech primitive, wherein the speech primitive is a basic unit of annotation of the target audio signal.

[0048] Another aspect of the embodiment provides an electronic device, comprising:

[0049] a memory configured to store a computer program;

[0050] a processor configured to execute the computer program stored in the memory, and the computer program, when executed, implements the cutting method for Chinese speech signals of any of the above embodiments.

[0051] Another aspect of the embodiment provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the cutting method for Chinese speech signals of any of the above embodiments.

[0052] In the embodiments of the present disclosure, for a target audio signal to be segmented, first, according to the audio amplitudes of a plurality of sampling points in the target audio signal, voice endpoint detection is performed on the target audio signal to obtain valid voice segments contained in the target audio signal. Through the voice endpoint detection, noise or silence segments in the target audio signal can be ruled out, and a more accurate voice signal can be obtained. Then, by using the feature that the formant energy of vowels is higher than that of other sounds, the formant energy of the voice signal in the voice segment is calculated, so as to determine the positions of each vowel contained in the voice segment according to the formant energy, and obtain a vowel position sequence, which can improve the accuracy of the extraction of the vowel positions. Since the vowel position sequence has been determined, in combination with the initial and final consonants corresponding to the Chinese text, the position segment where the initial and final consonant segmentation points and the syllable segmentation points are located can be obtained. Then, in combination with the amplitudes of the sampling points in the position segment, the specific initial and final consonant segmentation points and the syllable segmentation points are determined, and the segmentation of the target audio signal is realized according to the initial and final consonant segmentation points and the syllable segmentation points. That is, in the embodiments of the present disclosure, the noise and silence segments are ruled out first to obtain a plurality of voice segments, and then the voice segments are segmented, which helps to improve the segmentation accuracy. Since the detection of the formant energy is not easily disturbed by noise, the vowel position sequence is detected by using the formant energy, which can improve the accuracy of the detection of the vowel positions. Then, based on the vowel positions, in combination with the actual initial and final consonants of the Chinese text, the position segment where each segmentation point is located is determined, and the required segmentation points are determined from the position segment, which can further improve the accuracy of the initial and final consonant segmentation and the syllable segmentation, improve the accuracy of the initial and final consonant segmentation and the syllable segmentation of the continuous voice signal, and thus improve the accuracy of the phonological unit segmentation. Based on the accurate phonological units, the accuracy of the phonological units in the corpus and the extraction of the features based on the phonological units can be improved. The method of automatically segmenting the initial and final consonants can improve the efficiency of the segmentation of a large number of audio signals, and helps to improve the efficiency of the construction of a large-scale corpus. In the case of improving the accuracy of the phonological units in the corpus, the phonological units are used for speech synthesis, which can improve the naturalness of the speech synthesis. And the phonological units are used for speech recognition, which can improve the accuracy of the speech recognition.

[0053] The technical solutions of the present disclosure will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0054] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0055] The present disclosure can be more clearly understood and appreciated from the following detailed description, taken in conjunction with the following drawings of which:

[0056] Figure 1 A flowchart of a Chinese speech signal segmentation method provided for an exemplary embodiment of the present disclosure;

[0057] Figure 2 A flow chart of a segmentation process of a Chinese speech signal is provided for an exemplary embodiment of the present disclosure;

[0058] Figure 3 A schematic diagram of a segmentation point in a speech segment is provided for an exemplary embodiment of the present disclosure;

[0059] Figure 4 A structural schematic diagram of a segmentation device of a Chinese speech signal is provided for an exemplary embodiment of the present disclosure;

[0060] Figure 5 A structural schematic diagram of an application embodiment of an electronic device of the present disclosure. DETAILED DESCRIPTION

[0061] Various exemplary embodiments of the present disclosure will now be described in detail below with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0062] Those skilled in the art can understand that the terms "first", "second", and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they represent a necessary logical sequence between them.

[0063] It should also be understood that in the embodiments of the present disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two, or more.

[0064] It should also be understood that for any component, data, or structure mentioned in the embodiments of the present disclosure, unless specifically limited or the context gives a contrary implication, it can generally be understood as one or more.

[0065] In addition, the term "and / or" in the present disclosure is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects.

[0066] It should also be understood that the description of various embodiments of the present disclosure focuses on the differences between the various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0067] At the same time, it should be understood that, for the sake of brevity, the size of each part shown in the drawings is not drawn in accordance with the actual proportional relationship.

[0068] The following description of at least one example embodiment is merely exemplary in nature and is in no way intended to limit the disclosure, its application, or uses.

[0069] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0070] It should be noted that like reference numerals and letters refer to like items throughout the several views, and thus a discussion of the same item in one view does not normally need to be repeated in a similar view.

[0071] Embodiments of the present disclosure can be applied to terminal devices, computer systems, servers, and other electronic devices, which can operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known terminal devices, computer systems, environments, and / or configurations that can be suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computers, server computers, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputers, mainframe computers, and distributed cloud computing environments that include any of the above systems or the like.

[0072] Terminal devices, computer systems, servers, and other electronic devices can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like, that perform particular tasks or implement particular abstract data types. Computer systems / servers can be practiced in distributed cloud computing environments with remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including memory storage devices.

[0073] Voice segmentation refers to the process of segmenting a voice signal into a voice basic unit (i.e., the voice primitive involved in embodiments of the present disclosure), which can be a syllable or a smaller phoneme such as an initial consonant and vowel. In embodiments of the present disclosure, the voice primitive uses an initial consonant and vowel as the primitive.

[0074] In speech synthesis technology, synthesis is performed in units of voice primitives. In the process of constructing a corpus for speech synthesis, the voice in the corpus first needs to be segmented and labeled, the voice primitives are obtained through segmentation, and then each voice primitive is labeled with features, and a corpus for speech synthesis is constructed. Speech synthesis based on accurate voice primitives in the corpus can improve the naturalness of speech synthesis.

[0075] In speech recognition technology, speech units are used for recognition. In the process of constructing a speech corpus for speech recognition, the speech in the corpus needs to be segmented to obtain speech units contained in each speech, and subsequent feature extraction and feature labeling are based on speech units to construct a corpus for speech recognition. Speech recognition based on accurate speech units in the corpus can improve the accuracy of speech recognition.

[0076] To solve the problems in the related art, an embodiment of the present disclosure provides a segmentation method of a Chinese speech signal. Figure 1 FIG. 1 is a flowchart of a segmentation method of a Chinese speech signal according to an example embodiment of the present disclosure. The method provided in the embodiment of the present disclosure can be used in an electronic device, such as a mobile phone, a tablet computer, a smart speaker, etc. Figure 1 As shown in FIG. 1, the method comprises the following steps:

[0077] In step 101, a target audio signal containing speech corresponding to a target Chinese text is sampled to obtain signal amplitudes corresponding to a plurality of sampling points.

[0078] The target audio signal is a speech signal to be segmented, which contains speech corresponding to the target Chinese text. Optionally, the target Chinese text can be a sentence, a paragraph, or an article, etc., and the target audio signal is a continuous speech signal corresponding to the target Chinese text. Optionally, the target audio signal can be a sample speech signal obtained in the process of constructing a corpus.

[0079] For example, the target Chinese text can be "The Northern Song Dynasty Emperor Zhenzong made Bianliang. The king was virtuous and the ministers were wise, the people were peaceful and happy, and the West Xia invaded, causing the Central Plains to be in flames. In March, everything was recovering, and in the 800 li of Qin, green willows were in line."

[0080] The signal amplitude of the audio signal represents the intensity or energy of the corresponding sound. In the embodiment of the present disclosure, the target audio signal is segmented based on the signal amplitude of the target audio signal. First, the target audio signal can be sampled at a preset sampling rate to obtain a plurality of sampling points and signal amplitudes corresponding to the sampling points. Then, the target audio signal is segmented in combination with the sampling points, and the target audio signal can be segmented using a preset unit frame length, and the sampling points in the preset unit frame length are combined into a frame. For example, the preset unit frame length can be 20 ms, i.e., one frame is 20 ms.

[0081] For the above example, the preset sampling rate is 22050 Hz, and sampling the target audio signal corresponding to the above example Chinese text can obtain 992250 sampling points. The total length of the target audio signal is 45s, and one frame is 20ms, which can obtain 2250 frames, each frame containing 441 sampling points.

[0082] Step 102: Based on the signal amplitude corresponding to multiple sampling points, perform speech endpoint detection on the target audio signal to obtain multiple speech segments in the target audio signal. Among them, different speech segments correspond to different Chinese text segments, and the Chinese text segments belong to the target Chinese text.

[0083] The target audio signal consists of continuous speech audio corresponding to the target Chinese text. It contains silent segments or noise, such as pauses within paragraphs. To improve detection accuracy, silent or noisy segments are first identified in the target audio signal. Speech endpoint detection refers to detecting the presence or absence of speech in an audio signal within a noisy environment. It is used to accurately detect the start and end positions of valid speech segments in the audio signal. Performing speech endpoint detection on the target audio signal yields the start and end positions of multiple speech segments. The target Chinese text contains multiple text segments; each speech segment obtained through speech endpoint detection corresponds to a different Chinese text segment—that is, one speech segment corresponds to one Chinese text segment.

[0084] In this embodiment of the disclosure, speech endpoint detection is performed based on the energy characteristics of the audio signal. After obtaining the signal amplitude corresponding to several sampling points, the energy of each frame of audio signal can be determined based on the signal amplitude of each sampling point, thereby segmenting the target audio signal to obtain multiple speech segments.

[0085] If the noise energy characteristics of multiple consecutive speech frames meet the speech decision criteria, the multiple consecutive speech frames can be identified as a speech segment.

[0086] To illustrate, based on the above example, after segmenting the target audio signal, we can obtain 13 speech segments: "Emperor Renzong of the Northern Song Dynasty established Bianliang", "The emperor is upright and the ministers are virtuous", "The people are safe and sound", "How hateful", "Western Xia", "Invasion", "Caused", "Central Plains", "Blazing with fire", "The weather in March", "All things revive", "Eight hundred miles of Qin Chuan", "Green willows in rows".

[0087] Step 103: Take each of the multiple speech segments as the target speech segment, and determine the vowel position sequence corresponding to the target speech segment based on the formant energy of the speech signal in the target speech segment. The vowel position sequence contains the vowel position of each vowel in the target speech segment.

[0088] After detecting each speech segment, a rough detection of syllable position can be performed. In one possible implementation, a detection of vowels in the speech segment can be performed. In which, the energy of Chinese syllable is mainly concentrated in the vowel, and the formant frequency of the vowel is mainly concentrated between 290-4000Hz. Since the formant energy (FE) of the vowel is much larger than the FE parameter of non-vowel, and it is less affected by noise, the FE parameter can be used to detect the position of the vowel in the speech segment, and a vowel position sequence is obtained.

[0089] During the detection, for each speech segment, a corresponding vowel position sequence is detected. Each speech segment can be taken as a target speech segment, and the FE parameter of the speech signal in the target speech segment is determined, so as to determine the vowel position sequence of the target speech segment based on the FE parameter of the target speech segment, and the vowel position sequence contains the positions of all vowels in the target speech segment.

[0090] Illustratively, when the vowel detection is performed on the speech segment corresponding to "Min'ankang", the detected vowel position sequence contains the positions of the three vowels in "Min'ankang".

[0091] In step 104, the signal amplitude of the sampling points between the adjacent two vowel positions in the vowel position sequence and the initial and final consonants of the Chinese text segment corresponding to the target speech segment are used to determine the syllable segmentation point and the initial and final consonant segmentation point of the target speech segment.

[0092] After the vowel positions are determined, the positions of the syllable and / or the initial and final consonant are contained between the two vowels. In one possible implementation, the initial and final consonants of the Chinese text segment corresponding to the target speech segment are determined to obtain the initial and final consonants contained in the corresponding pinyin, so that the syllable segmentation point and / or the initial and final consonant segmentation point contained in the two vowels can be determined. Illustratively, the vowel position sequence of "Min'ankang" contains {P1, P2, P3}, in which, the initial and final consonant segmentation point of the initial consonant "m" and the vowel "in" is contained between the starting sampling point P0 and P1 of the speech segment corresponding to "Min'ankang"; the syllable segmentation point between "min" and "an" is contained between P1 and P2; the syllable segmentation point between "an" and "kang" and the initial and final consonant segmentation point between the initial consonant "k" and the vowel "ang" are contained between P2 and P3.

[0093] That is, the number and type of segmentation points contained in the speech segment (vowel segment) between the two vowel positions can be determined according to the initial and final consonants of the Chinese text segment corresponding to the target speech segment. In combination with the amplitude of the sampling points contained in the vowel segment, the vowel segment is searched to obtain the syllable segmentation point and / or the initial and final consonant segmentation point, so as to improve the detection accuracy.

[0094] That is, in the syllable segmentation method of the Chinese speech signal provided by the embodiments of the present disclosure, the initial and final consonants of the Chinese text segment corresponding to the target speech segment are used to determine the number and type of segmentation points contained in the speech segment (vowel segment) between the two vowel positions. Figure 2As shown, the voice endpoint detection process, the vowel segment detection process, and the consonant-vowel division process are included, wherein the voice endpoint detection process detects a plurality of Chinese text segments corresponding to voice segments, and then the vowel segment detection process is performed on the plurality of voice segments to obtain vowel segments included in each voice segment, so as to perform consonant-vowel division and syllable division based on the vowel segments to obtain syllable division points and consonant-vowel division points.

[0095] In step 105, based on the syllable division points and the consonant-vowel division points of the target voice segment, the target voice segment is divided to obtain a divided voice primitive. The voice primitive is a basic unit for labeling the target audio signal.

[0096] According to the syllable division points and the consonant-vowel division points included in the target voice segment, the target voice segment can be divided to obtain the voice primitive included in the target voice segment. After dividing all voice segments, all voice primitives included in the target audio data can be obtained, and the consonant-vowel and syllable division of continuous voice can be realized to obtain voice primitives in units of consonant-vowel.

[0097] The voice primitive can be used in the process of constructing a corpus. The target audio signal is labeled in units of voice primitives to obtain a sample audio signal for training, which is stored in the corpus.

[0098] That is, in the embodiment of the present disclosure, for the target audio signal to be segmented, first, according to the audio amplitude of a plurality of sampling points in the target audio signal, voice endpoint detection is performed on the target audio signal to obtain the valid voice segment contained in the target audio signal. Through voice endpoint detection, noise or silent segments in the target audio signal can be ruled out, and a more accurate voice signal can be obtained; then, by using the characteristic that the formant energy of vowels is higher than that of other sounds, the formant energy of the voice signal in the voice segment is calculated, so as to determine the positions of each vowel contained in the voice segment according to the formant energy, and obtain the vowel position sequence, which can improve the accuracy of vowel position extraction; since the vowel position sequence has been determined, in combination with the initial and final consonants corresponding to the Chinese text, the position segment where the initial and final consonant segmentation points and the syllable segmentation points are located can be obtained, and then in combination with the amplitudes of the sampling points in the position segment, the specific initial and final consonant segmentation points and syllable segmentation points are determined, and the segmentation of the target audio signal is realized according to the initial and final consonant segmentation points and the syllable segmentation points. That is, in the embodiment of the present disclosure, the noise and silent segments in the target audio signal are first ruled out to obtain each voice segment, and then the voice segment is segmented, which helps to improve the segmentation accuracy; and since the detection of formant energy is not easily disturbed by noise, the vowel position sequence is detected by using the formant energy, which can improve the accuracy of vowel position detection. Based on the vowel position, in combination with the actual initial and final consonants of the Chinese text, the position segment where each segmentation point is located is determined, the required segmentation point is determined from the position segment, which can further improve the accuracy of initial and final consonant and syllable segmentation, improve the accuracy of initial and final consonant and syllable segmentation of continuous voice signal, and thus improve the accuracy of phonological unit segmentation. Based on the accurate phonological unit, the accuracy of phonological unit in the corpus and the accuracy of the extracted features based on the phonological unit can be improved. The method of automatically segmenting initial and final consonants can improve the efficiency of segmenting a large number of audio signals, and help to improve the efficiency of constructing a large-scale corpus. In the case of improving the accuracy of phonological unit in the corpus, the phonological unit is used for speech synthesis, which can improve the naturalness of speech synthesis; and the phonological unit is used for speech recognition, which can improve the accuracy of speech recognition.

[0099] In the embodiment of the present disclosure, in the process of voice endpoint detection, the noise energy of the audio is calculated based on the audio amplitude, and the target audio signal is cut according to the change of the noise energy and the signal amplitude to obtain the voice segment. The process of performing voice endpoint detection on the target audio signal based on the signal amplitudes of a plurality of sampling points to obtain a plurality of voice segments in the target audio signal includes the following steps:

[0100] Step 1021, calculate the average value of the signal amplitudes of the sampling points in each frame of audio signal to obtain the frame average amplitude of each frame of audio signal.

[0101] First, the average value of the signal amplitudes of the sampling points in each frame of audio signal can be calculated to obtain the frame average amplitude corresponding to each audio frame.

[0102] The calculation of the frame average amplitude of the mth frame is as follows:

[0103]

[0104] Wherein, N is the sampling points contained in a frame, S[N(m-1)+i] represents the signal amplitude of the ith sampling point in the mth frame. Illustratively, in combination with the above example, N is 441.

[0105] In step 1022, if the frame average amplitudes of the continuous K frames of audio signals are less than the amplitude threshold value, the end of the continuous K frames of audio signals is taken as the starting point to traverse the audio frames contained in the target audio signal, wherein K is a positive integer and K is a preset value, and the amplitude threshold value is calculated based on the average of the amplitudes of a plurality of sampling points.

[0106] The initial noise or silence section is determined according to the frame average amplitudes of the continuous K frames of audio signals. When the frame average amplitudes of the continuous K frames of audio signals are all less than the amplitude threshold value, the voice section can be searched starting from the end of the continuous K frames of audio signals.

[0107] Wherein, the amplitude threshold value can be calculated according to the average of the amplitudes of all sampling points (i.e., the average total amplitude of the audio signal).

[0108] Wherein, the calculation of the average total amplitude of the target audio signal is as follows:

[0109]

[0110] Wherein, L is the total number of sampling points, and S(i) represents the signal amplitude of the ith sampling point among all sampling points. Illustratively, in combination with the above example, L is 992250.

[0111] Optionally, the amplitude threshold value can be 1 / 5 of the average total amplitude.

[0112] K is a preset positive integer. Illustratively, K is 10. When the A m of the continuous 10 frames is less than 1 / 5A, the voice section can be searched starting from the last sampling point in the continuous 10 frames.

[0113] In step 1023, during the traversal process, if the noise energy of each frame of audio signal in the continuous M frames of audio signals is greater than the energy threshold value, and the sign preserving rate of each frame of audio signal in the continuous M frames of audio signals is greater than the sign preserving rate threshold value, it is determined that the continuous M frames of audio signals are a voice section, wherein M is a positive integer and M is a preset value, the energy threshold value is calculated based on the average noise energy of the continuous K frames of audio signals, and the sign preserving rate threshold value is calculated based on the average sign preserving rate of the continuous K frames.

[0114] When searching for the speech segment, the noise energy of the continuous K frame audio signal and the noise preserving rate can be used as the basis for searching for the speech segment. First, the average noise energy of the continuous K frame audio signal can be calculated to determine the energy threshold required for judging the speech segment. The calculation method of the noise energy of each frame is as follows:

[0115]

[0116] wherein K is the number of sampling points contained in one frame, S(i) represents the signal amplitude of the i th sampling point, E n is the noise energy of the n th frame.

[0117] The calculation method of the average noise energy of the continuous K frame is as follows:

[0118]

[0119] The noise preserving rate threshold for judging can be determined according to the average noise preserving rate of the continuous K frame audio signal. The noise preserving rate refers to the maximum number of sampling points in which the sampling values in one frame continuously maintain the same sign, and the expression is as follows:

[0120]

[0121] wherein L m refers to the number of sampling points contained in the m th segment in which the sampling values in one frame continuously maintain the same sign, and each segment is composed of continuous sampling points with the same sign.

[0122] wherein each segment of the speech sequence satisfies the following condition:

[0123]

[0124] wherein S n,m (i) represents the signal amplitude of the i th sampling point of the m th segment of the n th frame of the target audio signal.

[0125] The noise preserving rate of each frame can be calculated according to the above method, and the average noise preserving rate of the continuous K frame audio signal is:

[0126]

[0127] In one possible implementation, an integer multiple of the average noise energy can be determined as the energy threshold, and an integer multiple of the average noise preserving rate can be determined as the noise preserving rate threshold. Illustratively, the energy threshold can be 5 times the average noise energy (5*E_arverage), and the noise preserving rate threshold can be 5 times the average noise preserving rate (5*Rks_arverage).

[0128] The search starts from the last sample point in the last K frames of audio signals, and when the noise energy of each of the M continuous frames of audio signals is greater than the energy threshold, and the noise preservation rate of each of the M continuous frames of audio signals is greater than the noise preservation rate threshold, the M continuous frames of audio signals are determined as a speech segment, that is, the first sample point in the M continuous frames of audio signals is the start position of the speech segment, and the last sample point is the end position of the speech segment.

[0129] Illustratively, M can be 5. When the noise energy of the 5 continuous frames of audio signals is greater than 5*E_arverage, and the noise preservation rate of the 5 continuous frames of audio signals is greater than 5*Rks_arverage, the 5 continuous frames of audio signals are determined as a speech segment.

[0130] Through the traversal mode, a plurality of speech segments contained in the target audio signal can be obtained.

[0131] In the embodiments of the present disclosure, the noise energy of each frame of audio signal and the noise preservation rate of each frame of audio signal are used for speech segment detection. The noise energy can reflect the intensity of the sound, and the noise preservation rate can reflect the change of the sound intensity in a period of time. The accuracy of speech detection can be improved by detecting speech based on the two.

[0132] In the above embodiments, speech endpoint detection is performed on the target audio signal to obtain a plurality of speech segments, and then the vowel position in the speech segment is detected using the FE parameter to roughly determine the position of the tone node and the segment of the consonant-vowel division point for each division. The process of determining the vowel position sequence corresponding to the speech segment based on the formant energy of the speech signal in the speech segment includes the following steps:

[0133] Step 1031, windowing and framing the target speech segment to obtain a short-time signal.

[0134] First, the speech signal corresponding to the target speech segment is windowed and framed to obtain a short-time signal x(n) corresponding to the target speech segment, where x(n) represents the short-time signal corresponding to the nth frame.

[0135] Step 1032, performing Fourier transform on the short-time signal to obtain a frequency domain signal.

[0136] The short-time signal is a time domain signal, and needs to be converted into a frequency domain signal by using Fourier transform. Optionally, the short-time time domain signal can be converted into a frequency domain signal X(K) by using Fast Fourier Transform (FFT).

[0137] Step 1033, calculating the formant energy of each frame of audio signal in the target speech segment based on the frequency domain signal.

[0138] Then, the FE parameter of the frequency domain signal X(K) in 290-4000Hz is calculated. The calculation method is as follows:

[0139]

[0140] FE is the resonance peak energy corresponding to the nth frame of audio signal.

[0141] In combination with the above example, the N=441 time domain data of each frame can be zero-padded to 512 data, and then the frequency domain signal X(k) is obtained by FFT transformation. The highest frequency is 11025Hz, the sampling frequency is 22050Hz, the frequency resolution is 43Hz, the short-time energy FE in the frequency range of 290-4000Hz is calculated, and the calculation method is as follows:

[0142]

[0143] In step 1034, the target speech segment is searched based on the resonance peak energy of each frame of audio signal.

[0144] In step 1035, in the resonance peak energy of the continuous S frames of audio signal is greater than the energy threshold, the sampling point in the continuous S frames of audio signal is determined as the vowel position.

[0145] The resonance peak energy corresponding to the vowel is greater than the resonance peak energy corresponding to other sounds. According to the resonance peak energy of each frame of audio signal, the search is performed in the speech segment. In one possible implementation, the resonance peak energy of the continuous S frames of audio signal can be judged.

[0146] The maximum value in the FE parameter corresponding to all frames can be used to determine the energy threshold. Optionally, the energy threshold can be 1 / 8f m , f m , which is the maximum value of the FE parameter corresponding to all frames in the speech segment.

[0147] In the case that the FE of the continuous S frames of audio signal is greater than the energy threshold, the sampling point in the continuous S frames of audio signal is determined as the vowel position. The target sampling point can be the first sampling point or the intermediate sampling point in the continuous S frames.

[0148] When the condition that the FE of the continuous S frames of audio signal is greater than the energy threshold is not met, and the FE of the continuous S frames of audio signal greater than the energy threshold is searched again, the target sampling point of the continuous S frames of audio signal searched again is determined as the next vowel position.

[0149] In the embodiment of the present disclosure, the resonance peak energy of the continuous multiple frames is used to detect the vowel position in the speech segment, which can improve the accuracy of vowel position detection.

[0150] After the vowel detection, the initial and final consonants can be segmented according to the detected vowel position sequence. The process includes the following steps:

[0151] 1041, based on the initial and final consonants of the target speech segment corresponding to the Chinese text segment, determine the initial type of the P-th syllable in the P-th vowel position point in the vowel position sequence.

[0152] The number of vowels can be obtained at the same time as the vowel detection, and then the corresponding target Chinese text is searched to find the Chinese syllable corresponding to the speech segment. Each vowel position in the target speech segment corresponds to a syllable. In a possible implementation, the Chinese text can be converted into Chinese pinyin to obtain the specific initial type of the last syllable in the syllable segmentation. The initial type includes zero initial, clear initial and dark initial.

[0153] In combination with the above example, the vowel segment detection of the speech segment corresponding to "Min'an Kang" in the 13 speech segments will obtain 3 vowel positions, and the number of vowels is 3, corresponding to the 3 syllables of "Min'an Kang", the Chinese pinyin is "min an kang", and the initial of the third syllable "kang" is "k".

[0154] 1042, based on the signal amplitude of each sampling point in the P-1-th vowel position point and the P-th vowel position point in the vowel position sequence, calculate the second-order difference value of the continuous frames between the P-1-th vowel position point and the P-th vowel position point.

[0155] After the vowel segment detection, the vowel position sequence {P1, P2, P3, …, Pn} is obtained, and the initial and final consonants need to be segmented between the position point 1 and the position point P1, the position point P1 and the position point P2, the position point P2 and the position point P3, …, the position point Pn-1 and the position point Pn.

[0156] Step a1, based on the signal amplitude of each sampling point in the P-1-th vowel position point and the P-th vowel position point in the vowel position sequence, frame the speech segment in the P-1-th vowel position point and the P-th vowel position point to obtain a plurality of speech frames.

[0157] During the segmentation process, the speech segment between the two position points is re-framed. When the speech segment between the P-1-th vowel position point and the P-th vowel position point is framed, the target zero-crossing point is searched from the P-1-th vowel position point, and the target zero-crossing point refers to the sampling point where the signal amplitude crosses zero and the future signal amplitude trend changes to positive.

[0158] During the search process, the i-th speech frame starting from the i-th target zero-crossing point is determined, and the i+1-th target zero-crossing point is searched starting from the end point of the i-th speech frame, and i is a positive integer.

[0159] That is, starting from the sampling point corresponding to the P-1th vowel position point, a target zero-crossing point is searched for as the starting point of a new frame, wherein the target zero-crossing point refers to a sampling point whose signal amplitude crosses zero and whose future change trend changes to a positive value, that is, the amplitude of the previous sampling point of the target zero-crossing point is negative, and the amplitude corresponding to the target zero-crossing point itself is a positive value, and the amplitude of the next sampling point is also a positive value.

[0160] With the target zero-crossing point as the starting point of a new frame, a corresponding speech frame is obtained according to a preset frame length, which can be 20 ms. Then, the next target zero-crossing point is searched for starting from the ending sampling point of the divided speech frame, and the next speech frame is obtained starting from the next target zero-crossing point.

[0161] In step a2, correlation operations are performed on adjacent frames in the plurality of speech frames to obtain correlation coefficients of the adjacent frames.

[0162] After the plurality of speech frames are obtained by the above method, correlation operations are performed on adjacent frames in the plurality of speech frames. It should be noted that the adjacent frames refer to two adjacent speech frames obtained according to the division order, and the sampling points in the adjacent frames can not be continuous.

[0163] The correlation coefficients of the adjacent frames are calculated as follows:

[0164]

[0165] wherein S j (n) represents the signal amplitude corresponding to the nth sampling point in the jth frame.

[0166] In step a3, second-order differences are taken on the correlation coefficients corresponding to three continuous speech frames to obtain second-order difference values of the plurality of three continuous speech frames.

[0167] In the embodiments of the present disclosure, second-order differences are taken on the correlation coefficients of the continuous frames to determine the segmentation points between syllables or initials and finals. The second-order difference is as follows:

[0168] G 2 (j) = |R(j+2)-R(j)| / H

[0169] wherein G 2 (j) represents a second-order difference value between the jth frame and the j+2th frame,

[0170] and

[0171] Second-order differences are taken on the three continuous speech frames respectively to obtain second-order difference values of the plurality of three continuous speech frames respectively.

[0172] 1043, determining the syllable division point and the phonetic coda division point between the P-1 vowel position point and the P vowel position point based on the initial type of the P-th syllable and the second-order difference value.

[0173] After obtaining the second-order difference value, when determining the division point, different determination methods will be used to determine the division point according to the category of the initial of the latter syllable (zero initial, voiced initial and unvoiced initial).

[0174] The P-1 vowel position point and the P vowel position point contain the syllable division point between the P-1 syllable and the P syllable, and the determination method can be determined according to the initial type of the latter syllable (i.e. the P-th syllable). For example, as described above, after detecting the vowels of "Minan Kang", the vowel position sequence {P1, P2, P3} is obtained, wherein the P1 and P2 position points contain the syllable division point between the syllables "min" and "an", and the determination method of the division point can be determined according to the initial type of the latter syllable, i.e. the initial type of "an".

[0175] Optionally, in the case that the initial type of the P-th syllable is zero initial, the middle frame of the continuous three frames corresponding to the maximum second-order difference value is determined as the syllable division point.

[0176] When the latter syllable is zero initial, there is a syllable division point between the P-1 vowel position point and the P vowel position point, and there is no phonetic coda division point. Therefore, the maximum value of the plurality of second-order difference values can be obtained, and the syllable division point can be determined according to the maximum value. The middle frame of the continuous three frames of speech frames corresponding to the maximum second-order difference value can be determined as the syllable division point. That is, when the second-order difference value of the cross-correlation between the k-th speech frame and the k+1-th speech frame and the cross-correlation between the k+1-th speech frame and the k+2-th speech frame is the maximum value, the k+1-th speech frame is the syllable division point.

[0177] For example, as described above, "an" is zero initial, and the middle frame of the continuous three frames of speech frames with the maximum second-order difference value between P1 and P2 can be determined as the syllable division point.

[0178] Optionally, in the case that the initial type of the P-th syllable is unvoiced initial, the middle frame of the continuous three frames of speech frames corresponding to the maximum second-order difference value is determined as the first division point, and the middle frame of the continuous three frames of speech frames corresponding to the second largest second-order difference value is determined as the second division point; the former division point of the first division point and the second division point is determined as the syllable division point, and the latter division point of the first division point and the second division point is determined as the phonetic coda division point.

[0179] When the post-coda is a clear initial, there are syllable division points between the P-1th and Pth vowel position points, and there is a consonant-vowel coda division point. The maximum second-order difference value and the second maximum second-order difference value can be obtained. The middle frame of the continuous three frames of speech frames corresponding to the maximum second-order difference value is one division point, and the middle frame of the continuous three frames of speech frames corresponding to the second maximum second-order difference value is another division point. Since the syllable division point is located on the time axis before the consonant-vowel coda division point, the front division point of the two division points (the first division point and the second division point) located on the front of the time axis is determined as the syllable division point, and the rear division point located on the rear of the time axis is determined as the consonant-vowel coda division point.

[0180] For example, as shown in the above example, P2 and P3 contain the syllable division points of the syllables "an" and "kang" and the consonant-vowel coda division point in "kang". The front division point can be determined as the division point between the syllables "an" and "kang", and the rear division point can be determined as the division point of the initial "k" and the vowel "ang". Figure 3

[0181] Optionally, when the initial type of the Pth syllable is a voiced initial, the first T maximum points in the second-order difference value are obtained, and the middle frames between the first T maximum points are combined to obtain a plurality of combined frame segments. Based on the zero-crossing rate average of the plurality of combined frame segments, the syllable division point and the consonant-vowel coda division point are determined.

[0182] When the post-coda is a voiced initial, the zero-crossing rate average is used for judgment. The first T maximum points in the second-order difference value are obtained first, and the speech frames are combined based on the first T maximum points to obtain combined frame segments.

[0183] The plurality of combined frame segments contain the minimum speech segments between every two adjacent points in the first T maximum points and the combined segments obtained by combining the minimum speech segments. When the minimum speech segments are combined, continuous two segments, continuous three segments, and all minimum speech segments can be combined.

[0184] For example, when T is 5, the first five maximum points in the second-order difference value are obtained first. One minimum speech segment can be obtained by combining every two sampling points in the first five maximum points, and four minimum speech segments can be obtained by combining the five maximum points. Then, the four minimum speech segments are combined. The combination methods can be two-by-two combination, continuous three-segment combination, and four-segment combination. The two-by-two combination can obtain three new combined segments, the continuous three-segment combination can obtain two new combined segments. That is, the combined frame segments obtained by combining the five maximum points contain four minimum speech segments, three combined segments obtained by two-by-two combination, two combined segments obtained by continuous three-segment combination, and one combined segment obtained by four-segment combination, a total of ten combined segments.

[0185] ​After that, the combined segments can be filtered to obtain the split segments. The process includes the following steps:

[0186] Step b1, calculating the average zero-crossing rate of the combined segment and the average zero-crossing rate of each minimum speech segment.

[0187] Wherein, the zero-crossing rate of each frame is calculated as follows:

[0188]

[0189] Wherein, n is the nth sampling point, sgn[] is the sign function, and w(n) is the window function. The window function is as follows:

[0190]

[0191] When calculating the average zero-crossing rate of the minimum speech segment, the average zero-crossing rate of all frames contained in the minimum speech segment can be calculated, and then the average zero-crossing rate is averaged to obtain the average zero-crossing rate.

[0192] Illustratively, when the minimum speech segment contains 3 frames, the average zero-crossing rate of the minimum speech segment is (Z j1 +Z j2 +Z j3 ) / 3.

[0193] The combined segment refers to the combined speech segment obtained by continuously combining the minimum speech segments. The average zero-crossing rate of the combined segment can be calculated, wherein the ratio of the sum of the zero-crossing rates of all frames contained in the combined segment to the sum of the frame numbers is the average zero-crossing rate of the combined segment.

[0194] Illustratively, when the combined segment is obtained by combining two consecutive minimum speech segments, in combination with the above example, the combined segment contains 6 frames, and the average zero-crossing rate is (Z j1 +Z j2 +Z j3 +Z j4 +Z j5 +Z j6 ) / 6.

[0195] Step b2, filtering the combined segment in the combined segment in the case where the deviation between the average zero-crossing rate of the combined segment and the average zero-crossing rate of the minimum speech segment contained in the combined segment is greater than the preset deviation threshold.

[0196] The combined segment is filtered according to the deviation between the average zero-crossing rate of the combined segment and the average zero-crossing rate of the minimum speech segment corresponding to the combined segment. Wherein, the preset deviation threshold corresponding to different combination modes can be different.

[0197] When performing bias comparison, the average zero-crossing rate of the combined segment is compared with the average zero-crossing rate of each of the smallest speech segments contained in the combined segment. When the bias of one of the smallest speech segments is greater than the bias threshold, the combined segment is filtered.

[0198] Illustratively, in combination with the above example, when two smallest speech segments are combined, the average zero-crossing rate of the combined segment obtained by combining the two adjacent segments is compared with the average zero-crossing rate of each of the two smallest speech segments, respectively. When the bias is greater than 30%, the combined segment is filtered.

[0199] When three smallest speech segments are combined, the average zero-crossing rate of the combined segment obtained by combining the three adjacent segments is compared with the average zero-crossing rate of each of the three smallest speech segments, respectively. When the bias is greater than 20%, the combined segment is filtered.

[0200] When four smallest speech segments are combined, the average zero-crossing rate of the combined segment obtained by combining the four adjacent segments is compared with the average zero-crossing rate of each of the four smallest speech segments, respectively. When the bias is greater than 10%, the combined segment is filtered.

[0201] Step b3, determining the maximum average zero-crossing rate of the average zero-crossing rate of the remaining combined frame segments, and determining the combined frame segment corresponding to the maximum average zero-crossing rate as the segmentation segment.

[0202] After filtering the combined segments in the combined frame segment, a set of possible segmentation segments is obtained. The maximum value of the average zero-crossing rate of the remaining combined frame segments is determined, and the combined frame segment corresponding to the maximum average zero-crossing rate is determined as the segmentation segment. The remaining combined frame segment is the combined frame segment remaining after filtering the combined segments in the above manner.

[0203] Step b4, determining the left end point of the segmentation segment as the syllable segmentation point, and determining the right end point of the segmentation segment as the initial-final segmentation point.

[0204] Illustratively, in the speech segment "green willows", the initial of the syllable "liu" is "l", which is a voiced initial. Finally, the left end point of the segmentation segment is the segmentation point between the syllables "lv" and "liu", and the right end point is the segmentation point of the initial "l" and the final "iu".

[0205] In the embodiments of the present disclosure, the determination mode of the segmentation point is determined according to the initial type of the segmented syllable. Different judgment conditions are used to determine the syllable segmentation point and the initial-final segmentation point for different types of initials, thereby improving the segmentation accuracy.

[0206] Figure 4 is a structural block diagram of a segmentation device for a Chinese speech signal provided by an exemplary embodiment of the present disclosure. As shown in Figure 4 The device comprises:

[0207] The sampling module 401 is configured to sample a target audio signal containing target Chinese text corresponding speech to obtain signal amplitudes corresponding to a plurality of sampling points;

[0208] The endpoint detection module 402 is configured to perform speech endpoint detection on the target audio signal based on the signal amplitudes corresponding to the plurality of sampling points to obtain a plurality of speech segments in the target audio signal, wherein different speech segments correspond to different Chinese text segments, and the Chinese text segments belong to the target Chinese text.

[0209] The vowel detection module 403 is configured to take each of the plurality of speech segments as a target speech segment, and determine a vowel position sequence corresponding to the target speech segment based on formant energy of speech signals in the target speech segment, wherein the vowel position sequence contains vowel positions of each vowel in the target speech segment.

[0210] The split point determination module 404 is configured to determine syllable split points and initial-final split points of the target speech segment based on signal amplitudes of sampling points between adjacent two vowel positions in the vowel position sequence and initial-final of a Chinese text segment corresponding to the target speech segment.

[0211] The splitting module 405 is configured to split the target speech segment based on the syllable split points and the initial-final split points of the target speech segment to obtain a split speech primitive, wherein the speech primitive is a basic unit for labeling the target audio signal.

[0212] In an exemplary embodiment, the endpoint detection module 402 is further configured to:

[0213] Calculate an average value of signal amplitudes of sampling points in each frame of audio signal to obtain a frame average amplitude of each frame of audio signal.

[0214] If frame average amplitudes of K consecutive frames of audio signal are less than an amplitude threshold value, traverse audio frames contained in the target audio signal from an end of the K consecutive frames of audio signal as a starting point, wherein K is a positive integer, K is a preset value, and the amplitude threshold value is calculated based on an average value of signal amplitudes of the plurality of sampling points.

[0215] If, in the traversal process, noise energy of each frame of audio signal in M consecutive frames of audio signal is greater than an energy threshold value, and a noise preservation rate of each frame of audio signal in the M consecutive frames of audio signal is greater than a noise preservation rate threshold value, determine the M consecutive frames of audio signal as the speech segment, wherein M is a positive integer, M is a preset value, the energy threshold value is calculated based on an average noise energy of the K consecutive frames of audio signal, and the noise preservation rate threshold value is calculated based on an average noise preservation rate of the K consecutive frames of audio signal.

[0216] In an example embodiment, the vowel detection module 403 is further configured to:

[0217] windowing and framing the target speech segment to obtain a short-time signal;

[0218] performing Fourier transform on the short-time signal to obtain a frequency domain signal;

[0219] calculating formant energy of each frame of audio signal in the target speech segment based on the frequency domain signal;

[0220] performing a traversal search on the target speech segment based on the formant energy of each frame of audio signal;

[0221] in the traversal search process, in a case where the formant energy of continuous S frames of audio signal is greater than an energy threshold, determining a sampling point in the continuous S frames of audio signal as a vowel position, wherein S is a preset positive integer.

[0222] In an example embodiment, the split point determination module 404 is further configured to:

[0223] determining an initial consonant type of a P-th syllable in which a P-th vowel position point in the vowel position sequence is located based on an initial consonant and a final consonant of a Chinese text segment corresponding to the target speech segment, wherein P is a positive integer;

[0224] calculating a second-order difference value of continuous frames between the P-1-th vowel position point and the P-th vowel position point based on signal amplitudes of each sampling point in the P-1-th vowel position point and the P-th vowel position point;

[0225] determining an initial consonant and final consonant split point and a syllable split point between the P-1-th vowel position point and the P-th vowel position point based on the initial consonant and final consonant type of the P-th syllable and the second-order difference value.

[0226] In an example embodiment, the split point determination module 404 is further configured to:

[0227] performing framing on a speech segment in the P-1-th vowel position point and the P-th vowel position point based on signal amplitudes of each sampling point in the P-1-th vowel position point and the P-th vowel position point to obtain a plurality of speech frames;

[0228] performing correlation operation on adjacent frames in the plurality of speech frames to obtain a correlation coefficient of adjacent frames;

[0229] performing second-order difference on the correlation coefficients corresponding to continuous three frames of speech frames to obtain a second-order difference value of the continuous three frames of speech frames.

[0230] In an example embodiment, the split point determination module 404 is further configured to:

[0231] searching a target zero-crossing point from the first P-1 vowel position point, the target zero-crossing point being a sample point at which the signal amplitude crosses zero and the future signal amplitude trend changes to positive;

[0232] In the searching process, the i-th speech frame starting from the i-th target zero-crossing point is determined, and the i+1-th target zero-crossing point is searched starting from the end point of the i-th speech frame, i being a positive integer.

[0233] In an exemplary embodiment, the segmentation point determination module 404 is further configured to:

[0234] In a case where the initial consonant type of the P-th syllable is zero initial consonant, the middle frame of the three continuous speech frames corresponding to the maximum second-order difference value is determined as the syllable segmentation point;

[0235] In a case where the initial consonant type of the P-th syllable is clear initial consonant, the middle frame of the three continuous speech frames corresponding to the maximum second-order difference value is determined as the first segmentation point, and the middle frame of the three continuous speech frames corresponding to the second maximum second-order difference value is determined as the second segmentation point; the former segmentation point of the first segmentation point and the second segmentation point is determined as the syllable segmentation point, and the latter segmentation point of the first segmentation point and the second segmentation point is determined as the initial-consonant-vowel-consonant segmentation point;

[0236] In a case where the initial consonant type of the P-th syllable is voiced initial consonant, the first T maximum points in the second-order difference values are obtained, and the middle frames between the first T maximum points are combined to obtain a plurality of combined frame segments; the syllable segmentation point and the initial-consonant-vowel-consonant segmentation point are determined based on the average zero-crossing rate of the plurality of combined frame segments, T being a preset positive integer.

[0237] In an exemplary embodiment, the plurality of combined frame segments include the minimum speech segments between every two adjacent points in the first T maximum points and the combined segments obtained by combining the minimum speech segments;

[0238] The segmentation point determination module 404 is further configured to:

[0239] The average zero-crossing rate of the combined segment and the average zero-crossing rate of each minimum speech segment are calculated;

[0240] In a case where the deviation between the average zero-crossing rate of the combined segment and the average zero-crossing rate of the minimum speech segment included in the combined segment is greater than a preset deviation threshold, the combined segment in the combined frame segment is filtered;

[0241] The maximum average zero-crossing rate of the average zero-crossing rates of the remaining combined frame segments is determined, and the combined frame segment corresponding to the maximum average zero-crossing rate is determined as the segmentation segment.

[0242] determining the left endpoint of the segmented section as the syllable segmentation point and determining the right endpoint of the segmented section as the initial consonant and vowel segmentation point.

[0243] The segmentation device for Chinese speech signals according to the embodiments of the present disclosure corresponds to the embodiments of the segmentation method for Chinese speech signals according to the present disclosure, and the related content can be mutually referred to. The beneficial technical effects of the segmentation device for Chinese speech signals according to the embodiments of the present disclosure can be referred to the corresponding beneficial technical effects of the above-mentioned exemplary method part, and will not be repeated here.

[0244] In addition, the embodiments of the present disclosure also provide an electronic device, comprising:

[0245] a memory configured to store a computer program;

[0246] a processor configured to execute the computer program stored in the memory, and when the computer program is executed, the segmentation method for Chinese speech signals according to any one of the embodiments of the present disclosure is implemented.

[0247] Figure 5 The structure schematic diagram of an application embodiment of the electronic device according to the present disclosure is shown in the following figure. Figure 5 The electronic device according to the embodiments of the present disclosure will be described below with reference to the figure. The electronic device can be any one or both of the first device and the second device, or a single device independent of them, which can communicate with the first device and the second device to receive the collected input signals therefrom.

[0248] As shown in the figure, Figure 5 the electronic device comprises one or more processors and a memory.

[0249] The processor can be a central processing unit (CPU) or other forms of processing units having data processing and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions.

[0250] The memory can comprise one or more computer program products, which can comprise various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory and / or the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory and / or the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor can run the program instructions to implement the segmentation method for Chinese speech signals according to the embodiments of the present disclosure and / or other desired functions.

[0251] In one example, the electronic device can further include an input device and an output device, which components are interconnected through a bus system and / or other forms of connection mechanism (not shown).

[0252] Further, the input device can include, for example, a keyboard, a mouse, and the like.

[0253] The output device can output various information including the determined distance information, direction information, and the like, to the outside. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0254] Of course, in order to simplify, Figure 5 Among the components of the electronic device, only some of the components related to the present disclosure are illustrated, and components such as a bus, an input / output interface, and the like are omitted. In addition to this, the electronic device can further include any other appropriate components according to a specific application.

[0255] In addition to the above-described method and device, an embodiment of the present disclosure can be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method of segmenting a Chinese speech signal according to various embodiments of the present disclosure described in the above parts of the specification.

[0256] The computer program product can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, and the like, and conventional procedural programming languages, such as the "C" programming language, or the like. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0257] In addition, an embodiment of the present disclosure can also be a computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the method of segmenting a Chinese speech signal according to various embodiments of the present disclosure described in the above parts of the specification.

[0258] The computer readable storage medium can be a combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0259] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium. When the program is executed, the steps of the method embodiments are executed; and the foregoing storage medium includes ROM, RAM, magnetic disk or optical disk and various storage media that can store program codes.

[0260] The above describes the basic principles of the present disclosure in combination with specific embodiments. However, it should be noted that the advantages, advantages, effects and the like mentioned in the present disclosure are only examples and are not limiting. These advantages, advantages, effects and the like cannot be considered as necessary for each embodiment of the present disclosure. In addition, the above specific details are only for the purpose of example and for the purpose of understanding, and the above details do not limit the present disclosure to the above specific details.

[0261] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between each embodiment can be referred to each other. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0262] The block diagrams of the devices, apparatuses, equipment, systems involved in the present disclosure are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any way. Words such as "include", "contain", "have" and the like are open-ended words, which mean "include but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0263] The methods and apparatus of the present disclosure can be implemented in numerous ways, such as with software, hardware, firmware, or any combination of software, hardware, firmware, and the like. The order of any steps described above is merely exemplary and the steps need not be performed in the order described, unless otherwise specified. Additionally, the steps of the methods need not be performed in the order described, unless otherwise specified. Furthermore, any steps described above can be combined, reordered, or otherwise altered, unless otherwise specified. Moreover, in some embodiments, the present disclosure can be implemented as a program stored in a memory of a machine readable medium, which program, when executed by a machine, causes the machine to perform the methods described above. Accordingly, the present disclosure also covers the recording medium storing the program.

[0264] It is also noted that the methods of the present disclosure can be implemented by way of machine, hardware, software, firmware, or any combination thereof. Any features described as modules, components or the like can be implemented together in an integrated way or separately as individual elements. Any such software can be implemented by way of instructions stored in a memory of a machine readable medium, which when executed by a processor of the machine, cause the processor to carry out any of the methods described above. Accordingly, the present disclosure also covers any machine readable medium having stored therein the instructions for carrying out any of the methods described above.

[0265] The above description is provided for the purpose of illustrating and describing the disclosed aspects. It is not intended to be limiting. Numerous modifications and variations are possible in light of the above teachings without departing from the scope of the disclosure. It is also intended that the present disclosure be construed to cover any such modifications and variations as long as they fall within the scope of the appended claims, and their equivalents.

[0266] The above description has been presented for the purpose of illustration and description. It is not intended to be limiting. Although various example aspects and embodiments have been discussed, it should be understood that modifications, substitutions, changes, additions and subcombinations are possible.

Claims

1. A method for segmenting a Chinese speech signal, characterized in that, The method comprises: sampling a target audio signal containing speech corresponding to a target Chinese text to obtain signal amplitudes corresponding to a plurality of sampling points; performing speech endpoint detection on the target audio signal based on the signal amplitudes corresponding to the plurality of sampling points to obtain a plurality of speech segments in the target audio signal, wherein different speech segments correspond to different Chinese text segments, and the Chinese text segments belong to the target Chinese text; determining a vowel position sequence corresponding to each speech segment in the plurality of speech segments based on the formant energy of the speech signal in the target speech segment, wherein the vowel position sequence contains the vowel positions of each vowel in the target speech segment; determining a syllable segmentation point and an initial-final segmentation point of the target speech segment based on the signal amplitudes of the sampling points between adjacent vowel positions in the vowel position sequence and the initial-final of the Chinese text segment corresponding to the target speech segment; segmenting the target speech segment based on the syllable segmentation point and the initial-final segmentation point to obtain a segmented speech primitive, wherein the speech primitive is a basic unit of the target audio signal.

2. The method of claim 1, wherein, The method comprises: calculating the average value of the signal amplitudes of the sampling points in each frame of audio signal to obtain the frame average amplitude of each frame of audio signal; if the frame average amplitudes of K consecutive frames of audio signal are less than an amplitude threshold value, traversing the audio frames contained in the target audio signal from the end of the K consecutive frames of audio signal as the starting point, wherein K is a positive integer and K is a preset value, and the amplitude threshold value is calculated based on the average value of the signal amplitudes of the plurality of sampling points; if, during the traversal process, the noise energy of each frame of audio signal in M consecutive frames of audio signal is greater than an energy threshold value, and the sign preserving rate of each frame of audio signal in the M consecutive frames of audio signal is greater than a sign preserving rate threshold value, determining the M consecutive frames of audio signal as the speech segment, wherein M is a positive integer and M is a preset value, the energy threshold value is calculated based on the average noise energy of the K consecutive frames of audio signal, and the sign preserving rate threshold value is calculated based on the average sign preserving rate of the K consecutive frames of audio signal.

3. The method of claim 1, wherein, The method comprises: windowing and framing the target speech segment to obtain a short-time signal; performing Fourier transform on the short-time signal to obtain a frequency domain signal; calculating the formant energy of each frame of audio signal in the target speech segment based on the frequency domain signal; performing traversal search on the target speech segment based on the formant energy of each frame of audio signal; during the traversal search process, if the formant energy of S consecutive frames of audio signal is greater than an energy threshold value, determining the sampling points in the S consecutive frames of audio signal as vowel positions, wherein S is a preset positive integer.

4. The method according to any one of claims 1 to 3, characterized in that, The signal amplitude of the sampling points between two adjacent vowel positions in the vowel position sequence and the initial and final consonants of the Chinese text segment corresponding to the target voice segment are used to determine the syllable segmentation point and the initial and final consonant segmentation point of the target voice segment, and the method comprises the following steps: Based on the initial and final consonants of the Chinese text segment corresponding to the target voice segment, the initial consonant type of the Pth syllable in which the Pth vowel position point in the vowel position sequence is located is determined, P being a positive integer; Based on the signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence, the second-order difference value of the continuous frames between the P-1th vowel position point and the Pth vowel position point is calculated; Based on the initial and final consonant types of the Pth syllable and the second-order difference value, the initial and final consonant segmentation point and the syllable segmentation point between the P-1th vowel position point and the Pth vowel position point are determined.

5. The method of claim 4, wherein, The signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence is used to calculate the second-order difference value of the continuous frames between the P-1th vowel position point and the Pth vowel position point, and the method comprises the following steps: Based on the signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence, the voice segment in the P-1th vowel position point and the Pth vowel position point is segmented into a plurality of voice frames; The adjacent frames in the plurality of voice frames are subjected to correlation operation to obtain the correlation coefficient of the adjacent frames; The correlation coefficients corresponding to the continuous three voice frames are subjected to second-order difference to obtain the second-order difference value of the continuous three voice frames.

6. The method of claim 5, wherein, The signal amplitude of each sampling point in the P-1th vowel position point and the Pth vowel position point in the vowel position sequence is used to calculate the second-order difference value of the continuous frames between the P-1th vowel position point and the Pth vowel position point, and the method comprises the following steps: A target zero-crossing point is searched from the P-1th vowel position point, the target zero-crossing point being a sampling point at which the signal amplitude crosses zero and the future signal amplitude trend changes to positive; During the searching process, the i th voice frame with the i th target zero-crossing point as the starting point is determined, and the i+1th target zero-crossing point is searched with the end point of the i th voice frame as the starting point, i being a positive integer.

7. The method of claim 5, wherein, Based on the initial and final consonant types of the Pth syllable and the second-order difference value, the initial and final consonant segmentation point and the syllable segmentation point between the P-1th vowel position point and the Pth vowel position point are determined, and the method comprises the following steps: In the case that the initial consonant type of the Pth syllable is zero initial, the middle frame of the continuous three voice frames corresponding to the maximum second-order difference value is determined as the syllable segmentation point; In the case that the initial consonant type of the Pth syllable is clear initial, the middle frame of the continuous three voice frames corresponding to the maximum second-order difference value is determined as the first segmentation point, and the middle frame of the continuous three voice frames corresponding to the second maximum second-order difference value is determined as the second segmentation point; the front segmentation point of the first segmentation point and the second segmentation point is determined as the syllable segmentation point, and the rear segmentation point of the first segmentation point and the second segmentation point is determined as the initial and final consonant segmentation point; In a case where the initial consonant type of the Pth syllable is a voiced initial consonant, a first T maximum value points in the second-order difference value are obtained, and intermediate frames between the first T maximum value points are combined to obtain a plurality of combined frame segments, T being a preset positive integer, the plurality of combined frame segments containing minimum speech segments between two adjacent points in the first T maximum value points and a combined segment obtained by combining the minimum speech segments; an average zero-crossing rate of the combined segment and an average zero-crossing rate of each minimum speech segment are calculated; in a case where a deviation between the average zero-crossing rate of the combined segment and the average zero-crossing rate of the minimum speech segment contained in the combined segment is greater than a preset deviation threshold, the combined segment in the combined frame segment is filtered; a maximum average zero-crossing rate of the average zero-crossing rates of the remaining combined frame segments is determined, and a combined frame segment corresponding to the maximum average zero-crossing rate is determined as a segmentation segment; a left end point of the segmentation segment is determined as the syllable segmentation point, and a right end point of the segmentation segment is determined as the initial-final consonant segmentation point.

8. A Chinese speech signal segmenting apparatus, characterized by comprising: Comprise: A sampling module is configured to sample a target audio signal containing a target Chinese text corresponding speech to obtain signal amplitudes corresponding to a plurality of sampling points; An endpoint detection module is configured to perform speech endpoint detection on the target audio signal based on the signal amplitudes corresponding to the plurality of sampling points to obtain a plurality of speech segments in the target audio signal, wherein different speech segments correspond to different Chinese text segments, and the Chinese text segments belong to the target Chinese text; A vowel detection module is configured to take each speech segment in the plurality of speech segments as a target speech segment, respectively, and determine a vowel position sequence corresponding to the target speech segment based on a formant energy of a speech signal in the target speech segment, wherein the vowel position sequence contains vowel positions of each vowel in the target speech segment; A segmentation point determination module is configured to determine a syllable segmentation point and an initial-final consonant segmentation point of the target speech segment based on signal amplitudes of sampling points between adjacent two vowel positions in the vowel position sequence and an initial-final consonant of a Chinese text segment corresponding to the target speech segment; A segmentation module is configured to segment the target speech segment based on the syllable segmentation point and the initial-final consonant segmentation point of the target speech segment to obtain a segmented speech primitive, wherein the speech primitive is a basic unit of labeling the target audio signal.

9. An electronic device, comprising: Comprise: A memory is configured to store a computer program; A processor is configured to execute the computer program stored in the memory, and when the computer program is executed, the segmentation method of the Chinese speech signal in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, the segmentation method of the Chinese speech signal in any one of claims 1-7 is implemented.

Images