Binary natural speech segmentation method and system based on time domain analysis

Abstract

The invention provides a binary natural speech segmentation method and system based on time domain analysis. The method comprises the following steps: acquiring a target binary natural voice file and generating corresponding binary natural voice data; performing code element-by-code analysis on the binary natural voice data, and identifying whether the current code element is an error code, a normal code or a cut-off position of a code word or a code block according to a relation between a time interval between the current code element and an adjacent code element and a preset time interval coefficient; and performing code word segmentation and code block segmentation on the binary natural voice data according to an identification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech signal processing, and particularly relates to a binary natural speech segmentation method and system based on time domain analysis. BACKGROUND

[0002] With the rapid development of speech signal processing technology, speech segmentation, as a core preprocessing link in the fields of speech recognition, speech synthesis, speaker separation and audio retrieval, has a decisive influence on the subsequent processing effect in terms of accuracy and real-time performance. The current speech segmentation method is mainly for ordinary speech.

[0003] Unlike ordinary speech, binary natural speech is composed of two independent syllables, has a relatively fixed structure and strong rhythm, and can be composed of two sounds "d" and "d" in different combination forms. Among them, the "d" sound occupies the first time length, and the "d" sound occupies the second time length. In the binary natural speech composed of "d" and "d" two sounds, each code word can be composed of "d" and "d" combination, or only different numbers of "d" sound, or different numbers of "d" sound, or "d" and "d" permutation combination, but generally, the different numbers are not more than 5. The regularity in time domain makes it difficult for traditional ordinary speech segmentation methods based on statistical model or spectral analysis to be directly applicable. The existing general speech segmentation technology often excessively relies on the statistical characteristics of the training data, and does not fully consider the time sequence regularity and combination limitation of the syllable in binary speech, which is easy to cause over-segmentation or merging error, thereby affecting the accuracy and efficiency of subsequent processing. SUMMARY

[0004] The present application provides a binary natural speech segmentation method and system based on time domain analysis, which is a segmentation method specially designed for the characteristics of binary natural speech, and can fully utilize the time domain structure characteristics to realize efficient, accurate and real-time syllable boundary detection and code word segmentation.

[0005] In a first aspect, the present application provides a binary natural speech segmentation method based on time domain analysis, comprising:

[0006] Obtaining a target binary natural speech file and generating corresponding binary natural speech data;

[0007] Performing symbol-by-symbol analysis on the binary natural speech data, and identifying whether the current symbol is an error code, a normal code, or located at the cutoff position of a code word or a code group according to the relationship between the time interval between the current symbol and its adjacent symbol and the preset time interval coefficient;

[0008] According to the identification result, the binary natural speech data is divided into code words and code groups.

[0009] Further, the method for determining whether the current symbol is an error code, a normal code or located at the cut-off position of a code word or a code group according to the relationship between the time interval between the current symbol and its adjacent symbol and the preset time interval coefficient comprises:

[0010] The time interval length between adjacent normal symbols is set as A0, the time interval length between adjacent normal code words is set as B0, and the time interval length between adjacent normal code groups is set as C0;

[0011] According to the preset time interval lengths A0, B0 and C0, the value intervals of the first time interval coefficient K0, the second time interval coefficient M0 and the third time interval coefficient N0 are respectively interval K, interval M and interval N, and satisfy

[0012] If the time interval A between the current symbol S and its adjacent symbol S+1 S <K0A S-1 , the current symbol S is determined as an error code; if A S =K0A S-1 , the current symbol S is determined as a normal code; wherein A S-1 is the time interval between the symbol S-1 and the current symbol S;

[0013] If A S >K0A S-1 and A S <M0B0, the symbol S+1 is determined as an error code; if A S >K0A S-1 and A S =M0B0, the current symbol S is determined as the last symbol of the code word;

[0014] If A S >K0A S-1 and A S >M0B0 and A S <N0C0, the symbol S+1 is determined as an error code; if A S >K0A S-1 and A S >M0B0 and A S =N0C0, the current symbol S is determined as the last code group of the code group; if A S >K0A S-1 and A S >M0B0 and A S >N0C0, the current symbol S is determined as the last symbol of the binary natural voice data.

[0015] Further, the code word splitting and code group splitting include: pre-code word splitting the code symbol according to the number of code symbols; and pre-code group splitting the code word according to the number of code words.

[0016] Further, the pre-code word splitting the code symbol according to the number of code symbols includes:

[0017] Supposing that one code word is composed of X code symbols, for the L1th code symbol, when T1 is an integer, the L1th code symbol is pre-determined as the last code symbol of the T1th code word; when is not an integer, calculating and pre-determining that the L1th code symbol belongs to the T2+1th code word. represents a floor operation;

[0018] Further, the pre-code group splitting the code word according to the number of code words includes:

[0019] Supposing that one code group is composed of Y code words, for the L2th code word, when T3 is an integer, the L2th code word is pre-determined as the last code word of the T3th code group; when is not an integer, calculating and pre-determining that the L2th code word belongs to the T4+1th code group.

[0020] In a second aspect, the present application provides a binary natural speech segmentation system based on time domain analysis, which comprises:

[0021] a data acquisition and preprocessing module, configured to acquire a target binary natural speech file and generate corresponding binary natural speech data;

[0022] a code symbol analysis module, configured to perform code symbol-by-code symbol analysis on the binary natural speech data, and identify whether a current code symbol is an error code, a normal code or located at a cutoff position of a code word or a code group according to a relationship between a time interval between the current code symbol and its adjacent code symbols and a preset time interval coefficient;

[0023] a segmentation module, configured to perform code word splitting and code group splitting on the binary natural speech data according to the identification result.

[0024] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of the first aspect.

[0025] In a fourth aspect, the present application provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of the first aspect.

[0026] The present application has the following beneficial effects:

[0027] (1) The amount of calculation data can be reduced when the audio is segmented from the time domain, and the segmentation of the audio signal can be realized.

[0028] (2) The audio segmentation can be automatically completed, and the segmentation speed is fast and the accuracy is high.

[0029] (3) The error code and the wrong code can be automatically identified, the identification rate of the error code is effectively improved, and the subsequent influence caused by the error code is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flowchart of a binary natural speech segmentation method based on time domain analysis provided by an embodiment of the present application is shown in the figure;

[0031] Figure 2 A schematic diagram of the time interval between adjacent code elements in the same code word provided by an embodiment of the present application is shown in the figure;

[0032] Figure 3 A schematic diagram of the time interval between adjacent code words provided by an embodiment of the present application is shown in the figure;

[0033] Figure 4 A schematic diagram of code word segmentation provided by an embodiment of the present application is shown in the figure;

[0034] Figure 5 A schematic diagram of code group segmentation provided by an embodiment of the present application is shown in the figure;

[0035] Figure 6 A structural schematic diagram of a binary natural speech segmentation system based on time domain analysis provided by an embodiment of the present application is shown in the figure;

[0036] Figure 7 A structural block diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0038] In the present application, generally, the relationship among the three of symbols, code words and code groups is as follows: X symbols form a code word, Y code words form a code group; the code word, code group and even the cutoff need to be determined according to the time interval among different symbols, different code words and different code groups. Generally, the value range of X is 1-4; Y=5.

[0039] As shown in Figure 1 The embodiment of the present application provides a binary natural speech segmentation method based on time domain analysis, which comprises the following steps:

[0040] S101: acquiring a target binary natural speech file and generating corresponding binary natural speech data; specifically, the binary natural speech data is a programmable data, and the generation process specifically comprises: generally, the acquired target binary natural speech file is a WAV file and an MP3 file; after the file is acquired, first, the file header, sampling rate, bit depth and channel number are parsed to extract the original audio data to obtain a binary data block. Then, if the file is an MP3 file, the file needs to be decoded (such as using a decoding library to decompress it into PCM data); since the WAV file itself is PCM data, it cannot be decoded. Then, the binary data is converted and formatted, specifically: the binary data is first converted into an integer array, and then the integer value is normalized to the interval of -1-1; finally, a programmable data in the form of a floating-point number array or an integer array is obtained. In addition, subsequent programming environments can be selected as needed, such as MATLAB (using functions such as audioread to complete all steps in one step) or C# (using libraries such as Naudio to complete step by step or in one step).

[0041] S102: performing symbol-by-symbol analysis on the binary natural speech data, and identifying whether the current symbol is an error code, a normal code or located at the cutoff position of a code word or a code group according to the relationship between the time interval between the current symbol and its adjacent symbols and the preset time interval coefficient;

[0042] Specifically, in combination with Figure 2 and Figure 3 As shown, this step specifically comprises the following sub-steps:

[0043] S1021: setting the time interval length between adjacent normal symbols as A0, the time interval length between adjacent normal code words as B0, and the time interval length between adjacent normal code groups as C0;

[0044] S1022: determining the value range of the first time interval coefficient K0, the second time interval coefficient M0 and the third time interval coefficient N0 as interval K, interval M and interval N respectively according to the preset time interval lengths A0, B0 and C0, and satisfying

[0045] In one embodiment, the time interval of adjacent code words in each code group is recorded, the second average time interval is determined according to the time intervals of all adjacent code words, and the second time interval coefficient M0 is determined according to the second average time interval and the preset code word time interval length B0.

[0046] In one embodiment, the time interval of adjacent code words in each code group is recorded, the second average time interval is determined according to the time intervals of all adjacent code words, and the second time interval coefficient M0 is determined according to the second average time interval and the preset code word time interval length B0.

[0047] In one embodiment, the time interval of adjacent code groups is recorded, the third average time interval is determined according to the time intervals of all adjacent code groups, and the third time interval coefficient N0 is determined according to the third average time interval and the preset code group time interval length C0.

[0048] In the embodiment of the present application, the interval K is set as [0.95, 1.05], the interval M is set as [0.9, 1.1], and the interval N is set as [0.85, 1.15].

[0049] It can be understood that, considering that the familiarity with the target text or the personal reading habits (accent, stuttering, etc.) of the user are different when reading the target text, the three time interval coefficients are set in order to improve the segmentation accuracy, and a wide margin is set to reduce the segmentation error caused by the reading habits as much as possible.

[0050] For example, in a theoretical case, the preset code word time interval length is 1 time unit, and in an actual scenario, the first average time interval calculated is 0.98 time units. At this time, it should be considered that the reading result of the user is correct. By setting the interval of the first time interval coefficient as [0.95, 1.05], the segmentation result obtained by the method of the present application is also consistent with the reading result in the actual scenario, that is, the segmentation accuracy meets the requirements.

[0051] S1023: If the time interval A S <K0A S-1 between the current code word S (i.e., the Sth code word) and its adjacent code word S+1 (i.e., the (S+1)th code word) is greater than the first time interval coefficient K0A S , it is determined that the current code word S is an error code; if A S-1 = K0A S-1 , it is determined that the current code word S is a normal code; wherein A S-1 is the time interval between the code word S-1 and the current code word S.

[0052] For example, the process of judging the first symbol is as follows: the length of the time interval between the first symbol and the second symbol is denoted as A1; specifically, the length A1 is determined according to the end timestamp of the first symbol and the start timestamp of the second symbol. If A1

[0053] S1024: If A S >K0A S-1 and A S <M0B0, then the symbol S+1 is determined as an error code; if A S >K0A S-1 and A S =M0B0, then the current symbol S is determined as the last symbol of the code word to which the symbol belongs.

[0054] For example, when A S >K0A S-1 , i.e., when A S >1.05A S-1 , further judgment is needed, specifically including: if A S <M0B0, i.e., if A S <0.9B0, then the symbol S+1 is determined as an error code; if A S =M0B0, i.e., if 0.95B0 S ≤1.1B0, then the symbol S is determined as the last symbol of a code word.

[0055] S1025: If A S >K0A S-1 and A S >M0B0 and A S <N0C0, then the symbol S+1 is determined as an error code; if A S >K0A S-1 and A S >M0B0 and A S =N0C0, then the current symbol S is determined as the last code group of the code group to which the symbol belongs; if A S >K0A S-1 and A S >M0B0 and A S >N0C0, then the current symbol S is determined as the last symbol of the binary natural speech data.

[0056] For example, when it is determined that the Sth symbol is the last symbol in the code group, i.e., A S >M0B0, further judgment is needed, which specifically includes: when A S <N0C0, i.e., A S <0.85C0, it is determined that the symbol S+1 is an error symbol; when A S =N0C0, i.e., 0.85C0≤A S ≤1.15C0, it is determined that the Sth symbol is the last symbol in a code group; when A S >N0C0, i.e., A S >1.15C0, it is determined that the Sth symbol is the last symbol in the entire target code text.

[0057] S103: performing code word segmentation and code group segmentation on the binary natural speech data according to the recognition result.

[0058] Specifically, when performing code word segmentation, it further includes: performing pre-code word segmentation on the symbols according to the number of symbols. For example, for the L1th symbol, when T1 is an integer, it is pre-determined that the L1th symbol is the last symbol of the T1th code word; when is not an integer, it is calculated that and it is pre-determined that the L1th symbol belongs to the T2+1th code word. represents a floor operation. For example, Figure 4 as shown.

[0059] When performing code group segmentation, it further includes: performing pre-code group segmentation on the code words according to the number of code words. For example, for the L2th code word, when T3 is an integer, it is pre-determined that the L2th code word is the last code word of the T3th code group; when is not an integer, it is calculated that and it is pre-determined that the L2th code word belongs to the T4+1th code group. For example, Figure 5 as shown.

[0060] Based on the same inventive concept, as shown in Figure 6 , the embodiment of the present application provides a binary natural speech segmentation system based on time domain analysis, which includes a data acquisition and preprocessing module, a symbol analysis module and a segmentation module.

[0061] Specifically, the data acquisition and preprocessing module is used to acquire the target binary natural speech file and generate the corresponding binary natural speech data; the code element analysis module is used to perform code element-by-code element analysis on the binary natural speech data, and based on the relationship between the time interval between the current code element and its adjacent code elements and the preset time interval coefficient, to identify whether the current code element is an error code, a normal code, or located at the cutoff position of a codeword or code group; the segmentation module is used to perform codeword segmentation and code group segmentation on the binary natural speech data according to the identification results.

[0062] It should be noted that the binary natural speech segmentation system provided in this embodiment of the invention is for implementing the above method, and its specific functions can be referred to the above method embodiment, which will not be repeated here.

[0063] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 701, a communication interface 702, a memory 703, and a communication bus 704. The processor 701, communication interface 702, and memory 703 communicate with each other via the communication bus 704. The processor 701 can call logical instructions in the memory 703 to execute a time-domain analysis-based binary natural speech segmentation method. This method includes: acquiring a target binary natural speech file and generating corresponding binary natural speech data; performing symbol-by-symbol analysis on the binary natural speech data, identifying whether the current symbol is an error, a normal symbol, or located at the cutoff position of a codeword or code group based on the relationship between the time interval between the current symbol and its adjacent symbols and a preset time interval coefficient; and performing codeword segmentation and code group segmentation on the binary natural speech data based on the identification results.

[0064] Furthermore, when the logical instructions in the aforementioned memory 703 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] The embodiment of the present application further provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the binary natural speech segmentation method based on time domain analysis provided by each method embodiment.

[0066] The embodiment of the present application further provides a non-transitory computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the binary natural speech segmentation method based on time domain analysis provided by each method embodiment is realized.

[0067] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software plus necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some part of the embodiment.

[0068] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A binary natural speech segmentation method based on time-domain analysis, characterized in that, include: Acquire the target binary natural speech file and generate the corresponding binary natural speech data; The binary natural speech data is analyzed bit by bit. Based on the relationship between the time interval between the current code and its adjacent code and the preset time interval coefficient, it is determined whether the current code is an error code, a normal code, or located at the cutoff position of a codeword or code group. Based on the recognition results, the binary natural speech data is segmented into codewords and code groups.

2. The binary natural speech segmentation method based on time-domain analysis according to claim 1, characterized in that, The method of determining whether the current code element is an error, a normal code element, or located at the cutoff position of a codeword or code group based on the relationship between the time interval between the current code element and its adjacent code elements and a preset time interval coefficient specifically includes: Set the time interval between adjacent normal code elements to A0, the time interval between adjacent normal codewords to B0, and the time interval between adjacent normal code groups to C0. Based on preset time interval lengths A0, B0, and C0, the value intervals of the first time interval coefficient K0, the second time interval coefficient M0, and the third time interval coefficient N0 are determined to be intervals K, M, and N, respectively, and satisfy the following conditions: If the time interval A between the current code element S and its adjacent code element S+1 is... S <K0A S-1 If A is an error, then the current symbol S is determined to be an error; S =K0A S-1 If so, then the current code element S is determined to be a normal code; where A S-1 The time interval between symbol S-1 and the current symbol S; If A S > K0A S-1 and A S < M0B0, then it is determined that symbol S + 1 is an error symbol; if A S > K0A S-1 and A S = M0B0, then it is determined that the current symbol S is the last symbol of the codeword to which it belongs; If A S > K0A S-1 and A S > M0B0 and A S < N0C0, then it is determined that the symbol S + 1 is an error symbol; if A S > K0A S-1 and A S > M0B0 and A S = N0C0, then it is determined that the current symbol S is the last symbol of the code group; if A S > K0A S-1 and A S > M0B0 and A S > N0C0, then it is determined that the current symbol S is the last symbol of the binary natural speech data.

3. The binary natural speech segmentation method based on time-domain analysis according to claim 1, characterized in that, When performing codeword segmentation and code group segmentation, the process includes: performing precodeword segmentation on code elements based on the number of code elements; and performing precode group segmentation on code words based on the number of code words.

4. The binary natural speech segmentation method based on time-domain analysis according to claim 3, characterized in that, The aforementioned method of precodeword segmentation based on the number of code elements specifically includes: Let a codeword consist of X code elements. Then, for the L1-th code element, when... When T1 is an integer, the L1th code element is pre-determined to be the last code element of the T1th codeword; when If the result is not an integer, then calculate... And pre-determine that the L1th code element belongs to the T2+1th codeword; This indicates the floor function.

5. The binary natural speech segmentation method based on time-domain analysis according to claim 3, characterized in that, The aforementioned method of segmenting codewords into precode groups based on the number of codewords specifically includes: Let a code group consist of Y codewords. Then, for the L2-th codeword, when... When T3 is an integer, the L2th codeword is pre-determined to be the last codeword of the T3th code group; when If the result is not an integer, then calculate... And pre-determine that the L2 codeword belongs to the T4+1 code group.

6. A binary natural speech segmentation system based on time-domain analysis, characterized in that, include: The data acquisition and preprocessing module is used to acquire the target binary natural speech file and generate the corresponding binary natural speech data. The code element analysis module is used to perform code element-by-code element analysis on the binary natural speech data. Based on the relationship between the time interval between the current code element and its adjacent code elements and the preset time interval coefficient, it identifies whether the current code element is an error code, a normal code, or is located at the cutoff position of a codeword or code group. The segmentation module is used to perform codeword segmentation and code group segmentation on the binary natural speech data based on the recognition results.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images