High-similarity synthetic speech analysis system and method

Abstract

The invention discloses a high-similarity synthetic speech analysis system, which comprises a recording input module, an analysis module, a basis output module, a machine judgment module, an expert judgment module and a result output module, and finally generates a judgment conclusion and a basis thereof by comprehensively analyzing various characteristics of recording. The analysis module is connected with the recording input module and is used for carrying out feature extraction and analysis on the input recording file, and the analysis content mainly comprises a authenticity analysis module, a paragraph analysis module, a polyphone analysis module, a pronunciation contradiction analysis module and a rhythm analysis module. The invention further discloses a high-similarity synthetic speech analysis method. According to the invention, the rhythm characteristic, the paradlingual sound production characteristic, the sound production contradictory characteristic and the like of the voice can be identified, the difference between the synthetic voice and the natural voice is revealed, the real voice and the synthetic voice are accurately identified, and the judgment and the basis of whether the voice in the recording file is the synthetic voice are given.

Description

Technical Field

[0001] The present invention relates to the technical field of speech analysis, and in particular to a high-similarity synthesized speech analysis system and method for distinguishing between real-person speech and synthesized speech. Background Art

[0002] With the development of speech synthesis technology, the similarity between synthesized speech and real human speech is increasing. In situations where high security is required, such as identity authentication and transaction confirmation, it becomes particularly important to distinguish between real human speech and synthesized speech.

[0003] Existing methods usually can only analyze based on speech features, ignoring multi-dimensional speech information. There is currently no effective means for recordings with low signal-to-noise ratios, low sampling rates, or recordings that are a mixture of synthetic speech and real speech that may appear in practice. Therefore, there is an urgent need for a system that combines multiple audio features to more accurately analyze and judge the authenticity of recordings. At the same time, in order to make up for machine errors and untimely iterations, expert judgment needs to be introduced. Summary of the invention

[0004] In order to solve the above technical problems, the present invention aims to provide a high-similarity synthetic speech analysis system and method, which can accurately identify the difference between real human speech and synthetic speech, and then give a judgment on whether the speech in the recording file is synthetic speech and its basis.

[0005] The technical solution adopted by the present invention to solve its technical problem is: A high-similarity synthetic speech analysis system includes a recording input module, an analysis module, a basis output module, a machine judgment module and an expert judgment module, and a result output module. By comprehensively analyzing various features of the recording, a judgment conclusion and its basis are finally generated.

[0006] The recording input module is the entrance of the whole system and is used to receive recording files. The recording input module transfers the recording files to the analysis module.

[0007] The analysis module is connected to the recording input module, and is used for extracting features and analyzing the input recording files, and analyzing the contents; it mainly includes an authenticity analysis module, a paralinguistic sound analysis module, a polyphone analysis module, a pronunciation contradiction analysis module and a rhythm analysis module, wherein the authenticity analysis module analyzes the formation mode of the recording files by extracting metadata and making judgments; the paralinguistic sound analysis module analyzes the non-semantic content of the recording files by identifying and counting paralinguistic sounds; the polyphone analysis module analyzes the pronunciation accuracy of the polyphones in the recording files by identifying and judging the polyphones; the pronunciation contradiction analysis module analyzes whether there are pronunciation modes that are difficult for humans to complete in the recording files by extracting the transition sections of the resonance peaks; the rhythm analysis module analyzes the naturalness of the recording by analyzing the pause duration and the stability of the voice; and the analysis results of the analysis module are output to the basis output module and the machine judgment module.

[0008] Furthermore, the authenticity analysis module is used to analyze and judge the formation method of the recording file, and includes two units, namely a metadata extraction unit and an authenticity judgment unit. After receiving the recording file from the recording input module, the metadata extraction unit is used to obtain relevant information of the recording, and then the authenticity judgment unit performs a logical judgment on the extraction result of the extraction unit.

[0009] Furthermore, the metadata extraction unit in the authenticity analysis module is used to output metadata information, including creation time, modification time, modification software, and modification history information.

[0010] Further, the paralinguistic sounds are used to extract, identify and count the paralinguistic sounds to determine the naturalness of the recording. After receiving the recording file from the recording input module, the paralinguistic information in the recording is extracted, including inhalation, exhalation, coughing, tut-tut, throat clearing, and nose sniffing. The category of the paralinguistic information is identified by the recognition unit, and finally the occurrence type and frequency information of the paralinguistic sounds in the recording file are obtained by the classification and statistics unit.

[0011] Furthermore, the polyphonetic character analysis module is used to analyze the polyphonetic characters in the recording and determine whether their pronunciation is reasonable. It includes a polyphonetic character extraction unit, a voice annotation unit and a judgment unit. The polyphonetic character extraction unit performs text recognition and polyphonetic character recognition. The recognition result is compared with the pronunciation of the syllable in the recording obtained by the pronunciation annotation unit, and the judgment unit obtains the recognition result of whether the polyphonetic character pronunciation is correct or not.

[0012] Furthermore, the pronunciation contradiction analysis module is used to extract the resonance peak spectrum between syllables in the recording due to incompatible phonation states, and identify whether there are resonance peaks with unnatural transitions, which are contrary to the law of human phonation. The pronunciation contradiction analysis module includes a pronunciation contradiction extraction unit and a statistical unit. After the recording is input, the pronunciation contradiction extraction unit identifies the contradictory pronunciation, and then the pronunciation contradiction statistical unit counts the category and duration of the extracted pronunciation contradiction phenomenon to obtain the pronunciation contradiction analysis module result.

[0013] Furthermore, the rhythm analysis module is used to extract the time lengths of sentence intervals between different groups in the recording after receiving the recording from the recording input module, and perform stability statistical analysis to determine the distribution and stability of the time intervals, and determine the naturalness of the recording by analyzing the pause duration and the stability of the voice.

[0014] The basis output module is used to collect the main extraction and analysis results of the analysis module and integrate them into output data for reference by the expert judgment module and output by the result output module.

[0015] Furthermore, the basis output module is used to obtain the metadata extraction unit result in the authenticity module, the paralanguage sound extraction unit extraction result in the paralanguage sound module, the classification statistics unit statistical result in the paralanguage sound module, the polyphone extraction unit extraction result in the polyphone module, the judgment result in the polyphone module, the pronunciation contradiction module result, and the rhythm module stability analysis result, and store the above files in sequence in a folder with the file name of "file name of the recording file + basis output".

[0016] The machine determination module is used to receive the analysis results from the analysis module, perform weighted evaluation according to the set logical relationship, and obtain the determination result. The result of the machine determination module will be transmitted to the result output module.

[0017] Furthermore, the machine judgment module combines the conclusions of the above modules and, according to the set logic, performs weighted judgment on the results of the authenticity judgment unit, the classification and statistics unit in the paralinguistic sound module, the judgment unit in the polyphone module, the statistics unit in the pronunciation contradiction module, and the stability analysis unit in the rhythm module to obtain a machine judgment opinion.

[0018] The expert determination module is designed to avoid the uncertainty of the results of the machine determination module. The expert determination module accepts the formatted main extraction and analysis results given by the output module, provides an input interface for experts to conduct manual review, and the output results of the expert determination module are output to the result output module.

[0019] The result output module is used to receive the result transmission of the basis output module, the expert judgment module, and the machine judgment module, and perform formatted sorting and output, which is the final result export of the present invention.

[0020] A high-similarity synthesized speech analysis method is applied to any one of the high-similarity synthesized speech analysis systems described above, and specifically comprises the following steps: Step S1. The high similarity synthetic speech analysis system receives the recording file and passes the recording file downward for analysis.

[0021] Step S2. The system extracts and analyzes the authenticity, paralinguistic sounds, polyphones, pronunciation contradictions and rhythmic features of the recording file input in step S1.

[0022] Specifically include: Step S2.1. The system analyzes and determines the formation method of the recording file. First, the metadata extraction unit obtains the relevant information of the recording, and then the authenticity determination unit performs a logical judgment on the extraction result of the extraction unit.

[0023] Step S2.2. The system extracts, recognizes and counts the paralinguistic sound information, including inhalation, exhalation, coughing, clicking, throat clearing and nasal sniffing, and identifies the category of the paralinguistic information through the recognition unit. Finally, the classification and statistics unit obtains the occurrence type and frequency information of the paralinguistic sound in the recording file.

[0024] Step S2.3. The system analyzes the polyphone in the recording to determine whether its pronunciation is reasonable. First, text recognition and polyphone recognition are performed, and then the recognition result is compared with the pronunciation of the syllable in the recording obtained by the pronunciation annotation unit, and finally the judgment unit obtains the recognition result of whether the polyphone pronunciation is correct or not.

[0025] Step S2.4. The system extracts the resonance peak spectrum between syllables in the recording due to incompatible phonation states, identifies whether there are resonance peaks with unnatural transitions, first identifies the contradictory pronunciations through the pronunciation contradiction extraction unit, and then uses the pronunciation contradiction statistics unit to count the categories and durations of the extracted pronunciation contradiction phenomena to obtain the pronunciation contradiction analysis module results.

[0026] Step S2.5. The system extracts the length of the intervals between different groups of sentences in the recording, and performs stability statistical analysis to determine the distribution and stability of the time intervals, and uses this to determine the naturalness of the recording.

[0027] Step S3. The system collects the main extraction and analysis results of the analysis step and integrates them into output data.

[0028] Step S4. Based on the analysis results, the system performs weighted evaluation according to the set logical relationship to obtain a judgment result.

[0029] Step S5. Based on the aggregated main extraction and analysis results, the system provides an input interface for experts to conduct manual review, and the experts make manual judgments.

[0030] Step S6. The system formats and organizes the output basis, expert judgment, and machine judgment results.

[0031] Beneficial effects of the present invention: The present invention can improve the accuracy of distinguishing real person speech and synthesized speech, and has the following beneficial effects: 1. Recognize the prosodic features, paralinguistic pronunciation features, and contradictory pronunciation features of speech, reveal the difference between synthetic speech and natural speech, and accurately identify real human speech and synthetic speech.

[0032] 2. Introduce an expert judgment module to make up for the problems of machine errors and untimely iterations, and at the same time provide the possibility for the application of this method in forensic identification.

[0033] 3. Output all features obtained from the investigation and analysis to avoid black box analysis and ensure the transparency and explainability of the system. Through the collaborative work of various modules and cross-validation of data, not only can the accuracy of the judgment of the authenticity of the recording be effectively improved, but the analysis process of each feature and its role in the final conclusion can also be clearly displayed in the output results. In this way, users can fully understand the specific basis of the recording analysis and the processing logic of each step, thereby enhancing their trust in the system's judgment results.

[0034] 4. The present invention can flexibly respond to different types of recording sample analysis requirements through modular design, and has high adaptability and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings of the present invention are used to provide a further understanding of the present invention, and illustrate the embodiments of the present invention and their description by way of example.

[0036] Figure 1 It is a diagram of the system components and their relationships.

[0037] Figure 2 It is a flow chart of the implementation of the invention.

[0038] Figure 3 It is a diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment, and other embodiments may also fall within the scope of the present invention as long as they conform to the gist of the present invention.

[0040] In a preferred embodiment of the present invention, in view of the above problems existing in the prior art, a high-similarity synthetic speech analysis system is provided. The module composition and relational structure of the system are shown in Figure 1 , and the implementation processing flow of the present invention is shown in Figure 2 . Specifically, in combination with Figure 3 and the embodiments below, the technical solution of the present invention will be described, specifically including: 1. Recording input module: The operator imports the speech to be inspected "exhibit.m4a" into the system. The recording input module copies and reads the recording file for preparation of analysis.

[0041] The content of the exhibit.m4a is 9 sentences narrated by a female voice, formed by splicing a natural speech segment and a synthetic speech segment. Among them, the natural speech is 25 seconds in total, followed by 15 seconds of synthetic speech spliced.

[0042] 2. Analysis module: The analysis module is connected to the recording input module and includes five parts: authenticity analysis module, paralanguage analysis module, polyphone analysis module, pronunciation contradiction analysis module, and prosody analysis module.

[0043] ① Authenticity analysis module: First, the metadata extraction unit obtains metadata information M1 such as the creation time, modification time, modification software, and modification history of the recording. The creation time of "exhibit.m4a" is 18:36:21 on September 3, 2024, the modification time is 09:15:48 on September 4, 2024, the modification software is "Audition CC", and the modification history is empty. The authenticity determination unit makes a logical judgment on the extraction result, and the logical judgment S1 is "the modification software is highly abnormal and the time information is highly abnormal".

[0044] ② Paralanguage analysis module: Extract the paralanguage information in the recording, including inhalation sounds, exhalation sounds, coughing sounds, clicking sounds, throat-clearing sounds, nose-blowing sounds A1 - An. In A1 - An of "exhibit.m4a", there is a broadband short-time noise at the starting position of the speech, which is identified as an inhalation sound by the recognition unit, and the classification and statistics unit records the occurrence type and frequency information S2 as "inhalation sound 1 time".

[0045] ③ Polyphone analysis module: Analyze the polyphones in "exhibit.m4a". After the polyphone extraction unit performs character recognition and polyphone recognition, the polyphones B1 - Bn obtained include "(rural) shed" and "say". Comparing the recognition result with the pronunciation of this syllable in the recording recognized by the pronunciation annotation unit, it is found that "(rural) shed" is pronounced in the third tone, and all 3 occurrences of "say" are pronounced in the first tone. Through the judgment unit, the recognition result S3 of whether the pronunciation of the above polyphones is correct in the context is "shed: incorrect; say: correct, correct, correct".

[0046] ④ Pronunciation contradiction analysis module: The pronunciation contradiction analysis module includes a pronunciation contradiction extraction unit and a statistics unit. There are 3 pronunciation contradictions in the "examination material.m4a" that can be identified by the pronunciation contradiction extraction unit, denoted as P1 - P3, all of which appear before the consonant [s]. The pronunciation contradiction statistics unit counts the categories and durations of the extracted pronunciation contradiction phenomena, and derives the result S4 of the pronunciation contradiction analysis module as "contradiction before stop sound for 50ms, contradiction before stop sound for 60ms, contradiction before stop sound for 50ms".

[0047] ⑤ Prosody analysis module: The prosody analysis module extracts a total of 12 pauses in the "examination material.m4a", including 8 sentence breaks, 3 mid - sentence pauses, and 1 start - of - sentence pause. Through the stability statistical analysis of their respective durations, it is found that the stability of the time intervals is poor, but it shows a certain distribution law of being more scattered at the front and more uniform at the back. Therefore, the naturalness S5 of the recording is obtained as "poor stability, there are distribution differences".

[0048] 3. Basis for output module: The results M1 of the metadata extraction unit in the authenticity module, the extraction results A1 - An of the paralinguistic sound extraction unit in the paralinguistic sound module, the statistical result S2 of the classification and statistics unit in the paralinguistic sound module, the extraction results B1 - Bn of the polyphone extraction unit in the polyphone module, the judgment result S3 in the polyphone module, the results P1 - Pn of the pronunciation contradiction module, and the stability analysis result S5 of the prosody module are gathered into a folder named "examination material basis output", and at the same time output to the form "examination material output.xls".

[0049] 4. Machine determination module: According to the set logical relationship, after weighted evaluation, the machine determination opinion O1 is obtained as "highly suspected of being synthesized".

[0050] 5. Expert determination module: The appraisal expert intervenes manually, and the output result O2 of the expert determination module is "the first part of the recording (from the start of the recording to about 25 seconds) is natural speech, and the second part is synthesized speech, and the two are spliced together".

[0051] 6. Result output module: The final result output, summarizes the results of O1 and O2 and outputs them after sorting, and continues to write to the 2 determination columns in the form "examination material output.xls".

[0052] In the embodiment of the present invention, through the collaborative work of the above - mentioned modules, the technical solution of the present invention comprehensively and deeply analyzes whether the voice in the recording file "examination material.m4a" is synthesized voice, and provides an accurate determination result.

Claims

1. A high-similarity synthetic speech analysis system, comprising a recording input module and a result output module, characterized in that: It also includes an analysis module, a basis output module, a machine judgment module and an expert judgment module, which comprehensively analyze various features of the recording and finally generate a judgment conclusion and its basis; the analysis module is connected to the recording input module, and is used to extract and analyze the features of the input recording file and analyze the content; it mainly includes an authenticity analysis module, a paralinguistic sound analysis module, a polyphone analysis module, a pronunciation contradiction analysis module and a rhythm analysis module, wherein the authenticity analysis module analyzes the formation mode of the recording file by extracting metadata and judging; the paralinguistic sound analysis module analyzes the non-semantic content in the recording file by identifying and counting paralinguistic sounds; the polyphone analysis module analyzes the pronunciation accuracy of polyphones in the recording file by identifying and judging polyphones; The pronunciation contradiction analysis module extracts the transition segment of the resonance peak to determine whether there is a pronunciation method that is difficult for humans to complete in the recording file; the rhythm analysis module analyzes the naturalness of the recording by analyzing the pause duration and the stability of the voice; the analysis module outputs the analysis results to the basis output module and the machine judgment module.

2. A high similarity synthetic speech analysis system according to claim 1, characterized in that: The authenticity analysis module is used to analyze and judge the formation method of the recording file, and includes two units, namely a metadata extraction unit and an authenticity judgment unit. After receiving the recording file from the recording input module, the metadata extraction unit first obtains the relevant information of the recording, and then the authenticity judgment unit performs a logical judgment on the extraction result of the extraction unit.

3. The high similarity synthetic speech analysis system according to claim 1, characterized in that: The paralinguistic sounds are used to extract, identify and count the paralinguistic sounds to determine the naturalness of the recording; After receiving the recording file from the recording input module, the paralinguistic information in the recording is extracted, including inhalation, exhalation, coughing, clicking, throat clearing, and sniffling, etc. The category of the paralinguistic information is identified by the recognition unit, and finally the occurrence type and frequency information of the paralinguistic sounds in the recording file are obtained by the classification and statistical unit.

4. The high similarity synthetic speech analysis system according to claim 1, characterized in that: The polyphone analysis module is used to analyze the polyphones in the recording and determine whether their pronunciation is reasonable; it includes a polyphone extraction unit, a voice annotation unit and a judgment unit. The polyphone extraction unit performs text recognition and polyphone recognition. The recognition result is compared with the pronunciation of the syllable in the recording obtained by the pronunciation annotation unit, and the judgment unit obtains the recognition result of whether the pronunciation of the polyphone is correct or not.

5. The high similarity synthetic speech analysis system according to claim 1, characterized in that: The pronunciation contradiction analysis module is used to extract the resonance peak spectrum between syllables in the recording due to incompatible vocalization states, and identify whether there are resonance peaks with unnatural transitions, which are contrary to the human vocalization law; the pronunciation contradiction analysis module includes a pronunciation contradiction extraction unit and a statistical unit. After the recording is input, the pronunciation contradiction extraction unit is used to identify the contradictory pronunciation, and then the pronunciation contradiction statistical unit counts the category and duration of the extracted pronunciation contradiction phenomenon to obtain the pronunciation contradiction analysis module result.

6. A high similarity synthetic speech analysis system according to claim 1, characterized in that: The rhythm analysis module is used to extract the time lengths of the sentence intervals between different groups in the recording after receiving the recording from the recording input module, and perform stability statistical analysis to determine the distribution and stability of the time intervals, and determine the naturalness of the recording by analyzing the pause duration and the stability of the voice.

7. The high similarity synthetic speech analysis system according to claim 1, characterized in that: The basis output module is used to collect the main extraction and analysis results of the analysis module and integrate them into output data for reference by the expert judgment module and output by the result output module; the basis output module is used to obtain the metadata extraction unit result in the authenticity module, the paralanguage sound extraction unit extraction result in the paralanguage sound module, the classification statistics unit statistical result in the paralanguage sound module, the polyphone extraction unit extraction result in the polyphone module, the judgment result in the polyphone module, the pronunciation contradiction module result, and the rhythm module stability analysis result, and store the above files in a folder with "the file name of the recording file + the basis output" as the file name in order.

8. The high similarity synthetic speech analysis system according to claim 1, characterized in that: The machine determination module is used to receive the analysis results from the analysis module, perform weighted evaluation according to the set logical relationship, and obtain the determination result; the result of the machine determination module will be transmitted to the result output module; the machine determination module combines the conclusions of the above modules, and according to the set logic, performs weighted judgment on the results of the authenticity judgment unit, the classification and statistics unit in the paralinguistic sound module, the judgment unit in the polyphone module, the statistics unit in the pronunciation contradiction module, and the stability analysis unit in the rhythm module to obtain the machine judgment opinion.

9. The high similarity synthetic speech analysis system according to claim 1, characterized in that: The expert determination module is designed to avoid the uncertainty of the results of the machine determination module. The expert determination module accepts the formatted main extraction and analysis results given by the output module, provides an input interface for experts to conduct manual review, and the output results of the expert determination module are output to the result output module.

10. A high-similarity synthetic speech analysis method, applied to a high-similarity synthetic speech analysis system as described above, specifically comprising the following steps: Step S1. The high similarity synthetic speech analysis system receives the recording file and passes the recording file downward for analysis; Step S2. The system extracts and analyzes the authenticity, paralinguistic sounds, polyphones, pronunciation contradictions and rhythmic features of the recording file input in step S1; Specifically include: Step S2.

1. The system analyzes and determines the formation mode of the recording file, firstly obtains the relevant information of the recording through the metadata extraction unit, and then performs a logical judgment on the extraction result of the extraction unit through the authenticity determination unit; Step S2.

2. The system extracts, identifies and counts the paralinguistic sound information, including inhalation, exhalation, coughing, tutting, throat clearing and nasal sniffing, and identifies the category of the paralinguistic information through the recognition unit. Finally, the classification and statistical unit obtains the type and frequency information of the paralinguistic sound in the recording file; Step S2.

3. The system analyzes the polyphone in the recording to determine whether its pronunciation is reasonable; first, text recognition and polyphone recognition are performed, and then the recognition result is compared with the pronunciation of the syllable in the recording obtained by the pronunciation annotation unit, and finally the judgment unit obtains the recognition result of whether the polyphone pronunciation is correct or not; Step S2.

4. The system extracts the resonance peak spectrum between syllables in the recording due to incompatible phonation states, identifies whether there are resonance peaks with unnatural transitions, first identifies the contradictory pronunciations through the pronunciation contradiction extraction unit, and then the pronunciation contradiction statistics unit counts the categories and durations of the extracted pronunciation contradiction phenomena to obtain the pronunciation contradiction analysis module results; Step S2.

5. The system extracts the length of the intervals between different groups of sentences in the recording, and performs stability statistical analysis to determine the distribution and stability of the time intervals, and is used to determine the naturalness of the recording; Step S3. The system collects the main extraction and analysis results of the analysis step and integrates them into output data; Step S4. The system performs weighted evaluation based on the analysis results and the set logical relationship to obtain a determination result; Step S5. Based on the collected main extraction and analysis results, the system provides an input interface for experts to conduct manual review, and the experts make manual judgments; Step S6. The system formats and organizes the output basis, expert judgment, and machine judgment results.

Images