Synthetic speech detection method

Abstract

It is disclosed a synthetic speech detection framework, based on breath-aware temporal discrepancy detection which is able to analyze the interplay between breathing patterns and phonetic production through five functional blocks, thereby significantly enhancing synthetic speech detection accuracy. These functional blocks include the Breathing-Speech Alignment Module (2), Breath-Speech Interaction Embeddings (3), Temporal Consistency Analyzer (4), Breath-Speech Coherence Module (5), and Deepfake Detection Classifier (6). By leveraging the physiological and phonetic cues that are difficult for synthetic speech generation models to replicate, the framework leverages physiological and phonetic cues, offering a robust solution to distinguish between organic and synthetic speech.

Description

TECHINCAL FIELD

[0001] The present disclosure relates to methods and systems for detecting synthetic speech.PRIOR ART

[0002] Synthetic speech, also known as deepfake speech, generated by Al-powered technology, for example through text-to-speech or voice assistant systems, aims to be identical to organic speech produced by a human. While such audio has many well intentioned usage, the potential for dangerous applications has created the need for accurate and automated demarcation of human-spoken from synthetically generated audio.

[0003] The research community has responded with challenges such as ASVspoof, ADD, and SASV. These competitions curate datasets of synthetic and organic speech and invite participants to create detection algorithms to test on these datasets. Subsequently, these datasets are the de facto standard for synthetic speech and give a baseline of comparison for all current and future synthetic speech detectors.

[0004] Most of the currently existing synthetic speech detectors, focus on low-level spectral (e.g., spectrogram, Mel-frequency cepstral coefficients, Linear Frequency Cepstral Coefficients, and constant-Q cepstral coefficients) imperfections, created during the audio generation pipeline and show starkly different classification results vs. human interpreters.

[0005] However, these low-level spectral detection-based methods often fall short in accurately detecting deepfake speech due to the complexities involved in replicating organic speech dynamics, particularly the interaction between breathing patterns and phonetic production, and will be rendered obsolete due to the rapid advancement of the speech generation field.

[0006] In fact, with advances in artificial intelligence and machine learning, the generation of synthetic speech has become increasingly sophisticated, which justifies the need to develop more advanced detection mechanisms that can effectively discern between organic and synthetic speech.

[0007] The present solution intended to innovatively overcome such issues.SUMMARY OF THE DISCLOSURE

[0008] The present disclosure concerns a novel synthetic speech detection framework based on breath-aware temporal discrepancy detection combined with phonetic insights. By focusing on the complex relationship between breathing patterns and phonetic production in natural speech dynamics, the accuracy in the detection of a synthetic speech is enhanced.

[0009] In fact, unlike traditional detection models that emphasize low-level spectral features, the proposed framework leverages the interaction between breath cycles and the phonetic structure of speech, which plays a crucial role in shaping speech rhythm, prosody, and articulation. Consequently, by targeting the physiological and phonetic cues that are difficult for synthetic generation models to replicate, the accuracy in detecting a synthetic speech can be significantly improved.

[0010] The framework consists of five functional blocks: a Breathing-Speech Alignment Module, Breath-Speech Interaction Embeddings, Temporal Consistency Analyzer, Breath-Speech Coherence Module, and Deepfake Detection Classifier. Each functional block plays a crucial role in capturing the intricate details of breath and phonetic interactions, focusing on detecting synthetic speech through temporal analysis, breath-phonetic interaction modeling, and deep feature extraction. By incorporating phonetic insights, these functional blocks are able to capture the intricate details of how breath and phonetic production interact in natural speech, making it possible to identify irregularities in synthetic speech.

[0011] With this context, it is the object of the present disclosure a synthetic speech detection method, defining a specific set of operations to be performed by the aforementioned functional blocks. In general terms, the method may comprise the following steps: Detecting breath events within a speech signal; said detecting may be carried out using a combination of a Convolutional Neural Network and a Bidirectional Long Short-Term Memory network to process the input speech signal; Aligning the detected breath events with natural phonetic units and linguistic boundaries by employing, for example, a Transformer-based model to map breath events to vowels, plosives, and fricatives; Extracting multi-dimensional features that model the interaction between breathing patterns and phonetic production; Evaluating temporal consistency between the detected breath events and phonetic production, for example by employing a self-attention mechanism within a Transformer-based network, resulting in a Phonetic Temporal Consistency Score; Analyzing the coherence between breath events, speech intensity, and phonetic energy patterns by measuring intensity fluctuations through root mean square energy and amplitude envelope features; and Integrating the extracted features and evaluated scores into a unified feature vector and classifying the speech sample as real or synthetic, for example using a Multilayer Perceptron or Transformer-based classifier. DESCRIPTION OF FIGURES

[0012] Figure 1 illustrates a representation of the synthetic speech detection framework of the present disclosure, where the reference signs represent: 1 - input raw speech waveform; 2 - Breathing-Speech Alignment module; a. - Time-stamped breath events aligned with said phonetic and linguistic segments; 3 - Breath-Speech Interaction Embeddings module; b. - Breath-speech embedding record; 4 - Temporal Consistency Analyzer module; c. - Phonetic temporal consistency score; 5 - Breath-Speech Coherence module; d. - Breath-Speech Coherence score; 6 - Synthetic Speech Detection Classifier module; 7 - Classification score. DETAILED DESCRIPTION

[0013] The more general configurations of the present disclosure are described in the Summary of the disclosure. Such configurations are detailed below in accordance with other advantageous and / or preferred embodiments of implementation of the present disclosure.

[0014] The present disclosure is directed towards a sophisticated framework for the detection of synthetic speech utilizing a Breath-Aware Temporal Discrepancy Detection methodology. This framework leverages the complex interplay between breathing patterns and phonetic articulation, which is challenging for synthetic speech generation models to accurately replicate, thereby significantly enhancing the accuracy of the detection.

[0015] This framework is constituted by five functional blocks.

[0016] The first functional block is a Breathing-Speech Alignment (BSA) module (2): this module is responsible for detecting breath events within input speech signals and aligning these events with natural phonetic and linguistic and linguistic segments, ensuring that breath timing matches natural speech structure. In the context of the present disclosure, a phonetic segment is an individual speech sound, that relates to vowels, plosives, and fricatives. As to a linguistic segment, it relates to sentences or clauses and the respective to pauses and breaks. The BSA module (2) may employ a combination of a Convolutional Neural Network (CNN) and a Bidirectional Long Short-Term Memory (BiLSTM) network to accurately detect breath events from raw speech waveforms (1). Following detection, a Transformer-based model may be employed to align these breath events with specific phonetic units including vowels, plosives, and fricatives. Breath is tightly linked to these sounds, as vowels require continuous airflow, while plosives and fricatives depend on precise breath control. The output of this module is a set of time-stamped breath events that are aligned with phonetic and linguistic segments (a.), being crucial for identifying discrepancies in synthetic speech where breath patterns do not naturally follow phonetic articulations.

[0017] The second functional block, is a Breath-Speech Interaction Embeddings (BSIE) module (3): This module is configured to extract detailed, multi-dimensional features that model the interaction between breathing patterns and phonetic production. The feature extraction process may involve several subcomponents: Breath-Event-Aware Acoustic Features unit: to capture acoustic changes around breath events, particularly focusing on the transitions between breath and specific phonetic sounds like vowels and consonants, by track variations in the harmonic-to-noise ratio (HNR) and pitch during these transitions; Temporal Alignment Variability Features unit: to analyze the timing of breaths relative to phonetic unit boundaries (e.g., after plosives or between vowel segments), capturing unnatural pauses or rigid patterns indicative of artificial speech; Speech Intensity-Breath Correlation Features unit: to model how speech intensity naturally varies with phonetic production and breath cycles, focusing on the decay in intensity before breaths and the surge after breaths. This is especially relevant for vowel sounds, which rely on steady airflow, and consonants, which require controlled bursts of air; Breath-Speech Frequency Alignment unit: to capture spectral shifts during breath events and to analyze the influence of breath noise on low-frequency bands, measuring how formant shifts in vowels are affected by breath dynamics, revealing synthetic speech's inability to replicate natural vowel transitions.

[0018] The module (3) focuses on both temporal and acoustic relationships, capturing how breath cycles impact specific phonetic features such as vowel clarity, consonant articulation, and prosodic patterns. The output of this module (3) is a high-dimensional Breath-Speech Interaction Embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features, and temporal variability, providing a detailed representation of natural vs. synthetic speech patterns.

[0019] The third functional block is a Temporal Consistency Analyzer (TCA) module (4): This module evaluates the temporal consistency between detected breath events and phonetic production, ensuring that breath timing follows realistic patterns observed in natural speech. A Transformer-based self-attention network may be employed to model the dependencies between phonetic segments and breath events, focusing on long-range temporal coherence. The module (4) calculates a Phonetic Temporal Consistency (PTC) score (c.) that evaluates how well the timing of breaths aligns with expected patterns for specific phonetic categories (e.g., pauses before vowels, short bursts of breath before plosives). Therefore, the output of this module is the PTC score (c.) that quantifies the degree to which breath events adhere to natural temporal patterns relative to phonetic units. In synthetic speech, misalignments between breath events and phoneme boundaries may lead to an unnatural score.

[0020] The fourth functional block is a Breath-Speech Coherence (BSC) module (5): This module determines if coherence between breath events, speech intensity, and phonetic energy patterns is maintained, which detects whether breath intensity naturally decays or surges in response to phonetic production. By applying features such as root mean square energy and amplitude envelope, the module (5) measures intensity fluctuations, and how phonetic intensity (especially for vowels and consonants) changes before and after breath events. The cross-correlation between breath events and speech intensity may be then computed, focusing on phonetic transitions where breath naturally modulates energy, such as the surge after inhalation when producing vowel sounds or the energy dip before exhalation in fricatives. Therefore, this module (5) outputs a Breath-Speech Coherence (BSC) score (d.) reflecting the natural alignment between breath timing, phonetic intensity, and energy patterns. Synthetic speech may exhibit irregular energy fluctuations around breaths, which this module can detect.

[0021] Finally, the fifth block is a Deepfake Detection Classifier (DDC) module (6): this module integrates the outputs from the previous modules (2, 3, 4, 5) into a final classification decision (7), that determines whether the input speech (1) is natural or synthetic, based on breath-phonetic interaction and temporal patterns. It may involve feature fusion, creating a unified feature vector that captures both temporal and phonetic dependencies. A Multilayer Perceptron or Transformer-based classifier may be employed to analyze and integrate these features, thereby being able to classify speech (1) samples as natural or synthetic, as it is trained to distinguish the subtle differences in breath-phonetic relationships that are difficult for synthetic speech generation models to replicate. Consequently, a binary classification score (7) is generated indicating whether the input speech is natural or a synthetic, based on phonetic and breath-speech interaction features.EMBODIMENTS

[0022] In a preferred embodiment of the synthetic speech detection method of the present disclosure, it is comprised by the following steps: i. processing, by a Breathing-Speech Alignment (BSA) module (2), a raw speech waveform (1), to detect breath events and to align said breath events with a natural speech structure, defined by phonetic and linguistic segments, thereby generating a set of time-stamped breath events aligned with said phonetic and linguistic segments (a.); ii. processing, by a Breath-Speech Interaction Embeddings (BSIE) module (3), the set of time-stamped breath events in orderto extract multi-dimensional features modeling said time-stamped breath events and phonetic production, thereby generating a breath-speech embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features and temporal variability; iii. evaluating temporal consistency between detected breath events and phonetic segments production, by a Temporal Consistency Analyzer (TCA) module (4), thereby generating a phonetic temporal consistency (PTC) score (c.); iv. measuring speech intensity fluctuations and phonetic energy pattern parameters, by a Breath-Speech Coherence (BSC) module (5), and cross-correlating said parameters with breath events, thereby generating a Breath-Speech Coherence (BSC) score (d.); v. fusing, by a Synthetic Speech Detection Classifier (SSDC) module (6) the outputs from the previous modules to generate a classification score (7) indicating whether the input raw speech waveform (1) is natural or synthetic.

[0023] In one embodiment of the method, the step of generating the set of time-stamped breath events executed by the BSA module (2) includes: employing a Convolutional Neural Network and a Bidirectional Long Short-Term Memory Network to process the raw speech waveform (1) in order to detect breath events.

[0024] In another embodiment of the method, the step of generating the set of time-stamped breath events executed by the BSA module (2) includes: implementing a transformer-based architecture to align the detected breath events with phonetic segments, such as vowels, plosives and fricatives, and with linguistic segments, such as linguistic boundaries defined by pauses and sentence breaks.

[0025] In another embodiment of the method, the step of generating a breath-speech embedding record (b.) by the BSIE module (3) includes: capturing acoustic changes around breath events, by a breath-event-aware acoustic features unit; analyzing time-stamped breath events relative to phonetic segment boundaries, such as after plosives or between vowel segments, to detect unnatural pauses or rigid breath patterns, by a temporal alignment variability features unit; modeling the variability of speech intensity with phonetic production and breath cycles defined by the time-stamped breath events, by a speech intensity-breath correlation features unit; capturing spectral shifts during breath events, by a breath-speech frequency alignment unit.

[0026] More particularly, capturing acoustic changes around breath events further may include: detecting transactions between breath events and phonetic segments such as vowels and consonants; and tracking variations in the harmonic-to-noise ration and pitch during these transactions.

[0027] In addition, capturing spectral shifts during breath events may further include measuring a relation between formant shifts F1, F2, in vowels, and breath dynamics.

[0028] In another embodiment of the method, the step of generating the PCT score (c.), by the TCA module (4) includes: employing a transformer-based self-attention network to model dependencies between phonetic segments and detected breath events.

[0029] In another embodiment of the method, the step of measuring speech intensity fluctuations and phonetic energy pattern parameters by the BSC module (5) includes applying root mean square energy and amplitude envelope features. More particularly, the cross-correlation between breath events and speech intensity is computed on phonetic transitions.

[0030] In another embodiment of the method, the step of fusing the outputs from the previous modules to generate the classification score (7) includes fusing: the set of time-stamped breath events aligned with said phonetic and linguistic segments (a.), from the BSA module (2), the breath-speech embedding record (b.), from the BSIE module (3), the PTC score (c.), from the TCA module (4), and the BSC score (d.), from the BSC module (5), into a unified feature vector, adapted to capture both temporal and phonetic dependencies.

[0031] More particularly, the step of generating the classification score (7), includes employing a classifier to process the unified feature vector and to classify input raw speech waveform (1) samples; the classifier being a Multilayer Perceptron or a Transformer-based classifier.

[0032] Of course, the preferred embodiments shown above are combinable, in the different possible forms, being herein avoided the repetition all such combinations.

Examples

embodiments

EMBODIMENTS

[0022]In a preferred embodiment of the synthetic speech detection method of the present disclosure, it is comprised by the following steps:

i. processing, by a Breathing-Speech Alignment (BSA) module (2), a raw speech waveform (1), to detect breath events and to align said breath events with a natural speech structure, defined by phonetic and linguistic segments, thereby generating a set of time-stamped breath events aligned with said phonetic and linguistic segments (a.); ii. processing, by a Breath-Speech Interaction Embeddings (BSIE) module (3), the set of time-stamped breath events in orderto extract multi-dimensional features modeling said time-stamped breath events and phonetic production, thereby generating a breath-speech embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features and temporal variability; iii. evaluating temporal consistency between detected breath events and phonetic segments production, by a Temporal Consistency Analyzer (...

TECHINCAL FIELD

[0001] The present disclosure relates to methods and systems for detecting synthetic speech.PRIOR ART

[0002] Synthetic speech, also known as deepfake speech, generated by Al-powered technology, for example through text-to-speech or voice assistant systems, aims to be identical to organic speech produced by a human. While such audio has many well intentioned usage, the potential for dangerous applications has created the need for accurate and automated demarcation of human-spoken from synthetically generated audio.

[0003] The research community has responded with challenges such as ASVspoof, ADD, and SASV. These competitions curate datasets of synthetic and organic speech and invite participants to create detection algorithms to test on these datasets. Subsequently, these datasets are the de facto standard for synthetic speech and give a baseline of comparison for all current and future synthetic speech detectors.

[0004] Most of the currently existing synthetic speech detectors, focus on low-level spectral (e.g., spectrogram, Mel-frequency cepstral coefficients, Linear Frequency Cepstral Coefficients, and constant-Q cepstral coefficients) imperfections, created during the audio generation pipeline and show starkly different classification results vs. human interpreters.

[0005] However, these low-level spectral detection-based methods often fall short in accurately detecting deepfake speech due to the complexities involved in replicating organic speech dynamics, particularly the interaction between breathing patterns and phonetic production, and will be rendered obsolete due to the rapid advancement of the speech generation field.

[0006] In fact, with advances in artificial intelligence and machine learning, the generation of synthetic speech has become increasingly sophisticated, which justifies the need to develop more advanced detection mechanisms that can effectively discern between organic and synthetic speech.

[0007] The present solution intended to innovatively overcome such issues.SUMMARY OF THE DISCLOSURE

[0008] The present disclosure concerns a novel synthetic speech detection framework based on breath-aware temporal discrepancy detection combined with phonetic insights. By focusing on the complex relationship between breathing patterns and phonetic production in natural speech dynamics, the accuracy in the detection of a synthetic speech is enhanced.

[0009] In fact, unlike traditional detection models that emphasize low-level spectral features, the proposed framework leverages the interaction between breath cycles and the phonetic structure of speech, which plays a crucial role in shaping speech rhythm, prosody, and articulation. Consequently, by targeting the physiological and phonetic cues that are difficult for synthetic generation models to replicate, the accuracy in detecting a synthetic speech can be significantly improved.

[0010] The framework consists of five functional blocks: a Breathing-Speech Alignment Module, Breath-Speech Interaction Embeddings, Temporal Consistency Analyzer, Breath-Speech Coherence Module, and Deepfake Detection Classifier. Each functional block plays a crucial role in capturing the intricate details of breath and phonetic interactions, focusing on detecting synthetic speech through temporal analysis, breath-phonetic interaction modeling, and deep feature extraction. By incorporating phonetic insights, these functional blocks are able to capture the intricate details of how breath and phonetic production interact in natural speech, making it possible to identify irregularities in synthetic speech.

[0011] With this context, it is the object of the present disclosure a synthetic speech detection method, defining a specific set of operations to be performed by the aforementioned functional blocks. In general terms, the method may comprise the following steps: Detecting breath events within a speech signal; said detecting may be carried out using a combination of a Convolutional Neural Network and a Bidirectional Long Short-Term Memory network to process the input speech signal; Aligning the detected breath events with natural phonetic units and linguistic boundaries by employing, for example, a Transformer-based model to map breath events to vowels, plosives, and fricatives; Extracting multi-dimensional features that model the interaction between breathing patterns and phonetic production; Evaluating temporal consistency between the detected breath events and phonetic production, for example by employing a self-attention mechanism within a Transformer-based network, resulting in a Phonetic Temporal Consistency Score; Analyzing the coherence between breath events, speech intensity, and phonetic energy patterns by measuring intensity fluctuations through root mean square energy and amplitude envelope features; and Integrating the extracted features and evaluated scores into a unified feature vector and classifying the speech sample as real or synthetic, for example using a Multilayer Perceptron or Transformer-based classifier. DESCRIPTION OF FIGURES

[0012] Figure 1 illustrates a representation of the synthetic speech detection framework of the present disclosure, where the reference signs represent: 1 - input raw speech waveform; 2 - Breathing-Speech Alignment module; a. - Time-stamped breath events aligned with said phonetic and linguistic segments; 3 - Breath-Speech Interaction Embeddings module; b. - Breath-speech embedding record; 4 - Temporal Consistency Analyzer module; c. - Phonetic temporal consistency score; 5 - Breath-Speech Coherence module; d. - Breath-Speech Coherence score; 6 - Synthetic Speech Detection Classifier module; 7 - Classification score. DETAILED DESCRIPTION

[0013] The more general configurations of the present disclosure are described in the Summary of the disclosure. Such configurations are detailed below in accordance with other advantageous and / or preferred embodiments of implementation of the present disclosure.

[0014] The present disclosure is directed towards a sophisticated framework for the detection of synthetic speech utilizing a Breath-Aware Temporal Discrepancy Detection methodology. This framework leverages the complex interplay between breathing patterns and phonetic articulation, which is challenging for synthetic speech generation models to accurately replicate, thereby significantly enhancing the accuracy of the detection.

[0015] This framework is constituted by five functional blocks.

[0016] The first functional block is a Breathing-Speech Alignment (BSA) module (2): this module is responsible for detecting breath events within input speech signals and aligning these events with natural phonetic and linguistic and linguistic segments, ensuring that breath timing matches natural speech structure. In the context of the present disclosure, a phonetic segment is an individual speech sound, that relates to vowels, plosives, and fricatives. As to a linguistic segment, it relates to sentences or clauses and the respective to pauses and breaks. The BSA module (2) may employ a combination of a Convolutional Neural Network (CNN) and a Bidirectional Long Short-Term Memory (BiLSTM) network to accurately detect breath events from raw speech waveforms (1). Following detection, a Transformer-based model may be employed to align these breath events with specific phonetic units including vowels, plosives, and fricatives. Breath is tightly linked to these sounds, as vowels require continuous airflow, while plosives and fricatives depend on precise breath control. The output of this module is a set of time-stamped breath events that are aligned with phonetic and linguistic segments (a.), being crucial for identifying discrepancies in synthetic speech where breath patterns do not naturally follow phonetic articulations.

[0017] The second functional block, is a Breath-Speech Interaction Embeddings (BSIE) module (3): This module is configured to extract detailed, multi-dimensional features that model the interaction between breathing patterns and phonetic production. The feature extraction process may involve several subcomponents: Breath-Event-Aware Acoustic Features unit: to capture acoustic changes around breath events, particularly focusing on the transitions between breath and specific phonetic sounds like vowels and consonants, by track variations in the harmonic-to-noise ratio (HNR) and pitch during these transitions; Temporal Alignment Variability Features unit: to analyze the timing of breaths relative to phonetic unit boundaries (e.g., after plosives or between vowel segments), capturing unnatural pauses or rigid patterns indicative of artificial speech; Speech Intensity-Breath Correlation Features unit: to model how speech intensity naturally varies with phonetic production and breath cycles, focusing on the decay in intensity before breaths and the surge after breaths. This is especially relevant for vowel sounds, which rely on steady airflow, and consonants, which require controlled bursts of air; Breath-Speech Frequency Alignment unit: to capture spectral shifts during breath events and to analyze the influence of breath noise on low-frequency bands, measuring how formant shifts in vowels are affected by breath dynamics, revealing synthetic speech's inability to replicate natural vowel transitions.

[0018] The module (3) focuses on both temporal and acoustic relationships, capturing how breath cycles impact specific phonetic features such as vowel clarity, consonant articulation, and prosodic patterns. The output of this module (3) is a high-dimensional Breath-Speech Interaction Embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features, and temporal variability, providing a detailed representation of natural vs. synthetic speech patterns.

[0019] The third functional block is a Temporal Consistency Analyzer (TCA) module (4): This module evaluates the temporal consistency between detected breath events and phonetic production, ensuring that breath timing follows realistic patterns observed in natural speech. A Transformer-based self-attention network may be employed to model the dependencies between phonetic segments and breath events, focusing on long-range temporal coherence. The module (4) calculates a Phonetic Temporal Consistency (PTC) score (c.) that evaluates how well the timing of breaths aligns with expected patterns for specific phonetic categories (e.g., pauses before vowels, short bursts of breath before plosives). Therefore, the output of this module is the PTC score (c.) that quantifies the degree to which breath events adhere to natural temporal patterns relative to phonetic units. In synthetic speech, misalignments between breath events and phoneme boundaries may lead to an unnatural score.

[0020] The fourth functional block is a Breath-Speech Coherence (BSC) module (5): This module determines if coherence between breath events, speech intensity, and phonetic energy patterns is maintained, which detects whether breath intensity naturally decays or surges in response to phonetic production. By applying features such as root mean square energy and amplitude envelope, the module (5) measures intensity fluctuations, and how phonetic intensity (especially for vowels and consonants) changes before and after breath events. The cross-correlation between breath events and speech intensity may be then computed, focusing on phonetic transitions where breath naturally modulates energy, such as the surge after inhalation when producing vowel sounds or the energy dip before exhalation in fricatives. Therefore, this module (5) outputs a Breath-Speech Coherence (BSC) score (d.) reflecting the natural alignment between breath timing, phonetic intensity, and energy patterns. Synthetic speech may exhibit irregular energy fluctuations around breaths, which this module can detect.

[0021] Finally, the fifth block is a Deepfake Detection Classifier (DDC) module (6): this module integrates the outputs from the previous modules (2, 3, 4, 5) into a final classification decision (7), that determines whether the input speech (1) is natural or synthetic, based on breath-phonetic interaction and temporal patterns. It may involve feature fusion, creating a unified feature vector that captures both temporal and phonetic dependencies. A Multilayer Perceptron or Transformer-based classifier may be employed to analyze and integrate these features, thereby being able to classify speech (1) samples as natural or synthetic, as it is trained to distinguish the subtle differences in breath-phonetic relationships that are difficult for synthetic speech generation models to replicate. Consequently, a binary classification score (7) is generated indicating whether the input speech is natural or a synthetic, based on phonetic and breath-speech interaction features.EMBODIMENTS

[0022] In a preferred embodiment of the synthetic speech detection method of the present disclosure, it is comprised by the following steps: i. processing, by a Breathing-Speech Alignment (BSA) module (2), a raw speech waveform (1), to detect breath events and to align said breath events with a natural speech structure, defined by phonetic and linguistic segments, thereby generating a set of time-stamped breath events aligned with said phonetic and linguistic segments (a.); ii. processing, by a Breath-Speech Interaction Embeddings (BSIE) module (3), the set of time-stamped breath events in orderto extract multi-dimensional features modeling said time-stamped breath events and phonetic production, thereby generating a breath-speech embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features and temporal variability; iii. evaluating temporal consistency between detected breath events and phonetic segments production, by a Temporal Consistency Analyzer (TCA) module (4), thereby generating a phonetic temporal consistency (PTC) score (c.); iv. measuring speech intensity fluctuations and phonetic energy pattern parameters, by a Breath-Speech Coherence (BSC) module (5), and cross-correlating said parameters with breath events, thereby generating a Breath-Speech Coherence (BSC) score (d.); v. fusing, by a Synthetic Speech Detection Classifier (SSDC) module (6) the outputs from the previous modules to generate a classification score (7) indicating whether the input raw speech waveform (1) is natural or synthetic.

[0023] In one embodiment of the method, the step of generating the set of time-stamped breath events executed by the BSA module (2) includes: employing a Convolutional Neural Network and a Bidirectional Long Short-Term Memory Network to process the raw speech waveform (1) in order to detect breath events.

[0024] In another embodiment of the method, the step of generating the set of time-stamped breath events executed by the BSA module (2) includes: implementing a transformer-based architecture to align the detected breath events with phonetic segments, such as vowels, plosives and fricatives, and with linguistic segments, such as linguistic boundaries defined by pauses and sentence breaks.

[0025] In another embodiment of the method, the step of generating a breath-speech embedding record (b.) by the BSIE module (3) includes: capturing acoustic changes around breath events, by a breath-event-aware acoustic features unit; analyzing time-stamped breath events relative to phonetic segment boundaries, such as after plosives or between vowel segments, to detect unnatural pauses or rigid breath patterns, by a temporal alignment variability features unit; modeling the variability of speech intensity with phonetic production and breath cycles defined by the time-stamped breath events, by a speech intensity-breath correlation features unit; capturing spectral shifts during breath events, by a breath-speech frequency alignment unit.

[0026] More particularly, capturing acoustic changes around breath events further may include: detecting transactions between breath events and phonetic segments such as vowels and consonants; and tracking variations in the harmonic-to-noise ration and pitch during these transactions.

[0027] In addition, capturing spectral shifts during breath events may further include measuring a relation between formant shifts F1, F2, in vowels, and breath dynamics.

[0028] In another embodiment of the method, the step of generating the PCT score (c.), by the TCA module (4) includes: employing a transformer-based self-attention network to model dependencies between phonetic segments and detected breath events.

[0029] In another embodiment of the method, the step of measuring speech intensity fluctuations and phonetic energy pattern parameters by the BSC module (5) includes applying root mean square energy and amplitude envelope features. More particularly, the cross-correlation between breath events and speech intensity is computed on phonetic transitions.

[0030] In another embodiment of the method, the step of fusing the outputs from the previous modules to generate the classification score (7) includes fusing: the set of time-stamped breath events aligned with said phonetic and linguistic segments (a.), from the BSA module (2), the breath-speech embedding record (b.), from the BSIE module (3), the PTC score (c.), from the TCA module (4), and the BSC score (d.), from the BSC module (5), into a unified feature vector, adapted to capture both temporal and phonetic dependencies.

[0031] More particularly, the step of generating the classification score (7), includes employing a classifier to process the unified feature vector and to classify input raw speech waveform (1) samples; the classifier being a Multilayer Perceptron or a Transformer-based classifier.

[0032] Of course, the preferred embodiments shown above are combinable, in the different possible forms, being herein avoided the repetition all such combinations.

Examples

embodiments

EMBODIMENTS

[0022]In a preferred embodiment of the synthetic speech detection method of the present disclosure, it is comprised by the following steps:

i. processing, by a Breathing-Speech Alignment (BSA) module (2), a raw speech waveform (1), to detect breath events and to align said breath events with a natural speech structure, defined by phonetic and linguistic segments, thereby generating a set of time-stamped breath events aligned with said phonetic and linguistic segments (a.); ii. processing, by a Breath-Speech Interaction Embeddings (BSIE) module (3), the set of time-stamped breath events in orderto extract multi-dimensional features modeling said time-stamped breath events and phonetic production, thereby generating a breath-speech embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features and temporal variability; iii. evaluating temporal consistency between detected breath events and phonetic segments production, by a Temporal Consistency Analyzer (...

Claims

1. A synthetic speech detection method, comprising the following steps: i. processing, by a Breathing-Speech Alignment (BSA) module (2), a raw speech waveform (1), to detect breath events and to align said breath events with a natural speech structure, defined by phonetic and linguistic segments, thereby generating a set of time-stamped breath events aligned with said phonetic and linguistic segments (a.); ii. processing, by a Breath-Speech Interaction Embeddings (BSIE) module (3), the set of time-stamped breath events in order to extract multi-dimensional features modeling said time-stamped breath events and phonetic production, thereby generating a breath-speech embedding record (b.), encapsulating phonetic-breath dynamics, acoustic features and temporal variability; iii. evaluating temporal consistency between detected breath events and phonetic segments production, by a Temporal Consistency Analyzer (TCA) module (4), thereby generating a phonetic temporal consistency (PTC) score (c.); iv. measuring speech intensity fluctuations and phonetic energy pattern parameters, by a Breath-Speech Coherence (BSC) module (5), and cross-correlating said parameters with breath events, thereby generating a Breath-Speech Coherence (BSC) score (d.); v. fusing, by a Synthetic Speech Detection Classifier (SSDC) module (6) the outputs from the previous modules to generate a classification score (7) indicating whether the input raw speech waveform (1) is natural or synthetic.

2. The method according to claim 1, wherein the step of generating the set of time-stamped breath events executed by the BSA module (2) includes: - employing a Convolutional Neural Network and a Bidirectional Long Short-Term Memory Network to process the raw speech waveform (1) in order to detect breath events.

3. The method according to claims 1 or 2, wherein the step of generating the set of time-stamped breath events executed by the BSA module (2) includes: - implementing a transformer-based architecture to align the detected breath events with phonetic segments, such as vowels, plosives and fricatives, and with linguistic segments, such as linguistic boundaries defined by pauses and sentence breaks.

4. The method according to any of the previous claims wherein the step of generating a breath-speech embedding record (b.) by the BSIE module (3) includes: - capturing acoustic changes around breath events, by a breath-event-aware acoustic features unit; - analyzing time-stamped breath events relative to phonetic segment boundaries, such as after plosives or between vowel segments, to detect unnatural pauses or rigid breath patterns, by a temporal alignment variability features unit; - modeling the variability of speech intensity with phonetic production and breath cycles defined by the time-stamped breath events, by a speech intensity-breath correlation features unit; - capturing spectral shifts during breath events, by a breath-speech frequency alignment unit.

5. The method according to claim 4, wherein capturing acoustic changes around breath events further includes: - detecting transactions between breath events and phonetic segments such as vowels and consonants; and - tracking variations in the harmonic-to-noise ration and pitch during these transactions.

6. The method according to claim 4 or 5, wherein capturing spectral shifts during breath events further includes measuring a relation between formant shifts F1, F2, in vowels, and breath dynamics.

7. The method according to any of the previous claims, wherein the step of generating the PCT score (c.), by the TCA module (4) includes: - employing a transformer-based self-attention network to model dependencies between phonetic segments and detected breath events.

8. The method according to any of the previous claims, wherein the step of measuring speech intensity fluctuations and phonetic energy pattern parameters by the BSC module (5) includes applying root mean square energy and amplitude envelope features.

9. The method according to claim 8, wherein the cross-correlation between breath events and speech intensity is computed on phonetic transitions.

10. The method according to any of the previous claims, wherein the step of fusing the outputs from the previous modules to generate the classification score (7) includes fusing: - the set of time-stamped breath events aligned with said phonetic and linguistic segments (a.), from the BSA module (2), - the breath-speech embedding record (b.), from the BSIE module (3), - the PTC score (c.), from the TCA module (4), and - the BSC score (d.), from the BSC module (5), into a unified feature vector, adapted to capture both temporal and phonetic dependencies.

11. The method according to claim 10, further wherein the step of generating the classification score (7), includes employing a classifier to process the unified feature vector and to classify input raw speech waveform (1) samples; the classifier being a Multilayer Perceptron or a Transformer-based classifier.

Images