Systems and methods for language-independent acoustic-based determination of mental state

Abstract

Systems and methods for language-independent acoustic-based automatic determination of mental state, said system comprising: Module for Waveform-based Optimal Neurological Depression Evaluation with Representations Leveraging Multilingual Self-Supervised Speech embeddings via Speaker Identity Invariant Training; analysing latent speech representations from initial encoder layers; determining proximity of analysed latent speech representations to fundamental vocal features; and module for adversarial disentanglement of speaker and depression characteristics, characterized in that, a temporal pooling layer configured to have a common output vector dimension for extracted feature representations, in that, from the extracted feature representations, non-augmented features are fed to a speaker classification layer and augmented features are fed to a depression classification layer where total loss is a combination of a speaker loss and Major Depressive Disorder loss.

Description

[0001] SYSTEMS AND METHODS FOR LANGUAGE-INDEPENDENT ACOUSTIC-BASED DETERMINATION OF MENTAL STATE FIELD OF THE INVENTION:

[0002] This invention relates to the field of artificial intelligence, machine learning, computational networks, and computer engineering.

[0003] Particularly, this invention relates to speech signal processing systems.

[0004] Specifically, this invention relates to language-independent and speakerinvariant acoustic signal transformation systems for generating paralinguistic mental- state indicators from raw speech waveforms.

[0005] BACKGROUND OF THE INVENTION:

[0006] Mental health disorders are reported to be increasing at an accelerating rate affecting 10-15% world population catalyzed by the COVID-19 pandemic. The current rate of growth has a major economic impact compared to cancer, cardiovascular diseases, diabetes, and respiratory diseases. Suicide, owing to mental health issues, is currently the second leading cause of death in 15 to 29-year-olds, resulting in enormous social disruption and losses in productivity. In response to these alarming circumstances, in 2016, the World Health Organization declared depression to be the leading cause of disability worldwide.

[0007] However, as severity increases, traditional health care systems are not equipped to handle the massive influx of individuals affected by mental health disorders. Early detection and early intervention techniques can drastically improve cure rate of affected individuals; thus, relieving economic burden and increasing productivity of and quality of life of affected individuals. Automated screeningand continuous monitoring have proven to be an effective alternative to facilitate screening.

[0008] Depression is a widespread and serious mental disorder that is often overlooked due to its similar signs and symptoms. Timely diagnosis is essential for successful intervention. The non-invasive and continuous tracking of physiological and psychological markers through smartphone tech and wearable devices has recently garnered significant attention from the research community. Consequently, there's a growing interest in fully automated depression screening tools. Notably, psychomotor retardation, a hallmark symptom of Major Depressive Disorder (MDD), is evident through unique verbal cues such as monotonic speech patterns, specific word choices, and uncommon speech interruptions. This has led to a surge in the adoption of automated speech-based screening methods. However, despite the depth of research, using conventional machine learning techniques for auto-detecting depression from speech is still problematic due to the scarcity of data.

[0009] Voice-based Artificial Intelligence (Al) based screening technologies are gaining popularity amount users detecting a variety of mental health diseases including depression and anxiety. It is deemed as an inexpensive scalable solution since it can be easily accessed through digital devices, for example, mobile phones. Leveraging voice as a “biomarker”, these technologies can also be more robust to dishonest or cheating behaviours during assessment or monitoring sessions rather than traditional outpatient screening mechanisms, for example, PHQ-9 (Patient Health Questionnaire 9) or GAD-7 (Generalized Anxiety Disorder 7).

[0010] Voice-based Artificial Intelligent (Al) screening systems and methods, usually, rely on scripted dialogs and collect audio-based answers (other signals mightalso be collected at the same time). Based on the answers, it may use models built on acoustic features, or acoustic plus NLP features to produce a final classification; this is well documented in prior art. In order to avoid taking not useful answers as input, certain statistics-based techniques might be used to test each answer before moving to next step. A simple solution is to measure the total time of the recorded answer, and a more complex one involves something like voice activation or leveraging ASR (Automatic Speech Recognition) to check whether there are sentences with enough words. If the preconfigured threshold is not met, user might get re -prompted with the same question (or a different one) and will have to answer again.

[0011] There are pitfalls with this kind of approach of the prior art:

[0012] First, it gives longer answers a natural bias which is not always correct. A relatively short answer might reflect truthfully a user’s mental health status, however a longer one, with more words, on the other hand, might lead to incorrect assessment when the user is dishonest.

[0013] Second, the ASR (Automatic Speech Recognition) technology used, in such system and methods, usually ignores certain speech behaviours, e.g. pauses, hesitations, jittering, and shimmering etc., and only produces the sentences / words it can recognize. Filtering, based on output from such ASR technology might, leave out some otherwise meaningful answers to screening models.

[0014] Finally, a user’s emotion status when answering a question might influence a mental health assessment result. Answer validation techniques, of the prior art, however, ignores emotion, as an input, and is not sufficient here either.

[0015] Therefore, a better mechanism is needed to help ensure relatively higher input quality, which will, in turn, improve model performance.Furthermore, in recent years, non-invasive and continuous monitoring of physiological and psychological data by smartphone-based technologies and wearable devices has aroused researchers' interests. Advancements in acoustic and speech processing makes a new frontier of behavioral health diagnosis using machine learning. It was found that the depressive patients showed reduced stress, mono-pitch, loudness decay and etc. in language, which was consistent with the clinical observations of depressed patients. These patients showed slower speech, more pauses, and low volume compared to ordinary people. Studies have shown that the speech biomarkers recognized by machine learning models have a positive correlation and have greater potential in detecting mental disorders such as depression. Researchers have spent tremendous amount of effort studying depression and its correlation to acoustics and semantic aspects of speech.

[0016] Al technologies, in this space, can be divided into three main categories:

[0017] 1. Semantic based: Transform speech data into text transcriptions by applying automatic speech recognition (ASR), then perform natural language process (NLP) on top to build natural language-based classification models.

[0018] 2. Acoustic based: Extract acoustic features from speech directly, and then build classification models based on them. These features can be either hand engineered features e.g. rhythmic features / spectral-based correlation features, or latent feature embeddings via pretrained models.

[0019] 3. Multi-modal based: There are also some tries in combining these two modalities to create a multi-modal Al model to increase the accuracy of the assessment.

[0020] These technologies and related studies often use voice recordings from users related to certain topic as the model input; for example, answers to certain fixedquestions about their rest schedule, or description of their health status, and the like. On the other hand, it is worth noting that major depressive disorder (MDD) usually involves the numbing of emotions, especially grief, fear, anger and shame. Many past reviews suggest that individuals with depression differ from non-depressed individuals, especially in their emotion preferences and use of emotion regulation strategies. Unfortunately, users’ emotion preference or emotion regulation capabilities are not paid enough attention in existing voicebased Al technologies from training data collection, to model training, to inferencing when model is applied in real applications.

[0021] It has become a known fact, based on previous research, that human voice conveys a lot of emotion information. For example, some prior art literatures have found that one can detect not just basic emotional tone in the voice (e.g., positive vs. negative feelings or excitement vs. calm) but fine emotional nuances. Covering more emotion effects actively or utilizing emotion information better from voice (especially valence) can potentially the accuracy of such Al models.

[0022] This patent discusses a system and method towards building such technology.

[0023] Furthermore, self- supervised learning (SSL) techniques aim to develop a versatile robust universal model that can be advantageous across numerous downstream tasks. In recent years, the progress of foundation models trained in a self-supervised way exhibits that the self- supervised representations related to acoustic word embeddings and learning with zero lexical resources. Such encoders in foundation models trained with SSL method contains information about both linguistics and paralinguistic parts of speech which can be applied to many areas especially in affective computing in SER (speech emotion recognition). Foundation models, like the Wav2Vec,huBERT, WavLM excel in learning intricate speech features, largely due to their training on extensive amounts of unlabelled data, ranging from 60k to 94k hours. Such models not only effectively model speech content via masked speech prediction but also expand their versatility to non-ASR tasks by integrating speech denoising creating state of the art (SOTA) benchmarks.

[0024] Learning feature representations related to the speaker such as X- vectors, i-vectors, and other speaker embeddings have shown potential to increase the accuracy of detecting depression from speech. However, training a model in a speaker-invariant way involves ensuring that the model generalizes across various speakers rather than overfitting to specific vocal characteristics. By doing so, the model focuses on understanding the content and nuances of the speech rather than the unique attributes of individual speakers. This approach ensures that the model is less prone to learning speaker-specific features, which could limit its applicability and accuracy when exposed to voices it hasn't encountered during training. In essence, a speaker-invariant training approach encourage the model to leam more subtle acoustic feature patterns related to psychomotor retardation, making the model more versatile and robust in real-world scenarios where diverse voices are encountered.

[0025] Prior art speech-based mental state assessment systems increasingly rely on foundation models trained using self- supervised learning techniques. While such models extract high-dimensional latent speech representations, it has been observed that naive fine-tuning of these representations on limited or laboratory-controlled datasets results in overfitting to speaker identity, recording conditions, or language-specific lexical cues. This reduces robustness and generalizability when such systems are deployed in real-world environments, including mobile and consumer-grade devices.Further, prior art fine-tuning strategies fail to explicitly constrain representation learning to paralinguistic acoustic features associated with psychomotor retardation, such as temporal irregularity, pitch monotony, reduced prosodic variation, and speech hesitation patterns. As a result, clinically relevant acoustic biomarkers are often masked by dominant speaker- specific characteristics.

[0026] Accordingly, there exists a technical need for a system and method that:

[0027] (i) mitigates data scarcity through representation reuse and augmentation, and

[0028] (ii) enforces invariance to speaker identity and language, while preserving depression-correlated paralinguistic features.

[0029] Speech-based mental state analysis systems often fail when trained on acoustically homogeneous datasets or laboratory-controlled recordings, resulting in degraded performance when deployed across heterogeneous recording devices, speaker demographics, or conversational contexts.

[0030] Additionally, acoustic features correlated with mental-state indicators are frequently confounded by speaker-dependent characteristics such as genderspecific pitch ranges, vocal tract length, or articulation patterns. Such confounding leads to biased representations that limit generalizability across populations and recording environments.

[0031] Accordingly, there exists a technical need for a system that acquires speech signals across diverse paradigms and recording devices, while computationally enforcing invariance to speaker-related factors during representation learning.

[0032] OBJECTS OF THE INVENTION:

[0033] An object of the invention is to achieve auto-detecting depression from speech,the detection being language-independent.

[0034] Another object of the invention is to training systems and methods, for determination of mental state, by way of acoustics, in a speaker-invariant way.

[0035] Yet another object of the invention is to provide a system and method that mitigates limited labeled data availability by leveraging self- supervised speech representations trained on large-scale unlabeled corpora and further refining such representations using controlled augmentation strategies.

[0036] Still another object of the invention is to suppress speaker- specific bias in learned representations by adversarially constraining the training process such that paralinguistic speech features correlated with mental- state indicators are preserved while speaker-identifying features are actively discouraged.

[0037] An additional object of the invention is to enable language-independent mental state determination by prioritizing low-level acoustic and temporal features extracted from early encoder layers of self- supervised speech models.

[0038] SUMMARY OF THE INVENTION:

[0039] According to the present invention, there are provided systems and methods for processing raw speech signals to generate a computational mental-state indicator, wherein the generated indicator is substantially invariant to speaker identity and spoken language.

[0040] Typically, this invention provides systems and methods for Waveform-based Optimal Neurological Depression Evaluation with Representations Leveraging Multilingual Self-Supervised Speech embeddings via Speaker Identity Invariant Training.In accordance with the present invention, systems and methods are disclosed for determining a state from speech signals in a manner that is substantially invariant to speaker identity and spoken language.

[0041] This invention proposes a robust approach for automatic depression detection that is both speaker- independent and language-independent. By analysing latent speech representations from initial encoder layers, the inventors theorize that these representations are language-neutral due to their proximity to fundamental vocal features. For the first time, this invention establishes vocal biomarkers associated with depression which are inherently objective and remain unaffected by lexical nuances like accents and languages as the datasets used in training and evaluation consist of samples with different age groups and demographics. In preferred embodiment, self- supervised representations, of both English and Mandarin Chinese corpuses, are fed to this invention’s system and method and trained in a speaker-invariant manner to minimize effects on masking acoustic abnormalities caused by psychomotor retardation.

[0042] In this invention, the inventors propose a novel and inventive approach for two-stage Speech-Based signal processing pipeline. This system method uses a two-pronged / two -staged strategy:

[0043] - first stage, self- supervised learning processes are utilized to build feature representations with augmentations to provide concatenated features; and - second stage, these concatenated features are then applied to a training model that makes use of a combined loss-based adversarial learning framework.The system operates without speech transcription, lexical analysis, or language-specific modeling, thereby enabling cross-lingual generalization.

[0044] In a first stage, raw audio waveforms are transformed into latent acoustic representations using a self- supervised speech encoder trained on unlabeled speech data, thereby extracting paralinguistic features associated with temporal dynamics, spectral consistency, and prosodic variation.

[0045] In a second stage, the extracted representations are processed using a speaker-invariant training architecture that applies adversarial optimization to minimize a primary loss function associated with mental-state classification while simultaneously maximizing a secondary loss function associated with speaker identification. This adversarial constraint forces the learned representations to suppress speaker-dependent features and retain paralinguistic characteristics correlated with psychomotor retardation.

[0046] Through this architecture, the system achieves robust mental-state determination across heterogeneous speakers and languages without reliance on lexical content or speech transcription.

[0047] The nuisance-attribute branches may be implemented as separate classification heads, separate neural networks, or a shared multi-head network architecture. The number of nuisance-attribute branches is not limited and may be selected based on the attributes to be suppressed during training.

[0048] In certain embodiments, the disclosed speech signal processing system is configured to learn paralinguistic representations that are invariant to multiple nuisance attributes simultaneously. Such nuisance attributes may include, without limitation, speaker identity, gender, language, dialect, or combinationsthereof. To this end, the training pipeline may operate as a multi- objective invariant training pipeline in which a foundation model is jointly optimized with respect to a primary paralinguistic classification task and one or more adversarial nuisance-attribute prediction tasks. Gradient inversion or gradient reversal mechanisms may be applied to each nuisance-attribute prediction task such that feature representations learned by the foundation model suppress information predictive of the nuisance attributes while preserving temporal, spectral, and cadence-related paralinguistic modulation patterns relevant to the primary task.

[0049] According to this invention, there is provided a speech signal processing system for generating a paralinguistic state indicator, the system comprising:

[0050] - an input module arranged to receive one or more raw audio segments captured as time-domain speech waveforms;

[0051] - a foundation model operatively coupled to the input module, the foundation model comprising a plurality of transformer encoder blocks, wherein the foundation model transforms each raw audio segment into a frame-level embedding matrix by applying a fixed frame length and frame stride to the raw audio segment;

[0052] - a temporal pooling layer operatively coupled to an output of the foundation model, wherein the temporal pooling layer aggregates the frame-level embedding matrix into a fixed-dimension representation vector that is independent of a duration of the raw audio segment, the fixed-dimension representation vector encoding cadence as a temporal modulation component derived from relative timing relationships among successive frame-level embeddings;

[0053] - an augmentation and permutation module operatively coupled to the temporal pooling layer, wherein the augmentation and permutation module generates additional representation vectors by reordering andconcatenating pooled representation vectors corresponding to affectively distinct speech segments originating from a common recording session; - a speaker-invariant training pipeline operatively coupled to the temporal pooling layer and the augmentation and permutation module, the speakerinvariant training pipeline comprising:

[0054] (a) a primary classification branch receiving the concatenated representation vectors generated by the augmentation and permutation module; and

[0055] (b) a secondary parameterized network branch receiving pooled representation vectors that have not undergone concatenation, the secondary parameterized network branch being configured to predict at least one nuisance attribute selected from speaker identity, gender, language, or combinations thereof;

[0056] - a gradient inversion pathway interposed between the secondary parameterized network branch and the foundation model, wherein gradients associated with prediction of the at least one nuisance attribute are inverted prior to updating parameters of the foundation model; and - a loss aggregation module operatively coupled to the primary classification branch and the secondary classification branch, wherein the loss aggregation module combines:

[0057] (i) a first loss value generated by the primary classification branch; and

[0058] (ii) a second loss value associated with prediction of the at least one nuisance attribute by the secondary parameterized network branch, in such a manner that the first loss value contributes positively to parameter updates of the foundation model, and the second loss value contributes negatively to the parameter updates of the foundation model,

[0059] wherein the foundation model is trained under the combined influence of the first and second loss values such that the pooled representation vectorspropagated to the primary parameterized network branch suppress information predictive of at least one nuisance attribute while retaining paralinguistic temporal, spectral, and cadence-related modulation components derived from the raw audio segments.

[0060] In at least an embodiment, the frame-level embedding matrix comprises a plurality of embedding vectors each corresponding to a fixed-duration time frame defined by the fixed frame length and frame stride.

[0061] In at least an embodiment, the cadence encoded in the fixed-dimension representation vector corresponds to temporal modulation information represented by inter-frame timing relationships among successive embedding vectors within the frame-level embedding matrix.

[0062] In at least an embodiment, the temporal pooling layer comprises a pooling structure selected from a weighted aggregation structure, an attention-based aggregation structure, or a hierarchical aggregation structure.

[0063] In at least an embodiment, the augmentation and permutation module includes a concatenation structure that concatenates pooled representation vectors originating from temporally separated segments of the common recording session.

[0064] In at least an embodiment, the augmentation and permutation module further includes a reordering structure that alters an ordering of pooled representation vectors prior to concatenation.

[0065] In at least an embodiment, the speaker-invariant training pipeline comprises a shared parameter interface through which the primary classification branch andthe secondary classification branch propagate parameter update signals to the foundation model.

[0066] In at least an embodiment, the gradient inversion pathway comprises an inversion layer positioned between the secondary classification branch and the foundation model such that gradients propagated through the inversion layer are sign-reversed.

[0067] In at least an embodiment, the loss aggregation module comprises a summation structure that algebraically combines the first loss value and the second loss value prior to propagation toward the foundation model.

[0068] In at least an embodiment, the cadence remains embedded within the fixed-dimension representation vector during propagation through the augmentation and permutation module and the speaker- invariant training pipeline.

[0069] In at least an embodiment, the secondary parameterized network branch is extended to comprise a tertiary classification branch configured to identify a language origin of the raw audio segments, such that the gradient inversion pathway inverts parameter updates propagated from both the secondary and tertiary branches.

[0070] In at least an embodiment, the plurality of distinct language corpuses comprises at least an English language corpus and a Mandarin Chinese language corpus, and wherein the foundation model learns vocal biomarkers that are robust across said distinct languages.

[0071] In at least an embodiment, the tertiary classification branch generates a language identification loss by attempting to classify the language origin of the embedding matrix, and wherein the gradient inversion pathway forces thefoundation model to discard language- specific phonetic information required for said classification.

[0072] In at least an embodiment, feature representations generated by the foundation model are derived from initial encoder layers of the transformer encoder blocks to ensure proximity to fundamental vocal features that are inherently languageneutral.

[0073] According to this invention, there is provided a computer-implemented method for generating a paralinguistic state indicator from a speech signal, the method comprising:

[0074] - receiving one or more raw audio segments as time-domain speech waveforms;

[0075] - transforming each raw audio segment, using a foundation model comprising a plurality of transformer encoder blocks, into a framelevel embedding matrix by applying a fixed frame length and frame stride to the raw audio segment;

[0076] - aggregating the frame-level embedding matrix using a temporal pooling operation to generate a fixed-dimension representation vector independent of a duration of the raw audio segment, wherein the fixed-dimension representation vector encodes cadence as a temporal modulation component derived from relative timing relationships among successive frame-level embeddings;

[0077] - generating additional representation vectors by reordering and concatenating pooled representation vectors corresponding to affectively distinct speech segments originating from a common recording session;

[0078] - propagating concatenated representation vectors to a primary parameterized network branch and non-concatenated pooledrepresentation vectors to a secondary parameterized network branch configured to predict at least one nuisance attribute selected from speaker identity, gender, language, or combinations thereof; - inverting parameter update gradients propagated from the secondary classification branch toward the foundation model; and

[0079] - combining a first loss value associated with the primary classification branch and a second loss value associated with the secondary classification branch to update parameters of the foundation model.

[0080] In at least an embodiment, each frame-level embedding corresponds to a fixed-duration segment of the speech waveform defined by the fixed frame length.

[0081] In at least an embodiment, aggregating the frame-level embedding matrix comprises applying a pooling structure selected from weighted pooling, attention-based pooling, or hierarchical pooling.

[0082] In at least an embodiment, reordering pooled representation vectors comprises altering a sequence order of the pooled representation vectors prior to concatenation.

[0083] In at least an embodiment, concatenating pooled representation vectors comprises concatenating representation vectors derived from temporally separated speech segments within the common recording session.

[0084] In at least an embodiment, inverting parameter update gradients comprises propagating gradients through an inversion layer positioned between the secondary classification branch and the foundation model.In at least an embodiment, cadence remains embedded within the fixed-dimension representation vector throughout aggregation, reordering, concatenation, and gradient inversion.

[0085] In at least an embodiment, the secondary parameterized network branch is extended to comprise a tertiary classification branch configured to identify a language origin of the raw audio segments, and wherein parameter update gradients propagated from both the secondary and tertiary branches are inverted prior to updating parameters of the foundation model (102).

[0086] In at least an embodiment, the raw audio segments represent diverse speakers and at least two distinct languages, and wherein the foundation model (102) generates latent speech representations that encode paralinguistic abnormalities associated with depression irrespective of the speaker’s voice or language spoken.

[0087] In at least an embodiment, maximizing the prediction loss associated with the tertiary language identification task comprises propagating gradients through a gradient inversion pathway to penalize the foundation model (102) for correctly identifying the language origin.

[0088] In at least an embodiment, the method further comprises augmenting the latent speech representations by concatenating representations of affectively distinct speech segments to balance sample counts between depressed and healthy classes across the distinct languages.

[0089] BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS:

[0090] The invention will now be described in relation to the accompanying drawings, in which:FIGURE 1 is an illustration of the proposed system of this invention.

[0091] DETAILED DESCRIPTION OF THE ACCOMPANYING DRAWINGS:

[0092] According to this invention, there is provided systems and methods for language-independent acoustic-based determination of mental state.

[0093] In a given training session with English and Mandarin depression speech corpus, this framework is intended to maximize loss on speaker identification and language identification while concurrently minimizing loss function related to detection of Major Depressive Disorder (MDD). This approach ensures the model to learn paralinguistic features in speech with high correlation with Major Depressive Disorder (MDD). The model structure is illustrated in FIGURE 1 where feature representations go through a data augmentation process and adversarial disentanglement of speaker and depression characteristics.

[0094] In at least one embodiment, robustness to limited labeled data is achieved by reusing latent representations generated by the self- supervised encoder across multiple downstream training samples through controlled permutation and concatenation of affectively distinct speech segments. This augmentation strategy increases effective sample diversity without altering the underlying acoustic distribution.

[0095] In certain embodiments, the speaker-invariant training pipeline operates as a multi- objective adversarial learning framework comprising a primary parameterized network branch and a plurality of secondary parameterized network branches. The primary branch is configured to perform a paralinguistic classification task, such as detection of an affective or psychomotor state, based on pooled representation vectors generated by the foundation model. Eachsecondary or tertiary branch is configured to predict a respective nuisance attribute, such as speaker identity, gender, or language origin, from the same or related pooled representation vectors.

[0096] In such embodiments, a gradient inversion pathway or gradient reversal layer is interposed between the foundation model and each nuisance-attribute prediction branch. During backpropagation, gradients propagated from the nuisanceattribute branches toward the foundation model are inverted or sign-reversed prior to updating parameters of the foundation model. As a result, the foundation model is penalized for encoding features that are predictive of speaker identity, gender, language origin, or other nuisance attributes, thereby encouraging the emergence of invariant paralinguistic representations.

[0097] In embodiments directed to language invariance, the nuisance-attribute prediction branch associated with language origin is configured to classify language-dependent acoustic patterns present in the pooled representation vectors. The gradient inversion pathway forces the foundation model to discard or attenuate language- specific phonetic, prosodic, or phonological characteristics that enable language identification, while retaining languageagnostic temporal and spectral modulation patterns. The disclosed system does not rely on textual transcriptions or lexical tokens, and language invariance is achieved at the level of acoustic and paralinguistic representations rather than semantic or lexical content.

[0098] In some embodiments, the paralinguistic state indicator corresponds to a depression-related state characterized by psychomotor retardation, reduced speech energy, altered cadence, or abnormal temporal modulation patterns. Such indicators may be derived from speech without reliance on semantic content, enabling objective assessment across speakers and languages. Theprimary classification branch may be trained to detect such paralinguistic abnormalities while remaining invariant to speaker- specific and languagespecific characteristics.

[0099] In at least one embodiment, speaker-invariant learning is enforced by introducing a gradient-reversal or adversarial loss pathway that penalizes accurate speaker classification. During optimization, gradients associated with speaker prediction (109) are inverted prior to backpropagation, thereby discouraging the encoder from encoding speaker-identifying features.

[0100] By coupling this adversarial pathway with a depression-classification pathway, the system is computationally constrained to encode paralinguistic speech features that are predictive of mental- state indicators while remaining invariant to speaker identity and language.

[0101] In one aspect, the invention provides a speech-processing system configured to acquire speech signals across multiple interaction paradigms, including structured phonation tasks and unstructured conversational speech, and across multiple affective conditions.

[0102] In another aspect, the system employs a multi-stage representation pipeline that normalizes variable-length speech segments using temporal pooling, followed by adversarial optimization to suppress speaker-identifying features while preserving paralinguistic temporal and spectral characteristics.

[0103] In yet another aspect, the system generates a continuous-valued mental-state indicator representing a degree of deviation in paralinguistic speech patterns relative to a reference distribution, without producing a medical diagnosis or treatment recommendation.FIGURE 1 is an illustration of the proposed system of this invention.

[0104] Input raw audio segment (101) is fed into foundation model (102) and extracted feature representations are fed into a temporal pooling layer (103) to have a common output vector dimension. The non augmented samples are fed to a speaker classification layer (105) and the augmented features are fed to a depression classification layer (110) where the total loss (107) is a combination of a speaker loss and MDD loss.

[0105] In some embodiments, the raw audio segments used for training and evaluation are sourced from a plurality of distinct speech corpora. Such corpora may comprise speech recordings originating from multiple speakers and multiple languages, including but not limited to English, Mandarin Chinese, and other natural languages. The use of multilingual corpora enables the foundation model to leam paralinguistic features that generalize across language boundaries and are not dependent on language-specific phonetic inventories or linguistic content. The raw audio segments may be segmented into temporally distinct speech portions prior to embedding generation, and such segments may originate from common or different recording sessions.

[0106] In at least an embodiment, an input module (101) is configured such that input raw audio segment / s are fed into a foundation model (102) and extracted feature representations are fed into a temporal pooling layer (103) to have a common output vector dimension. From the extracted feature representations, non- augmented features are fed to a speaker classification layer (105) and augmented features are fed to a depression classification layer (110) where total loss (107) is a combination of a speaker loss and Major Depressive Disorder (MDD) loss.In at least one embodiment, speech data are acquired using a plurality of interaction paradigms, including:

[0107] (i) structured phonation tasks involving standardized textual or phonemic prompts; and

[0108] (ii) unstructured speech tasks involving spontaneous conversational responses.

[0109] Speech signals may be acquired under differing emotional valence conditions using predefined stimuli to induce affective variation, thereby increasing acoustic diversity.

[0110] Recordings may be captured using consumer-grade mobile devices, including smartphones and tablets, to introduce realistic variability in microphone characteristics, sampling rates, and background noise.

[0111] In at least one embodiment, speech segments of variable duration are normalized using adaptive temporal pooling (103). Empirical analysis indicates that pooling speech segments exceeding a predefined temporal threshold yields stable paralinguistic feature representations, thereby reducing variance attributable to short-term speech fluctuations.

[0112] In one implementation, a minimum effective speech duration is enforced prior to feature aggregation to ensure capture of temporally stable acoustic markers. This converts your “80 seconds” finding into generalizable enablement, not a hard limitation.

[0113] In one embodiment, acoustic features are extracted using a predefined feature extraction toolkit and grouped into spectral, prosodic, and glottal categories. These features may be statistically aggregated at a speaker level to generate acompact representation.

[0114] Feature attribution analysis may be applied to identify features exhibiting high discriminative contribution, particularly those associated with temporal energy distribution, pitch variability, and voicing stability.

[0115] In a preferred embodiment, raw speech signals are processed using a selfsupervised speech encoder to generate frame-level latent representations capturing spectral and articulatory dynamics.

[0116] The representations are temporally pooled (103) and optionally concatenated (103) with auxiliary embeddings encoding speaker-related attributes. The concatenated representation is then processed by an adversarial training pipeline comprising:

[0117] (i) a primary branch (110) optimized to generate a mental- state indicator; and

[0118] (ii) a secondary branch (105) optimized to predict speaker identity. During training, gradients associated with the speaker prediction (109) task are reversed or weighted negatively, thereby discouraging encoding of speakeridentifying features.

[0119] The pooled acoustic representation further encodes temporal modulation characteristics of the speech signal, referred to herein as cadence, as described further below as “Temporal Modulation and Cadence”.

[0120] In one embodiment, the training objective is defined as:

[0121] L_total = L_primary - k-L secondary

[0122] where,

[0123] L_primary corresponds to a loss associated with generating a mental-state indicator, andL_secondary corresponds to a loss associated with predicting speaker identity.

[0124] The weighting parameter X controls the degree of speaker invariance enforced during training.

[0125] In at least one embodiment, system performance is evaluated across demographic subgroups to verify invariance to speaker attributes such as gender or age. Representation fairness is assessed by measuring disparity ratios across subgroups.

[0126] In further embodiments, model performance is evaluated across different interaction paradigms and affective conditions to verify that representation learning remains stable irrespective of emotional stimulus or speech content.

[0127] In one embodiment, the system outputs a continuous-valued score representing the degree of deviation of extracted speech features from a reference distribution. This score may be correlated with externally provided reference scales during training but is generated solely based on speech-derived features during inference.

[0128] Acoustic Signal Acquisition and Segment Transformation

[0129] As illustrated in FIG.1, the system receives a set of raw audio segments (XI, X2,.. ..X_n) captured from a portable electronic device (101). In a preferred embodiment, these segments include 13 unique voice responses captured across three emotional valences (positive, neutral, and negative).

[0130] Foundation Model (102) and Feature Extraction Subsystem

[0131] The raw audio (101) is processed through a foundation model (102), specifically a large-scale pretrained speech representation model such as Whisper-large- v2.• Encoder Configuration: The model utilizes 12 transformer blocks (104) to extract high-dimensional acoustic embeddings.

[0132] • Temporal Resolution: Signals are processed using a sliding window of 25 milliseconds with a 10 millisecond stride, resulting in up to 3000 frames per input.

[0133] • Vector Dimension: Each embedding is transformed into a hidden size of 1280 (or 768 in alternative embodiments).

[0134] Temporal Pooling and Gender Integration (103)

[0135] To address variable audio lengths, an Adaptive Average Pooling Layer transforms extracted embeddings into a uniform vector dimension. Parallel to this, gender-related embeddings (extracted via a fine-tuned Wav2Vec 2.0 model) are concatenated with the primary features to neutralize gender as a confounding factor in paralinguistic analysis.

[0136] TEMPORAL MODULATION AND CADENCE:

[0137] As used herein, the term “cadence” refers to a computationally derived temporal modulation characteristic of a speech signal, expressed as a component of a multi-dimensional acoustic representation, and not as a linguistic, semantic, or perceptual attribute.

[0138] In particular, cadence is obtained from relative timing relationships among successive speech frames generated from a time-domain audio waveform, and reflects variations in energy distribution, spectral transition density, and inter-frame temporal spacing across a sequence of frames.

[0139] Cadence, as defined herein, captures rhythmic modulation patterns that emerge from the temporal arrangement of voiced and unvoiced segments, pauses, and transitional regions in the speech signal.

[0140] In one embodiment, cadence is derived by the foundation model (102) as part of a frame-level embedding matrix, wherein each embedding encodeslocalized spectral and temporal information corresponding to a fixed-duration frame of the speech waveform. The cadence component is subsequently formed by a temporal pooling operation (103) that aggregates the frame-level embeddings over time, such that relative temporal modulation patterns across frames are retained in the pooled representation vector.

[0141] Cadence is therefore represented as a latent temporal modulation component embedded within the pooled acoustic representation, and is distinguishable from static spectral features in that it encodes relationships between successive frames rather than isolated frame characteristics.

[0142] Importantly, cadence as used in this disclosure does not correspond to speaking rate, prosody labels, stress patterns, or linguistic rhythm, and is not manually annotated or inferred through human judgment. Instead, cadence is generated automatically through signal transformation operations performed by interconnected computational modules, and exists solely as a machine-interpretable representation derived from the speech waveform.

[0143] In certain embodiments, cadence is preserved through augmentation and permutation operations applied to the pooled representation, such that temporal modulation characteristics remain invariant to speaker identity or recording conditions, while remaining discriminative with respect to paralinguistic state inference.

[0144] Adversarial Disentanglement and Training Pipeline (107)

[0145] As shown in Figure 1, the system splits into two specific classification branches to achieve speaker invariance:

[0146] 1. Speaker Classifier (105) Branch: Utilizes linear layers to predict speaker identity from non-concatenated features.

[0147] 2. Depression Classifier (110) Branch: Utilizes two transformer encoder layers (128-dimensional hidden state, 4 attention heads) to predict MDD from concatenated, shuffled features.Technical Transformation (Combined Loss):

[0148] The system optimizes the model parameters using a combined adversarial loss (107) function:

[0149] Ltotal — Ld * Ls

[0150] Where La is the cross-entropy loss for determining mental-state, Lsis the speaker prediction loss, and X (initial value 1 x 10-3) is a hyperparameter governing the contribution of speaker-loss. This adversarial configuration forces the model to actively discard information related to speaker- specific features.

[0151] Sub-Question Shuffling and Data Augmentation (106)

[0152] To overcome the technical problem of data scarcity, the system implements a sub-question Shuffling technique (106). Features are randomly shuffled and permuted until the sample sizes for "depressed" and "healthy" classes are balanced, ensuring the classifier does not overfit to majority class distribution.

[0153] The system was validated across English (DAIC-WOZ) and Mandarin (Oizys) datasets, achieving a state-of-the-art Fl-avg score of 0.83 and AUC of 0.88. The technical effect is the objective detection of psychomotor retardation — indicated by spectral-temporal energy changes in mid-frequency bands, independent of the speaker’s language, age, or identity.

[0154] Output Generation

[0155] The system outputs a continuous-valued paralinguistic deviation score representing divergence from a reference acoustic distribution.

[0156] The output is generated solely from speech-derived features

[0157] In at least an embodiment, a variety of datasets are used, the datasets comprising, but not limited to, DAIC-WOZ Dataset, and Oizys Dataset.Speech based depression detection systems are often benchmarked using the performance on The Distress analysis interview corpus wizard of Oz (DAICWOZ), the dataset contains 189 participants. During the data collection process each participant completed patient health questioner (PHQ-8) and assigned a depression score form a self-rating index. The Dataset consist 100 speakers used for training and 30 speakers used for evaluation containing 58 hours of combined speech sampled at 16kHz.

[0158] Oizys is a Mandarin Chinese speech corpus collected by Voice Health Tech annotated partnering with Peking University Sixth Hospital, The Fifth people's Hospital of Tangshan, and Weihai Mental Health Center etc. Unlike DAIC-WOZ dataset Oizys is a much diverse dataset containing 45,552 audio samples consists of 3343 unique speakers for training and evaluation as well as 161 speakers for independent validation. During the interview process the participants were diagnosed by a license psychiatrist following DSM-5 process and conclude if the participant was depressed or healthy, depression severity was estimated using Hamilton Depression Rating Scale (HAM-D) method. The participants were asked to use their own mobile device record answers for 13 uniquely designed questions complied according to the privacy and ethical standard capturing a wide range of acoustic properties in speech. Oizys is the only dataset in the world that has the scale and diversity on the voice protocols used to capture data containing Diadochokinetic tasks, sustained vowel, stroop test as well as open speech capturing data from ages ranging from 18-65-year-old participants with an average age of 34 where 64.8% of whom are female and 35.2% of whom are male.

[0159] In at least an embodiment, the system and method of this invention proposes data processing modules, feature extraction modules, and augmentation modules as used here.In accordance with the input module (101), the system and method, of this invention, takes raw audio segments as inputs. Using the Speech-Based Depression Detection (SDD) model, as delineated in Figure 1, audio representation features are extracted from the raw audio segments which are input through the input module (101). During an initialisation stage, the system accepts a unique voice sample from a recording session denoted as X, that consists of 13 unique voice samples from a dataset (preferably, the Oyisiz dataset). This sample is an aggregation of 13 individual voice responses, formulated as a result of a participants answering a set of questions. Hence, this sample is represented as X = {x1, , x13}. A foundation model (102) takes in the raw audio segment / s (101), and gives out an embedding matrix.

[0160] Given Fi frame length and Fsframe stride and Fcdenote the frame count.

[0161] The shape of embedding matrix is [Fc, D], the relationship between frame count for a given signal with a duration dFc = + 1].

[0162]

[0163] According to a non-limiting exemplary embodoment, the embedding dimension D is 768 and the convolution stride Fi is 20ms and Fsis 320ms. The raw audio segments (101), after going through the foundation model (101), are fed in an adaptive average pooling layer to get the Fc to be 1 to let all the outputs to have a same dimensional output vector. This forms the output features.

[0164] In at least an embodiment, the output features are fed into a speaker invariant training pipeline (108) uniform speaker with a combined loss function (107) forcing the system and method, of this invention, to learn paralinguistic related features disentangling the model from learning speaker related features.

[0165] Uniform speaker disentanglement (USD) minimizes prediction loss for a primary task of depression prediction (111) and simultaneously maximizes lossof a secondary task of speaker prediction (109).

[0166]

[0167] In the calculated total loss, the system and method, of this invention, combines cross entropy loss for speaker prediction (109), multiplied with X, where lambda is a hyperparameter governing the contribution of speaker loss, with le-3 being selected as the initial hyper parameter value. In the formulation of the detailed equation, d denotes depression detection, s denotes speaker diarization, and CE denotes Cross Entropy Loss.

[0168] Ld= CE(0dX,yd) . (2)

[0169] In the equation 2, Ld denotes the Cross Entropy Loss for depression detection, with 0d representing parameters of the depression detection model, X representing the input features, and yd representing the true labels for depression.

[0170] Ls= CE(6sX,ys) . (3)

[0171] Similarly, in the equation 3 Lsdenotes the Cross Entropy Loss for speaker diarization, with 0Srepresenting the parameters of the speaker diarization, X the input features, and ysthe true labels for speaker recognition.

[0172] >

[0173]

[0174] The equation 5 computes the gradient of the total loss with respect to the parameters of the depression detection model.

[0175]

[0176] Finally, the parameters d of the depression detection model are updated using a learning rate a and the calculated gradient from equation 5. Thisupdate aims to minimize loss for depression detection while maximizing loss for speaker diarization; thereby, achieving a desired balance between the primary task and the secondary task. By leveraging this set of equations, the optimization step formulates a robust approach to simultaneously address the objectives of accurate depression prediction (111) and effective speaker diarization. The inclusion of the trade-off hyperparameter X offers flexibility in controlling the balance between the two competing objectives, catering to varying requirements and scenarios in practical applications.

[0177] The system and method, of this invention, is configured such that it compels the model to prioritize depression-discriminative data related to paralinguistic features while disregarding certain speaker- specific details; thereby, ensuring the model remains unaffected by variations in individual speaker characteristics.

[0178] Referring to Figure 1, the augmented output features, from the foundation model (101), are reused for the primary task of depression prediction (111). The aforementioned Oizys dataset consists of 13 questions with each question exploring different affective state being Neutral, Negative, and Positive. Even though, ‘depression’ is modeled as a binary classification problem, depression inherently contains samples with different severity levels. Different voice tasks, for example read speech vs spontaneous speech detecting in the detection of depression, have varying impacts on diverse age groups. Therefore, in order to balance the depressed and healthy sample count an alternative strategy is used where different permutations of samples from X with different affective states from the depressed class is used as an augmentation till the total sample size between the depressed and healthy class is equal.

[0179] The permuted feature representation arrays from X are concatenated togetherand passed through to a depression detection classifier (110) configuration of two transformer encoder layers, each operating in a 128-dimensional space and utilizing four attention heads to effectively manage sequence information, These layers are paired with a full-connected layer to facilitate the final classification output.

[0180] According to non-limiting exemplary embodiments, experiments were conducted using the aforementioned DAIC-WOZ datasets and Oizys datasets.

[0181] In order to create a balanced representation of the dataset a sub-data split was created Oizys_train, Oizys_validation, and Oizys_test. For the English dataset, DAIC- WOZ with 189 samples containing train, dev and test set, train set was used in training while the dev set was used to evaluate the model final performance.

[0182] Detailed information on the training and validation set size is represented in the Table 1.

[0183]

[0184] Table 1: Details about the datasets used for experiments.

[0185] The DAIC-WOZ dataset contains audio recordings of the participant and the interviewer’s speech. The speech segments related to the participants were extracted using the time-stamps provided with the corpus. Hence speech segments per speaker represented as array. XDW = {x1, ... , x"} the speech segments varies from length and contains free form speech.

[0186] Using the DAIC-WOZ datasets in the non-limiting exemplary embodiment, the system and method, of this invention, was trained using raw-audio features as input features, DAIC-WOZ dataset contains 50+ hours of audio samples across 189 participants however depressed participants represent 28% while healthy samples represent 72% in the total train set. Data processing and augmentation technique was applied to address the data imbalance issue. The speech segments per participant were randomly shuffled and new permutations were obtained as an augmentation strategy to balance the total depressed and healthy sample count. The permutations of the samples were generated preserving the order of the sequence to ensure the flow of speech. The extracted foundation model (101) features were channeled through an adaptive pooling layer to achieve a standardized, common as delineated in the speaker invariant training framework depicted in Figure 1. The output features concatenated by speaker and passed along to depression classifier while non-concatenated features used in the speaker classifier (105) yielded in optimal Fl scores.

[0187] Using the Oizys datasets in the non-limiting exemplary embodiment, the system and method, of this invention, was trained with similar architecture with slight variation in the data augmentation technique, Unlike the DAIC-WOZ dataset, Oizys datasets consist of 13 responses per speaker which have an affective state of expression being positive negative and neutral. Oizys_train consists of 8429depressed samples and 8762 healthy samples, In order to preserve the original variation of the emotional responses the question order was preserved while premutation process. The Extracted Foundation models (101) were later channeled through the speaker invariant training pipeline as depicted in Figure 1.

[0188] Depression classification, when modeled as a binary classification problem, is often evaluated using the Fl score which is the harmonic mean of the precision and recall. However, in clinical settings, additional metrics are often used to evaluate the algorithm performance.

[0189] Given that,

[0190] Tp: True Positives,

[0191] Tn : True Negatives,

[0192] Fp: False positives,

[0193] Fn: False negatives;

[0194] Sensitivity of the model is defined as a True positive rate or recall Sensitivity =Tp.

[0195] Tp+Fn

[0196] Another important metric is the True negative rate

[0197] Tn

[0198] Specificity = - Tn + Fp

[0199] In this context the system and method, of this invention, defines Precession as:

[0200] P nrecesi ■on = -TP .

[0201] Tp+Fp

[0202] The AUC is a metric that evaluates the performance of a binary classification system as its discrimination threshold is varied. It represents the probability that a randomly selected positive instance will rank higher than a randomly selected negative instance. Mathematically, AUC is the integral of the ROC curve. The ROC curve plots Sensitivity (or True Positive Rate) against 1-Specificity (orFalse Positive Rate) for various threshold values. The AUC will be a value between 0 and 1, with 1 indicating a perfect classifier and 0.5 indicating a classifier that is no better than random chance.

[0203] Baseline results are shown in the Table 2.

[0204]

[0205] Table 2: Depression detection performance on Oiz.ys, DAIC-WOZ datasets, baselines is evaluated based on Fl-AVG, Fl -MAX, Sensitivity, Specificity.

[0206] Baseline Models without augmentations.

[0207] Benchmark methods, on depression classification evaluated as part of the literature review, are summarized in the Table 3.

[0208]

[0209]

[0210] Table 3: Depression detection performance on Oiz.ys, DAIC-WOZ with speaker invariant training pipeline, datasets are evaluated based on Fl -AVG, Fl -MAX, Sensitivity, Specificity.

[0211] The effect on data augmentation for mixed model is presented in the section Table 4.

[0212]

[0213] Table 4: Performance in detecting depression across various models and SOT A baselines is evaluated based on Fl-AVG, Fl (ND), F1(D) , utilizing the DAIC-WoZ dataset. The SOTA baseline results are sourced from either reproduced values or directly reported from the relevant studies.Comparison of the methods are only performed on the DIAC — WOZdevset using Fl Scores and additionally to Fl scores sensitivity, specificity and AUC is recorded for each experiment.

[0214] For creating the baseline model results were generated by channeling the selfsupervised representations through a depression classifier model which consist of a fully connected layer. No augmentations were applied to the raw data therefore it’s evident to result in unbalanced performance in metrics

[0215] Comparing to the baseline model where only the self- supervised representation are used for training results, the invention’s system can observe an overall a +10% improvement in the external independent test set denoted as Oizystestwhile Trained using the speaker invariant training pipeline.

[0216] In the Oizys_test dataset, the invention’s system observed a marked enhancement in the performance metrics. The sensitivity exhibited a notable increase +0.10, while the specificity showed a commendable rise of +0.14. Additionally, the AUC score demonstrated a significant gain of +0.10. These findings underscore the efficacy of the modifications implemented in the model, emphasizing its potential in depression detection tasks.

[0217] When elevating depression detection performance on various models and the SOTA baseline. The results are based on Fl-Avg and F1(D) and Fl(ND) for DAIC-WOZ dataset. The DepAudioNet model considered to be the baseline model in MDD classification. The model is trained using Mel- spectrograms and archive 0.6 Fl-Avg and also it is observed that F1(D) and F1(H) is imbalanced due to the nature of the dataset.

[0218] In the recent publications use of self- supervised representations extracted fromraw audio file have gained significant attention as they are more robust and have better generalization capabilities compared to traditional hand engineered features such as mel- spectrograms and formants.

[0219] In the pilot experiments comparing the performance on DAIC-WOZ with other implementations, the invention’s system has been able to gain +10% improvement in Fl scores when the concatenated self-supervised features are channeled through a speaker invariant training process. It's also significant that the SOTA methods struggles when maintaining a balanced representation of both F1(D) and Fl(ND) metrices. This is dues to the highly unbalanced nature of the dataset. This can be overcome augmentation techniques where the sample counts between depressed and non- depressed sample counts are balanced.

[0220] Figure 2 is an illustration of the Self supervised representations for DAIC-WOZ and Oizys datasets, the figure shows the embedding space for the 2 corpuses share significant differences.

[0221] DAIC-WOZ and Oizys are fundamentally different corpuses. When both models are trained on the datasets individually shows excellent generalization ability while evaluating the evaluation metrics.

[0222] In order to further analyze the generalization ability in a mixed language setting Oyzis_train and DAIC-WOZ_train datasets the system shuffled together and independently evaluated. The self- supervised representations extracted from the foundation models were fed into a program to visualize the embedding space. Embedding analysis is vital for understanding the behavior of classifiers. Embedding point clouds generated via Uniform Manifold Approximation and Projection (UMAP) highlights the acoustic featuredifferences between the datasets highlighted in the Figure 2. Further experiments are required to evaluate and map the acoustic differences between the corpuses.

[0223] The TECHNICAL ADVANCEMENT, of this invention, lies in providing systems and methods which provision speech-based state detection from invariant training using self- supervised representations to maximize the contribution of para-linguistic features in depression diagnosis. The model performance evaluated on English and Mandarin Chinese corpuses signifies the deep learning method used can be applied to fundamentally different language corpuses when detecting MDD using speech. This invention’s approach, trained in an adversarial manner, surpasses the baseline model performance by +10% archiving SOTA on publicly available DAIC-WOZ dataset with only using raw audio acoustic speech form. The data augmentation and concatenating the features by speaker represent more information regarding psychomotor retardation compared to random sampling approach applied in DepAudioNet experiments.

[0224] While this detailed description has disclosed certain specific embodiments for illustrative purposes, various modifications will be apparent to those skilled in the art which do not constitute departures from the spirit and scope of the invention as defined in the following claims, and it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation.

Claims

CLAIMS,1. A speech signal processing system for generating a paralinguistic state indicator, the system comprising:- an input module (101) arranged to receive one or more raw audio segments captured as time-domain speech waveforms;- a foundation model (102) operatively coupled to the input module (101), the foundation model comprising a plurality of transformer encoder blocks (104), wherein the foundation model transforms each raw audio segment into a frame-level embedding matrix by applying a fixed frame length and frame stride to the raw audio segment;- a temporal pooling layer (103) operatively coupled to an output of the foundation model (102), wherein the temporal pooling layer aggregates the frame-level embedding matrix into a fixed-dimension representation vector that is independent of a duration of the raw audio segment, the fixed- dimension representation vector encoding cadence as a temporal modulation component derived from relative timing relationships among successive frame-level embeddings;- an augmentation and permutation module (106) operatively coupled to the temporal pooling layer (103), wherein the augmentation and permutation module generates additional representation vectors by reordering and concatenating pooled representation vectors corresponding to affectively distinct speech segments originating from a common recording session; - a speaker-invariant training pipeline (108) operatively coupled to the temporal pooling layer (103) and the augmentation and permutation module (106), the speaker-invariant training pipeline comprising:(a) a primary classification branch (110) receiving the concatenated representation vectors generated by the augmentation and permutation module (106); and(b) a secondary parameterized network branch (105) receiving pooled representation vectors that have not undergone concatenation, the secondary parameterized network branch being configured to predict at least one nuisance attribute selected from speaker identity, gender, language, or combinations thereof;- a gradient inversion pathway (109) interposed between the secondary parameterized network branch (105) and the foundation model (102), wherein gradients associated with prediction of the at least one nuisance attribute are inverted prior to updating parameters of the foundation model (102); and- a loss aggregation module (107) operatively coupled to the primary classification branch (110) and the secondary classification branch (105), wherein the loss aggregation module combines:(i) a first loss value generated by the primary classification branch (110);and(ii) a second loss value associated with prediction of the at least one nuisance attribute by the secondary parameterized network branch (105),in such a manner that the first loss value contributes positively to parameter updates of the foundation model (102), and the second loss value contributes negatively to the parameter updates of the foundation model (102), wherein the foundation model (102) is trained under the combined influence of the first and second loss values such that the pooled representation vectors propagated to the primary parameterized network branch (110) suppressinformation predictive of at least one nuisance attribute while retaining paralinguistic temporal, spectral, and cadence-related modulation components derived from the raw audio segments.

2. The system as claimed in claim 1 , wherein the frame-level embedding matrix comprises a plurality of embedding vectors each corresponding to a fixed-duration time frame defined by the fixed frame length and frame stride.

3. The system as claimed in claim 1, wherein the cadence encoded in the fixed- dimension representation vector corresponds to temporal modulation information represented by inter-frame timing relationships among successive embedding vectors within the frame-level embedding matrix.

4. The system as claimed in claim 1, wherein the temporal pooling layer (103) comprises a pooling structure selected from a weighted aggregation structure, an attention-based aggregation structure, or a hierarchical aggregation structure.

5. The system as claimed in claim 1, wherein the augmentation and permutation module (106) includes a concatenation structure that concatenates pooled representation vectors originating from temporally separated segments of the common recording session.

6. The system as claimed in claim 1 , wherein the augmentation and permutation module (106) further includes a reordering structure that alters an ordering of pooled representation vectors prior to concatenation.

7. The system as claimed in claim 1, wherein the speaker-invariant training pipeline (108) comprises a shared parameter interface through which the primary classification branch (110) and the secondary classification branch (105) propagate parameter update signals to the foundation model (102).

8. The system as claimed in claim 1, wherein the gradient inversion pathway (109) comprises an inversion layer positioned between the secondary classification branch (105) and the foundation model (102) such that gradients propagated through the inversion layer are sign-reversed.

9. The system as claimed in claim 1, wherein the loss aggregation module (107) comprises a summation structure that algebraically combines the first loss value and the second loss value prior to propagation toward the foundation model (102).

10. The system as claimed in claim 1, wherein the cadence remains embedded within the fixed-dimension representation vector during propagation through the augmentation and permutation module (106) and the speaker-invariant training pipeline (108).

11. The system as claimed in claim 1, wherein the secondary parameterized network branch (105) is extended to comprise a tertiary classification branch configured to identify a language origin of the raw audio segments, such that the gradient inversion pathway (109) inverts parameter updates propagated from both the secondary and tertiary branches.

12. The system as claimed in claim 11, wherein the plurality of distinct language corpuses comprises at least an English language corpus and a Mandarin Chinese language corpus, and wherein the foundation model (102) learns vocal biomarkers that are robust across said distinct languages.

13. The system as claimed in claim 11, wherein the tertiary classification branch generates a language identification loss by attempting to classify the language origin of the embedding matrix, and wherein the gradient inversion pathway (109) forces the foundation model (102) to discard languagespecific phonetic information required for said classification.

14. The system as claimed in claim 11, wherein feature representations generated by the foundation model (102) are derived from initial encoder layers of the transformer encoder blocks (104) to ensure proximity to fundamental vocal features that are inherently language-neutral.

15. A computer-implemented method for generating a paralinguistic state indicator from a speech signal, the method comprising:- receiving one or more raw audio segments as time-domain speech waveforms;- transforming each raw audio segment, using a foundation model comprising a plurality of transformer encoder blocks, into a frame-level embedding matrix by applying a fixed frame length and frame stride to the raw audio segment;- aggregating the frame-level embedding matrix using a temporal pooling operation to generate a fixed-dimension representation vector independent of a duration of the raw audio segment, wherein the fixed-dimension representation vector encodes cadence as a temporal modulation component derived from relative timing relationships among successive frame-level embeddings;- generating additional representation vectors by reordering and concatenating pooled representation vectors corresponding to affectively distinct speech segments originating from a common recording session; - propagating concatenated representation vectors to a primary parameterized network branch and non-concatenated pooled representation vectors to a secondary parameterized network branch configured to predict at least one nuisance attribute selected from speaker identity, gender, language, or combinations thereof;- inverting parameter update gradients propagated from the secondary classification branch toward the foundation model; and- combining a first loss value associated with the primary classification branch and a second loss value associated with the secondary classification branch to update parameters of the foundation model.

16. The method as claimed in claim 15, wherein each frame-level embedding corresponds to a fixed-duration segment of the speech waveform defined by the fixed frame length.

17. The method as claimed in claim 15, wherein aggregating the frame-level embedding matrix comprises applying a pooling structure selected from weighted pooling, attention-based pooling, or hierarchical pooling.

18. The method as claimed in claim 15, wherein reordering pooled representation vectors comprises altering a sequence order of the pooled representation vectors prior to concatenation.

19. The method as claimed in claim 15, wherein concatenating pooled representation vectors comprises concatenating representation vectors derived from temporally separated speech segments within the common recording session.

20. The method as claimed in claim 15, wherein inverting parameter update gradients comprises propagating gradients through an inversion layer positioned between the secondary classification branch and the foundation model.

21. The method as claimed in claim 15, wherein cadence remains embedded within the fixed-dimension representation vector throughout aggregation, reordering, concatenation, and gradient inversion.

22. The method as claimed in claim 15, wherein the secondary parameterized network branch is extended to comprise a tertiary classification branch configured to identify a language origin of the raw audio segments, and wherein parameter update gradients propagated from both the secondary and tertiary branches are inverted prior to updating parameters of the foundation model (102).

23. The method as claimed in claim 22, wherein the raw audio segments represent diverse speakers and at least two distinct languages, and wherein the foundation model (102) generates latent speech representations thatencode paralinguistic abnormalities associated with depression irrespective of the speaker’s voice or language spoken.

24. The method as claimed in claim 22, wherein maximizing the prediction loss associated with the tertiary language identification task comprises propagating gradients through a gradient inversion pathway to penalize the foundation model (102) for correctly identifying the language origin.

25. The method as claimed in claim 22, further comprising augmenting the latent speech representations by concatenating representations of affectively distinct speech segments to balance sample counts between depressed and healthy classes across the distinct languages.

Images