Ataxia dysarthria voice airflow synchronous analysis and training system and method

Abstract

The invention discloses an ataxia dysarthria voice and airflow synchronous analysis and training system and method, and belongs to the technical field of medical rehabilitation, and the system comprises a data collection module, a signal preprocessing module, a voice and airflow synchronization analysis module, a personalized training generation module and a training feedback and evaluation module. Based on a differential geometry principle, a Riemannian manifold theory is innovatively introduced, voice and air flow signals are respectively mapped to different Riemannian manifold spaces, accurate time sequence alignment of the voice-air flow signals is realized by constructing a geodesic flow field, a double-flow network structure is designed to extract voice-air flow synchronization features, and the accuracy of voice-air flow synchronization is improved. According to the method, an accurate dysarthria evaluation result is generated, the system generates a personalized training scheme according to the evaluation result, a training closed loop is formed through real-time feedback and dynamic adjustment, accurate evaluation and effective training of ataxia dysarthria are achieved, and compared with a traditional method, the accuracy of dysarthria analysis is improved by about 35%, and the training efficiency is improved by 30%.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation technology, specifically to a system and method for synchronous analysis and training of speech airflow in ataxia-associated articulation disorders based on the principles of differential geometry. Background Technology

[0002] Ataxia is a neurological disorder caused by lesions in the cerebellum or its conduction pathways, primarily manifesting as impaired motor coordination. Patients with ataxia often also experience dysarthria, characterized by unclear speech, unstable speech rate, and abnormal pitch, severely impacting their daily communication and quality of life.

[0003] Currently, rehabilitation training for ataxia-related dysarthria mainly relies on the subjective experience assessment of speech therapists and traditional speech training methods. These methods suffer from problems such as inconsistent assessment standards, difficulty in quantifying training effects, and insufficient personalization. Some existing electronic training systems primarily focus on the analysis of speech signals, neglecting the significant impact of oropharyngeal airflow and speech coordination on articulation, thus failing to accurately locate and effectively improve dysarthria in ataxia patients.

[0004] Traditional methods for analyzing articulation disorders typically employ linear analysis models in Euclidean space, which struggle to accurately represent the nonlinear characteristics of speech and airflow signals. Furthermore, existing systems lack sufficient temporal correlation analysis of speech and airflow signals, failing to precisely capture the temporal coordination issues encountered by ataxia patients during articulation. Moreover, current training methods lack fine-grained adaptation to individual differences, resulting in limited training effectiveness.

[0005] Therefore, there is an urgent need for a system that can accurately analyze the speech-airflow synchronization relationship and provide personalized training programs to help ataxia patients effectively improve their articulation disorders. Summary of the Invention

[0006] The purpose of this invention is to provide a system and method for synchronous analysis and training of speech and airflow in ataxia-induced articulation disorders. Based on the principles of differential geometry, it achieves synchronous analysis of speech and airflow signals, accurately assesses the characteristics of articulation disorders, generates personalized training programs, and effectively improves the articulation ability of ataxia patients.

[0007] This invention proposes a speech airflow synchronization analysis and training system for ataxia-associated articulation disorders, comprising:

[0008] The data acquisition module is used to collect the patient's voice signals and airflow signals from the mouth and nose;

[0009] The signal preprocessing module is electrically connected to the data acquisition module and is used to perform noise reduction and feature extraction on the speech signal and the nasal airflow signal to obtain speech feature vector and airflow feature vector.

[0010] The speech airflow synchronization analysis module, electrically connected to the signal preprocessing module, is used for:

[0011] The speech feature vectors are mapped to the first Riemannian manifold space to form a speech manifold representation;

[0012] The airflow feature vector is mapped to the second Riemannian manifold space to form an airflow manifold representation;

[0013] Construct a geodesic flow field between the first Riemannian manifold space and the second Riemannian manifold space to achieve temporal alignment between the speech manifold representation and the airflow manifold representation;

[0014] Based on the temporally aligned speech manifold representation and the airflow manifold representation, speech-airflow synchronization features are extracted through a two-stream network structure to generate speech-airflow synchronization evaluation results.

[0015] The personalized training generation module is electrically connected to the speech-airflow synchronicity analysis module and is used to generate a personalized training plan for the patient's articulation disorder characteristics based on the speech-airflow synchronicity assessment results.

[0016] The training feedback and evaluation module is electrically connected to the personalized training generation module and the data acquisition module. It is used to collect training data in real time during the patient's execution of the personalized training plan, evaluate the training effect, and feed back the training results to the personalized training generation module for dynamic adjustment of the personalized training plan.

[0017] Preferably, the data acquisition module includes:

[0018] A voice acquisition unit is used to acquire the patient's voice signals through a microphone array;

[0019] An airflow acquisition unit is used to acquire airflow signals from the patient's mouth and nose via a separate mouth and nose airflow sensor.

[0020] The synchronization control unit is electrically connected to the voice acquisition unit and the airflow acquisition unit, and is used to synchronize the sampling clocks of the voice signal and the nasal airflow signal to ensure the time consistency of the acquired data.

[0021] Preferably, the signal preprocessing module includes:

[0022] The speech preprocessing unit is used to perform noise reduction, segmentation and feature extraction on the speech signal to generate the speech feature vector, wherein the speech feature vector includes Mel frequency cepstral coefficients, fundamental frequency, speech intensity and spectral centroid parameters;

[0023] An airflow preprocessing unit is used to perform noise reduction, segmentation, and feature extraction on the oral and nasal airflow signal to generate the airflow feature vector, wherein the airflow feature vector includes airflow intensity, airflow rate change, oral and nasal airflow ratio, and airflow vibration parameters.

[0024] Preferably, the speech airflow synchronization analysis module includes:

[0025] The manifold mapping unit is used to construct the first Riemannian manifold space and the second Riemannian manifold space, and to implement the mapping of the speech feature vector and the airflow feature vector to the corresponding manifold space;

[0026] The time alignment unit, connected to the manifold mapping unit, is used to construct the geodesic flow field and solve for the optimal time alignment path along the integral curve of the geodesic flow field;

[0027] A dual-stream network unit, connected to the temporal alignment unit, is used to process the aligned manifold representation through a Riemann deep learning network and extract speech-airflow synchronization features.

[0028] A synchronous evaluation unit, connected to the dual-stream network unit, is used to generate the speech-airflow synchronization evaluation result based on the speech-airflow synchronization characteristics.

[0029] Preferably, the manifold mapping unit is specifically used for:

[0030] Construct a first Riemannian manifold space and a second Riemannian manifold space with local inner product structure;

[0031] A tangent space is constructed at each point of each manifold to represent the local direction of change of the signal;

[0032] Calculate geodesics on the manifold to characterize the optimal transition paths between signal states;

[0033] Analyze the curvature tensor of the manifold to detect anomalous articulation patterns.

[0034] Preferably, the timing alignment unit is specifically used for:

[0035] Define the energy density function on the product space of the two manifolds;

[0036] The gradient flow field of the energy function is used as a geodesic flow field;

[0037] A multi-scale time window strategy is adopted to iteratively solve for the optimal alignment path from coarse to fine;

[0038] The time window size is adaptively adjusted based on the local rate of change of the signal.

[0039] Preferably, the dual-stream network unit includes:

[0040] A speech stream network is used to extract speech features on the first Riemannian manifold;

[0041] An airflow network is used to extract airflow features on the second Riemannian manifold;

[0042] Cross-flow mapping layer is used to establish feature mapping relationships between two manifolds;

[0043] The feature fusion layer is used to fuse two-stream features into a common representation space.

[0044] Preferably, the personalized training generation module includes:

[0045] The disorder analysis unit is used to analyze the speech-airflow synchronicity assessment results and identify the type and severity of the patient's articulation disorder.

[0046] The training goal setting unit, connected to the obstacle analysis unit, is used to set short-term training goals and long-term rehabilitation goals based on the patient's articulation disorder characteristics.

[0047] The training task generation unit, connected to the training target setting unit, is used to design targeted training tasks, including training on airflow control of the lips, tongue, soft palate, and epiglottis.

[0048] The virtual guidance model generation unit, connected to the training task generation unit, is used to create a visual airflow control guidance model to assist patients in training.

[0049] Preferably, the training feedback and evaluation module includes:

[0050] A real-time monitoring unit is used to collect patients' speech and airflow data during training;

[0051] The effect evaluation unit, connected to the real-time monitoring unit, is used to calculate the manifold distance between the current articulation and the target articulation, and to evaluate the stability and accuracy of airflow control.

[0052] The feedback generation unit, connected to the effect evaluation unit, is used to generate real-time visual and auditory feedback to guide the patient to adjust pronunciation and airflow control.

[0053] The progress tracking unit, connected to the effect evaluation unit, is used to record historical training data and generate training progress reports and improvement suggestions.

[0054] A method for synchronizing speech airflow in ataxia-associated articulation disorders includes the following steps:

[0055] Collect the patient's speech and nasal / oral airflow signals;

[0056] The speech signal and the nasal airflow signal are preprocessed to extract speech feature vectors and airflow feature vectors;

[0057] The speech feature vectors are mapped to the first Riemannian manifold space to form a speech manifold representation;

[0058] The airflow feature vector is mapped to the second Riemannian manifold space to form an airflow manifold representation;

[0059] Construct a geodesic flow field between the first Riemannian manifold space and the second Riemannian manifold space to achieve temporal alignment between the speech manifold representation and the airflow manifold representation;

[0060] Based on the temporally aligned speech manifold representation and the airflow manifold representation, speech-airflow synchronization features are extracted through a two-stream network structure to generate speech-airflow synchronization evaluation results.

[0061] Based on the speech-airflow synchronicity assessment results, a personalized training program is generated targeting the patient's articulation disorder characteristics.

[0062] Guide patients to implement the personalized training program, collect training data in real time, and evaluate the training effect;

[0063] Based on the training effect evaluation results, the personalized training plan is dynamically adjusted to form a training closed loop.

[0064] The present invention has the following beneficial effects:

[0065] 1. This invention is the first to apply Riemannian manifold theory in differential geometry to speech disorder analysis. By constructing Riemannian manifold representations of speech and airflow, it accurately captures the nonlinear characteristics of the signal, improving the analysis accuracy by about 35% compared to traditional methods.

[0066] 2. This invention innovatively proposes a time-series alignment algorithm based on geodesic flow field, which solves the problem of accurate synchronization of voice-airflow signals and improves the recognition accuracy of time-series coordination problems by about 40%.

[0067] 3. This invention designs a two-stream network structure based on Riemann deep learning, which realizes the deep fusion of speech-airflow multimodal information, providing a more comprehensive and accurate assessment of articulation disorders, improving the comprehensiveness of the assessment by about 45%.

[0068] 4. Based on accurate assessment results, this invention can generate highly personalized training programs and significantly improve training efficiency through real-time feedback and dynamic adjustment mechanisms, reducing the average time for patients to reach the same training goals by 30%.

[0069] 5. This invention realizes a complete closed-loop system of assessment-training-feedback, providing a one-stop rehabilitation solution for ataxia and dysarthria, and greatly improving patient participation and the sustainability of training effects. Attached Figure Description

[0070] Figure 1 This is a system overall framework diagram of the present invention;

[0071] Figure 2 This is a structural diagram of the data acquisition module of the present invention;

[0072] Figure 3 This is a structural diagram of the signal preprocessing module of the present invention;

[0073] Figure 4 This is a structural diagram of the speech airflow synchronization analysis module of the present invention;

[0074] Figure 5 This is a flowchart of the manifold mapping unit of the present invention;

[0075] Figure 6 This is a flowchart of the timing alignment unit of the present invention;

[0076] Figure 7 This is a structural diagram of the dual-stream network unit of the present invention;

[0077] Figure 8 This is a structural diagram of the personalized training generation module of the present invention;

[0078] Figure 9 This is a structural diagram of the training feedback and evaluation module of the present invention;

[0079] Figure 10 This is a flowchart of the method of the present invention. Detailed Implementation

[0080] Please refer to Figure 1 - Figure 10 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0081] like Figure 1 As shown, the ataxia-associated speech disorder speech airflow synchronization analysis and training system provided by the present invention includes: a data acquisition module 1, a signal preprocessing module 2, a speech airflow synchronization analysis module 3, a personalized training generation module 4, and a training feedback and evaluation module 5.

[0082] Data acquisition module 1 is used to acquire the patient's speech signal and nasal / oral airflow signal. Signal preprocessing module 2, electrically connected to data acquisition module 1, performs noise reduction and feature extraction on the acquired speech and nasal / oral airflow signals to obtain speech feature vectors and airflow feature vectors. Speech-airflow synchronization analysis module 3, electrically connected to signal preprocessing module 2, performs manifold mapping, temporal alignment, and synchronization feature extraction on the speech and airflow feature vectors to generate speech-airflow synchronization assessment results. Personalized training generation module 4, electrically connected to speech-airflow synchronization analysis module 3, generates personalized training plans tailored to the patient's articulation disorder characteristics based on the speech-airflow synchronization assessment results. Training feedback and evaluation module 5, electrically connected to personalized training generation module 4 and data acquisition module 1, collects training data in real time during the patient's implementation of the personalized training plan, evaluates the training effect, and provides feedback on the training results to personalized training generation module 4 for dynamic adjustment of the personalized training plan.

[0083] Preferably, such as Figure 2 As shown, the data acquisition module 1 includes a voice acquisition unit 11, an airflow acquisition unit 12, and a synchronization control unit 13. The voice acquisition unit 11 is used to acquire the patient's voice signal through a microphone array. In one embodiment of the invention, the microphone array uses an 8-channel configuration with a sampling rate of 48kHz and a dynamic range of 120dB, capable of clearly capturing the patient's voice details. The airflow acquisition unit 12 is used to acquire the patient's nasal and oral airflow signals through a separate nasal and oral airflow sensor. In a preferred embodiment of the invention, the airflow sensor has a sampling rate of 1kHz, a pressure accuracy of ±0.01kPa, and a flow rate range of 0-200L / min, capable of accurately measuring airflow changes during the patient's vocalization process. The synchronization control unit 13 is electrically connected to the voice acquisition unit 11 and the airflow acquisition unit 12, used to synchronize the sampling clocks of the voice signal and the nasal and oral airflow signals, ensuring the time consistency of the acquired data. In an embodiment of the invention, the synchronization control unit 13 employs a hardware clock synchronization mechanism to ensure that the sampling error is less than 0.1ms, laying the foundation for subsequent accurate analysis.

[0084] like Figure 3As shown, the signal preprocessing module 2 includes a speech preprocessing unit 21 and an airflow preprocessing unit 22. The speech preprocessing unit 21 performs noise reduction, segmentation, and feature extraction on the speech signal to generate a speech feature vector. The speech feature vector includes Mel-frequency cepstral coefficients, fundamental frequency, speech intensity, and spectral centroid parameters. In a specific embodiment of the invention, the Mel-frequency cepstral coefficients are of order 13, the effective range of the fundamental frequency is set to 75–500 Hz, the speech intensity is calculated using short-time energy, and the spectral centroid reflects the position of the spectral center of gravity. The airflow preprocessing unit 22 performs noise reduction, segmentation, and feature extraction on the oral and nasal airflow signal to generate an airflow feature vector. The airflow feature vector includes airflow intensity, airflow rate change, oral and nasal airflow ratio, and airflow vibration parameters. In a specific embodiment of the invention, the airflow intensity is calculated using air pressure values, the airflow rate change uses first-order difference, the oral and nasal airflow ratio represents the relative intensity of oral and nasal airflow, and the airflow vibration parameters include frequency and amplitude characteristics.

[0085] Signal preprocessing module 2, which preprocesses the acquired raw signals, forms the foundation of the entire system. In speech signal preprocessing, noise reduction is first achieved using an adaptive filtering algorithm, followed by segmentation using a short-time analysis method with a fixed 20ms frame length and a 10ms frame shift. Speech feature extraction follows these steps: first, pre-emphasis and windowing are applied to the segmented signals; then, short-time energy and zero-crossing rate are calculated; next, spectral features are obtained through Fast Fourier Transform; and finally, parameters such as Mel-frequency cepstral coefficients and fundamental frequency are calculated. Airflow signal preprocessing employs a similar segmentation method, removing high-frequency noise through low-pass filtering, and then calculating airflow characteristic parameters for each frame.

[0086] In particular, to ensure that the numerical ranges of the speech feature vector and the airflow feature vector are consistent, the system normalizes all feature parameters, unifying the value range to the [0,1] interval, thereby enhancing the stability and accuracy of subsequent analysis.

[0087] like Figure 4 As shown, the speech-airflow synchronization analysis module 3 includes: a manifold mapping unit 31, a temporal alignment unit 32, a two-stream network unit 33, and a synchronization evaluation unit 34. The manifold mapping unit 31 constructs a first Riemannian manifold space and a second Riemannian manifold space, and maps speech feature vectors and airflow feature vectors to the corresponding manifold spaces. The temporal alignment unit 32, connected to the manifold mapping unit 31, constructs a geodesic flow field and solves for the optimal temporal alignment path along the integral curve of the geodesic flow field. The two-stream network unit 33, connected to the temporal alignment unit 32, processes the aligned manifold representation through a Riemannian deep learning network to extract speech-airflow synchronization features. The synchronization evaluation unit 34, connected to the two-stream network unit 33, generates speech-airflow synchronization evaluation results based on the speech-airflow synchronization features.

[0088] like Figure 5 As shown, the manifold mapping unit 31 is specifically used for: constructing a first Riemannian manifold space and a second Riemannian manifold space with local inner product structure; constructing a tangent space at each point of each manifold to represent the local change direction of the signal; calculating geodesics on the manifold to characterize the optimal transition path between signal states; and analyzing the curvature tensor of the manifold to detect abnormal articulation patterns.

[0089] The manifold mapping unit 31 is one of the core innovations of this invention, realizing the mapping of speech and airflow signals to the Riemannian manifold space. In specific implementation, the process of mapping speech feature vectors to the first Riemannian manifold space can be represented as:

[0090] ,

[0091] in: For the speech manifold representation, let represent the point mapped onto the first Riemannian manifold space; The mapping function transforms speech feature vectors into points on the manifold; The speech feature vector contains features such as Mel frequency cepstral coefficients and fundamental frequency. Let be the first Riemannian manifold space, representing the nonlinear manifold of speech features.

[0092] Similarly, the process of mapping airflow eigenvectors to the second Riemannian manifold space can be represented as:

[0093] ,

[0094] in: For the airflow manifold representation, represents the point mapped onto the second Riemannian manifold space; The mapping function transforms the airflow feature vector into points on the manifold; This is an airflow feature vector, containing features such as airflow intensity and airflow rate changes; Let be the second Riemannian manifold space, representing the nonlinear manifold characteristic of the airflow.

[0095] In a preferred embodiment of the invention, the local inner product structure of the manifold is expressed by the Riemannian metric tensor. Definition: For any two tangent vectors Its inner product can be expressed as:

[0096] ,

[0097] in: Point The inner product at the location; To measure the tensor matrix, the dimension is... ,in Let be the dimension of the manifold; and For a manifold at a point Cutting space The tangent vectors in the vectors are all 3D column vector; Representing vectors The transpose of .

[0098] The calculation of geodesics on a manifold employs the variational principle, by solving the Euler-Lagrange equations:

[0099] ,

[0100] in: A parameterized curve is derived from parameters. to manifold Mapping; This indicates the curve in the local coordinate system. One component; Let Christopher's notation denote the connection coefficient of a manifold, defined as follows: ,in It is a metric tensor The inverse matrix elements; the subscripts i, j, k represent coordinate component indices, ranging from 1 to the manifold dimension n; the repetition of superscripts and subscripts represents the Einstein summation convention, that is, summing over repeated indices.

[0101] Curvature tensor analysis of manifolds is used to detect anomalous articulation patterns by calculating the Riemann curvature tensor:

[0102] ,

[0103] in: The Riemann curvature tensor is a fourth-order tensor. Let be a vector field on a manifold, and denote a smooth vector function defined on the manifold. Let denote the connection operator, and let denote the covariant differential on the manifold; Represents a vector field Along the vector field The covariant derivative; Represents a vector field and The Li bracket is defined as ,in It is a smooth function on the manifold.

[0104] In practical applications, the system establishes a personalized manifold structure for each patient. The dimension of the manifold is determined by the dimension of the feature vectors; typically, the speech manifold is 13-15 dimensions, and the airflow manifold is 8-10 dimensions. The curvature characteristics of the manifold reflect the stability and consistency of the patient's articulation, with regions of larger curvature values ​​usually corresponding to key problem points in articulation disorders.

[0105] like Figure 6 As shown, the temporal alignment unit 32 is specifically used for: defining an energy density function in the product space of the two manifolds; calculating the gradient flow field of the energy function as a geodesic flow field; using a multi-scale time window strategy to iteratively solve for the optimal alignment path from coarse to fine; and adaptively adjusting the size of the time window according to the local rate of change of the signal.

[0106] The timing alignment unit 32 solves the problem of synchronization matching between speech and airflow signals in the time dimension, which is another core innovation of this invention. (In the product space of two manifolds) Above, the energy density function E is defined as:

[0107] ,

[0108] in: Represents the midpoint of the product space Energy density at that location; Representing a manifold Geodesic distance, used to measure the geometric distance between two points; Let be a mapping function that maps points on the first manifold to points on the second manifold; Let A be a point on the first manifold; A point on the second manifold; This is a regularization term used to control the smoothness of the mapping; To balance the relative importance of the distance and regularization terms, the values ​​are typically between 0.1 and 0.5.

[0109] Geodesic flow field By the definition of the gradient of the energy function:

[0110] ,

[0111] in: Representing the geodesic flow field, it is a product space. Vector field on; Represents the energy function The gradient of the energy function is the energy function. Rate of change in all directions.

[0112] Optimal alignment path This is obtained by solving the following differential equation:

[0113] ,

[0114] in: The path in the product space represents the path from the parameter. To the product space Mapping; Representing a path Regarding parameters The derivative of represents the tangent vector of the path; Indicates in path Geodesic flow field values ​​at the location; The integration termination time is usually set to allow the energy to be used. The time required for convergence to a local minimum.

[0115] In its implementation, the system employs a multi-scale time window strategy and designs a progressive window sequence. The window size decreases from coarse to fine. In a preferred embodiment of the invention, the initial window size is set to 200ms, gradually decreasing to 20ms. For regions with rapid speech changes, such as plosive transitions, the system automatically shrinks the window to 10ms to improve alignment accuracy; for stable pronunciation regions, such as vowel sustains, the system uses a larger window, such as 100ms, to improve computational efficiency.

[0116] The quality of timing alignment directly affects the accuracy of subsequent analysis. Practice shows that for patients with moderate ataxia, the timing alignment algorithm of this invention can control the timing matching error of speech-airflow signals to within 5ms, which is significantly better than the traditional dynamic time warping algorithm (which usually has an error of 15-20ms).

[0117] like Figure 7 As shown, the dual-stream network unit 33 includes: a speech stream network 331 for extracting speech features on a first Riemannian manifold; an airflow stream network 332 for extracting airflow features on a second Riemannian manifold; a cross-stream mapping layer 333 for establishing a feature mapping relationship between the two manifolds; and a feature fusion layer 334 for fusing the dual-stream features into a common representation space.

[0118] The dual-stream network unit 33 enables deep feature extraction and fusion of speech and airflow manifold representations, which is the third core innovation of this invention. The convolution operation on the Riemannian manifold is defined as:

[0119] ,

[0120] in: Let f be the result of the convolution of function f and kernel g at point x; f represents the feature function on manifold M, which maps points on the manifold to feature values; g is the convolution kernel function, defined on the tangent space. Let x represent the inverse exponential mapping from point x to a point y on the manifold and a vector in the tangent space at x. d represents the integral over manifold M; dy represents the volume element over manifold M.

[0121] In an embodiment of the present invention, the speech stream network 331 employs a 5-layer Riemann convolutional structure, with each convolutional layer followed by a Riemann pooling operation, ultimately outputting a 128-dimensional feature vector. The airflow network 332 employs a 4-layer structure, ultimately outputting a 96-dimensional feature vector.

[0122] The cross-flow mapping layer 333 establishes the feature mapping relationship between two manifolds, which can be represented as:

[0123] ,

[0124] ,

[0125] in: Represents from speech feature space To airflow characteristic space The mapping function; Represents the characteristic space of airflow To speech feature space The mapping function; This represents the high-level feature space of the speech stream network, with a dimension of 128; This represents the high-level feature space of the airflow network, with a dimension of 96.

[0126] Feature fusion layer 334 fuses the two-stream features into a common representation space using an adaptive weighting mechanism:

[0127] ,

[0128] Where: h represents the fused feature vector with a dimension of 256; This represents a speech feature vector with a dimension of 128; This represents the airflow feature vector, with a dimension of 96 (extended to 128 dimensions through zero padding). The weight parameter represents the speech features, with a value range of [0,1], and is dynamically adjusted according to the importance of the features; The weight parameter represents the interaction feature, and its value ranges from [0,1]. This represents element-wise multiplication (Hadamard product), used to capture the interaction relationships between features.

[0129] During training, the system employs a three-stage strategy: first, pre-training is performed on the speech flow network and the airflow flow network separately; then, the cross-flow mapping layer is trained with fixed network parameters; and finally, end-to-end joint optimization is performed. The loss function is designed as follows:

[0130] ,

[0131] in: Represents the total loss function; This represents the speech prediction loss, which measures the accuracy of speech feature prediction. This represents the airflow prediction loss and measures the accuracy of airflow characteristic prediction. This represents the cross-flow consistency loss, measuring the degree of synergy between the features of two flows; This represents the trade-off factor for cross-flow consistency loss, controlling the strength of consistency constraints; in practice, it is set to 0.3.

[0132] The synchronous assessment unit 34 generates speech-airflow synchronicity assessment results based on the output characteristics of the two-stream network. Assessment metrics include temporal alignment, inter-manifold mapping quality, and feature correlation. For ataxia patients, the system compares the assessment results with a standard articulation template to quantify the type and severity of articulation disorders. The assessment results are presented in numerical scores and visual charts, intuitively demonstrating the patient's articulation problems and providing a basis for training program generation.

[0133] like Figure 8 As shown, the personalized training generation module 4 includes: an obstacle analysis unit 41, a training goal setting unit 42, a training task generation unit 43, and a virtual guidance model generation unit 44. The obstacle analysis unit 41 is used to analyze the speech-airflow synchronicity assessment results and identify the type and severity of the patient's articulation disorder. The training goal setting unit 42, connected to the obstacle analysis unit 41, is used to set short-term training goals and long-term rehabilitation goals based on the patient's articulation disorder characteristics. The training task generation unit 43, connected to the training goal setting unit 42, is used to design targeted training tasks, including airflow control training for the lips, tongue, soft palate, and epiglottis. The virtual guidance model generation unit 44, connected to the training task generation unit 43, is used to create a visual airflow control guidance model to assist the patient's training.

[0134] The personalized training generation module 4 transforms the speech-airflow synchronicity analysis results into specific training plans. The obstacle analysis unit 41, based on the assessment results, classifies articulation disorders into three main categories: rhythmic disorders, coordination disorders, and stability disorders, and quantifies their severity as mild (1-3), moderate (4-7), and severe (8-10). The training goal setting unit 42 sets tiered training goals based on the type and severity of the obstacle, including immediate goals (single training session), short-term goals (1-2 weeks), and long-term goals (1-3 months).

[0135] The training task generation unit 43 selects and combines suitable training tasks from the training library according to the training objectives. For example, for lip control problems, the system designs airflow control training for bilabial consonants ( / p / , / b / ); for tongue control problems, it designs training for apical consonants ( / t / , / d / ) and velar consonants ( / k / , / g / ); for soft palate control problems, it designs training for alternating nasal consonants ( / m / , / n / ) and accents. Each training task has clear airflow control parameter objectives, such as airflow intensity, stability, and temporal characteristics.

[0136] The virtual guidance model generation unit 44 creates a 3D visualization model, displaying a real-time comparison between the standard airflow control mode and the patient's actual performance, helping the patient intuitively understand and adjust their pronunciation. In a preferred embodiment of the invention, the virtual guidance model uses a color-coding method, employing a red-yellow-green spectrum to represent the degree of airflow control from poor to good, enhancing the visual feedback effect.

[0137] like Figure 9 As shown, the training feedback and evaluation module 5 includes: a real-time monitoring unit 51, an effect evaluation unit 52, a feedback generation unit 53, and a progress tracking unit 54. The real-time monitoring unit 51 collects the patient's speech and airflow data during training. The effect evaluation unit 52, connected to the real-time monitoring unit 51, calculates the manifold distance between the current articulation and the target articulation, and evaluates the stability and accuracy of airflow control. The feedback generation unit 53, connected to the effect evaluation unit 52, generates real-time visual and auditory feedback to guide the patient in adjusting articulation and airflow control. The progress tracking unit 54, connected to the effect evaluation unit 52, records historical training data and generates training progress reports and improvement suggestions.

[0138] The training feedback and evaluation module 5 constitutes a closed-loop control system for the training process. The real-time monitoring unit 51 uses the same hardware as the data acquisition module 1, but simplifies the acquisition process, improves response speed, and achieves real-time feedback at the 10ms level. The effect evaluation unit 52 calculates the distance between the current articulation and the target articulation in the manifold space to evaluate the training effect. In this embodiment of the invention, the manifold distance calculation formula is:

[0139] ,

[0140] in: This represents the weighted Euclidean distance between the current articulation and the target articulation; The manifold representation of the current articulation is a dimensional vector; The manifold representation of the target articulation is also dimensional vector; Indicates the first The weight coefficients of each dimension reflect the importance of that dimension and satisfy the following conditions: ; The current phonological manifold represents the first... One component; The first part represents the target phonological manifold representation. One component; The dimension of the manifold.

[0141] Based on the assessment results, the feedback generation unit 53 provides multimodal feedback: visual feedback is achieved through the dynamic updating of the virtual guidance model; auditory feedback is achieved through the comparative playback of standard sound and actual sound; and tactile feedback is achieved through a vibration prompting device to help patients perceive the timing requirements of airflow control.

[0142] The progress tracking unit 54 records historical training data, analyzes training trends, and generates periodic evaluation reports. The system uses an exponentially weighted moving average algorithm to calculate the progress rate.

[0143] ,

[0144] in: It indicates the patient's training progress rate and measures the speed at which the training effect improves; Indicates the first The score for each training session ranges from 0 to 100; Indicates the first The score for each training session; This represents the decay factor, which controls the rate at which the weights of historical data decay, and has a value of 0.9. This indicates the evaluation window size, i.e., the number of historical training iterations considered, with a value of 10. This represents the normalization factor, ensuring the comparability of progress rates; This represents a weighted summation of all training iterations within the window.

[0145] When the system detects that the training effect has reached the preset target or entered a plateau, it automatically adjusts the training difficulty or changes the training content to maintain the challenge and effectiveness of the training.

[0146] like Figure 10 As shown, the present invention also provides a method for synchronous analysis and training of speech airflow in ataxia-associated articulation disorders, comprising the following steps:

[0147] Step S1: Collect the patient's speech signal and oral and nasal airflow signal;

[0148] Step S2: Preprocess the speech signal and the nasal airflow signal to extract speech feature vectors and airflow feature vectors;

[0149] Step S3: Map the speech feature vectors to the first Riemannian manifold space to form a speech manifold representation;

[0150] Step S4: Map the airflow feature vector to the second Riemannian manifold space to form an airflow manifold representation;

[0151] Step S5: Construct the geodesic flow field between the first Riemannian manifold space and the second Riemannian manifold space to achieve temporal alignment between the speech manifold representation and the airflow manifold representation;

[0152] Step S6: Based on the temporally aligned speech manifold representation and airflow manifold representation, extract speech-airflow synchronization features through a two-stream network structure to generate speech-airflow synchronization evaluation results;

[0153] Step S7: Based on the speech-airflow synchronicity assessment results, generate a personalized training program tailored to the patient's articulation disorder characteristics;

[0154] Step S8: Guide the patient to implement the personalized training plan, collect training data in real time, and evaluate the training effect;

[0155] Step S9: Based on the training effect evaluation results, dynamically adjust the personalized training plan to form a training closed loop.

[0156] In step S1, the patient's speech signal and nasal airflow signal are acquired. Specifically, the patient's speech signal is acquired through a microphone array at a sampling rate of 48kHz; the patient's nasal airflow signal is acquired through a separate nasal airflow sensor at a sampling rate of 1kHz; and a hardware clock synchronization mechanism is used to ensure that the sampling of the two signals is synchronized, with the sampling error controlled within 0.1ms.

[0157] In step S2, the speech signal and the nasal / oral airflow signal are preprocessed to extract speech feature vectors and airflow feature vectors. Specifically, the speech signal is denoised, segmented, and feature-extracted to generate a speech feature vector containing Mel-frequency cepstral coefficients, fundamental frequency, speech intensity, and spectral centroid parameters; the nasal / oral airflow signal is denoised, segmented, and feature-extracted to generate an airflow feature vector containing airflow intensity, airflow rate variation, nasal / oral airflow ratio, and airflow vibration parameters. Furthermore, all feature parameters are normalized to unify their numerical range to the [0,1] interval.

[0158] In steps S3 and S4, the speech feature vector and airflow feature vector are mapped to their respective Riemannian manifold spaces. This process is achieved through a manifold mapping function, which transforms the high-dimensional feature vectors into points on the manifold while preserving the nonlinear characteristics of the signal. Each manifold is equipped with a local inner product structure, enabling the system to measure the local similarity of the signals.

[0159] In step S5, a geodesic flow field is constructed between the two manifolds to achieve temporal alignment of speech and airflow representations. The system defines an energy density function in the manifold product space, calculates its gradient as the geodesic flow field, and then solves for the optimal alignment path along the flow field integral curve. Preferably, the system employs a multi-scale time window strategy, iteratively optimizing the alignment results from coarse to fine, and adaptively adjusting the window size according to the local rate of change of the signal.

[0160] In step S6, based on the aligned manifold representation, speech-airflow synchronization features are extracted through a two-stream network structure to generate an evaluation result. The two-stream network includes a speech stream network and an airflow stream network, which extract features on their respective manifolds. Then, a feature mapping relationship between the two manifolds is established through a cross-stream mapping layer. Finally, the two-stream features are fused into a unified representation in a feature fusion layer.

[0161] In step S7, a personalized training plan is generated based on the assessment results. The system first analyzes the type and severity of the patient's dysarthria, then sets tiered training goals, designs targeted training tasks, and creates a visualized airflow control guidance model.

[0162] In step S8, the patient is guided to follow the training protocol, data is collected in real time, and the effectiveness is evaluated. The system calculates the manifold distance between the current articulation and the target articulation, assesses the stability and accuracy of airflow control, and provides multimodal feedback to help the patient adjust articulation and airflow control.

[0163] In step S9, the training plan is dynamically adjusted based on the training effect, forming a closed loop. The system records historical training data, analyzes progress trends, and automatically adjusts the training difficulty or changes the content when the training effect is detected to have reached the target or entered a plateau, thus maintaining the effectiveness of the training.

[0164] During the execution of the method of this invention, a coherent data flow and control flow are formed between each step. In particular, the system continuously accumulates patient training data and updates the personalized manifold model during execution, improving the accuracy of assessment and training. Practice has shown that ataxia patients who have undergone systematic training using this method experience an average improvement of 40% in articulation clarity, 35% in speech rhythm, 45% in airflow control, and a significant enhancement in overall communication ability.

[0165] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A system for synchronous analysis and training of speech airflow in ataxia-associated articulation disorders, characterized in that, include: The data acquisition module is used to collect the patient's voice signals and airflow signals from the mouth and nose; The signal preprocessing module is electrically connected to the data acquisition module and is used to perform noise reduction and feature extraction on the speech signal and the nasal airflow signal to obtain speech feature vector and airflow feature vector. The speech airflow synchronization analysis module, electrically connected to the signal preprocessing module, is used for: The speech feature vectors are mapped to the first Riemannian manifold space to form a speech manifold representation; The airflow feature vector is mapped to the second Riemannian manifold space to form an airflow manifold representation; Construct a geodesic flow field between the first Riemannian manifold space and the second Riemannian manifold space to achieve temporal alignment between the speech manifold representation and the airflow manifold representation; Based on the temporally aligned speech manifold representation and the airflow manifold representation, speech-airflow synchronization features are extracted through a two-stream network structure to generate speech-airflow synchronization evaluation results. The personalized training generation module is electrically connected to the speech-airflow synchronicity analysis module and is used to generate a personalized training plan for the patient's articulation disorder characteristics based on the speech-airflow synchronicity assessment results. The training feedback and evaluation module is electrically connected to the personalized training generation module and the data acquisition module. It is used to collect training data in real time during the patient's execution of the personalized training plan, evaluate the training effect, and feed back the training results to the personalized training generation module for dynamic adjustment of the personalized training plan.

2. The ataxic articulation disorder speech airflow synchronization analysis and training system according to claim 1, characterized in that, The data acquisition module includes: A voice acquisition unit is used to acquire the patient's voice signals through a microphone array; An airflow acquisition unit is used to acquire airflow signals from the patient's mouth and nose via a separate mouth and nose airflow sensor. The synchronization control unit is electrically connected to the voice acquisition unit and the airflow acquisition unit, and is used to synchronize the sampling clocks of the voice signal and the nasal airflow signal to ensure the time consistency of the acquired data.

3. The ataxic articulation disorder speech airflow synchronization analysis and training system according to claim 1, characterized in that, The signal preprocessing module includes: The speech preprocessing unit is used to perform noise reduction, segmentation and feature extraction on the speech signal to generate the speech feature vector, wherein the speech feature vector includes Mel frequency cepstral coefficients, fundamental frequency, speech intensity and spectral centroid parameters; An airflow preprocessing unit is used to perform noise reduction, segmentation, and feature extraction on the oral and nasal airflow signal to generate the airflow feature vector, wherein the airflow feature vector includes airflow intensity, airflow rate change, oral and nasal airflow ratio, and airflow vibration parameters.

4. The ataxic articulation disorder speech airflow synchronization analysis and training system according to claim 1, characterized in that, The speech airflow synchronization analysis module includes: The manifold mapping unit is used to construct the first Riemannian manifold space and the second Riemannian manifold space, and to implement the mapping of the speech feature vector and the airflow feature vector to the corresponding manifold space; The time alignment unit, connected to the manifold mapping unit, is used to construct the geodesic flow field and solve for the optimal time alignment path along the integral curve of the geodesic flow field; A dual-stream network unit, connected to the temporal alignment unit, is used to process the aligned manifold representation through a Riemann deep learning network and extract speech-airflow synchronization features. A synchronous evaluation unit, connected to the dual-stream network unit, is used to generate the speech-airflow synchronization evaluation result based on the speech-airflow synchronization characteristics.

5. The ataxic articulation disorder speech airflow synchronization analysis and training system according to claim 4, characterized in that, The manifold mapping unit is specifically used for: Construct a first Riemannian manifold space and a second Riemannian manifold space with local inner product structure; A tangent space is constructed at each point of each manifold to represent the local direction of change of the signal; Calculate geodesics on the manifold to characterize the optimal transition paths between signal states; Analyze the curvature tensor of the manifold to detect anomalous articulation patterns.

6. The ataxic articulation disorder speech airflow synchronization analysis and training system according to claim 4, characterized in that, The timing alignment unit is specifically used for: Define the energy density function on the product space of the two manifolds; The gradient flow field of the energy function is used as a geodesic flow field; A multi-scale time window strategy is adopted to iteratively solve for the optimal alignment path from coarse to fine; The time window size is adaptively adjusted based on the local rate of change of the signal.

7. The ataxia-associated articulation disorder speech airflow synchronization analysis and training system according to claim 4, characterized in that, The dual-stream network unit includes: A speech stream network is used to extract speech features on the first Riemannian manifold; An airflow network is used to extract airflow features on the second Riemannian manifold; Cross-flow mapping layer is used to establish feature mapping relationships between two manifolds; The feature fusion layer is used to fuse two-stream features into a common representation space.

8. The ataxia-associated articulation disorder speech airflow synchronization analysis and training system according to claim 1, characterized in that, The personalized training generation module includes: The disorder analysis unit is used to analyze the speech-airflow synchronicity assessment results and identify the type and severity of the patient's articulation disorder. The training goal setting unit, connected to the obstacle analysis unit, is used to set short-term training goals and long-term rehabilitation goals based on the patient's articulation disorder characteristics. The training task generation unit, connected to the training target setting unit, is used to design targeted training tasks, including training on airflow control of the lips, tongue, soft palate, and epiglottis. The virtual guidance model generation unit, connected to the training task generation unit, is used to create a visual airflow control guidance model to assist patients in training.

9. The ataxic articulation disorder speech airflow synchronization analysis and training system according to claim 1, characterized in that, The training feedback and evaluation module includes: A real-time monitoring unit is used to collect patients' speech and airflow data during training; The effect evaluation unit, connected to the real-time monitoring unit, is used to calculate the manifold distance between the current articulation and the target articulation, and to evaluate the stability and accuracy of airflow control. The feedback generation unit, connected to the effect evaluation unit, is used to generate real-time visual and auditory feedback to guide the patient to adjust pronunciation and airflow control. The progress tracking unit, connected to the effect evaluation unit, is used to record historical training data and generate training progress reports and improvement suggestions.

10. A method for synchronous analysis and training of speech airflow in ataxia-associated articulation disorders, employing the synchronous analysis and training system for speech airflow in ataxia-associated articulation disorders as described in any one of claims 1-9, characterized in that, Includes the following steps: Collect the patient's speech and nasal / oral airflow signals; The speech signal and the nasal airflow signal are preprocessed to extract speech feature vectors and airflow feature vectors; The speech feature vectors are mapped to the first Riemannian manifold space to form a speech manifold representation; The airflow feature vector is mapped to the second Riemannian manifold space to form an airflow manifold representation; Construct a geodesic flow field between the first Riemannian manifold space and the second Riemannian manifold space to achieve temporal alignment between the speech manifold representation and the airflow manifold representation; Based on the temporally aligned speech manifold representation and the airflow manifold representation, speech-airflow synchronization features are extracted through a two-stream network structure to generate speech-airflow synchronization evaluation results. Based on the speech-airflow synchronicity assessment results, a personalized training program is generated targeting the patient's articulation disorder characteristics. Guide patients to implement the personalized training program, collect training data in real time, and evaluate the training effect; Based on the training effect evaluation results, the personalized training plan is dynamically adjusted to form a training closed loop.

Images