Intelligent classification system and method for dysphagia based on multimodality of ultrasound images and electromyographic signals

Abstract

The present invention discloses an intelligent classification system and method for dysphagia based on multimodality of ultrasound images and electromyographic signals. The method comprises obtaining original ultrasound image sequences and original electromyographic signal sequences, performing time alignment and adaptive noise suppression processing; performing multidimensional signal quality scoring and adaptive enhancement processing on denoised data; extracting and combining time domain, frequency domain and time-frequency domain features from enhanced data; calculating feature weights based on feature correlation and quality scoring, and performing multi-layer fusion; performing feature classification and evaluating reliability using a hierarchical classifier; and comprehensively evaluating classification performance and optimizing system parameters. The present invention improves the accuracy and reliability of dysphagia diagnosis through multimodal data fusion and adaptive processing mechanisms, and has good clinical application value.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical diagnosis, and in particular to an intelligent classification system and method for dysphagia based on multimodality of ultrasonic images and electromyographic signals. Background Art

[0002] Dysphagia is a common functional disease that can lead to serious complications such as difficulty eating, malnutrition, and aspiration pneumonia, which seriously affects the patient's quality of life and prognosis. Clinical studies have shown that early accurate diagnosis and timely intervention are of great significance to improving patient prognosis. Traditional diagnostic methods mainly rely on clinical experience and a single examination method, which is difficult to fully reflect the dynamic changes of swallowing function. With the development of medical imaging and bioelectric signal processing technology, multimodal analysis methods based on ultrasound images and electromyographic signals provide a new technical path for the intelligent diagnosis of dysphagia, which has the advantages of being non-invasive, real-time, and objective.

[0003] At present, the diagnosis and research of dysphagia mainly focuses on the signal analysis of a single modality. Ultrasound imaging technology uses a B-ultrasound probe to collect dynamic images of tongue muscle movement and pharyngeal structure, and uses methods such as image segmentation and motion trajectory analysis to evaluate swallowing function; electromyography technology records the electrical activity of the neck muscle group through surface electrodes, and uses methods such as time domain feature extraction and threshold judgment to identify swallowing movements. These methods have achieved certain results under specific conditions, but due to the limitations of single modality information, it is difficult to fully reflect the physiological mechanism of the swallowing process. At the same time, traditional signal processing methods mostly use fixed parameter filtering algorithms and simple feature extraction strategies, which have poor adaptability to noise interference and individual differences.

[0004] In specific practice, the existing technology still has the following problems: First, the time alignment problem of multimodal data. Due to the differences in sampling frequency and trigger delay of different devices, the time synchronization of ultrasound image sequence and electromyographic signal sequence is poor, which affects the accuracy of feature extraction; second, signal quality evaluation lacks scientific quantitative standards. Existing methods often rely on a single indicator or empirical threshold, which is difficult to accurately reflect the reliability of data; third, there are redundancy and information loss problems in feature engineering. Traditional feature extraction methods do not capture the local features and time-varying characteristics of signals sufficiently, and the feature selection process lacks in-depth analysis of the correlation between features; finally, in the classification decision link, a single classifier is difficult to adapt to complex disease manifestations, lacks a reliability evaluation mechanism, and lacks an effective feedback mechanism for system optimization. These technical problems seriously restrict the accuracy and reliability of intelligent diagnosis of dysphagia. Summary of the invention

[0005] The purpose of the invention is to provide an intelligent classification system and method for dysphagia based on multimodality of ultrasound images and electromyography signals, in order to solve at least one technical problem existing in the prior art.

[0006] The technical solution is an intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals, comprising the following steps:

[0007] S1, obtaining an original ultrasound image sequence and an original electromyographic signal sequence, performing time alignment processing on the original ultrasound image sequence and the original electromyographic signal sequence to obtain aligned data; performing multi-scale decomposition and adaptive noise suppression processing on the aligned data to obtain denoised data;

[0008] S2. Based on the denoised data, a multidimensional signal quality score is calculated to obtain a quality score matrix; according to the quality score matrix, the denoised data is adaptively enhanced to obtain enhanced data;

[0009] S3. Based on the enhanced data, extract the time domain features, frequency domain features and time-frequency domain features respectively to obtain the time domain feature vector, frequency domain feature vector and time-frequency feature vector; combine the time domain feature vector, frequency domain feature vector and time-frequency feature vector to generate a feature matrix;

[0010] S4. Based on the feature matrix and the quality score matrix, calculate the feature correlation index to obtain correlation index data; calculate the feature weight according to the correlation index data and the quality score matrix to obtain a feature weight vector; use the feature weight vector to perform multi-layer fusion processing on the feature matrix to generate a fused feature vector;

[0011] S5. Based on the fused feature vector, feature classification is performed through a hierarchical classifier to obtain classification result data; reliability assessment is performed on the classification result data to generate a confidence score; comprehensive judgment is made based on the classification result data and the confidence score, and the final classification result is output.

[0012] The intelligent classification system for dysphagia based on multimodality of ultrasound images and electromyographic signals includes:

[0013] at least one processor; and,

[0014] a memory communicatively connected to at least one of the processors; wherein,

[0015] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals.

[0016] Beneficial effects: The present invention solves the problems of strong subjectivity, low accuracy, poor reliability, etc. of traditional diagnostic methods, and provides an objective, accurate and reliable basis for diagnosis through intelligent analysis of multimodal data; it can more comprehensively capture abnormal characteristics of swallowing function, improve the accuracy and reliability of diagnosis, and provide strong technical support for clinical practice; through a closed-loop optimization mechanism, the system can continuously improve and adapt to different clinical environments, and has good practicality and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flow chart of the method of the present invention.

[0018] Figure 2 This is a flow chart of step S1 of the present invention.

[0019] Figure 3 This is a flow chart of step S2 of the present invention.

[0020] Figure 4 This is a flow chart of step S3 of the present invention.

[0021] Figure 5 This is a flow chart of step S4 of the present invention.

[0022] Figure 6 This is a flow chart of step S5 of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, the present application proposes an intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals, comprising the following steps:

[0024] S1, obtaining an original ultrasound image sequence and an original electromyographic signal sequence, performing time alignment processing on the original ultrasound image sequence and the original electromyographic signal sequence to obtain aligned data; performing multi-scale decomposition and adaptive noise suppression processing on the aligned data to obtain denoised data;

[0025] S2. Based on the denoised data, a multidimensional signal quality score is calculated to obtain a quality score matrix; according to the quality score matrix, the denoised data is adaptively enhanced to obtain enhanced data;

[0026] S3. Based on the enhanced data, extract the time domain features, frequency domain features and time-frequency domain features respectively to obtain the time domain feature vector, frequency domain feature vector and time-frequency feature vector; combine the time domain feature vector, frequency domain feature vector and time-frequency feature vector to generate a feature matrix;

[0027] S4. Based on the feature matrix and the quality score matrix, calculate the feature correlation index to obtain correlation index data; calculate the feature weight according to the correlation index data and the quality score matrix to obtain a feature weight vector; use the feature weight vector to perform multi-layer fusion processing on the feature matrix to generate a fused feature vector;

[0028] S5. Based on the fused feature vector, feature classification is performed through a hierarchical classifier to obtain classification result data; reliability assessment is performed on the classification result data to generate a confidence score; comprehensive judgment is made based on the classification result data and the confidence score, and the final classification result is output.

[0029] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0030] S11, the acquisition device respectively acquires the original ultrasound image sequence and the original electromyographic signal sequence during the swallowing process of the patient, performs image digitization processing on the original ultrasound image sequence, performs analog-to-digital conversion processing on the original electromyographic signal sequence, and generates digital signal data;

[0031] S12, constructing a time warping matrix, and projecting the digitized signal data onto a unified time axis, calculating the Euclidean distance between the original ultrasound image sequence and the original electromyographic signal sequence; aligning the original ultrasound image sequence and the original electromyographic signal sequence by minimizing the Euclidean distance, and generating aligned data;

[0032] S13. Use wavelet packet transform to perform multi-scale decomposition on the aligned data to obtain a coefficient matrix; calculate the energy distribution of each decomposition layer based on the coefficient matrix and determine the adaptive threshold function; based on the adaptive threshold function, process the coefficient matrix and obtain denoised data through wavelet packet reconstruction.

[0033] In one embodiment of the present application, the timing alignment algorithm: D(t) = β1·||U(t) - U'(t)|| 2 + β2·||E(t) - E'(t)|| 2 + β3·∫|ΔU(t) - ΔE(t)|dt; where U(t) and E(t) are the original ultrasound and electromyography signal sequences, U'(t) and E'(t) are the time-warped sequences, ΔU(t) and ΔE(t) are the first-order derivatives of the signals, β1, β2, and β3 are adaptive weight coefficients, which are determined by dynamic programming optimization; ||·|| 2 represents the Euclidean distance norm, ∫ represents the time integral, t is the time variable, and D(t) is the alignment distance function to be minimized.

[0034] Adaptive threshold function: λ(j, k) = σj, k [log(Nj, k) / log(k)] [1 + exp(-Ej, k / Emax)]; where σj, k is the noise standard deviation of the kth subband of the jth layer, Nj, k is the number of coefficients of the subband, Ej, k is the energy value of the subband, and Emax is the maximum energy value of all subbands; j is the number of decomposition levels, and k is the subband index; λ(j, k) is an adaptive threshold used for soft threshold processing of wavelet coefficients.

[0035] This embodiment achieves accurate synchronization and high-quality preprocessing of multimodal data by performing time alignment and adaptive noise suppression processing on the original ultrasound image sequence and electromyographic signal sequence. Specifically, the time alignment adopts the Euclidean distance minimization strategy to ensure the accurate correspondence between different modal data in the time dimension and avoid the time deviation caused by the asynchrony of the acquisition equipment; the multi-scale decomposition method of wavelet packet transform makes full use of the local characteristics of the signal, and realizes accurate identification and suppression of noise through adaptive threshold functions, while retaining the effective components of the signal. This embodiment not only improves the accuracy of subsequent feature extraction, but also reduces the impact of environmental noise on classification results. In the specific scenario of swallowing disorder diagnosis, it effectively solves practical problems such as equipment delay and environmental interference in the process of multimodal data acquisition, laying a solid data foundation for subsequent feature extraction and classification. Compared with the traditional fixed threshold denoising method, this embodiment can adaptively adjust the processing parameters according to the signal characteristics, improve data quality, and reduce the misdiagnosis rate.

[0036] According to one aspect of the present application, step S11 is further:

[0037] S111, obtaining an original ultrasound image sequence, performing inter-frame difference calculation on the original ultrasound image sequence to obtain change detection data; performing threshold detection on the change detection data to obtain swallowing event data; performing grayscale quantization on the original ultrasound image sequence to obtain quantized data; performing spatial sampling on the quantized data to obtain sampled data; performing data compression encoding on the sampled data to generate image digitized data;

[0038] S112, obtaining an original electromyographic signal sequence, performing envelope detection on the original electromyographic signal sequence to obtain envelope data; performing peak recognition on the envelope data to obtain swallowing mark data; performing anti-aliasing filtering on the original electromyographic signal sequence to obtain filtered data; performing sampling and holding on the filtered data to obtain sampled and held data; performing quantization encoding on the sampled and held data to generate electromyographic digital data;

[0039] S113, time-aligning the swallowing event data with the swallowing mark data to obtain synchronous mark data; time-aligning the image digitized data and the electromyography digitized data according to the synchronous mark data to obtain aligned data; unifying the data format of the aligned data to generate digital signal data.

[0040] This embodiment achieves high-quality digital conversion of the original ultrasound image sequence and the electromyographic signal sequence through a multi-level digital processing mechanism and a timing synchronization strategy. Inter-frame difference and threshold detection methods are used to identify swallowing events, and grayscale quantization and spatial sampling are combined to achieve accurate digitization of the image; at the same time, the swallowing markers of the electromyographic signal are extracted through envelope detection and peak recognition methods, and the reliable conversion of the signal is achieved by combining anti-aliasing filtering and sampling and holding. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problems of timing inconsistency and unstable data quality in the multimodal data acquisition process. By synchronizing the time alignment of the marker data, the timing consistency of data of different modalities is ensured, providing a high-quality digital data foundation for subsequent feature extraction and analysis. Compared with the traditional simple digitization method, this embodiment can better retain the effective information of the signal, while improving the timing consistency of the data, providing reliable data support for accurate diagnosis.

[0041] According to one aspect of the present application, step S13 is further:

[0042] S131, selecting a wavelet basis function for the aligned data to obtain basis function parameters; decomposing the aligned data into high-frequency and low-frequency components according to the basis function parameters to obtain frequency component data; recursively decomposing the frequency component data to generate a coefficient matrix.

[0043] S132. Calculate the energy value of the coefficients of each decomposition layer in the coefficient matrix to obtain energy data; establish an energy distribution model based on the energy data to obtain distribution characteristic data; use the distribution characteristic data to construct an adaptive threshold function to generate threshold function parameters.

[0044] S133, screening the coefficients of the coefficient matrix according to the threshold function parameters to obtain noise reduction coefficient data; rescaling the noise reduction coefficient data to obtain reconstructed coefficient data; and performing inverse wavelet packet transform using the reconstructed coefficient data to generate denoised data.

[0045] This embodiment achieves efficient noise reduction and signal enhancement of multimodal data through adaptive wavelet packet transform and energy distribution optimization. Combined with basis function optimization selection and multi-layer recursive decomposition, refined decomposition of the signal is achieved; through energy distribution model and adaptive threshold function, accurate identification and suppression of noise are achieved. In the swallowing disorder diagnosis scenario, this embodiment effectively solves various noise interference problems existing in the clinical environment, and improves the quality of the signal through adaptive noise reduction processing. The adaptive threshold strategy based on energy distribution can dynamically adjust the noise reduction parameters according to the signal characteristics, avoiding the signal distortion that may be caused by the traditional fixed threshold method. Compared with traditional noise reduction methods, this embodiment can more effectively suppress noise interference while retaining effective signal components, providing a high-quality signal foundation for subsequent feature extraction.

[0046] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0047] S21, calculating the signal-to-noise ratio index of the denoised data to obtain signal-to-noise ratio data; calculating the signal continuity index of the denoised data to obtain continuity data; analyzing the spectrum purity index of the denoised data to obtain spectrum purity data; performing weighted combination of the signal-to-noise ratio data, the continuity data and the spectrum purity data to generate a quality score matrix;

[0048] S22. Based on the quality score matrix, an enhancement weight matrix is ​​constructed; the denoised data is weighted by using the enhancement weight matrix to obtain preliminary enhanced data; the preliminary enhanced data is adaptively equalized to generate enhanced data.

[0049] In one embodiment of the present application, the quality scoring algorithm is: Q(x, t) = α1(t)·SNR(x) + α2(t)·C(x) + α3(t)·SP(x); wherein SNR(x) = 10·log10(Ps / Pn), SNR(x) is a signal-to-noise ratio indicator, Ps is a signal power, and Pn is a noise power; C(x) = 1 - |ΔS(x)| / S(x), C(x) is a signal continuity indicator, ΔS(x) is a signal difference, and S(x) is a signal amplitude; SP(x) = Em / Et, SP(x) is a spectrum purity indicator, Em is a main frequency band energy, and Et is a total energy; αk(t) is an adaptive weight coefficient, which is dynamically adjusted by fuzzy rules; x is a signal sampling point, and t is a time variable; Q(x, t) is a final quality score.

[0050] This embodiment realizes the quality assessment and targeted enhancement of multimodal data by constructing a multidimensional signal quality scoring system and an adaptive enhancement processing mechanism. The three key indicators of signal-to-noise ratio, signal continuity and spectral purity are weighted and fused to form a comprehensive quality scoring matrix, which can accurately reflect the reliability and validity of the data. Based on the adaptive enhancement processing of the quality scoring matrix, by constructing an enhancement weight matrix and adaptive equalization processing, differentiated enhancement of signals of different quality levels is achieved, ensuring the overall improvement of data quality. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem of unstable signal quality caused by individual differences among patients, changes in the examination environment and other factors, and improves the reliability of subsequent analysis. Compared with the traditional fixed parameter enhancement method, this embodiment can adaptively adjust the enhancement strategy according to the dynamic changes of signal quality, improve the signal-to-noise ratio and signal integrity of the data, and provide high-quality data support for accurate diagnosis.

[0051] According to one aspect of the present application, step S21 is further:

[0052] S211, performing segment processing on the denoised data to obtain signal segment data; extracting principal components based on the signal segment data by adaptive singular value decomposition to obtain principal component data; calculating the signal-to-noise ratio according to the deviation between the principal component data and the signal segment data to generate signal-to-noise ratio data;

[0053] S212, constructing a Hilbert space mapping based on the denoised data to obtain phase space trajectory data; calculating continuity characteristics of the phase space trajectory data to obtain trajectory feature data; evaluating signal continuity based on the trajectory feature data to generate continuity data;

[0054] S213, performing wavelet packet energy entropy analysis on the denoised data to obtain energy entropy data; constructing a spectrum purity evaluation model based on the energy entropy data to obtain evaluation model data; calculating a spectrum purity index based on the evaluation model data to generate spectrum purity data;

[0055] S214. Based on the signal-to-noise ratio data, continuity data and spectral purity data, a fuzzy reasoning system is constructed to obtain fuzzy rule data; using the fuzzy rule data, adaptive weight calculation is performed to obtain weight coefficient data; according to the weight coefficient data, indicator fusion is performed to generate a quality score matrix.

[0056] In one embodiment of the present application, the adaptive singular value decomposition algorithm: SVD(X, t) = μ(t)·U(t)Σ(t)V(t) T + λ(t)·||X – X*|| F ; where U(t) and V(t) are left and right singular vector matrices, Σ(t) is the singular value matrix, ||·|| Fis the Frobenius norm, X* is the reconstruction matrix, μ(t) is the time-varying weight coefficient, λ(t) is the adaptive regularization parameter; t is the time variable, X is the signal matrix; SVD(X, t) is the adaptive decomposition result, T Indicates transpose.

[0057] This embodiment achieves comprehensive evaluation of signal quality and reliability quantification through multi-dimensional quality assessment and fuzzy fusion mechanism. A complete quality assessment system is constructed by using adaptive singular value decomposition to calculate the signal-to-noise ratio, Hilbert space mapping to evaluate signal continuity, and wavelet packet energy entropy to analyze spectral purity. Scientific quantification of quality scores is achieved through multi-index fusion through the fuzzy reasoning system. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem that the traditional single indicator is difficult to fully reflect the signal quality, and provides precise guidance for subsequent signal enhancement through multi-dimensional quality assessment. Compared with the traditional simple quality assessment method, this embodiment can evaluate the signal quality more comprehensively and accurately, improve the reliability of quality assessment, and provide a scientific basis for adaptive signal enhancement.

[0058] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0059] S31, extracting the envelope of the enhanced data to obtain envelope data; based on the envelope data, calculating the mean, variance, skewness and kurtosis to obtain statistical feature data; normalizing the statistical feature data to generate a time domain feature vector;

[0060] S32, performing fast Fourier transform on the enhanced data to obtain spectrum data; based on the spectrum data, calculating the energy distribution of the characteristic frequency band to obtain frequency band energy data; performing normalization processing on the frequency band energy data to generate a frequency domain feature vector;

[0061] S33, performing Hilbert transform on the enhanced data to obtain analytical signal data; based on the analytical signal data, calculating the instantaneous frequency and instantaneous amplitude to obtain time-frequency feature data; performing normalization processing on the time-frequency feature data to generate a time-frequency feature vector;

[0062] S34, concatenating and combining the time domain feature vector, the frequency domain feature vector and the time-frequency feature vector according to a preset rule to generate a feature matrix.

[0063] In one embodiment of the present application, the process of generating a time domain feature vector is specifically: F(t) = γ1·H[x(t)] + γ2·Ψ[ENV(t)] / Ψt + γ3·VAR[x(t,τ)]; wherein H[x(t)] is the Hilbert transform result, ENV(t) is the signal envelope, VAR[x(t,τ)] is the sliding window variance, and τ is the window length; γ1, γ2, and γ3 are feature weight coefficients; t is a time variable; F(t) is the extracted time domain feature vector, and Ψ is a partial derivative.

[0064] This embodiment achieves comprehensive capture of the signal features of the swallowing process by systematically extracting time domain, frequency domain and time-frequency domain features and performing reasonable feature combinations. A variety of transformation methods such as envelope extraction, fast Fourier transform and Hilbert transform are used to extract feature information from different dimensions, and the comparability of features is ensured through normalization. The construction of the feature matrix adopts preset rules for splicing and combination to ensure the complementarity and integrity between features. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem that a single feature cannot fully reflect the state of swallowing function. Through the combination of multi-dimensional features, it can accurately capture abnormal manifestations in the swallowing process. Compared with the traditional single feature extraction method, this embodiment provides a richer and more accurate feature description, improves the recognition accuracy of abnormal patterns, and provides a more reliable basis for clinical diagnosis.

[0065] According to one aspect of the present application, step S31 is further:

[0066] S311, constructing a Hilbert transform complex plane for the enhanced data to obtain analytical signal data; using adaptive bilateral filtering to smooth the analytical signal data to obtain smoothed signal data; extracting a signal envelope according to the smoothed signal data to generate envelope data.

[0067] S312, establishing a piecewise linear approximation model for the envelope data to obtain linear piecewise data; calculating the jump point characteristics of the linear piecewise data to obtain jump characteristic data; calculating statistical parameters in combination with the envelope data and the jump characteristic data to generate statistical characteristic data.

[0068] S313, constructing a probability distribution model for the statistical feature data to obtain distribution parameter data; optimizing the distribution parameter data using a kernel density estimation method to obtain optimized parameter data; performing maximum and minimum normalization on the optimized parameter data to generate a time domain feature vector.

[0069] In one embodiment of the present application, the adaptive bilateral filtering algorithm: BF(I, x) = k(x) -1·∫∫Gσs(||ξ- x||)·Gσr(|I(ξ) - I(x)|)·I(ξ)dξ; where Gσs is the spatial Gaussian kernel function, Gσr is the range Gaussian kernel function, σs and σr are adaptive bandwidth parameters, k(x) is the normalization coefficient, ξ is the integral variable; x is the spatial position, I is the input signal; BF(I, x) is the filtered output.

[0070] This embodiment achieves accurate extraction of time domain features and effective construction of feature space through adaptive envelope extraction and statistical feature optimization. Hilbert transform and adaptive bilateral filtering are combined for envelope extraction, and the time domain characteristics of the signal are captured through piecewise linear approximation and jump feature analysis. The kernel density estimation method is used to optimize the probability distribution model and achieve feature normalization. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem that traditional time domain feature extraction methods are insensitive to local changes in the signal. Through refined feature extraction, the changes in time domain features during the swallowing process are accurately captured. Compared with traditional time domain feature extraction methods, this embodiment can better retain the local characteristics of the signal and improve the ability to distinguish features.

[0071] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0072] S41. Based on the characteristic components in the characteristic matrix, calculate the Pearson correlation coefficient to obtain correlation coefficient data; based on the characteristic components in the characteristic matrix, calculate the mutual information value to obtain mutual information data; perform weighted combination of the correlation coefficient data and the mutual information data to generate correlation index data;

[0073] S42, constructing a weight calculation model using the correlation index data and the quality score matrix; calculating the feature importance score according to the weight calculation model to obtain importance data; normalizing the importance data to generate a feature weight vector;

[0074] S43. Based on the feature weight vector, perform weighted processing on the feature matrix to obtain preliminary fused data; perform dimensionality reduction processing on the preliminary fused data to obtain reduced-dimensional feature data; perform nonlinear transformation on the reduced-dimensional feature data to generate a fused feature vector.

[0075] In one embodiment of the present application, the process of generating correlation index data is specifically as follows: R(i, j) = ω1·ρ(i, j) + ω2·I(Xi; Xj) + ω3·exp(-||Xi - Xj|| 2 / σ 2 ); where ρ(i, j) is the Pearson correlation coefficient, I(Xi; Xj) is the mutual information value, ||Xi - Xj|| 2 is the eigenvector distance, σ2 is the kernel function bandwidth parameter; ω1, ω2, ω3 are adaptive weight coefficients; i, j are feature indexes; R(i, j) is the comprehensive correlation index.

[0076] Feature fusion algorithm: Z(f) = Φ(W1·f1 + W2·f2 + ... + Wn·fn)·exp(-λ·L(f)); where fn is the nth eigenvector, Wn is the corresponding attention weight matrix, Φ(·) is the nonlinear activation function, L(f) is the manifold regularization term, λ is the regularization coefficient; f is the feature index; Z(f) is the fused feature vector.

[0077] This embodiment achieves optimal selection and effective fusion of features through feature correlation analysis and multi-layer fusion processing. The feature correlation is evaluated by combining the Pearson correlation coefficient and the mutual information value, and a feature weight calculation model is constructed based on the quality score matrix to ensure the scientificity and reliability of feature selection. The multi-layer fusion processing adopts a combination strategy of weighting, dimensionality reduction and nonlinear transformation, which effectively reduces feature redundancy and improves the expressiveness of features. In the swallowing disorder diagnosis scenario, this embodiment solves the problems of correlation and redundancy that may exist between multidimensional features, and improves the generalization ability of the classification model through scientific feature selection and fusion. Compared with traditional feature selection methods, this embodiment can better retain effective information, while reducing computational complexity and improving the practicality of the system.

[0078] According to one aspect of the present application, step S41 is further:

[0079] S411, constructing an adaptive segmentation window for the feature matrix to obtain segmentation feature data; using local polynomial regression to smooth the segmentation feature data to obtain smoothed feature data; calculating the Pearson correlation coefficient of the smoothed feature data to generate correlation coefficient data.

[0080] S412, performing kernel function transformation on the feature matrix to obtain high-dimensional feature mapping data; calculating the mutual information between features using a sorting-based entropy estimation method to obtain initial mutual information data; performing bias correction on the initial mutual information data to generate mutual information data.

[0081] S413, construct an adaptive weight learning network, input correlation coefficient data and mutual information data, and obtain weight learning data; use the gradient projection method to optimize the weight learning data to obtain optimized weight data; perform indicator fusion based on the optimized weight data to generate correlation indicator data.

[0082] In one embodiment of the present application, the local polynomial regression algorithm: LPR(x, h) = [Wh(x) T Kh(x)Wh(x)] -1Wh(x) T Kh(x)Y(x)·exp(-Δ(x) / σ 2 ), where Wh(x) is the design matrix, Kh(x) is the kernel weight matrix, h is the bandwidth parameter, Y(x) is the observation vector, Δ(x) is the local residual, σ 2 is the scale parameter; x is the feature vector; LPR(x, h) is the regression estimate.

[0083] This embodiment achieves accurate quantification and reliability assessment of the relationship between features through the fusion of adaptive correlation analysis and information theory. Combined with local polynomial regression and adaptive segmented window technology, dynamic evaluation of feature correlation is achieved; through kernel function transformation and sorting-based entropy estimation method, accurate calculation of mutual information between features is achieved. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem that complex dependencies between features are difficult to accurately quantify. By constructing an adaptive weight learning network and combining the gradient projection optimization method, the scientific fusion of correlation indicators is achieved, avoiding the defect of traditional correlation analysis methods that are insensitive to nonlinear relationships. Compared with traditional correlation analysis methods, this embodiment can more accurately capture the complex relationships between features, improve the scientificity and reliability of feature selection, and provide a reliable basis for subsequent feature fusion.

[0084] According to one aspect of the present application, step S43 is further:

[0085] S431, construct a multi-scale feature pyramid model, perform hierarchical processing on the feature matrix and the feature weight vector to obtain hierarchical feature data; use the attention mechanism to adjust the weight of the hierarchical feature data to obtain adjusted feature data; perform weighted summation on the adjusted feature data to generate preliminary fusion data;

[0086] S432, based on the preliminary fusion data, construct a manifold learning framework to obtain local feature structure data; use a spectral clustering method to perform dimension reduction mapping on the local feature structure data to obtain mapping feature data; optimize the mapping feature data based on the distance-preserving principle to generate dimension reduction feature data;

[0087] S433. Construct a deep autoencoding network to encode the reduced-dimensional feature data to obtain feature-encoded data; use a back-propagation algorithm to optimize the feature-encoded data to obtain optimized feature data; use an activation function to perform a nonlinear transformation on the optimized feature data to generate a fused feature vector.

[0088] In one embodiment of the present application, the attention mechanism algorithm: A(Q, K, V) = softmax(Q·K T / sqrt(d))·V + α·tanh(W·[Q;K;V]); where Q is the query matrix, K is the key matrix, V is the value matrix, d is the feature dimension, W is the learnable weight matrix, α is the adaptive coefficient, [;] represents matrix concatenation; softmax and tanh are activation functions; A(Q, K, V) is the attention weighted output.

[0089] Manifold learning algorithm: M(X) = argmin{∑||xi - xj|| 2 ·Wij + λ·tr(X T LX)}; where Wij is the local neighbor weight matrix, L is the Laplace matrix, tr(·) is the trace of the matrix, λ is the regularization parameter, xi and xj are data points; X is the high-dimensional feature matrix; M(X) is the feature representation after dimensionality reduction.

[0090] This embodiment realizes the hierarchical fusion and nonlinear transformation of features through a multi-scale feature pyramid and a deep autoencoder network. The attention mechanism is used to adaptively adjust the weights of hierarchical features, and the dimension reduction mapping is performed through the manifold learning framework to achieve the optimized reconstruction of the feature space. The introduction of the deep autoencoder network makes the feature fusion process have stronger nonlinear expression capabilities and can capture complex feature association patterns. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem of limited expression capabilities of traditional feature fusion methods, and improves the expression capabilities of features through multi-level feature fusion and nonlinear transformation. Compared with traditional feature fusion methods, this embodiment can better retain the essential information of features, while achieving effective reduction of dimensions and improving the efficiency and accuracy of subsequent classification.

[0091] like Figure 6 As shown, according to one aspect of the present application, step S5 is further:

[0092] S51, dividing the fused feature vector into feature space to obtain multi-layer feature data; constructing basic classifiers for the multi-layer feature data respectively to obtain classifier group data; organizing the classifier group data according to a tree structure to generate classification result data;

[0093] S52. Based on the classification result data, calculate the probability distribution to obtain probability distribution data; perform entropy calculation on the probability distribution data to obtain entropy data; calculate the classification reliability based on the entropy data to generate a confidence score;

[0094] S53, performing weighted combination of the classification result data and the confidence score to obtain judgment basis data; filtering the judgment basis data according to a preset threshold to obtain filtering result data; determining the classification category based on the filtering result data to generate a final classification result.

[0095] In one embodiment of the present application, a hierarchical classifier algorithm: P(y|x) = ∑αk·Pk(y|x)·exp(-Dk(x,ck)); wherein Pk(y|x) is the probability output of the kth base classifier, Dk(x,ck) is the distance from the sample to the class center, αk is the classifier weight coefficient, optimized by the differential evolution algorithm; y is the category label, x is the input feature; P(y|x) is the final classification probability.

[0096] Reliability evaluation algorithm: C(x) = η1·[1 - H(p)] / log(M) + η2·exp(-V(p) / μ) + η3·S(x); where H(p) is the probability distribution entropy, M is the number of categories, V(p) is the probability variance, S(x) is the sample sparsity, and μ is the variance threshold parameter; η1, η2, and η3 are weight coefficients; x is the input sample; and C(x) is the reliability score.

[0097] This embodiment achieves accurate classification of dysphagia and quantitative evaluation of the reliability of the results through hierarchical classifiers and reliability assessment mechanisms. The classifier group organized in a tree structure realizes multi-level division and progressive classification of the feature space. The reliability assessment mechanism combined with probability distribution and entropy calculation provides credibility support for the classification results. In the dysphagia diagnosis scenario, this embodiment effectively solves the problem that traditional single classifiers are difficult to deal with complex diseases. Through multi-layer classification and reliability assessment, the accuracy and credibility of the diagnosis are improved. Compared with traditional classification methods, this embodiment can better handle boundary conditions and uncertainties, reduce the misdiagnosis rate, and provide more reliable support for clinical decision-making.

[0098] According to one aspect of the present application, step S51 is further:

[0099] S511. Construct a multi-scale feature decomposition network, perform hierarchical processing on the fused feature vector, and obtain initial hierarchical data; use an adaptive clustering algorithm to spatially partition the initial hierarchical data to obtain partitioned feature data; optimize the partitioned feature data based on the information gain criterion to generate multi-layer feature data.

[0100] S512. Establish a heterogeneous integrated learning framework for multi-layer feature data to obtain basic model data; use a differential evolution algorithm to optimize the parameter configuration of the basic model data to obtain optimized model data; use a cross-validation method to evaluate the performance of the optimized model data and generate classifier group data.

[0101] S513, construct a dynamic decision tree structure, organize the classifier group data hierarchically, and obtain initial tree structure data; use an adaptive pruning algorithm to optimize the initial tree structure data to obtain optimized tree structure data; perform decision fusion on the optimized tree structure data based on a voting mechanism to generate classification result data.

[0102] In one embodiment of the present application, the adaptive clustering algorithm: C(X, k) = ∑[exp(-||xi - μk|| 2 / 2σk 2 )·πk]·[1 + β·H(pk)]; where μk is the center of the kth class, σk is the intra-class variance, πk is the prior probability, H(pk) is the intra-class entropy, β is the adaptive coefficient, xi is the sample point, X is the feature set, k is the number of categories, and C(X, k) is the clustering objective function.

[0103] Differential evolution algorithm: DE(x, F, CR) = xi + F·(xr1 - xr2) + CR·(xbest - xi)·exp(-g / gmax); where F is the scaling factor, CR is the crossover rate, xr1 and xr2 are randomly selected individuals, xbest is the current optimal solution, g is the current generation, and gmax is the maximum generation; x is the parameter vector; DE(x, F, CR) is the new solution after evolution.

[0104] This embodiment realizes the hierarchical construction and decision fusion of classifiers through multi-scale feature decomposition and heterogeneous ensemble learning. Combined with the adaptive clustering algorithm and the information gain criterion, the optimal partitioning of the feature space is realized; the basic model parameters are optimized through the differential evolution algorithm, and the reliability of the classifier performance is guaranteed by combining the cross-validation method. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problem that a single classifier is difficult to cope with the classification of complex diseases. By constructing a dynamic decision tree structure and combining an adaptive pruning algorithm, the optimized organization and decision fusion of the classifier are realized, and the accuracy and reliability of classification are improved. Compared with traditional classification methods, this embodiment has stronger generalization ability and robustness, and can better adapt to the individual differences of different patients.

[0105] According to one aspect of the present application, it also includes: S6, obtaining the final classification result, confidence score and quality score matrix, comprehensively evaluating the classification performance, and obtaining evaluation report data; adjusting system parameters according to the evaluation report data to generate optimized parameter data.

[0106] Step S61, calculate the classification accuracy of the final classification result to obtain accuracy data; calculate the system stability index in combination with the confidence score and the quality score matrix to obtain stability data; perform error pattern statistics on the final classification result to obtain error statistical data; integrate the accuracy data, stability data and error statistical data to generate evaluation report data.

[0107] Step S62: extract performance bottleneck indicators based on the evaluation report data to obtain optimization target data; perform parameter sensitivity analysis on the optimization target data to obtain sensitivity data; calculate parameter adjustment amounts based on the sensitivity data to generate optimization parameter data.

[0108] Step S63, distribute the optimized parameter data to each processing module to obtain parameter distribution data; verify the validity of the parameter distribution data to obtain verification result data; update the system parameter configuration according to the verification result data to complete the parameter optimization.

[0109] In one embodiment of the present application, the parameter optimization algorithm is: θ*(t) = argmin{L(θ) + λ1·||∇θ||1 + λ2·R(θ)}; wherein L(θ) is the performance loss function, ||∇θ||1 is the L1 norm of the parameter gradient, R(θ) is the regularization term, λ1 and λ2 are regularization coefficients; θ is the system parameter vector; θ*(t) is the optimized parameter configuration.

[0110] This embodiment achieves continuous optimization and performance improvement of the system through comprehensive performance evaluation and parameter optimization mechanisms. A comprehensive evaluation system is constructed by combining classification accuracy, system stability and error pattern analysis. The parameter optimization strategy based on the evaluation results ensures the effectiveness of the optimization through sensitivity analysis and verification. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the performance degradation problem that may occur in the long-term operation of the system, and ensures the stability of the diagnostic performance through continuous optimization. Compared with the traditional fixed parameter system, this embodiment can be adaptively adjusted according to the actual application effect, which improves the adaptability and reliability of the system and provides better protection for clinical applications.

[0111] According to one aspect of the present application, step S61 is further:

[0112] S611. Construct a multidimensional confusion matrix model, perform category statistics on the final classification results, and obtain category statistical data; use the bootstrap resampling method to perform confidence interval estimation on the category statistical data to obtain interval estimation data; calculate the classification performance index based on the interval estimation data to generate accuracy data.

[0113] S612. Construct a dynamic stability assessment model, map the confidence score and quality score matrix to a unified assessment space, and obtain mapped score data; use an adaptive weighted method to fuse the mapped score data to obtain fused score data; calculate the fluctuation characteristics of the fused score data based on a time series sliding window to generate stability data.

[0114] S613. Establish an error pattern clustering model, extract features from samples with classification errors, and obtain error feature data; use a hierarchical clustering algorithm to perform pattern mining on the error feature data to obtain pattern clustering data; combine expert rules to semantically annotate the pattern clustering data to generate error statistics.

[0115] S614. Construct a multi-criteria decision-making fusion framework, establish an evaluation index system for accuracy data, stability data and error statistics data, and obtain index system data; use fuzzy comprehensive evaluation methods to quantitatively evaluate the index system data to obtain comprehensive evaluation data; perform graded processing on the comprehensive evaluation data according to the evaluation standards to generate evaluation report data.

[0116] In one embodiment of the present application, the dynamic stability evaluation algorithm: S(t) = ω1·exp(-||yt – y(t-1)|| 2 / τ) + ω2·(1 - H(pt) / log(K)) + ω3·R(t); where yt is the system output at time t, H(pt) is the probability entropy, K is the number of categories, R(t) is the time series correlation coefficient, τ is the time scale parameter, ω1, ω2, ω3 are adaptive weights; t is the time variable; S(t) is the stability score.

[0117] This embodiment achieves comprehensive evaluation and optimization guidance of system performance through multi-dimensional performance evaluation and error pattern analysis. The bootstrap resampling method is used for confidence interval estimation, the stability performance of the system is analyzed through the dynamic stability evaluation model, and the potential problems of the system are identified in combination with error pattern clustering analysis. In the swallowing disorder diagnosis scenario, this embodiment effectively solves the problems of incomplete system performance evaluation and unclear optimization direction. By constructing a multi-criteria decision fusion framework and combining it with a fuzzy comprehensive evaluation method, scientific evaluation and classification of system performance are achieved, providing clear direction guidance for system optimization. Compared with traditional performance evaluation methods, this embodiment can more comprehensively evaluate system performance, provide more operational optimization suggestions, and effectively improve the overall performance and reliability of the system.

[0118] In another embodiment of the present application, step S1 may also be: acquiring the original ultrasound image sequence and the original electromyography signal sequence, performing coarse alignment preprocessing based on cross-correlation on the acquired sequences to obtain preliminary alignment data; performing precise alignment processing on the preliminary alignment data using a multi-scale DTW algorithm to obtain precise alignment data; performing multi-scale decomposition and adaptive noise suppression processing on the precise alignment data to obtain denoised data.

[0119] S1a, the acquisition device respectively acquires the original ultrasound image sequence and the original electromyographic signal sequence of the patient during swallowing, performs digital preprocessing on the acquired sequences to obtain digital signal data; performs data format unification processing on the digital signal data to generate standardized signal data.

[0120] S1b, calculating the mutual correlation coefficient sequence of the standardized signal data to obtain correlation coefficient data; determining the maximum correlation position according to the correlation coefficient data to obtain time offset data; and performing rough alignment processing on the signal sequence using the time offset data to generate preliminary alignment data.

[0121] S1c, construct a multi-scale wavelet decomposition framework, perform multi-scale decomposition on the preliminary alignment data, and obtain multi-scale feature data; construct a DTW cost matrix for the multi-scale feature data to obtain cost matrix data; use a dynamic programming algorithm to calculate the optimal alignment path to obtain alignment path data; perform precise alignment processing on the signal according to the alignment path data to generate precise alignment data.

[0122] S1d. Calculate the alignment error index for the precise alignment data to obtain error index data; calculate the alignment stability index based on the time domain characteristics of the signal to obtain stability index data; fuse the error index data and the stability index data to generate an alignment quality score.

[0123] In the process of dysphagia diagnosis, there are natural timing differences and nonlinear deformations in the acquisition of ultrasound images and electromyographic signals. This embodiment introduces a coarse alignment preprocessing based on cross-correlation to first achieve preliminary synchronization of signals at the macro level, which can quickly locate the corresponding relationship of key time points, reduce the search space for subsequent precise alignment, and reduce the computational complexity. The multiscale dynamic time warping (DTW) algorithm can adaptively handle the nonlinear time deformation caused by physiological differences in the swallowing process by constructing a warping path on different time scales, especially in the transition between the four stages of swallowing preparation, oral phase, pharyngeal phase and esophageal phase, and can accurately capture the characteristics of timing changes. At the same time, by calculating the alignment error index and stability index, a quantitative evaluation mechanism for alignment quality is established, providing a reliable data basis for subsequent feature extraction. This embodiment not only improves the accuracy of timing alignment, but also ensures stability under different pathological conditions, and improves the quality of multimodal data fusion.

[0124] In another embodiment of the present application, step S3 can also be: obtain enhanced data, first extract time domain features to obtain time domain feature vectors; extract frequency domain features based on the time domain feature vectors to obtain frequency domain feature vectors; extract time-frequency features by combining the time domain feature vectors and the frequency domain feature vectors to obtain time-frequency feature vectors; perform redundancy analysis and adaptive selection on the acquired feature vectors to generate an optimized feature matrix.

[0125] S3a, performing time domain segmentation processing on the enhanced data to obtain segmented data; calculating statistical feature parameters of the segmented data to obtain statistical feature data; performing feature importance evaluation on the statistical feature data to generate a time domain feature vector.

[0126] S3b, based on the time domain feature vector, the enhanced data is divided into frequency bands to obtain frequency band data; the energy distribution characteristics of each frequency band are calculated to obtain frequency band energy data; feature significance analysis is performed on the frequency band energy data to generate a frequency domain feature vector.

[0127] S3c, combining the time domain feature vector and the frequency domain feature vector to construct a joint analysis window to obtain the analysis window data; performing a time-frequency joint transformation on the analysis window data to obtain time-frequency distribution data; extracting the characteristic parameters of the time-frequency distribution data to generate a time-frequency feature vector.

[0128] S3d. Calculate the mutual information value between all feature vectors to obtain feature correlation data; construct a feature redundancy evaluation model based on the feature correlation data to obtain redundancy scoring data; use the redundancy scoring data to screen and combine features to generate an optimized feature matrix.

[0129] This embodiment adopts a progressive strategy of extracting frequency domain features guided by time domain features in view of the diagnostic characteristics of dysphagia. First, the key stages of swallowing are identified through time domain segmentation processing. These time domain features include the muscle contraction pattern and pharyngeal tissue movement characteristics during swallowing. Adaptive frequency band division is performed based on these time domain features, so that the extraction of frequency domain features is more in line with physiological characteristics and can accurately reflect the frequency characteristics of different swallowing stages. Through the joint analysis window of time domain features and frequency domain features, the precise extraction of instantaneous frequency features is achieved, which is of great significance for identifying abnormal patterns in dysphagia. At the same time, a feature redundancy analysis mechanism is introduced to screen out the most discriminative feature combination by calculating the mutual information value between features and constructing a redundancy evaluation model. This embodiment not only improves the physiological relevance of the features, but also reduces the computational complexity through feature optimization, so that the feature vector can better express the pathological characteristics of dysphagia.

[0130] In another embodiment of the present application, step S5 can also be: obtaining a fused feature vector, constructing a dynamic feature weight adjustment mechanism, and obtaining a weight adjustment feature; performing sample distribution balancing processing on the weight adjustment feature to obtain balanced feature data; using a hierarchical classifier to classify the balanced feature data to obtain classification result data; combining the classification result data to construct a model adaptive update mechanism to generate a final classification result.

[0131] S5a, calculating the feature importance score of the fused feature vector to obtain importance data; constructing a dynamic weight model based on the importance data and classification performance feedback to obtain weight parameter data; using the weight parameter data to weight the importance of the features to generate weight-adjusted features.

[0132] S5b, analyzing the category distribution characteristics of the weight adjustment features to obtain distribution characteristic data; constructing oversampling and undersampling strategies based on the distribution characteristic data to obtain sampling strategy data; balancing the samples according to the sampling strategy data to generate balanced characteristic data.

[0133] S5c, using the balanced feature data to train the basic classifier group to obtain base classifier data; constructing an online learning framework to perform performance evaluation on the base classifier data to obtain performance evaluation data; dynamically updating the classifier parameters according to the performance evaluation data to generate the final classification result.

[0134] In view of the large individual differences and diverse symptom manifestations of dysphagia patients, this embodiment introduces a dynamic feature weight adjustment mechanism based on a quality score matrix. First, through feature importance evaluation, adaptive weight allocation is performed on the feature manifestations of different patients to ensure that the classifier can highlight the individual pathological characteristics. In the sample distribution balance processing, the resampling strategy is guided by the quality score matrix, which effectively solves the problem of uneven distribution of different types of dysphagia samples. Through the dynamic adjustment of the classifier performance evaluation criteria, the adaptive optimization of the model parameters is achieved, so that the classifier can better adapt to the dynamic changes of the patient's status. This embodiment not only improves the accuracy of classification, but also enhances the generalization ability of the model for new samples. At the same time, through the reliability evaluation mechanism, it provides doctors with a more reliable diagnostic reference. In addition, the model adaptive update mechanism can continuously optimize the classification performance, so that the system maintains a high diagnostic accuracy during long-term use. The whole process forms a closed-loop self-optimization system, which improves the clinical practicality of intelligent diagnosis of dysphagia.

[0135] In another embodiment of the present application, step S21 may also be: obtaining denoised data, calculating the signal-to-noise ratio index to obtain signal-to-noise ratio data; combining energy entropy analysis to obtain energy entropy data; obtaining time-frequency purity data based on time-frequency analysis; and adaptively fusing these three types of data to generate a quality score matrix.

[0136] S21a, adaptively segmenting the denoised data according to the physiological characteristics of swallowing to obtain signal segment data; performing singular value decomposition on the signal segment data to obtain principal component data; calculating the deviation between the principal component data and the original data to generate signal-to-noise ratio data.

[0137] S21b, performing wavelet packet decomposition on the signal segment data to obtain coefficient matrix data; calculating the energy entropy characteristics of the coefficient matrix data to obtain energy entropy data; inputting the energy entropy data into a feature selection module for subsequent feature importance evaluation.

[0138] S21c. Perform time-frequency analysis on the signal segment data using Hilbert transform to obtain time-frequency spectrum data; evaluate the purity characteristics of the time-frequency spectrum data to obtain time-frequency purity data; and use the time-frequency purity data for quality control of subsequent feature extraction.

[0139] This embodiment introduces a multi-dimensional signal quality assessment mechanism to address the problem that swallowing disorder signals are susceptible to noise interference and signal quality directly affects the accuracy of diagnosis. First, the signal is divided into segments closely related to the swallowing process through an adaptive segmentation method based on swallowing physiological characteristics. This physiologically oriented segmentation strategy ensures the physiological relevance of quality assessment. The segmented signal is subjected to singular value decomposition, and a more accurate signal-to-noise ratio assessment is obtained by calculating the deviation between the principal component and the original data, which can better reflect the characteristics of swallowing signals than the traditional signal-to-noise ratio calculation method. At the same time, wavelet packet decomposition is introduced for energy entropy analysis. The energy entropy characteristics of the coefficient matrix can effectively evaluate the complexity and information richness of the signal, which is of great significance for identifying abnormal patterns of swallowing disorders. The time-frequency analysis is performed through Hilbert transform to evaluate the time-frequency purity of the signal, which can accurately reflect the changes in instantaneous frequency characteristics during the swallowing process. The quality assessment indicators of these three dimensions are adaptively fused to form a unified quality scoring matrix, which provides a reliable quality control basis for subsequent feature extraction and classification. This embodiment not only improves the accuracy of signal processing, but also provides doctors with a more reliable diagnostic basis, and enhances the reliability and practicality of the system in clinical applications.

[0140] According to one aspect of the present application, step S43 may also be:

[0141] S43a. Perform weighted processing on the feature matrix in combination with the quality score matrix to obtain weighted feature data; construct a multi-scale feature pyramid model, perform hierarchical processing on the weighted feature data, and obtain hierarchical feature data.

[0142] S43b. Analyze the local structure of hierarchical feature data based on the manifold learning algorithm to obtain local feature structure data; optimize the local feature structure data in combination with the global feature distribution to obtain optimized structure data; use the optimized structure data to guide feature dimensionality reduction to generate reduced dimensionality feature data.

[0143] S43c, fuse the local feature structure data with the global feature distribution information to obtain feature representation data; perform nonlinear transformation on the feature representation data to obtain transformed feature data; optimize the transformed feature data in combination with the quality score matrix to generate a fused feature vector.

[0144] In view of the multi-scale and complex hierarchical relationship of swallowing disorder signal features, this embodiment proposes a feature fusion strategy combined with a quality score matrix. First, the feature matrix is ​​weighted by the quality score matrix to ensure the dominant role of high-quality features in the fusion process. A multi-scale feature pyramid model is constructed for hierarchical processing. This hierarchical structure can retain swallowing feature information at different scales. By analyzing the local structure of the features through the manifold learning algorithm, the local dynamic change characteristics during the swallowing process can be accurately captured, especially the abnormal movement patterns shown by patients with swallowing disorders. The local structure is optimized in combination with the global feature distribution. This local-global combination strategy not only maintains the local detail features, but also maintains the consistency of the global features. In the process of feature dimensionality reduction, the discriminative ability of the features after dimensionality reduction is ensured by optimizing the guidance of the structural data. Finally, the local feature structure is fused with the global distribution information, the expression ability of the features is enhanced by nonlinear transformation, and the quality score matrix is ​​combined for optimization to generate the final fused feature vector. This embodiment not only improves the expression ability of the features, but also enhances the model's ability to recognize different types of swallowing disorders, and improves the accuracy and reliability of classification.

[0145] In a further embodiment, step S5 can also be: obtaining a fused feature vector and a quality score matrix, optimizing the features based on a dynamic feature weight adjustment mechanism to obtain weight-adjusted features; performing sample distribution balancing processing on the weight-adjusted features to obtain balanced feature data; classifying the balanced feature data using a hierarchical classifier and performing reliability assessment in combination with the quality score matrix to obtain classification result data and a confidence score; constructing a model adaptive update mechanism based on the classification result data and the confidence score to generate a final classification result.

[0146] S5i. Construct a feature importance evaluation model based on the quality score matrix and local feature structure data to obtain importance score data; dynamically adjust feature weights according to the importance score data to generate weight adjustment features.

[0147] S5ii. Use the quality score matrix to guide the sample distribution balance processing to obtain sample weight data; resample the weight adjustment features according to the sample weight data to generate balanced feature data.

[0148] S5iii. Construct classifier performance evaluation criteria in combination with the quality score matrix to obtain evaluation criterion data; use the evaluation criterion data to guide classifier parameter optimization to obtain optimized parameter data; update the classifier model according to the optimized parameter data to generate the final classification result.

[0149] In the swallowing disorder diagnosis scenario, this embodiment establishes a quality-driven classification framework by integrating the quality scoring matrix throughout the entire process of feature weight adjustment, sample balance and classifier optimization. Specifically, when evaluating the importance of features, an evaluation model is constructed by combining the quality scoring matrix and local feature structure data, so that the adjustment of feature weights not only takes into account signal quality factors, but also maintains the local structural relationship of features, and can more accurately identify key feature patterns in swallowing disorders. In the sample balancing process, the quality scoring matrix directly guides the allocation of sample weights, ensuring the dominant role of high-quality samples in model training, while also avoiding the negative impact of low-quality samples on classifier performance. In the classifier optimization stage, the quality scoring matrix participates in the construction of performance evaluation criteria, providing more reliable guidance for model parameter optimization. This embodiment improves the classification system's recognition accuracy for swallowing disorders of varying degrees, especially for complex cases with unstable signal quality.

[0150] In order to solve the reliability problem of dysphagia diagnosis results, a confidence scoring mechanism was added and combined with the classification result data to build a more complete model adaptive update mechanism. First, by combining the quality scoring matrix to build a performance evaluation criterion, a quantitative evaluation of the reliability of the classification results was achieved. In the evaluation process, not only the accuracy of the classification was considered, but also the confidence of the results was included, which provided doctors with more comprehensive diagnostic reference information. At the same time, by guiding the optimization of classifier parameters through the evaluation criterion data, the system can adaptively adjust the classification strategy according to the characteristics of different cases. This reliability-driven optimization mechanism is particularly suitable for handling difficult cases in dysphagia. It can timely identify and mark low-confidence classification results, remind doctors to conduct further examinations, and improve the practicality and credibility of the system in clinical applications. In the sample distribution balance processing link, a weight allocation mechanism based on quality scoring was introduced. This improvement is particularly aimed at the common class imbalance and quality unevenness problems in dysphagia data. By guiding the sample distribution balance processing through the quality scoring matrix, the system can simultaneously consider the class distribution and quality characteristics of the sample and generate more reasonable sample weight data. During the resampling process, the system processes the weight adjustment features according to the sample weight data. This quality-based resampling strategy not only balances the distribution of samples of different categories, but also ensures that the resampled data set maintains a high quality level. It improves the model's ability to identify rare types of swallowing disorders, while avoiding the over-amplification of low-quality samples during the resampling process, providing a better data foundation for subsequent classifier training.

[0151] According to one aspect of the present application, a multimodal intelligent classification method for dysphagia based on ultrasound images and electromyographic signals comprises the following steps:

[0152] Step 1: Data collection.

[0153] Ultrasound equipment is used to obtain dynamic images of the larynx, tongue, and pharynx during swallowing. Ultrasound images provide spatial and temporal information about swallowing movements, and can capture the dynamic process of muscle contraction and food passing through the pharynx. High-density surface electromyography (HD-sEMG) sensors are used to collect myoelectric activity during swallowing in the neck area. EMG signals reflect the electrical activity of muscles during swallowing and can provide electrophysiological information about swallowing movements.

[0154] Step 2: Data preprocessing.

[0155] The collected ultrasound images are processed by noise removal, image enhancement, contrast adjustment, etc. to improve image quality. Common image processing methods include edge detection, filtering, denoising, etc. The electromyographic signals are denoised, signal filtered (such as low-pass and high-pass filters), and normalized. Filters are used to remove interference components in the electromyographic signals (such as motion artifacts, electrical noise, etc.). Signals in different time periods are processed in segments to facilitate subsequent feature extraction.

[0156] Step 3: Feature extraction.

[0157] Extract dynamic features of the larynx, tongue, and pharynx during swallowing, such as muscle contraction and relaxation, and time / space information of the swallowing process. Use image processing methods (such as edge detection and region segmentation) to extract geometric features, motion features, and relative position information of key areas. Convolutional neural networks (CNNs) can be used to extract more advanced visual features (such as muscle contraction patterns, food movement trajectories, etc.) from images. Extract time domain features: such as root mean square (RMS), integrated electromyography (IEMG), maximum amplitude, etc., to describe the intensity and duration of muscle activity. Extract frequency domain features: such as power spectral density (PSD), frequency band features, etc., to capture the frequency distribution of muscle activity. Extract time-frequency domain features: Extract the time-frequency features of the signal through methods such as short-time Fourier transform (STFT) to analyze the temporal changes of muscle activity. Extract spatial features: Based on high-density electromyography, analyze the spatial distribution characteristics between different electrodes and identify the activity patterns of different muscle groups. Fusion of ultrasound image features with electromyographic signal features to construct a unified feature representation.

[0158] Step 4: Model training.

[0159] Select machine learning models: supervised learning methods such as support vector machine (SVM), logistic regression, K-nearest neighbor (KNN), Gaussian mixture model (GMM), etc., are used to classify multimodal features. Deep learning methods: use convolutional neural network (CNN) to extract high-level features of ultrasound images, combined with long short-term memory network (LSTM) for time series data modeling, and extract deep time series features from electromyographic signals and ultrasound images. Use the data of existing dysphagia patients and healthy controls for data and label training. Labels can be based on clinical diagnosis (such as whether there is dysphagia, swallowing stage classification, etc.) or movement patterns in ultrasound images and electromyographic signals.

[0160] Step 5: Model evaluation.

[0161] Use accuracy, recall, F1 score, ROC curve, AUC, etc. to evaluate the performance of the classification model. For multi-classification problems, confusion matrix, weighted F1 score, etc. can be used for comprehensive evaluation. Use cross-validation method to verify the model to avoid overfitting and ensure the generalization ability of the model.

[0162] This embodiment uses the location annotation of the hyoid bone and infrahyoid muscle and the electromyographic signal characteristics in the ultrasound video to realize the automatic classification of healthy people and patients with dysphagia. It combines multiple machine learning algorithms, including ARMA model, hidden Markov model (HMM) and expectation maximization (EM) algorithm, logistic regression, support vector machine (SVM), K-nearest neighbor, CNN and LSTM, and can non-invasively quantify the swallowing function of patients, improve diagnostic efficiency and reduce patient discomfort.

[0163] According to one aspect of the present application, a swallowing disorder intelligent classification system based on multimodality of ultrasound images and electromyographic signals includes:

[0164] at least one processor; and,

[0165] a memory communicatively connected to at least one of the processors; wherein,

[0166] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent classification method for dysphagia based on ultrasound images and electromyographic signal multimodality as described in any of the above embodiments.

[0167] The present invention realizes the intelligent diagnosis and classification of dysphagia by constructing a complete closed loop of multimodal data processing, feature extraction, classification decision and optimization feedback. It integrates key links such as data preprocessing, quality assessment, feature engineering, classification decision and performance optimization to form a systematic technical solution. In the dysphagia diagnosis scenario, the present invention comprehensively solves the problems of strong subjectivity, low accuracy and poor reliability of traditional diagnostic methods, and provides an objective, accurate and reliable diagnostic basis through intelligent analysis of multimodal data. Compared with traditional single modality or simple fusion methods, the present invention can more comprehensively capture the abnormal characteristics of swallowing function, improve the accuracy and reliability of diagnosis, and provide strong technical support for clinical practice. Through the closed-loop optimization mechanism, the system can continuously improve and adapt to different clinical environments, and has good practicality and promotion value.

[0168] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. An intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals, characterized in that: The steps include: S1, obtaining an original ultrasound image sequence and an original electromyographic signal sequence, and performing time sequence alignment processing on the original ultrasound image sequence and the original electromyographic signal sequence to obtain aligned data; Perform multi-scale decomposition and adaptive noise suppression on the aligned data to obtain denoised data; S2. Based on the denoised data, a multidimensional signal quality score is calculated to obtain a quality score matrix; according to the quality score matrix, the denoised data is adaptively enhanced to obtain enhanced data; S3. Based on the enhanced data, extract the time domain features, frequency domain features and time-frequency domain features respectively to obtain the time domain feature vector, frequency domain feature vector and time-frequency feature vector; combine the time domain feature vector, frequency domain feature vector and time-frequency feature vector to generate a feature matrix; S4. Based on the feature matrix and the quality score matrix, calculate the feature correlation index to obtain correlation index data; calculate the feature weight according to the correlation index data and the quality score matrix to obtain a feature weight vector; Using the feature weight vector, perform multi-layer fusion processing on the feature matrix to generate a fused feature vector; S5. Based on the fused feature vector, feature classification is performed through a hierarchical classifier to obtain classification result data; Conduct reliability assessment on the classification result data and generate a confidence score; make a comprehensive judgment based on the classification result data and the confidence score and output the final classification result; Step S4 is further as follows: S41. Calculate the Pearson correlation coefficient based on the characteristic components in the characteristic matrix to obtain correlation coefficient data; Based on the characteristic components in the characteristic matrix, the mutual information value is calculated to obtain the mutual information data; the correlation coefficient data and the mutual information data are weightedly combined to generate the correlation index data; S42. Use the correlation index data and the quality score matrix to construct a weight calculation model; According to the weight calculation model, the feature importance score is calculated to obtain the importance data; the importance data is normalized to generate a feature weight vector; S43, based on the feature weight vector, weighting the feature matrix to obtain preliminary fused data; performing dimensionality reduction processing on the preliminary fused data to obtain reduced-dimensional feature data; performing nonlinear transformation on the reduced-dimensional feature data to generate a fused feature vector; Step S43 is further as follows: S431, construct a multi-scale feature pyramid model, perform hierarchical processing on the feature matrix and the feature weight vector to obtain hierarchical feature data; use the attention mechanism to adjust the weight of the hierarchical feature data to obtain adjusted feature data; perform weighted summation on the adjusted feature data to generate preliminary fusion data; S432, based on the preliminary fusion data, construct a manifold learning framework to obtain local feature structure data; The spectral clustering method is used to reduce the dimension of the local feature structure data to obtain the mapping feature data; the mapping feature data is optimized based on the distance-preserving principle to generate the reduced dimension feature data; S433. Construct a deep autoencoding network to encode the reduced-dimensional feature data to obtain feature-encoded data; use a back-propagation algorithm to optimize the feature-encoded data to obtain optimized feature data; use an activation function to perform a nonlinear transformation on the optimized feature data to generate a fused feature vector.

2. The intelligent classification method for dysphagia based on ultrasound images and electromyographic signals according to claim 1 is characterized in that: Step S1 is further as follows: S11, the acquisition device respectively acquires the original ultrasound image sequence and the original electromyographic signal sequence during the swallowing process of the patient, performs image digitization processing on the original ultrasound image sequence, performs analog-to-digital conversion processing on the original electromyographic signal sequence, and generates digital signal data; S12, constructing a time warping matrix, and projecting the digitized signal data onto a unified time axis, calculating the Euclidean distance between the original ultrasound image sequence and the original electromyographic signal sequence; aligning the original ultrasound image sequence and the original electromyographic signal sequence by minimizing the Euclidean distance, and generating aligned data; S13, performing multi-scale decomposition on the aligned data using wavelet packet transform to obtain a coefficient matrix; According to the coefficient matrix, the energy distribution of each decomposition layer is calculated and the adaptive threshold function is determined; Based on the adaptive threshold function, the coefficient matrix is ​​processed and the denoised data is obtained through wavelet packet reconstruction.

3. The intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals according to claim 2 is characterized in that: Step S2 is further as follows: S21, calculating the signal-to-noise ratio index of the denoised data to obtain signal-to-noise ratio data; calculating the signal continuity index of the denoised data to obtain continuity data; analyzing the spectrum purity index of the denoised data to obtain spectrum purity data; The signal-to-noise ratio data, continuity data and spectral purity data are weighted and combined to generate a quality score matrix; S22. Based on the quality score matrix, construct an enhanced weight matrix; The denoised data is weighted using the enhancement weight matrix to obtain preliminary enhanced data; The preliminary enhanced data is adaptively equalized to generate enhanced data.

4. The intelligent classification method for dysphagia based on ultrasound images and electromyographic signals according to claim 3 is characterized in that: Step S3 is further as follows: S31, extracting the envelope of the enhanced data to obtain envelope data; based on the envelope data, calculating the mean, variance, skewness and kurtosis to obtain statistical feature data; Normalize the statistical feature data to generate a time domain feature vector; S32, performing fast Fourier transform on the enhanced data to obtain spectrum data; based on the spectrum data, calculating the energy distribution of the characteristic frequency band to obtain frequency band energy data; performing normalization processing on the frequency band energy data to generate a frequency domain feature vector; S33, performing Hilbert transform on the enhanced data to obtain analytical signal data; based on the analytical signal data, calculating the instantaneous frequency and instantaneous amplitude to obtain time-frequency feature data; performing normalization processing on the time-frequency feature data to generate a time-frequency feature vector; S34, concatenating and combining the time domain feature vector, the frequency domain feature vector and the time-frequency feature vector according to a preset rule to generate a feature matrix.

5. The intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals according to claim 4 is characterized in that: Step S5 is further as follows: S51, dividing the fused feature vector into feature space to obtain multi-layer feature data; constructing basic classifiers for the multi-layer feature data respectively to obtain classifier group data; organizing the classifier group data according to a tree structure to generate classification result data; S52. Calculate probability distribution based on the classification result data to obtain probability distribution data; Calculate the entropy value of the probability distribution data to obtain entropy value data; calculate the classification reliability based on the entropy value data to generate a confidence score; S53, performing weighted combination of the classification result data and the confidence score to obtain judgment basis data; The judgment basis data is screened according to a preset threshold value to obtain screening result data; According to the screening result data, the classification category is determined and the final classification result is generated.

6. The intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals according to claim 5, characterized in that: Step S11 is further as follows: S111, obtaining an original ultrasound image sequence, performing inter-frame difference calculation on the original ultrasound image sequence to obtain change detection data; performing threshold detection on the change detection data to obtain swallowing event data; performing grayscale quantization on the original ultrasound image sequence to obtain quantized data; Performing spatial sampling on the quantized data to obtain sampled data; Perform data compression encoding on the sampled data to generate image digital data; S112, obtaining an original electromyographic signal sequence, performing envelope detection on the original electromyographic signal sequence to obtain envelope data; performing peak recognition on the envelope data to obtain swallowing mark data; performing anti-aliasing filtering on the original electromyographic signal sequence to obtain filtered data; The filtered data is sampled and held to obtain sampled and held data; the sampled and held data is quantized and encoded to generate electromyographic digital data; S113, time-aligning the swallowing event data with the swallowing mark data to obtain synchronization mark data; time-aligning the image digitized data and the electromyography digitized data according to the synchronization mark data to obtain alignment data; The data format of the alignment data is unified to generate digital signal data.

7. The intelligent classification method for dysphagia based on ultrasound images and electromyographic signals multimodality according to claim 5, characterized in that: Step S21 is further as follows: S211, performing segment processing on the denoised data to obtain signal segment data; extracting principal components based on the signal segment data by adaptive singular value decomposition to obtain principal component data; calculating the signal-to-noise ratio according to the deviation between the principal component data and the signal segment data to generate signal-to-noise ratio data; S212, constructing a Hilbert space mapping based on the denoised data to obtain phase space trajectory data; Calculate the continuity characteristics of the phase space trajectory data to obtain trajectory characteristic data; evaluate the signal continuity based on the trajectory characteristic data to generate continuity data; S213, performing wavelet packet energy entropy analysis on the denoised data to obtain energy entropy data; Based on the energy entropy data, a spectrum purity evaluation model is constructed to obtain evaluation model data; based on the evaluation model data, the spectrum purity index is calculated to generate spectrum purity data; S214, constructing a fuzzy reasoning system based on the signal-to-noise ratio data, the continuity data and the spectrum purity data to obtain fuzzy rule data; Using fuzzy rule data, adaptive weight calculation is performed to obtain weight coefficient data; According to the weight coefficient data, the indicators are fused to generate a quality score matrix.

8. An intelligent classification system for dysphagia based on multimodality of ultrasound images and electromyographic signals, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent classification method for dysphagia based on multimodality of ultrasound images and electromyographic signals as described in any one of claims 1 to 7.

Images