An artificial intelligence-based hearing impairment recovery condition evaluation method

Abstract

The present application relates to the technical field of artificial intelligence, in particular to a hearing impairment recovery condition evaluation method based on artificial intelligence; comprising the following steps executed in sequence: S1: collecting hearing threshold data at different frequency points, the hearing threshold data including initial hearing threshold before treatment and current hearing threshold after treatment, recording the age, hearing impairment type and speech recognition rate in a noisy environment of the patient at the same time, constructing a training data set, and generating an initial hearing threshold curve; S2: performing multi-band hearing threshold feature reorganization on the training data set to obtain a multi-band feature matrix; S3: constructing an artificial intelligence evaluation model to obtain an optimal model; S4: performing hearing evaluation by using the optimal model to output a predicted recovery level; adopting adaptive convolution kernel generation technology, generating a dynamic convolution kernel based on the initial hearing threshold curve of the patient, thereby enhancing the recognition ability of the model to individualized recovery patterns and focusing on key impaired frequency bands.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an artificial intelligence-based method for assessing the recovery of hearing loss. Background Technology

[0002] With the accelerating aging of the global population, hearing loss has become a significant health issue, particularly among the elderly, where its incidence has risen dramatically. Hearing loss not only impacts an individual's social life and quality of life but also negatively affects their mental health. To address this issue, hearing restoration technologies and assistive devices have been developed, but many challenges remain.

[0003] Traditional hearing recovery assessment methods neglect the physiological connections between frequency bands and fail to effectively utilize the physiological characteristics of the hearing threshold band, resulting in insufficient refinement of frequency feature processing and an inability to accurately capture cross-frequency band recovery trends. Summary of the Invention

[0004] The purpose of this invention is to provide an artificial intelligence-based method for assessing hearing loss recovery that can precisely process frequency characteristics.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An artificial intelligence-based method for assessing hearing loss recovery includes the following steps performed sequentially:

[0007] S1: Collect hearing threshold data at different frequency points. This hearing threshold data includes the initial hearing threshold before treatment and the current hearing threshold after treatment. At the same time, record the patient's age, hearing impairment type, and speech recognition rate in noisy environments, construct a training dataset, and generate an initial hearing threshold curve.

[0008] S2: Multi-band hearing threshold feature reconstruction for the training dataset: Divide the training dataset into three physiologically relevant sub-bands: low frequency band, mid frequency band and high frequency band. Calculate the average hearing threshold improvement value of each sub-band and integrate the age normalization factor, hearing impairment type coding and noise speech recognition rate normalization value to obtain the multi-band feature matrix.

[0009] S3: Build an AI evaluation model and obtain the best model;

[0010] The specific steps for constructing the artificial intelligence evaluation model in step S3 are as follows:

[0011] S3-1: Smooth the initial hearing threshold curve to obtain a smoothed hearing threshold curve vector, and fuse the smoothed hearing threshold curve vector with the multi-band feature matrix to obtain a dynamic convolution kernel matrix;

[0012] S3-2: The dynamic convolution kernel matrix is ​​used to perform convolution operation on the multi-band feature matrix. After the first modulation, the time-frequency feature map is obtained. The multi-band feature matrix is ​​then subjected to max pooling downsampling and the second modulation. The matrix after the second modulation is added element by element to the time-frequency feature map to obtain the time-frequency enhanced feature matrix.

[0013] S3-3: Perform sub-band feature vector mapping to obtain query vector and key vector. Calculate the attention weight matrix between sub-bands using the Softmax function. Multiply the attention weight matrix with the transposed time-frequency enhancement feature matrix and obtain the enhanced interaction feature matrix based on the synergistic effect of the sub-band improvement value vectors.

[0014] S3-4: Calculate the Bark-scale perceptual weight matrix of each sub-band using the Sigmoid function, multiply the Bark-scale perceptual weight matrix with the enhanced interactive feature matrix, and combine it with the damage type coding modulation to obtain the perceptual weighted feature matrix.

[0015] S3-5: The multi-band feature matrix, the time-frequency enhancement feature matrix, and the perceptual weighted feature matrix are tensor-concatenated along the row direction and scaled based on trainable scaling coefficients to obtain a concatenated composite feature matrix. The concatenated composite feature matrix is ​​flattened into a one-dimensional feature vector and a fused feature vector is obtained by capturing cross-scale nonlinear interactions based on a lightweight multilayer perceptron.

[0016] S3-6: Concatenate the fused feature vector with the sub-band improvement value vector, and calculate the predicted recovery level category based on the class probability distribution of the fully connected layer and the Softmax function. Calculate the difference-sensitive loss function, and use the difference-sensitive loss function to backpropagate the model until the validation loss no longer decreases or the validation accuracy no longer improves within the preset period. Then, output the best model.

[0017] S4: Use this optimal model for hearing assessment and output the predicted recovery level.

[0018] Preferably, in step S2, the low frequency band includes three frequency points: 250Hz, 500Hz and 1kHz, the mid frequency band includes two frequency points: 2kHz and 4kHz, and the high frequency band includes two frequency points: 6kHz and 8kHz.

[0019] The age normalization factor is the patient's age divided by 100;

[0020] This type of hearing impairment is represented using unique-heat coding;

[0021] The noise speech recognition rate is a percentage value obtained through speech recognition tests in noisy environments.

[0022] Preferably, in step S3-1, the initial hearing threshold curve is smoothed by applying a Savitzky-Golay filter, and the extended vector of the average hearing threshold improvement value of each sub-band is fused to obtain the smoothed hearing threshold curve vector.

[0023] The smooth hearing threshold curve vector is fused with the multi-band feature matrix, and a linear transformation is performed based on the learnable weight matrix and bias vector. The Sigmoid function is then applied to generate the dynamic convolution kernel matrix.

[0024] Preferably, in step S3-2, the first modulation is frequency domain modulation of the matrix after convolution operation combined with the subband improvement value, and the activation function is applied to obtain the time-frequency feature map;

[0025] The second modulation is to modulate the pooling features by generating a gated vector based on the subband improvement value and the dynamic convolution kernel matrix. The dynamic convolution kernel matrix is ​​then used to perform a time-frequency joint convolution with the time-frequency feature map, so that the convolution operation is notified to slide along the frequency band dimension and the time dimension, which is used to capture the local correlation between the subband improvement value and individual attributes in the treatment cycle.

[0026] Preferably, in step S3-3, the specific operation of sub-band feature vector mapping is as follows: the feature vectors of each sub-band in the time-frequency feature map are fused with the sub-band improvement value and the multi-band feature matrix, and a linear transformation is performed based on the query weight matrix and the key weight matrix to obtain the query vector and the key vector, thereby enhancing the vector's ability to represent the recovery association.

[0027] Preferably, in steps S3-6, when backpropagating the model using the difference-sensitive loss function, the adaptive moment estimator optimizer is used to adjust the trainable parameters in the model. In each iteration, the model calculates the predicted recovery level from forward propagation and calculates the loss value based on the difference between the true label and the predicted result. The parameters are updated using the gradient descent algorithm to minimize the loss.

[0028] Preferably, in step S4, when using the optimal model for hearing assessment, the patient's initial hearing threshold, current hearing threshold, age, hearing impairment type, and noise speech recognition rate need to be input.

[0029] Preferably, in step S4, the output predicted recovery level includes four categories: no improvement, partial improvement, significant improvement, and complete recovery.

[0030] By adopting the aforementioned design scheme, the beneficial effects of the present invention are as follows: This application proposes a multi-band feature reconstruction method based on the theory of auditory critical bands, which divides hearing data into physiologically relevant low-frequency bands, mid-frequency bands, and high-frequency bands in order to better capture recovery patterns between frequency bands and retain cross-frequency band recovery correlations;

[0031] Adaptive convolution kernel generation technology is used to generate dynamic convolution kernels based on the patient's initial hearing threshold curve, thereby enhancing the model's ability to identify individualized recovery patterns and focusing on key damaged frequency bands;

[0032] By introducing a cross-band attention mechanism, the model can calculate the recovery correlation between different frequency bands, enhance its ability to capture recovery patterns across frequency bands, and especially adjust the focus of the recovery pattern according to the specific type of hearing loss of the patient.

[0033] By adopting the Bark scale-aware weighting scheme, higher weights are given to the recovery of key frequency bands in speech, enabling the model to more accurately focus on the impact of recovery on speech recognition and improve classification performance. Attached Figure Description

[0034] Figure 1 This is a bar chart comparing the accuracy of the evaluation method of this invention with other evaluation methods;

[0035] Figure 2 This is a distribution diagram of the hearing threshold improvement values ​​for each frequency sub-band of the present invention;

[0036] Figure 3 This is the probability density distribution of the improvement values ​​for each frequency subband in this invention;

[0037] Figure 4 This is a distribution chart of hearing recovery levels in patients of different age groups according to the present invention;

[0038] Figure 5 The bar chart shows the comparison of F1 scores between the present invention's method and conventional methods for different damage types. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0040] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0041] An artificial intelligence-based method for assessing hearing loss recovery includes the following steps performed sequentially:

[0042] S1: Collect hearing threshold data at different frequency points. This hearing threshold data includes the initial hearing threshold before treatment and the current hearing threshold after treatment. At the same time, record the patient's age, hearing impairment type, and speech recognition rate in noisy environments, construct a training dataset, and generate an initial hearing threshold curve.

[0043] In this embodiment, the process of collecting hearing recovery data is completed through clinical hearing tests and patient follow-up records. Specifically, it includes performing binaural hearing tests on patients in a standard soundproof room using a pure-tone audiometer, collecting hearing threshold data at frequencies of 250 Hz, 500 Hz, 1000 Hz, 2000 Hz, 4000 Hz, 6000 Hz, and 8000 Hz, and simultaneously recording the patient's age, type of hearing impairment, and speech recognition rate in noisy environments.

[0044] Data acquisition covers the initial hearing threshold before treatment and the current hearing threshold after treatment, ensuring the integrity of the time series of data.

[0045] The training dataset was constructed based on a multi-center clinical study, collecting a large amount of patient follow-up data, including multiple test records during the hearing recovery process, to cover different recovery stages and individual differences. Data annotation was performed by clinical experts according to the magnitude of hearing threshold changes and clinical recovery standards. The annotation categories were divided into four levels: no improvement, partial improvement, significant improvement, and complete recovery. No improvement indicates a hearing threshold change of less than 5 dB, partial improvement indicates an improvement of 5 to 15 dB, significant improvement indicates an improvement of 15 to 30 dB, and complete recovery indicates that the hearing threshold has returned to the normal range, i.e., less than 25 dB.

[0046] S2: Multi-band hearing threshold feature reconstruction of the training dataset: Based on the theory of auditory critical bands, the eight frequency points are reconstructed into three physiologically relevant sub-bands, including low-frequency, mid-frequency, and high-frequency bands. The average hearing threshold improvement value of each sub-band is calculated, and age normalization factors, hearing impairment type coding, and noise speech recognition rate normalization values ​​are integrated to form a multi-band feature matrix. This enhances the physical interpretability of the features and preserves cross-band recovery correlations. Represented as:

[0047] ;

[0048] Among them, the multi-band feature matrix The dimension is By fusing frequency domain improvements and individual differences, the physical interpretability of features is enhanced, while cross-frequency band recovery correlations are preserved. The age normalization factor is calculated by dividing the patient's age by 100. One-hot encoding, representing the type of hearing impairment, is used to discretely characterize the impairment category. The normalized value of speech recognition rate in a noisy environment is a percentage obtained through speech recognition tests in a noisy environment. Subband The average hearing threshold improvement value, measured in dBHL, reflects the overall recovery level of this frequency band, and is calculated as follows: , This represents the sub-band index, with values ​​for low-frequency band, mid-frequency band, or high-frequency band, corresponding to the physiological critical frequency band. Subband The number of frequency points included is used to calculate the mean. Represents frequency The initial hearing threshold, measured in dBHL, represents the hearing level before treatment. Represents frequency The current hearing threshold, measured in dBHL, represents the hearing level after treatment. This indicates a frequency point, with values ​​ranging from 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 6kHz, and 8kHz. These represent the frequency points for hearing tests. When it is a low frequency band, the corresponding This represents the average improvement in hearing threshold in the low-frequency band. When it is in the mid-frequency band, the corresponding This represents the average improvement in hearing threshold in the mid-frequency band. When it is a high-frequency band, the corresponding This represents the average improvement in hearing threshold in the high-frequency band;

[0049] In practical implementation, based on the theory of auditory critical bands, that is, the human ear's perception of frequency is not uniform, but rather in units of critical bands. The low frequency band covers the fundamental frequency of speech and some formants, the mid frequency band corresponds to the key area of ​​speech intelligibility, and the high frequency band reflects auditory sensitivity in noisy environments. Therefore, the low frequency band (250Hz-1kHz) includes three frequency points: 250Hz, 500Hz, and 1kHz; the mid frequency band (2kHz-4kHz) includes two frequency points: 2kHz and 4kHz; and the high frequency band (6kHz-8kHz) includes two frequency points: 6kHz and 8kHz.

[0050] It should be noted that the frequency points were divided into three physiologically relevant sub-bands through critical band recombination, and the average improvement value of each sub-band was calculated. The age normalization factor, damage type coding, and noise speech recognition rate normalization value were integrated to form a multi-band feature matrix. Simultaneously, it captures frequency domain recovery trends and individual attribute interactions, such as age normalization factor. Eliminating the influence of age dimension, and using unique heat encoding for hearing loss type. Discrete representation of damage categories enables the model to distinguish recovery patterns for different damage types, achieving the effect of integrating frequency domain and attribute information.

[0051] S3: Train the artificial intelligence evaluation model, which includes the following steps:

[0052] S3-1: Dynamic convolution kernel matrix generation: Traditional convolution kernels have fixed weights, which cannot adapt to the heterogeneity of individual hearing curves, making it difficult for feature extraction to focus on the patient's key damaged frequency bands and reducing the model's ability to recognize personalized recovery patterns.

[0053] This invention, based on the patient's initial hearing threshold curve, generates an adaptive convolution kernel through smoothing filtering and parametric transformation. The kernel weights are positively correlated with the initial hearing loss distribution, thereby achieving individualized feature extraction and enhancing the model's ability to focus on key damaged frequency bands. The specific steps are as follows:

[0054] Initial hearing threshold curve smoothing: The initial hearing threshold curve is smoothed by applying a Savitzky-Golay filter, and the extended vectors of the average hearing threshold improvement values ​​of each sub-band are fused to obtain a smoothed hearing threshold curve vector. This smoothed vector is used to eliminate measurement noise. By fusing the initial hearing threshold and the improvement trend after smoothing, the curve's representation of the recovery mode is enhanced. This process eliminates measurement noise and enhances the physiological correlation between the curve and the recovery mode. Expressed by the following formula:

[0055] ;

[0056] in, The multi-band feature fusion coefficient is preferably set to 0.1. This coefficient is used to adjust the contribution of the sub-band improvement value to the smoothing curve.

[0057] The extended vector representing the average hearing threshold improvement value of each sub-band;

[0058] The Savitzky-Golay filter function is represented by a quadratic polynomial fitting and a sliding window calculation, with a window length of 5 and a polynomial order of 2.

[0059] This represents the initial hearing threshold curve vector, with dimension . It is composed of the initial hearing threshold at each frequency point, i.e. ;

[0060] Representing frequency point The initial hearing threshold, measured in dBHL, represents the hearing level before treatment;

[0061] This indicates the frequency index, with values ​​of 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 6kHz, and 8kHz, corresponding to the standard hearing test frequencies.

[0062] This indicates the transpose operation.

[0063] In its implementation, the Savitzky-Golay filter uses a sliding window of length 5 to adjust the initial hearing threshold curve vector. Local quadratic polynomial fitting is performed, and the polynomial is fitted using the least squares method within each window, thereby outputting a smooth value.

[0064] In one implementation, there are four types of damage: conductive, sensorineural, mixed, and unknown. Characterizes sensorineural hearing loss.

[0065] In practical implementation, the extended vector of the average hearing threshold improvement value in each sub-band In the calculation process, this is achieved by mapping each frequency point to the average improvement value of its respective sub-band, that is, The low-frequency band (250Hz-1kHz) corresponds to the first three frequency points, the mid-frequency band (2kHz-4kHz) corresponds to the fourth and fifth frequency points, and the high-frequency band (6kHz-8kHz) corresponds to the last two frequency points. By improving the sub-band trend, the physiological correlation of smoothing the hearing threshold curve is enhanced, thus... While eliminating noise, prior information about the recovery direction is fused.

[0066] Dynamic convolution kernel parameterization generation: By fusing the smoothed hearing threshold curve vector with the multi-band feature matrix, and performing a linear transformation based on the learnable weight matrix and bias vector, a dynamic convolution kernel matrix is ​​generated using the Sigmoid function. The dimension of this dynamic convolution kernel matrix is... This is used to adaptively extract key frequency band information, thereby making the convolution kernel weights positively correlated with the initial hearing loss distribution, achieving individualized feature extraction, and a dynamic convolution kernel matrix. Expressed by the following formula:

[0067] ;

[0068] In the formula, This indicates a concatenation operation;

[0069] This represents the flattening operation function, which rearranges the input matrix into a one-dimensional vector in row-major order.

[0070] This represents the Sigmoid activation function, which constrains the output value to... interval;

[0071] This represents the dynamic convolution weight matrix, which consists of trainable parameters.

[0072] This represents the dynamic convolution bias vector, which is a trainable parameter.

[0073] It should be noted that by using smoothing filtering and parameterized transformation, the convolution kernel weights are made positively correlated with the initial hearing loss distribution. This allows the smoothed hearing threshold curve to be fused with the multi-band feature matrix, generating a dynamic convolution kernel matrix. It can adaptively adjust the convolution kernel according to the individual's hearing loss pattern and recovery trend, thus focusing more on the key damaged frequency bands. In addition, it introduces a multi-band feature matrix. Subsequently, the convolution kernel not only considers the initial hearing threshold, but also integrates prior information on the recovery direction, making feature extraction more targeted. By incorporating the dynamic information of hearing recovery into the convolution kernel generation process, the model's personalization ability can be enhanced, thereby capturing patient-specific patterns more accurately.

[0074] S3-2: Time-frequency feature extraction:

[0075] Reconstructed features contain both time-domain and frequency-domain information, but conventional convolutional neural networks process the spatiotemporal dimension separately, severing the time-frequency coupling characteristics of hearing recovery, which makes the model unable to effectively capture the dynamic change patterns of frequency band recovery during treatment.

[0076] This invention employs dynamic convolution kernels for joint time-frequency convolution, extracts joint time-frequency features through convolution operations, and introduces residual connections to preserve the original feature information, thereby enhancing the model's ability to capture dynamic time-frequency interaction patterns. The specific steps are as follows:

[0077] S3-2-1: Time-frequency joint convolution operation:

[0078] The multi-band feature matrix is ​​convolved using a dynamic convolution kernel matrix, and the first modulation is performed in the frequency domain by combining the sub-band improvement value. The time-frequency feature map is then obtained by applying the ReLU activation function. This allows for the capture of dynamic interaction patterns between multi-band improvement values ​​and individual attributes in both the time and frequency domains, characterizing the local correlation of frequency band recovery during treatment, enhancing the ability to extract features from key sub-bands, and generating time-frequency feature maps. Expressed by the following formula:

[0079] ;

[0080] In the formula, The frequency domain modulation scaling parameter is preferably set to 0.1, which controls the modulation intensity of the subband improvement value on the convolutional features;

[0081] This represents the constructor for a diagonal matrix, which converts an input vector into a diagonal matrix.

[0082] This indicates that the activation function of the linear unit is modified by introducing a nonlinear transformation;

[0083] This represents a convolution operation with a sliding step of 1. The convolution kernel slides along the row and column directions of the feature matrix, with the kernel size matching the input feature map.

[0084] S3-2-2: Residual connectivity feature enhancement:

[0085] Max pooling downsampling is performed on the multi-band feature matrix, and a gate vector is generated based on the sub-band improvement value and the dynamic convolution kernel matrix to perform a second modulation on the pooled features. The time-frequency enhanced feature matrix is ​​obtained by adding it element-wise to the time-frequency feature map. This preserves the global information of the original multi-band features and enhances the expressive power of key recovered frequency bands, resulting in a time-frequency enhanced feature matrix. Expressed by the following formula:

[0086] ;

[0087] In the formula, This indicates a max pooling operation, using... Pooling window, step size is ;

[0088] For the gated vector, ;

[0089] is the gated transformation weight matrix, which consists of trainable parameters;

[0090] The gated transformation bias vector is a trainable parameter;

[0091] This indicates element-wise multiplication.

[0092] It should be noted that, through the dynamic convolution kernel matrix Performing time-frequency joint convolution allows the convolution operation to slide simultaneously along both the frequency band and time dimensions, capturing the local correlation between sub-band improvement values ​​and individual attributes during the treatment cycle. The method uses the average improvement value of sub-bands as a frequency domain prior to enhance the feature extraction capability of convolution operations for sub-bands with larger recovery amplitudes, making the model more focused on patient-specific recovery patterns.

[0093] S3-3: Feature Interaction Enhancement Module

[0094] Traditional fully connected layers directly compress features, which loses details of the imbalance in recovery between frequency bands and cannot quantify the mutual influence of the recovery levels of different sub-bands, making it difficult for the model to capture the correlation patterns in recovery between frequency bands.

[0095] This invention employs a cross-band attention mechanism, which models the correlation between frequency bands by calculating the attention weight matrix between sub-bands, thereby enhancing the feature interaction expression capability and enabling the model to capture the mutual influence of the sub-band recovery degree. The specific steps are as follows:

[0096] S3-3-1: Subband eigenvector mapping:

[0097] By fusing the feature vectors of each sub-band in the time-frequency feature map with the sub-band improvement value and the multi-band feature matrix, and performing a linear transformation based on the query weight matrix and the key weight matrix, query vectors and key vectors are obtained, enhancing the vectors' ability to represent the recovered associations. This is expressed as:

[0098] ;

[0099] ;

[0100] In the formula, Subband The query vector is used to calculate the attention weights;

[0101] Subband The key vector is used to calculate the attention weights;

[0102] The query transformation weight matrix represents a trainable parameter that maps the fused sub-band features to the query space.

[0103] The key transformation weight matrix is ​​a trainable parameter that maps the fused subband features to the key space.

[0104] Represents the multi-band feature matrix The Column vector.

[0105] S3-3-2: Cross-band attention weight calculation:

[0106] Based on the dot product similarity of the query vector and key vector, the attention weight matrix between sub-bands is calculated using the Softmax function, thereby quantifying the mutual influence of different sub-band recovery levels. Attention weight matrix for other subbands Represented as:

[0107] ;

[0108] In the formula, express Transpose of;

[0109] Subband The key vector;

[0110] This indicates other sub-band indices that are distinct from 'b', and its value is... Iterate through all sub-bands;

[0111] This represents the vector dimension scaling factor, with a preferred value of [value to be filled in]. , used for stable gradient calculation;

[0112] This represents the natural exponential function.

[0113] S3-3-3: Feature Interaction Enhancement: The enhanced interaction feature matrix is ​​obtained by performing matrix multiplication between the attention weight matrix and the transposed time-frequency enhanced feature matrix, and based on the synergistic effect of sub-band improvement values. This is used to characterize the sub-band features after cross-band attention modulation, thereby enhancing the recovery correlation patterns between frequency bands, capturing the mutual influence of the recovery degree between sub-bands, and strengthening the interactive feature matrix. Represented as:

[0114] ;

[0115] In the formula, Represents the attention weight matrix, subband Attention weight matrix for other subbands It is the b-th row vector of the attention weight matrix;

[0116] Let be the subband improvement value vector, represented as ;

[0117] for Transpose of;

[0118] Indicates the transpose operation;

[0119] The scaling parameter is preferably set to 10;

[0120] This represents element-wise multiplication;

[0121] for The transpose of .

[0122] It should be noted that the sub-band improvement value is incorporated as a frequency domain prior into the attention calculation, through... The term emphasizes the correlation between similar subbands with improved values, which is consistent with the principle of band collaboration in auditory physiology. It reflects the synergistic effect of band recovery, enabling the model to capture the collaborative or competitive patterns of band recovery. For example, when the high frequency band is improved, the attention of the mid frequency band may be enhanced, thus more accurately reflecting the overall recovery situation. It can enhance the model's ability to capture the correlation between band recovery. In modeling the impact of high frequency recovery on speech recognition rate, the attention mechanism can automatically strengthen the interaction of key frequency bands and improve the discriminativeness of feature expression.

[0123] S3-4: Auditory perception weighting: The recovery of key frequency bands in speech is more important for classification, but the conventional equal weighting loss function cannot reflect this prior, making it difficult for the model to distinguish the differences in the contribution of different frequency band recovery to the classification results, thereby reducing the accuracy of evaluation.

[0124] This application employs Bark scale-aware weighting, calculates the frequency band weight matrix using the sigmoid function based on the sub-band center frequency, and performs weighted transformation on the interactive features, thereby increasing the contribution of key frequency band features in speech and enhancing the model's focus on important frequency bands. The specific steps are as follows:

[0125] S3-4-1: Bark Scale Weight Calculation: Based on the Bark value corresponding to the center frequency of the sub-band, the Bark scale perception weight matrix of each sub-band is calculated using the Sigmoid function, forming a diagonal weight matrix that reflects the critical prior of the audio segment, expressed as:

[0126] ;

[0127] In the formula, This represents the Bark scale-aware weight matrix, with dimension 1. , is a diagonal matrix used to weight the features of different sub-bands;

[0128] This represents the constructor for a diagonal matrix, which converts an input vector into a diagonal matrix.

[0129] Subband The perceptual weight scalar, with a value range of This reflects the importance of the sub-band, and the calculation method is expressed as follows: ;

[0130] This represents the perceptual weight vector for the three sub-bands, corresponding to the low-frequency band, mid-frequency band, and high-frequency band, respectively. This represents the sensing weight for the low-frequency band. This represents the sensing weight in the mid-frequency band. Represents the sensing weights of the high-frequency band;

[0131] This represents the weight steepness control parameter, with the preferred value being [value to be filled in]. Adjust the slope of the function;

[0132] Subband The Bark value corresponding to the center frequency, specifically taken as... Indicates low frequency band, Indicates mid-frequency band, Represents the high-frequency band, calculated based on the Bark scale of auditory perception;

[0133] Represents the natural constant.

[0134] It should be noted that the Bark scale is a psychoacoustic scale based on the critical frequency band division of the human ear, converting frequencies into perceptually relevant units. The key frequency band for speech (1-4kHz) mainly falls in the mid-frequency band, and its recovery is crucial for speech recognition. The value is determined based on the center frequency of the sub-band. Specifically, the center of the low-frequency band is about 500Hz, which corresponds to a Bark value of 1.3; the center of the mid-frequency band is 3kHz, which corresponds to 3.5; and the center of the high-frequency band is 7kHz, which corresponds to 8.5.

[0135] S3-4-2: Perceptual Weighted Feature Transformation: The perceptual weighted feature matrix is ​​obtained by multiplying the Bark-scale perceptual weight matrix with the enhanced interaction feature matrix and combining it with damage type coding modulation. Its row vectors correspond to the weighted sub-band features, enhancing the contribution of key speech frequency bands and damage-related frequency bands, and perceptually weighted feature matrix. Represented as:

[0136] ;

[0137] In the formula, Represents the Bark scale perception weight matrix, which is a trainable parameter used to assign frequency band importance to features based on auditory perception criteria;

[0138] The impairment type modulation parameter is preferably set to 0.2 to decouple the hearing impairment type encoding from the perceptual weights.

[0139] Unique thermal coding combined with hearing loss type Bark-scale perceptual weight matrix Personalized modulation can be performed, for example, for patients with noise-induced hearing loss, mid-frequency band weighting can be achieved through... The adjustments further enhance the model to reflect its dependence on speech clarity, combining a general auditory model with individual injury pathology. This makes the weighting process not only dependent on frequency location but also responding to differences in injury mechanisms, enabling the model to distinguish the different contributions of "age-related injury" and "noise injury" to mid-frequency band recovery.

[0140] S3-5: Multi-scale feature fusion: Single-scale features are difficult to represent the diversity of recovery levels, such as the difference between short-term significant improvement and long-term slow recovery. Conventional methods only use single-level features and cannot capture information from different levels of abstraction at the same time, resulting in insufficient ability of the model to represent complex recovery patterns.

[0141] This invention constructs a multi-scale fused feature vector by aggregating original multi-band features, time-frequency features, and weighted features, performing tensor concatenation and flattening along the row direction, thereby preserving information at different levels of abstraction and enhancing the model's ability to represent complex recovery patterns. The specific steps are as follows:

[0142] S3-5-1: Multi-scale feature concatenation: By tensor concatenating the multi-band feature matrix, time-frequency enhanced feature matrix, and perceptually weighted feature matrix along the row direction, and scaling them based on trainable scaling factors, a concatenated composite feature matrix is ​​obtained. Its row vectors integrate original frequency band improvement, time-frequency dynamic interaction, and perceptual weighted features to represent multi-scale recovery patterns, thereby fusing features at different levels of abstraction, enhancing the model's ability to represent complex recovery patterns, and splicing together a composite feature matrix. Represented as:

[0143] ;

[0144] In the formula, The scaling factor for the original multi-band features is a trainable parameter.

[0145] , where is the scaling factor for the time-frequency features, and is a trainable parameter;

[0146] The scaling factor for the perceptual weighted features is a trainable parameter;

[0147] This indicates a concatenation operation.

[0148] S3-5-2: Feature Vector Flattening: By flattening the concatenated composite feature matrix into a one-dimensional feature vector, and capturing cross-scale nonlinear interactions based on a lightweight multilayer perceptron, a fused feature vector is obtained. Its elements contain linear and nonlinear combinations of multi-scale features, solving the problem of single feature scale and fusing feature vectors. Represented as:

[0149] ;

[0150] In the formula, It is a lightweight multilayer perceptron with a hidden layer dimension of 8;

[0151] This represents the flattening operation function, which rearranges the input matrix into a one-dimensional vector in row-major order.

[0152] It should be noted that, through This feature scaling and stitching is implemented, where trainable parameters automatically learn the contribution weights of features at each scale. For example, for patients in short-term recovery, It may be higher, dominated by time-frequency dynamic characteristics, for patients with long-term recovery. It may be higher; the perceived weighted features reflect the cumulative effect.

[0153] It should also be noted that, The project captures cross-scale nonlinear interactions, such as the coupling relationship between "initial improvement in the low-frequency band" and "long-term recovery in the mid-frequency band," through a lightweight multilayer perceptron. It models nonlinear dependencies between scales. For example, when there is a significant short-term improvement in the high-frequency band but a slow long-term recovery in the mid-frequency band, the lightweight multilayer perceptron can learn discriminative representations of such contradictory patterns, which is crucial for fine-grained classification of "partial improvement" and "significant improvement."

[0154] S3-6: Classification Prediction and Loss Function Calculation:

[0155] S3-6-1: Classification Prediction Output: The predicted recovery level category is output by concatenating the fused feature vector with the sub-band improvement value vector and calculating the class probability distribution based on the fully connected layer and the Softmax function. This allows the classifier to consider both data-driven patterns and clinical priors, predicting the recovery level category. Represented as:

[0156] ;

[0157] In the formula, the predicted recovery level category It is a scalar, and its value is... arrive The integer value represents no improvement, 2 represents partial improvement, 3 represents significant improvement, and 4 represents complete recovery.

[0158] This represents the maximum value index function, which returns the index of the class with the highest probability.

[0159] This represents the Softmax function, which transforms an input vector into a probability distribution.

[0160] This represents the classification weight matrix, which consists of trainable parameters.

[0161] This represents the classification bias vector, which is a trainable parameter.

[0162] S3-6-2: Design of the Difference-Sensitive Loss Function: By introducing a class-adaptive temperature coefficient and class-related Bark-aware weights into the standard cross-entropy loss function, the difference-sensitive loss function is calculated. It is used for backpropagation during model training; it makes the loss function more sensitive to the minority class, alleviating the problem of imbalanced data, and is a difference-sensitive loss function. Represented as:

[0163] ;

[0164] In the formula, This indicates the total number of recovery level categories, with a value of [value]. The corresponding levels are no improvement, partial improvement, significant improvement, and complete recovery.

[0165] For category The mean of the Bark perceptual weights is obtained by analyzing the Bark scale perceptual weight matrix for all training samples of this class. The average of the categories is obtained, and the calculation method is expressed as follows: ;

[0166] For categories in the training set The number of samples;

[0167] For category The set of sample indexes;

[0168] No. Bark scale-aware weight matrix for each sample;

[0169] For matrix trace operations;

[0170] It is a category The mean of Bark's perception weights;

[0171] This indicates the indicator function, which represents the true recovery level label of the sample. equals category The function value is Otherwise ;

[0172] Indicates the true recovery level label of the sample;

[0173] This represents the category index, with values ​​ranging from 1 to... The integers correspond to different recovery levels;

[0174] This represents a logarithmic function, with the default base being the natural constant.

[0175] Indicates the category of the fully connected layer output. The logits value is a scalar; the logits value refers to the unnormalized predicted score.

[0176] Indicates category The adaptive temperature coefficient is a scalar used to adjust the output distribution of the Softmax function, and its calculation method is expressed as follows: ;

[0177] Indicates category The adaptive temperature coefficient is a scalar.

[0178] To distinguish it from the category index of c;

[0179] Indicates the category of the fully connected layer output. The logits value is a scalar;

[0180] This represents the temperature control coefficient, with a preferred value of [value missing]. Adjust the range of temperature value;

[0181] Indicates category The sample size ranking value is a scalar; the class with the smallest sample size is selected. Take less And so on.

[0182] It should be noted that, Applying stronger weights to minority classes, such as the "complete recovery" class which typically has few samples, exponentially amplifies the corresponding mid-frequency band weights. This couples auditory perception weights with sample size ranking, allowing the model to focus on key speech frequency bands even when data is imbalanced, forcing the model to pay more attention to the discriminative features of high-frequency band complete recovery.

[0183] S3-7: Model Iterative Update: The model iterative update process is based on backpropagation optimization using the difference-sensitive loss function, and the adaptive moment estimator optimizer is used to adjust the trainable parameters in the model.

[0184] In each iteration, the model calculates the predicted recovery level from forward propagation, calculates the loss value based on the difference between the true label and the predicted result, and updates the parameters through the gradient descent algorithm to minimize the loss.

[0185] The training process uses mini-batch stochastic gradient descent with a batch size of 32 and an initial learning rate of 0.001. A learning rate decay strategy is applied, reducing the learning rate to half its original value every 50 iterations to achieve stable convergence.

[0186] The decision to stop iteration is based on monitoring the loss and accuracy on the validation set. Training is stopped early when the validation loss no longer decreases or the validation accuracy no longer improves within 20 consecutive iterations to prevent overfitting. At the same time, the maximum iteration period is set to 5000 to ensure that the model completes training within a reasonable time and obtains the best model.

[0187] S4: Assessment of Hearing Loss Recovery: After model training is completed, the assessment of hearing loss recovery is achieved by loading the optimal model parameters and performing forward propagation on new patient data. The specific process is as follows:

[0188] S4-1: Input the new patient's initial hearing threshold, current hearing threshold, age, hearing impairment type, and noise speech recognition rate, and generate a multi-band feature matrix through a multi-band hearing threshold feature reconstruction step;

[0189] S4-2: This matrix enters the artificial intelligence evaluation model and sequentially goes through dynamic convolution kernel generation, time-frequency feature extraction, feature interaction enhancement module, auditory perception weighting and multi-scale feature fusion processing to obtain the fused feature vector;

[0190] S4-3: The recovery level is predicted by the classification prediction layer and is divided into four categories: no improvement, partial improvement, significant improvement or complete recovery.

[0191] The evaluation results are presented in the form of a probability distribution, with the category corresponding to the highest probability being taken as the final evaluation level. Detailed analysis of the recovery degree of each sub-band is also provided, such as the improvement values ​​of the low-frequency band, mid-frequency band, and high-frequency band, to help clinicians understand the basis of the model's decision-making.

[0192] In one embodiment, such as Figure 1 As shown in the figure, the advantages of the proposed AI-based hearing loss recovery assessment method in classification accuracy are verified by comparing the method with four conventional machine learning methods, including traditional convolutional neural networks, support vector machines, random forests, and logistic regression. These comparative methods all use the same input features, namely the raw hearing threshold data without multi-band reconstruction, and are evaluated on the same training and test sets to ensure fairness in the comparison. The experimental results show that the classification accuracy of the proposed method is significantly higher than the other comparative methods across all four recovery levels, especially in the key categories of "significant improvement" and "complete recovery." The traditional convolutional neural network method performs relatively well in the "partial improvement" and "significant improvement" categories, but its accuracy drops significantly in the "complete recovery" category, possibly due to sample imbalance leading to insufficient model recognition ability for a minority of categories. The support vector machine and logistic regression methods show relatively low accuracy across all recovery levels and exhibit unstable performance, indicating that these traditional methods struggle to effectively capture the complex patterns of hearing recovery.

[0193] In one embodiment, such as Figure 2 The analysis shows the distribution characteristics of hearing threshold improvement values ​​in different frequency sub-bands, providing a basis for multi-band feature reconstruction in the method of this invention. The experiment uses box plots and kernel density estimation plots to illustrate the distribution of improvement values ​​in three physiologically relevant sub-bands: low-frequency, mid-frequency, and high-frequency. The box plots show that the mid-frequency band has the widest distribution range and the highest median, indicating that this band has the largest recovery amplitude and significant individual differences, consistent with the theory in auditory physiology that the mid-frequency band corresponds to the key region of speech intelligibility. The low-frequency band has a relatively concentrated distribution of improvement values ​​and a moderate improvement amplitude, reflecting the recovery characteristics of the fundamental frequency and resonance band. The high-frequency band has a narrower distribution range and a relatively low median, indicating that high-frequency hearing recovery is relatively difficult, consistent with clinical observations. The kernel density estimation plot further reveals the probability density distribution of improvement values ​​in each sub-band. The mid-frequency band exhibits a wide bimodal distribution, indicating the existence of two different recovery modes, while the low-frequency and high-frequency bands show a unimodal distribution, indicating a relatively uniform recovery mode.

[0194] In one embodiment, such as Figure 3As shown, the distribution of hearing recovery effects in patients of different age groups was analyzed to verify the necessity of the age normalization factor in the method of this invention. The experiment divided patients into five age groups, and a stacked bar chart was used to display the distribution ratio of each age group across the four recovery levels. A clear age trend can be observed in the chart: the proportion of "significant improvement" and "complete recovery" is significantly higher in the younger patient group, while the proportion of "no improvement" and "partial improvement" gradually increases with age. In patients aged 20 to 30, the sum of the proportions of "complete recovery" and "significant improvement" reaches a high level, indicating that younger patients have greater recovery potential. Conversely, in patients aged 61 to 70, the proportions of "no improvement" and "partial improvement" increase significantly, reflecting the important influence of age on hearing recovery. This age-related distribution of recovery patterns verifies the importance of introducing an age normalization factor in the feature reconstruction of this invention, enabling the model to better adapt to the recovery characteristics of patients of different ages.

[0195] In this embodiment, as Figure 4 As shown, the performance of the proposed method and the traditional convolutional neural network method are compared under different types of hearing loss. The experiment selected four common types of hearing loss: conductive hearing loss, sensorineural hearing loss, mixed hearing loss, and unknown types, using the F1 score as the evaluation metric. The experimental results clearly show that the proposed method achieves higher F1 scores than the traditional convolutional neural network method across all hearing loss types. The advantage of the proposed method is particularly evident in the complex types of sensorineural and mixed hearing loss, with a significant lead in F1 scores, indicating that the dynamic convolutional kernel and cross-band attention mechanism in the proposed method can effectively handle complex damage patterns. For conductive hearing loss, the performance difference between the two methods is relatively small, but the proposed method still maintains its advantage. For unknown types of hearing loss, the proposed method also shows better adaptability, demonstrating its strong generalization ability. These results verify that the proposed method, through multi-band feature reconstruction and personalized feature extraction, can better adapt to the assessment needs of different types of hearing loss.

[0196] In summary, this application proposes a multi-band feature reconstruction method based on the auditory critical band theory, which divides hearing data into physiologically relevant low-frequency, mid-frequency, and high-frequency bands to better capture recovery patterns between frequency bands and preserve cross-band recovery correlations.

[0197] Adaptive convolution kernel generation technology is used to generate dynamic convolution kernels based on the patient's initial hearing threshold curve, thereby enhancing the model's ability to identify individualized recovery patterns and focusing on key damaged frequency bands;

[0198] By introducing a cross-band attention mechanism, the model can calculate the recovery correlation between different frequency bands, enhance its ability to capture recovery patterns across frequency bands, and especially adjust the focus of the recovery pattern according to the specific type of hearing loss of the patient.

[0199] By adopting the Bark scale-aware weighting scheme, higher weights are given to the recovery of key frequency bands in speech, enabling the model to more accurately focus on the impact of recovery on speech recognition and improve classification performance.

[0200] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based hearing impairment recovery condition evaluation method, characterized by: Comprise the following steps executed in turn: S1: Collecting the hearing threshold data of different frequency points, the hearing threshold data including the initial hearing threshold before treatment and the current hearing threshold after treatment, recording the age of the patient, the type of hearing impairment and the speech recognition rate in the noise environment at the same time, constructing the training data set and generating the initial hearing threshold curve; S2: For the training data set, the multi-band hearing threshold feature is reorganized: the training data set is divided into three physiological related sub-bands of low frequency band, medium frequency band and high frequency band, the average hearing threshold improvement value of each sub-band is calculated, and the age normalization factor, the hearing impairment type code and the noise speech recognition rate normalization value are integrated to obtain the multi-band feature matrix; S3: Constructing an artificial intelligence evaluation model to obtain the best model; The specific steps of step S3 for constructing an artificial intelligence evaluation model are as follows: S3-1: Smooth the initial hearing threshold curve to obtain a smooth hearing threshold curve vector, fuse the smooth hearing threshold curve vector with the multi-band feature matrix to obtain a dynamic convolution kernel matrix; S3-2: Convolve the multi-band feature matrix with the dynamic convolution kernel matrix to obtain a time-frequency feature map after the first modulation, downsample the multi-band feature matrix by maximum pooling and perform the second modulation, and add the second modulation matrix and the time-frequency feature map element by element to obtain a time-frequency enhanced feature matrix; S3-3: Perform sub-band feature vector mapping to obtain a query vector and a key vector, calculate the attention weight matrix between sub-bands through the Softmax function, multiply the attention weight matrix with the transposed time-frequency enhanced feature matrix, and obtain an enhanced interaction feature matrix based on the synergistic effect of the sub-band improvement value vector; S3-4: Calculate the Bark scale perception weight matrix of each sub-band through the Sigmoid function, multiply the Bark scale perception weight matrix with the enhanced interaction feature matrix, and combine the damage type code modulation to obtain a perception weighted feature matrix; S3-5: Tensor splice the multi-band feature matrix, the time-frequency enhanced feature matrix and the perception weighted feature matrix along the row direction, scale based on the trainable scaling coefficient to obtain a spliced composite feature matrix, flatten the spliced composite feature matrix into a one-dimensional feature vector, and capture the cross-scale nonlinear interaction based on the lightweight multilayer perceptron to obtain a fusion feature vector; S3-6: Concatenate the fusion feature vector and the sub-band improvement value vector, calculate the class probability distribution output predicted recovery level class based on the fully connected layer and the Softmax function, calculate the difference sensitive loss function, and use the difference sensitive loss function to back propagate the model until the validation loss no longer decreases or the validation accuracy no longer improves within the preset period, and output the best model; S4: Using the best model for hearing evaluation, output the predicted recovery level. 2.The artificial intelligence-based hearing loss recovery condition evaluation method of claim 1, wherein: In step S2, the low frequency band includes three frequency points of 250Hz, 500Hz and 1kHz, the medium frequency band includes two frequency points of 2kHz and 4kHz, and the high frequency band includes two frequency points of 6kHz and 8kHz; The age normalization factor is the age of the patient divided by 100; The hearing impairment type code is expressed by one-hot encoding; The noise speech recognition rate is a percentage value obtained by a speech recognition test in a noisy environment. 3.The artificial intelligence-based hearing loss recovery condition evaluation method of claim 2, wherein: In step S3-1, the initial threshold curve is smoothed by applying a Savitzky-Golay filter, and the extended vector of the average threshold improvement value of each sub-band is fused to obtain the smoothed threshold curve vector. The smoothed threshold curve vector and the multi-frequency band feature matrix are fused, and linear transformation is performed based on a learnable weight matrix and a bias vector to generate the dynamic convolution kernel matrix. 4.The artificial intelligence-based hearing loss recovery condition evaluation method of claim 3, wherein: In step S3-2, the first modulation is a frequency domain modulation of the matrix after convolution combined with the sub-band improvement value, and an activation function is applied to obtain a time-frequency feature map; The second modulation is to generate a gating vector based on the sub-band improvement value and the dynamic convolution kernel matrix to modulate the pooled features, and the time-frequency joint convolution is performed between the dynamic convolution kernel matrix and the time-frequency feature map, so that the convolution operation slides along the frequency band dimension and the time dimension, which is used to capture the local correlation between the sub-band improvement value and the individual attribute in the treatment period. 5.The artificial intelligence-based hearing loss recovery condition evaluation method of claim 4, wherein: In step S3-3, the specific operation of sub-band feature vector mapping is to fuse each sub-band feature vector in the time-frequency feature map with the sub-band improvement value and the multi-frequency band feature matrix, and perform linear transformation based on a query weight matrix and a key weight matrix to obtain the query vector and the key vector, and enhance the representation ability of the vector for recovery association. 6.The artificial intelligence-based hearing loss recovery condition evaluation method of claim 5, wherein: In step S3-6, when the model is back-propagated using the difference-sensitive loss function, the adaptive matrix estimator optimizer is used to adjust the trainable parameters in the model, and in each iteration, the model calculates the predicted recovery level from forward propagation, and calculates the loss value based on the difference between the true label and the predicted result, and updates the parameters through the gradient descent algorithm to minimize the loss. 7.The artificial intelligence-based hearing loss recovery condition assessment method of claim 6, wherein: In step S4, when the best model is used for hearing evaluation, the initial threshold value, the current threshold value, the age, the hearing impairment type and the noise speech recognition rate of the patient need to be input. 8.The artificial intelligence-based hearing loss recovery condition evaluation method of claim 7, wherein: The output predicted recovery level includes four categories: no improvement, partial improvement, significant improvement and complete recovery.

Images