Hearing impairment recovery condition evaluation method based on artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a hearing impairment recovery condition evaluation method based on artificial intelligence. Comprising the following steps that S1, listening threshold data of different frequency points are collected, the listening threshold data comprise an initial listening threshold before treatment and a current listening threshold after treatment, meanwhile, the age, the listening impairment type and the speech recognition rate in the noise environment of a patient are recorded, a training data set is constructed, and an initial listening threshold curve is generated; s2, performing multi-band hearing threshold feature recombination on the training data set to obtain a multi-band feature matrix; s3, constructing an artificial intelligence evaluation model, and obtaining an optimal model; s4, performing hearing evaluation by adopting the optimal model, and outputting a prediction recovery level; a self-adaptive convolution kernel generation technology is adopted, and a dynamic convolution kernel is generated based on an initial hearing threshold curve of a patient, so that the recognition capability of the model on an individualized recovery mode is enhanced, and a key damaged frequency band is focused.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a hearing impairment recovery condition evaluation method based on artificial intelligence. BACKGROUND

[0002] With the acceleration of global population aging, hearing impairment has become one of the important problems affecting human health, especially in the elderly population, the incidence of hearing impairment has increased significantly. Hearing impairment not only affects the social and life quality of individuals, but also has a negative impact on their mental health. In order to improve this problem, hearing recovery technology and auxiliary equipment have gradually developed, but there are still many challenges.

[0003] The traditional hearing recovery evaluation method ignores the physiological correlation between frequency bands and fails to effectively utilize the physiological characteristics of the auditory critical frequency band, resulting in insufficient fine processing of frequency characteristics and inability to accurately capture the recovery trend across frequency bands. SUMMARY

[0004] The purpose of the present application is to provide a hearing impairment recovery condition evaluation method based on artificial intelligence which can finely process frequency characteristics.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: A hearing impairment recovery condition evaluation method based on artificial intelligence, comprising the following steps executed in sequence: S1: Collecting hearing threshold data at different frequency points, including initial hearing threshold before treatment and current hearing threshold after treatment, recording the patient's age, hearing impairment type and speech recognition rate in a noisy environment, constructing a training data set and generating an initial hearing threshold curve; S2: For the training data set, reorganize the multi-frequency band hearing threshold features: divide the training data set into three physiologically related sub-bands of low frequency band, middle frequency band and high frequency band, calculate the average hearing threshold improvement value of each sub-band, and integrate the age normalization factor, hearing impairment type code and noise speech recognition rate normalization value to obtain a multi-frequency band feature matrix; S3: Constructing an artificial intelligence evaluation model to obtain the best model; S4: Using the best model for hearing evaluation and outputting the predicted recovery level.

[0006] Preferably, in step S2, the low frequency band includes three frequency points of 250Hz, 500Hz and 1kHz, the middle frequency band includes two frequency points of 2kHz and 4kHz, and the high frequency band includes two frequency points of 6kHz and 8kHz; The age normalization factor is the patient's age divided by 100; The hearing impairment type code is represented by one-hot encoding. The noise speech recognition rate is a percentage value obtained by speech recognition test in a noisy environment.

[0007] Preferably, the specific steps of constructing the artificial intelligence evaluation model in step S3 are as follows: S3-1: smoothing the initial hearing threshold curve to obtain a smoothed hearing threshold curve vector, fusing the smoothed hearing threshold curve vector with the multi-band feature matrix to obtain a dynamic convolution kernel matrix; S3-2: performing convolution operation on the multi-band feature matrix using the dynamic convolution kernel matrix, obtaining a time-frequency feature map after first modulation, performing maximum pooling downsampling on the multi-band feature matrix and second modulation, element-wise adding the second modulation matrix and the time-frequency feature map to obtain a time-frequency enhanced feature matrix; S3-3: performing sub-band feature vector mapping to obtain a query vector and a key vector, calculating a sub-band attention weight matrix through a Softmax function, performing matrix multiplication on the attention weight matrix and the transposed time-frequency enhanced feature matrix, and obtaining an enhanced interaction feature matrix based on the synergistic effect of the sub-band improvement value vector; S3-4: calculating a Bark scale perception weight matrix of each sub-band through a Sigmoid function, multiplying the Bark scale perception weight matrix and the enhanced interaction feature matrix, and combining the damage type coding modulation to obtain a perceptual weighted feature matrix; S3-5: tensor splicing the multi-band feature matrix, the time-frequency feature matrix and the perceptual weighted feature matrix along the row direction, and scaling based on the trainable scaling coefficient to obtain a spliced composite feature matrix, flattening the spliced composite feature matrix into a one-dimensional feature vector, and capturing cross-scale nonlinear interaction based on a lightweight multilayer perceptron to obtain a fusion feature vector; S3-6: splicing the fusion feature vector and the sub-band improvement value vector, and calculating the class probability distribution to output the predicted recovery level class based on the fully connected layer and the Softmax function, calculating the difference sensitive loss function, and using the difference sensitive loss function to back propagate the model until the verification loss no longer decreases or the verification accuracy no longer improves within the preset period, and outputting the best model.

[0008] Preferably, in step S3-1, the initial hearing threshold curve is smoothed by applying a Savitzky-Golay filter, and an extension vector of the average hearing threshold improvement value of each sub-band is fused to obtain the smoothed hearing threshold curve vector; The smoothed hearing threshold curve vector is fused with the multi-band feature matrix, and linear transformation is performed based on a learnable weight matrix and a bias vector, and the Sigmoid function is applied to generate the dynamic convolution kernel matrix.

[0009] Preferably, in step S3-2, the first modulation is a frequency domain modulation of the matrix combined with the sub-band improvement value after convolution operation, and a time-frequency feature map is obtained by applying an activation function; The second modulation is to generate a gating vector based on the sub-band improvement value and the dynamic convolution kernel matrix to modulate the pooled features, and the dynamic convolution kernel matrix is used for time-frequency joint convolution with the time-frequency feature map, so that the convolution operation informs the sliding along the frequency band dimension and the time dimension, which is used to capture the local correlation of the sub-band improvement value and the individual attribute in the treatment cycle.

[0010] Preferably, in step S3-3, the specific operation of the sub-band feature vector mapping is to fuse each sub-band feature vector in the time-frequency feature map with the sub-band improvement value and the multi-band feature matrix, and perform linear transformation based on the query weight matrix and the key weight matrix to obtain the query vector and the key vector, and enhance the representation ability of the recovery association.

[0011] Preferably, in step S3-6, when the difference sensitive loss function is used for back propagation of the model, an adaptive matrix estimation optimizer is used to adjust the trainable parameters in the model, in each iteration, the model calculates the predicted recovery level from the forward propagation, and calculates the loss value based on the difference between the true label and the prediction result, and updates the parameters through the gradient descent algorithm to minimize the loss.

[0012] Preferably, in step S4, when the best model is used for hearing evaluation, the initial hearing threshold, the current hearing threshold, the age, the type of hearing impairment and the noise speech recognition rate of the patient need to be input.

[0013] Preferably, in step S4, the output predicted recovery level includes four categories of no improvement, partial improvement, significant improvement and complete recovery.

[0014] By adopting the foregoing design scheme, the application has the following beneficial effects: the application proposes a multi-band feature reorganization method based on the auditory critical band theory, which divides the hearing data into low frequency bands, medium frequency bands and high frequency bands according to the physiological correlation, so as to better capture the recovery mode between frequency bands and retain the recovery association across frequency bands; The adaptive convolution kernel generation technology is adopted to generate a dynamic convolution kernel based on the initial hearing threshold curve of the patient, so as to enhance the identification ability of the model to the individualized recovery mode and focus on the key damaged frequency band; The cross-band attention mechanism is introduced to calculate the recovery association between different frequency bands, and the capture ability of the model to the recovery mode between frequency bands is strengthened, and the focus of the recovery mode can be adjusted according to the specific hearing impairment type of the patient; The Bark scale perceptual weighting scheme is adopted to give higher weight to the recovery of the speech key frequency band, so that the model can more accurately focus on the influence of the recovery on speech recognition, and the classification performance is improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A bar chart for comparing the accuracy of the evaluation method of the present application with other evaluation methods; Figure 2 A distribution diagram of the hearing threshold improvement value of each frequency sub-band of the present application; Figure 3 A probability density distribution of the improvement value of each frequency sub-band of the present application; Figure 4 A distribution diagram of the hearing recovery level of patients in different age groups of the present application; Figure 5 A bar chart for comparing the F1 score of the present technology method with the conventional method under different damage types. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0017] The terms "first", "second", "third" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0018] An artificial intelligence-based hearing impairment recovery condition evaluation method, comprising the following steps executed in sequence: S1: Collecting hearing threshold data at different frequency points, the hearing threshold data including initial hearing threshold before treatment and current hearing threshold after treatment, recording the age, hearing impairment type and speech recognition rate in a noisy environment of the patient, constructing a training data set, and generating an initial hearing threshold curve; In the present embodiment, the hearing recovery data collection process is completed through clinical hearing test and patient follow-up record, specifically including using a pure tone audiometer to perform binaural hearing test on the patient in a standard soundproof room, collecting hearing threshold data at frequency points of 250 Hz, 500 Hz, 1000 Hz, 2000 Hz, 4000 Hz, 6000 Hz and 8000 Hz, and recording the age, hearing impairment type and speech recognition rate in a noisy environment of the patient.

[0019] Data collection covers initial hearing thresholds before treatment and current hearing thresholds after treatment, ensuring the time series integrity of data.

[0020] The construction of the training dataset is based on a multi-center clinical study, collecting a large number of patient follow-up data, including multiple test records during the hearing recovery process, to cover different recovery stages and individual differences. Data labeling is performed by clinical experts according to the amplitude of hearing threshold change and clinical recovery standards, with four levels of no improvement, partial improvement, significant improvement, and complete recovery. Among them, no improvement means that the hearing threshold change is less than 5 decibels, partial improvement means that the hearing threshold improvement is between 5 and 15 decibels, significant improvement means that the hearing threshold improvement is between 15 and 30 decibels, and complete recovery means that the hearing threshold is restored to the normal range, i.e. less than 25 decibels.

[0021] S2: Multi-band hearing threshold feature reorganization on the training dataset: Based on the theory of auditory critical band, eight frequency points are reorganized into three physiologically related sub-bands, including low frequency band, middle frequency band and high frequency band. By calculating the average hearing threshold improvement value of each sub-band, and integrating the age normalization factor, hearing impairment type coding and noise speech recognition rate normalization value, a multi-band feature matrix is formed, thereby enhancing the physical interpretability of the features and preserving the cross-band recovery correlation. The multi-band feature matrix is represented as: ; wherein the multi-band feature matrix has a dimension of By fusing frequency domain improvement values and individual differences, the physical interpretability of the features is enhanced, while the cross-band recovery correlation is preserved, is the age normalization factor, calculated as the patient's age divided by 100, represents the one-hot encoding of hearing impairment type, used to discretely represent the damage category, represents the speech recognition rate normalization value in a noisy environment, which is a percentage value obtained through speech recognition tests in a noisy environment, represents the average hearing threshold improvement value of sub-band , with a unit of dBHL, reflecting the overall recovery degree of the frequency band, and the calculation method is represented as , represents the sub-band index, taking the values of low frequency band, middle frequency band or high frequency band, corresponding to the physiological critical frequency band, represents the number of frequency points contained in sub-band , used for calculating the mean value, represents the initial hearing threshold of frequency , with a unit of dBHL, representing the pre-treatment hearing level, represents the initial hearing threshold of frequency The current hearing threshold of the patient, in dBHL, characterizes the hearing level after treatment, The frequency point, the value range is 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 6kHz, 8kHz, represents the hearing test frequency point, when The corresponding The average hearing threshold improvement value of the low frequency band is represented by The corresponding The average hearing threshold improvement value of the middle frequency band is represented by The corresponding The average hearing threshold improvement value of the high frequency band is represented by In the specific implementation, according to the critical frequency band theory, that is, the perception of human ears to frequency is not uniform, but in critical frequency bands, the low frequency band covers the speech fundamental frequency and part of the formant, the middle frequency band corresponds to the key area of speech intelligibility, and the high frequency band reflects the auditory sensitivity in a noisy environment, therefore, the low frequency band (250Hz-1kHz) contains three frequency points of 250Hz, 500Hz and 1kHz, the middle frequency band (2kHz-4kHz) contains two frequency points of 2kHz and 4kHz, and the high frequency band (6kHz-8kHz) contains two frequency points of 6kHz and 8kHz.

[0022] It should be noted that the frequency points are divided into three physiological related sub-bands by critical frequency band recombination, and the average improvement value of each sub-band is calculated, the age normalization factor, the damage type code and the noise speech recognition rate normalization value are integrated, and a multi-frequency band feature matrix The frequency domain recovery trend and individual attribute interaction are captured at the same time, such as the age normalization factor The influence of the age dimension is eliminated, and the exclusive hot coding of the hearing loss type Discretely represents the damage category, so that the model can distinguish the recovery mode of different damage types, and realizes the effect of integrating frequency domain and attribute information.

[0023] S3: training the artificial intelligence evaluation model, specifically including the following steps: S3-1: dynamic convolution kernel matrix generation: the traditional convolution kernel weight is fixed, which cannot adapt to the heterogeneity of individual hearing curves, resulting in that the feature extraction cannot focus on the key damaged frequency band of the patient, and the recognition ability of the model to the personalized recovery mode is reduced.

[0024] Based on the initial hearing threshold curve of the patient, the adaptive convolution kernel is generated through smoothing filtering and parameterized transformation, so that the convolution kernel weight is positively correlated with the initial hearing loss distribution, thereby realizing individualized feature extraction and enhancing the focusing ability of the model to the key damaged frequency band, and the specific steps are as follows: Smooth processing of initial threshold curve: the initial threshold curve is smoothed by applying a Savitzky-Golay filter, and an extended vector of average threshold improvement values of each sub-band is fused to obtain a smoothed threshold curve vector, which is used to eliminate measurement noise and enhance the physiological correlation of the curve to the recovery mode by fusing the initial threshold and the improved trend after smoothing; the smoothed threshold curve vector is expressed by the following formula: ; wherein, is a multi-band feature fusion coefficient, preferably set to 0.1, which is used to adjust the contribution degree of the sub-band improvement value to the smoothed curve; represents an extended vector of average threshold improvement values of each sub-band; represents a Savitzky-Golay filter function, which is calculated by using a quadratic polynomial fitting and a sliding window, the window length is 5, and the polynomial order is 2; represents an initial threshold curve vector, the dimension is , which is composed of initial threshold values of each frequency point, that is, ; represents the initial threshold value of the frequency point , the unit is dBHL, which represents the hearing level before treatment; represents the frequency point index, the value is 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 6kHz, 8kHz, which corresponds to the standard hearing test frequency point; represents a transposition operation.

[0025] In a specific implementation, the Savitzky-Golay filter performs local quadratic polynomial fitting on the initial threshold curve vector by using a sliding window with a length of 5, and a polynomial is fitted in each window by using the least square method, and then a smoothed value is output.

[0026] In an implementation, there are four types of damage, including conductive, sensorineural, mixed and unknown, such as, represents sensorineural hearing loss.

[0027] In a specific implementation, in the calculation process of the extended vector of average threshold improvement values of each sub-band , each frequency point is mapped to the average improvement value of the sub-band to which it belongs, that is, , where the low frequency band (250Hz-1kHz) corresponds to the first three frequency points, the middle frequency band (2kHz-4kHz) corresponds to the fourth and fifth frequency points, and the high frequency band (6kHz-8kHz) corresponds to the last two frequency points. By improving the trend of the sub-band, the physiological correlation of the smoothed audiometric curve is enhanced, making In the process of eliminating noise, the prior information of the recovery direction is fused.

[0028] Dynamic convolution kernel parameterization generation: by fusing the smoothed audiometric curve vector with the multi-band feature matrix and performing linear transformation based on the learnable weight matrix and bias vector, a dynamic convolution kernel matrix is generated by applying the Sigmoid function. The dimension of the dynamic convolution kernel matrix is , which is used to adaptively extract key frequency band information; thus the convolution kernel weight is positively correlated with the initial hearing loss distribution, realizing individualized feature extraction, and the dynamic convolution kernel matrix is expressed by the following formula: ; In the formula, represents the concatenation operation; represents the flattening operation function, which rearranges the input matrix into a one-dimensional vector in row-major order; represents the Sigmoid activation function, which restricts the output value to the interval; represents the dynamic convolution weight matrix, which is a trainable parameter; represents the dynamic convolution bias vector, which is a trainable parameter.

[0029] It should be noted that through smoothing filtering and parameterization transformation, the convolution kernel weight is positively correlated with the initial hearing loss distribution. The smoothed audiometric curve and the multi-band feature matrix are fused to generate a dynamic convolution kernel matrix , which can adaptively adjust the convolution kernel according to the individual's hearing loss pattern and recovery trend, thereby focusing more on the key damaged frequency band. In addition, after introducing the multi-band feature matrix , the convolution kernel not only considers the initial hearing threshold, but also fuses the prior information of the recovery direction, making the feature extraction more targeted. The dynamic information of hearing recovery is integrated into the convolution kernel generation process, which can enhance the individualization ability of the model, thereby more accurately capturing the patient-specific pattern.

[0030] S3-2: Time-frequency feature extraction: The reorganized features contain time and frequency domain information, but the conventional convolutional neural network processes the time and space dimensions separately, which breaks the time-frequency coupling characteristics of hearing recovery, causing the model to be unable to effectively capture the dynamic change pattern of the frequency band recovery during treatment.

[0031] The application adopts a dynamic convolution kernel to perform joint time-frequency convolution, extracts a time-frequency joint feature through convolution operation, and introduces a residual connection to retain original feature information, thereby enhancing the model's ability to capture time-frequency dynamic interaction patterns, and the specific steps are as follows: S3-2-1: Time-frequency joint convolution operation: The dynamic convolution kernel matrix is used to perform convolution operation on the multi-band feature matrix, and the frequency domain of the sub-band improvement value is combined for first modulation, and a ReLU activation function is applied to obtain a time-frequency feature map , thereby capturing the dynamic interaction patterns of the multi-band improvement value and the individual attribute in the time domain and the frequency domain, representing the local correlation of the frequency band recovery in the treatment process, and strengthening the feature extraction capability of the key sub-band, the time-frequency feature map is expressed by the following formula: ; In the formula, is a frequency domain modulation scaling parameter, which is preferably set to 0.1, and controls the modulation strength of the sub-band improvement value on the convolution feature; represents a diagonal matrix constructor, which converts an input vector into a diagonal matrix; represents a rectified linear unit activation function, which introduces a nonlinear transformation; represents a convolution operation, and the sliding step is , which slides in the row direction and the column direction of the feature matrix respectively, and the convolution kernel size matches the input feature map.

[0032] S3-2-2: Residual connection feature enhancement: The multi-band feature matrix is down-sampled by maximum pooling, and a gating vector is generated based on the sub-band improvement value and the dynamic convolution kernel matrix to modulate the pooled feature, and the time-frequency feature map is added element by element to obtain a time-frequency enhanced feature matrix , thereby retaining the global information of the original multi-band feature and enhancing the expression capability of the key recovery frequency band, the time-frequency enhanced feature matrix is expressed by the following formula: ; In the formula, represents a maximum pooling operation, which uses a pooling window, and the step is ; is a gating vector, ; is a gating transformation weight matrix, which is a trainable parameter; is a trainable parameter. represents element-wise multiplication.

[0033] It is to be noted that the dynamic convolution kernel matrix is used to perform the time-frequency joint convolution, so that the convolution operation slides along the frequency band dimension and the time dimension at the same time, and the local correlation between the subband improvement value and the individual attribute in the treatment cycle is captured, and the term takes the subband average improvement value as the frequency domain prior, enhances the feature extraction capability of the convolution operation on the subband with large recovery amplitude, and makes the model focus more on the patient-specific recovery mode.

[0034] S3-3: Feature interaction enhancement module: The traditional full connection layer directly compresses the features, which loses the details of the imbalance between the subbands, cannot quantify the mutual influence of the recovery degree of different subbands, and causes the model to be difficult to capture the recovery correlation between the subbands.

[0035] The cross-frequency attention mechanism is used in the application, the inter-subband attention weight matrix is calculated, the correlation between the subbands is modeled, and thus the feature interaction expression capability is enhanced, so that the model can capture the mutual influence of the recovery degree of the subbands, and the specific steps are as follows: S3-3-1: Subband feature vector mapping: The subband feature vectors in the time-frequency feature map are fused with the subband improvement value and the multi-band feature matrix, and linear transformation is performed based on the query weight matrix and the key weight matrix to obtain the query vector and the key vector, so as to enhance the representation capability of the vector to the recovery correlation, and is expressed as: ; ; In the formula, represents the query vector of the subband , which is used to calculate the attention weight; represents the key vector of the subband , which is used to calculate the attention weight; represents the query transformation weight matrix, which is a trainable parameter, and maps the fused subband features to the query space; represents the key transformation weight matrix, which is a trainable parameter, and maps the fused subband features to the key space; represents the column vector of the multi-band feature matrix .

[0036] S3-3-2: Cross-frequency attention weight calculation:​ Based on the dot product similarity of the query vector and key vector, the attention weight matrix between sub-bands is calculated using the Softmax function, thereby quantifying the mutual influence of different sub-band recovery levels. Attention weight matrix for other subbands Represented as: ; In the formula, express Transpose of; Subband The key vector; This indicates other sub-band indices that are distinct from 'b', and its value is... Iterate through all sub-bands; This represents the vector dimension scaling factor, with a preferred value of [value to be filled in]. , used for stable gradient calculation; This represents the natural exponential function.

[0037] S3-3-3: Feature Interaction Enhancement: The enhanced interaction feature matrix is ​​obtained by performing matrix multiplication between the attention weight matrix and the transposed time-frequency enhanced feature matrix, and based on the synergistic effect of sub-band improvement values. This is used to characterize the sub-band features after cross-band attention modulation, thereby enhancing the recovery correlation patterns between frequency bands, capturing the mutual influence of the recovery degree between sub-bands, and strengthening the interactive feature matrix. Represented as: ; In the formula, Represents the attention weight matrix, subband Attention weight matrix for other subbands It is the b-th row vector of the attention weight matrix; Let be the subband improvement value vector, represented as ; for transpose; Indicates the transpose operation; The scaling parameter is preferably set to 10; This represents element-wise multiplication; for The transpose of .

[0038] It should be noted that the sub-band improvement value is incorporated as a frequency domain prior into the attention calculation, through... The term emphasizes the correlation between similar subbands with improved values, which is consistent with the principle of band collaboration in auditory physiology. It reflects the synergistic effect of band recovery, enabling the model to capture the collaborative or competitive patterns of band recovery. For example, when the high frequency band is improved, the attention of the mid frequency band may be enhanced, thus more accurately reflecting the overall recovery situation. It can enhance the model's ability to capture the correlation between band recovery. In modeling the impact of high frequency recovery on speech recognition rate, the attention mechanism can automatically strengthen the interaction of key frequency bands and improve the discriminativeness of feature expression.

[0039] S3-4: Auditory perception weighting: The recovery of key frequency bands in speech is more important for classification, but the conventional equal weighting loss function cannot reflect this prior, making it difficult for the model to distinguish the differences in the contribution of different frequency band recovery to the classification results, thereby reducing the accuracy of evaluation.

[0040] This application employs Bark scale-aware weighting, calculates the frequency band weight matrix using the sigmoid function based on the sub-band center frequency, and performs weighted transformation on the interactive features, thereby increasing the contribution of key frequency band features in speech and enhancing the model's focus on important frequency bands. The specific steps are as follows: S3-4-1: Bark Scale Weight Calculation: Based on the Bark value corresponding to the center frequency of the sub-band, the Bark scale perception weight matrix of each sub-band is calculated using the Sigmoid function, forming a diagonal weight matrix that reflects the critical prior of the audio segment, expressed as: ; In the formula, This represents the Bark scale-aware weight matrix, with dimension 1. , is a diagonal matrix used to weight the features of different sub-bands; This represents the constructor for a diagonal matrix, which converts an input vector into a diagonal matrix. Subband The perceptual weight scalar, with a value range of This reflects the importance of the sub-band, and the calculation method is expressed as follows: ; This represents the perceptual weight vector for the three sub-bands, corresponding to the low-frequency band, mid-frequency band, and high-frequency band, respectively. This represents the sensing weight for the low-frequency band. This represents the sensing weight in the mid-frequency band. Represents the sensing weights of the high-frequency band; This represents the weight steepness control parameter, with the preferred value being [value to be filled in]. Adjust the slope of the function; Subband The Bark value corresponding to the center frequency, specifically taken as... Indicates low frequency band, Indicates mid-frequency band, Represents the high-frequency band, calculated based on the Bark scale of auditory perception; Represents the natural constant.

[0041] It should be noted that the Bark scale is a psychoacoustic scale based on the critical frequency band division of the human ear, converting frequencies into perceptually relevant units. The key frequency band for speech (1-4kHz) mainly falls in the mid-frequency band, and its recovery is crucial for speech recognition. The value is determined based on the center frequency of the sub-band. Specifically, the center of the low-frequency band is about 500Hz, which corresponds to a Bark value of 1.3; the center of the mid-frequency band is 3kHz, which corresponds to 3.5; and the center of the high-frequency band is 7kHz, which corresponds to 8.5.

[0042] S3-4-2: Perceptual Weighted Feature Transformation: The perceptual weighted feature matrix is ​​obtained by multiplying the Bark-scale perceptual weight matrix with the enhanced interaction feature matrix and combining it with damage type coding modulation. Its row vectors correspond to the weighted sub-band features, enhancing the contribution of key speech frequency bands and damage-related frequency bands, and perceptually weighted feature matrix. Represented as: ; In the formula, Represents the Bark scale perception weight matrix, which is a trainable parameter used to assign frequency band importance to features based on auditory perception criteria; The impairment type modulation parameter is preferably set to 0.2 to decouple the hearing impairment type encoding from the perceptual weights.

[0043] Unique thermal coding combined with hearing loss type Bark-scale perceptual weight matrix Personalized modulation can be performed, for example, for patients with noise-induced hearing loss, mid-frequency band weighting can be achieved through... The adjustments further enhance the model to reflect its dependence on speech clarity, combining a general auditory model with individual injury pathology. This makes the weighting process not only dependent on frequency location but also responding to differences in injury mechanisms, enabling the model to distinguish the different contributions of "age-related injury" and "noise injury" to mid-frequency band recovery.

[0044] S3-5: Multi-scale feature fusion: Single-scale features are difficult to represent the diversity of recovery levels, such as the difference between short-term significant improvement and long-term slow recovery. Conventional methods only use single-level features and cannot capture information from different levels of abstraction at the same time, resulting in insufficient ability of the model to represent complex recovery patterns.

[0045] This invention constructs a multi-scale fused feature vector by aggregating original multi-band features, time-frequency features, and weighted features, performing tensor concatenation and flattening along the row direction, thereby preserving information at different levels of abstraction and enhancing the model's ability to represent complex recovery patterns. The specific steps are as follows: S3-5-1: Multi-scale feature concatenation: By tensor concatenating the multi-band feature matrix, time-frequency feature matrix, and perceptually weighted feature matrix along the row direction, and scaling them based on trainable scaling factors, a concatenated composite feature matrix is ​​obtained. Its row vectors integrate original frequency band improvement, time-frequency dynamic interaction, and perceptual weighted features to represent multi-scale recovery patterns, thereby fusing features at different levels of abstraction, enhancing the model's ability to represent complex recovery patterns, and splicing together a composite feature matrix. Represented as: ; In the formula, The scaling factor for the original multi-band features is a trainable parameter. , where is the scaling factor for the time-frequency features, and is a trainable parameter; The scaling factor for the perceptual weighted features is a trainable parameter; This indicates a concatenation operation.

[0046] S3-5-2: Feature Vector Flattening: By flattening the concatenated composite feature matrix into a one-dimensional feature vector, and capturing cross-scale nonlinear interactions based on a lightweight multilayer perceptron, a fused feature vector is obtained. Its elements contain linear and nonlinear combinations of multi-scale features, solving the problem of single feature scale and fusing feature vectors. Represented as: ; In the formula, It is a lightweight multilayer perceptron with a hidden layer dimension of 8; This represents the flattening operation function, which rearranges the input matrix into a one-dimensional vector in row-major order.

[0047] It should be noted that, through This feature scaling and stitching is implemented, where trainable parameters automatically learn the contribution weights of features at each scale. For example, for patients in short-term recovery, It may be higher, dominated by time-frequency dynamic characteristics, for patients with long-term recovery. It may be higher; the perceived weighted features reflect the cumulative effect.

[0048] It should also be noted that, The project captures cross-scale nonlinear interactions, such as the coupling relationship between "initial improvement in the low-frequency band" and "long-term recovery in the mid-frequency band," through a lightweight multilayer perceptron. It models nonlinear dependencies between scales. For example, when there is a significant short-term improvement in the high-frequency band but a slow long-term recovery in the mid-frequency band, the lightweight multilayer perceptron can learn discriminative representations of such contradictory patterns, which is crucial for fine-grained classification of "partial improvement" and "significant improvement."

[0049] S3-6: Classification Prediction and Loss Function Calculation: S3-6-1: Classification Prediction Output: The predicted recovery level category is output by concatenating the fused feature vector with the sub-band improvement value vector and calculating the class probability distribution based on the fully connected layer and the Softmax function. This allows the classifier to consider both data-driven patterns and clinical priors, predicting the recovery level category. Represented as: ; In the formula, the predicted recovery level category It is a scalar, and its value is... arrive The integer value represents no improvement, 2 represents partial improvement, 3 represents significant improvement, and 4 represents complete recovery. This represents the maximum value index function, which returns the index of the class with the highest probability. This represents the Softmax function, which transforms an input vector into a probability distribution. This represents the classification weight matrix, which consists of trainable parameters. This represents the classification bias vector, which is a trainable parameter.

[0050] S3-6-2: Design of the Difference-Sensitive Loss Function: By introducing a class-adaptive temperature coefficient and class-related Bark-aware weights into the standard cross-entropy loss function, the difference-sensitive loss function is calculated. It is used for backpropagation during model training; it makes the loss function more sensitive to the minority class, alleviating the problem of imbalanced data, and is a difference-sensitive loss function. Represented as: ; In the formula, This indicates the total number of recovery level categories, with a value of [value]. The corresponding levels are no improvement, partial improvement, significant improvement, and complete recovery. For category The mean of the Bark perceptual weights is obtained by analyzing the Bark scale perceptual weight matrix for all training samples of this class. The average of the categories is obtained, and the calculation method is expressed as follows: ; For categories in the training set The number of samples; For category The set of sample indexes; No. Bark scale-aware weight matrix for each sample; For matrix trace operations; It is a category The mean of Bark's perception weights; This indicates the indicator function, which represents the true recovery level label of the sample. equals category The function value at time Otherwise ; Indicates the true recovery level label of the sample; This represents the category index, with values ​​ranging from 1 to... The integers correspond to different recovery levels; This represents a logarithmic function, with the default base being the natural constant. Indicates the category of the fully connected layer output. The logits value is a scalar; the logits value refers to the unnormalized predicted score. Indicate category The adaptive temperature coefficient is a scalar used to adjust the output distribution of the Softmax function, and its calculation method is expressed as follows: ; Indicate category The adaptive temperature coefficient is a scalar. To distinguish it from the category index of c; Indicates the category of the fully connected layer output. The logits value is a scalar; This represents the temperature control coefficient, with a preferred value of [value missing]. Adjust the range of temperature value; Indicate category The sample size ranking value is a scalar; the class with the smallest sample size is selected. Take less And so on.

[0051] It should be noted that, Applying stronger weights to minority classes, such as the "complete recovery" class which typically has few samples, exponentially amplifies the corresponding mid-frequency band weights. This couples auditory perception weights with sample size ranking, allowing the model to focus on key speech frequency bands even when data is imbalanced, and forcing the model to pay more attention to the discriminative features of high-frequency band complete recovery.

[0052] S3-7: Model Iterative Update: The model iterative update process is based on backpropagation optimization using the difference-sensitive loss function, and the adaptive moment estimator optimizer is used to adjust the trainable parameters in the model. In each iteration, the model calculates the predicted recovery level from forward propagation, calculates the loss value based on the difference between the true label and the predicted result, and updates the parameters through the gradient descent algorithm to minimize the loss.

[0053] The training process uses mini-batch stochastic gradient descent with a batch size of 32 and an initial learning rate of 0.001. A learning rate decay strategy is applied, reducing the learning rate to half its original value every 50 iterations to achieve stable convergence.

[0054] The decision to stop iteration is based on monitoring the loss and accuracy on the validation set. Training is stopped early when the validation loss no longer decreases or the validation accuracy no longer improves within 20 consecutive iterations to prevent overfitting. At the same time, the maximum iteration period is set to 5000 to ensure that the model completes training within a reasonable time and obtains the best model.

[0055] S4: Assessment of Hearing Loss Recovery: After model training is completed, the assessment of hearing loss recovery is achieved by loading the optimal model parameters and performing forward propagation on new patient data. The specific process is as follows: S4-1: Input the new patient's initial hearing threshold, current hearing threshold, age, hearing impairment type, and noise speech recognition rate, and generate a multi-band feature matrix through a multi-band hearing threshold feature reconstruction step; S4-2: This matrix enters the artificial intelligence evaluation model and sequentially goes through dynamic convolution kernel generation, time-frequency feature extraction, feature interaction enhancement module, auditory perception weighting and multi-scale feature fusion processing to obtain the fused feature vector; S4-3: The recovery level is predicted by the classification prediction layer and is divided into four categories: no improvement, partial improvement, significant improvement or complete recovery.

[0056] The evaluation results are presented in the form of a probability distribution, with the category corresponding to the highest probability being taken as the final evaluation level. Detailed analysis of the recovery degree of each sub-band is also provided, such as the improvement values ​​of the low-frequency band, mid-frequency band, and high-frequency band, to help clinicians understand the basis of the model's decision-making.

[0057] In one embodiment, such as Figure 1 As shown in the figure, the advantages of the proposed AI-based hearing loss recovery assessment method in classification accuracy are verified by comparing the method with four conventional machine learning methods, including traditional convolutional neural networks, support vector machines, random forests, and logistic regression. These comparative methods all use the same input features, namely the raw hearing threshold data without multi-band reconstruction, and are evaluated on the same training and test sets to ensure fairness in the comparison. The experimental results show that the classification accuracy of the proposed method is significantly higher than the other comparative methods across all four recovery levels, especially in the key categories of "significant improvement" and "complete recovery." The traditional convolutional neural network method performs relatively well in the "partial improvement" and "significant improvement" categories, but its accuracy drops significantly in the "complete recovery" category, possibly due to sample imbalance leading to insufficient model recognition ability for a minority of categories. The support vector machine and logistic regression methods show relatively low accuracy across all recovery levels and exhibit unstable performance, indicating that these traditional methods struggle to effectively capture the complex patterns of hearing recovery.

[0058] In one embodiment, such as Figure 2 The analysis shows the distribution characteristics of hearing threshold improvement values ​​in different frequency sub-bands, providing a basis for multi-band feature reconstruction in the method of this invention. The experiment uses box plots and kernel density estimation plots to illustrate the distribution of improvement values ​​in three physiologically relevant sub-bands: low-frequency, mid-frequency, and high-frequency. The box plots show that the mid-frequency band has the widest distribution range and the highest median, indicating that this band has the largest recovery amplitude and significant individual differences, consistent with the theory in auditory physiology that the mid-frequency band corresponds to the key region of speech intelligibility. The low-frequency band has a relatively concentrated distribution of improvement values ​​and a moderate improvement amplitude, reflecting the recovery characteristics of the fundamental frequency and resonance band. The high-frequency band has a narrower distribution range and a relatively low median, indicating that high-frequency hearing recovery is relatively difficult, consistent with clinical observations. The kernel density estimation plot further reveals the probability density distribution of improvement values ​​in each sub-band. The mid-frequency band exhibits a wide bimodal distribution, indicating the existence of two different recovery modes, while the low-frequency and high-frequency bands show a unimodal distribution, indicating a relatively uniform recovery mode.

[0059] In one embodiment, such as Figure 3As shown, the distribution of hearing recovery effects in patients of different age groups was analyzed to verify the necessity of the age normalization factor in the method of this invention. The experiment divided patients into five age groups, and a stacked bar chart was used to display the distribution ratio of each age group across the four recovery levels. A clear age trend can be observed in the chart: the proportion of "significant improvement" and "complete recovery" is significantly higher in the younger patient group, while the proportion of "no improvement" and "partial improvement" gradually increases with age. In patients aged 20 to 30, the sum of the proportions of "complete recovery" and "significant improvement" reaches a high level, indicating that younger patients have greater recovery potential. Conversely, in patients aged 61 to 70, the proportions of "no improvement" and "partial improvement" increase significantly, reflecting the important influence of age on hearing recovery. This age-related distribution of recovery patterns verifies the importance of introducing an age normalization factor in the feature reconstruction of this invention, enabling the model to better adapt to the recovery characteristics of patients of different ages.

[0060] In this embodiment, as Figure 4 As shown, the performance of the proposed method and the traditional convolutional neural network method are compared under different types of hearing loss. The experiment selected four common types of hearing loss: conductive hearing loss, sensorineural hearing loss, mixed hearing loss, and unknown types, using the F1 score as the evaluation metric. The experimental results clearly show that the proposed method achieves higher F1 scores than the traditional convolutional neural network method across all hearing loss types. The advantage of the proposed method is particularly evident in the complex types of sensorineural and mixed hearing loss, with a significant lead in F1 scores, indicating that the dynamic convolutional kernel and cross-band attention mechanism in the proposed method can effectively handle complex damage patterns. For conductive hearing loss, the performance difference between the two methods is relatively small, but the proposed method still maintains its advantage. For unknown types of hearing loss, the proposed method also shows better adaptability, demonstrating its strong generalization ability. These results verify that the proposed method, through multi-band feature reconstruction and personalized feature extraction, can better adapt to the assessment needs of different types of hearing loss.

[0061] In summary, this application proposes a multi-band feature reconstruction method based on the auditory critical band theory, which divides hearing data into physiologically relevant low-frequency, mid-frequency, and high-frequency bands to better capture recovery patterns between frequency bands and preserve cross-band recovery correlations. Adaptive convolution kernel generation technology is used to generate dynamic convolution kernels based on the patient's initial hearing threshold curve, thereby enhancing the model's ability to identify individualized recovery patterns and focusing on key damaged frequency bands; By introducing a cross-band attention mechanism, the model can calculate the recovery correlation between different frequency bands, enhance its ability to capture recovery patterns across frequency bands, and especially adjust the focus of the recovery pattern according to the specific type of hearing loss of the patient. By adopting the Bark scale-aware weighting scheme, higher weights are given to the recovery of key frequency bands in speech, enabling the model to more accurately focus on the impact of recovery on speech recognition and improve classification performance.

[0062] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for assessing hearing loss recovery based on artificial intelligence, characterized in that: The steps are as follows, performed sequentially: S1: Collect hearing threshold data at different frequency points. This hearing threshold data includes the initial hearing threshold before treatment and the current hearing threshold after treatment. At the same time, record the patient's age, hearing impairment type, and speech recognition rate in noisy environments, construct a training dataset, and generate an initial hearing threshold curve. S2: Multi-band hearing threshold feature reconstruction for the training dataset: Divide the training dataset into three physiologically relevant sub-bands: low frequency band, mid frequency band and high frequency band. Calculate the average hearing threshold improvement value of each sub-band and integrate the age normalization factor, hearing impairment type coding and noise speech recognition rate normalization value to obtain the multi-band feature matrix. S3: Build an AI evaluation model and obtain the best model; S4: Use this optimal model for hearing assessment and output the predicted recovery level.

2. The method for assessing hearing loss recovery based on artificial intelligence as described in claim 1, characterized in that: In step S2, the low frequency band includes three frequency points: 250Hz, 500Hz and 1kHz; the mid frequency band includes two frequency points: 2kHz and 4kHz; and the high frequency band includes two frequency points: 6kHz and 8kHz. The age normalization factor is the patient's age divided by 100; This type of hearing impairment is represented using unique-heat coding; The noise speech recognition rate is a percentage value obtained through speech recognition tests in noisy environments.

3. The method for assessing hearing loss recovery based on artificial intelligence as described in claim 2, characterized in that: The specific steps for constructing the artificial intelligence evaluation model in step S3 are as follows: S3-1: Smooth the initial hearing threshold curve to obtain a smoothed hearing threshold curve vector, and fuse the smoothed hearing threshold curve vector with the multi-band feature matrix to obtain a dynamic convolution kernel matrix; S3-2: The dynamic convolution kernel matrix is ​​used to perform convolution operation on the multi-band feature matrix. After the first modulation, the time-frequency feature map is obtained. The multi-band feature matrix is ​​then subjected to max pooling downsampling and the second modulation. The matrix after the second modulation is added element by element to the time-frequency feature map to obtain the time-frequency enhanced feature matrix. S3-3: Perform sub-band feature vector mapping to obtain query vector and key vector. Calculate the attention weight matrix between sub-bands using the Softmax function. Multiply the attention weight matrix with the transposed time-frequency enhancement feature matrix and obtain the enhanced interaction feature matrix based on the synergistic effect of the sub-band improvement value vectors. S3-4: Calculate the Bark-scale perceptual weight matrix of each sub-band using the Sigmoid function, multiply the Bark-scale perceptual weight matrix with the enhanced interactive feature matrix, and combine it with the damage type coding modulation to obtain the perceptual weighted feature matrix. S3-5: Tensor concatenate the multi-band feature matrix, the time-frequency feature matrix, and the perceptual weighted feature matrix along the row direction, and scale them based on trainable scaling factors to obtain a concatenated composite feature matrix. Flatten the concatenated composite feature matrix into a one-dimensional feature vector, and capture cross-scale nonlinear interactions based on a lightweight multilayer perceptron to obtain a fused feature vector. S3-6: Concatenate the fused feature vector with the sub-band improvement value vector, and calculate the predicted recovery level category based on the class probability distribution using the fully connected layer and the Softmax function. Calculate the difference-sensitive loss function, and use this difference-sensitive loss function to backpropagate the model until the validation loss no longer decreases or the validation accuracy no longer improves within a preset period. Then, output the best model.

4. The method for assessing hearing loss recovery based on artificial intelligence as described in claim 3, characterized in that: In step S3-1, the initial hearing threshold curve is smoothed by applying the Savitzky-Golay filter, and the extended vector of the average hearing threshold improvement value of each sub-band is fused to obtain the smoothed hearing threshold curve vector. The smooth hearing threshold curve vector is fused with the multi-band feature matrix, and a linear transformation is performed based on the learnable weight matrix and bias vector. The Sigmoid function is then applied to generate the dynamic convolution kernel matrix.

5. The artificial intelligence-based hearing loss recovery assessment method as described in claim 4, characterized in that: In step S3-2, the first modulation is to combine the matrix after convolution with the frequency domain modulation of the subband improvement value, and apply the activation function to obtain the time-frequency feature map; The second modulation is to modulate the pooling features by generating a gated vector based on the subband improvement value and the dynamic convolution kernel matrix. The dynamic convolution kernel matrix is ​​then used to perform a time-frequency joint convolution with the time-frequency feature map, so that the convolution operation is notified to slide along the frequency band dimension and the time dimension, which is used to capture the local correlation between the subband improvement value and individual attributes in the treatment cycle.

6. The method for assessing hearing loss recovery based on artificial intelligence as described in claim 5, characterized in that: In step S3-3, the specific operation of sub-band feature vector mapping is as follows: the feature vectors of each sub-band in the time-frequency feature map are fused with the sub-band improvement value and the multi-band feature matrix, and a linear transformation is performed based on the query weight matrix and the key weight matrix to obtain the query vector and the key vector, thereby enhancing the vector's ability to represent the recovery of correlation.

7. The artificial intelligence-based hearing loss recovery assessment method as described in claim 6, characterized in that: In steps S3-6, when backpropagating the model using the difference-sensitive loss function, the adaptive moment estimator optimizer is used to adjust the trainable parameters in the model. In each iteration, the model calculates the predicted recovery level from forward propagation and calculates the loss value based on the difference between the true label and the predicted result. The parameters are updated through the gradient descent algorithm to minimize the loss.

8. The artificial intelligence-based hearing loss recovery assessment method as described in claim 7, characterized in that: In step S4, when using this optimal model for hearing assessment, it is necessary to input the patient's initial hearing threshold, current hearing threshold, age, type of hearing impairment, and noise speech recognition rate.

9. The method for assessing hearing loss recovery based on artificial intelligence as described in claim 8, characterized in that: In step S4, the output predicted recovery level includes four categories: no improvement, partial improvement, significant improvement, and complete recovery.

Images