Noise reduction and audio enhancement system of wearable hearing aid device

Abstract

The invention relates to the technical field of voice processing, and discloses a noise reduction and audio enhancement system of wearable hearing aid equipment. Comprising an area identification module used for identifying a target area from sub-areas; the segmentation screening module is used for screening a target sample from the audio samples; the noise reduction adjustment module is used for converting the target sample into a noise reduction sample through a noise reduction model; the enhanced splicing module is used for splicing and combining the enhanced samples into ultimate audio; according to the method and the device, a local regionalization audio signal receiving effect is realized, high burden caused by undifferentiated processing of full-space audio signals is avoided, and the noise reduction intensity can be adaptively and dynamically adjusted according to different audio characteristics by dynamically adjusting the noise reduction model, so that the noise reduction efficiency is improved. The problem of insufficient pertinence caused by a traditional one-step noise reduction mode is avoided, meanwhile, the phenomenon of frequency band breakage or abrupt jump of the audio data is effectively avoided, and the use effect of the hearing aid device is improved.

Description

Technical Field

[0001] This invention relates to the field of speech processing technology, and more specifically, to a noise reduction and audio enhancement system for wearable hearing aids. Background Technology

[0002] Wearable hearing aids are an important assistive tool for people with hearing impairments. Their core mission is to effectively improve speech credibility and auditory comfort in complex and ever-changing acoustic environments, helping users regain clear communication abilities. By performing noise reduction and enhancement operations on the received audio data, they can reduce the negative interference caused by unnecessary noise in the external environment, while improving the fluency and authenticity of the audio data. Therefore, targeted noise reduction and enhancement operations are required for audio data.

[0003] Reference patent application CN119920263A discloses a training method, apparatus, electronic device, and storage medium for a speech enhancement model. The method includes: acquiring audio data, which includes at least a first audio signal containing environmental noise and a second audio signal not containing environmental noise; extracting first audio features from the first audio signal and second audio features from the second audio signal; using a pre-set first large model to denoise the first and second audio features, obtaining first denoised features of the first audio signal and second denoised features of the second audio signal; converting the frequency domains of the first and second denoised features to obtain a first denoised signal corresponding to the first audio signal and a second denoised signal corresponding to the second audio signal; calculating a loss function between the first and second denoised signals; and adjusting the enhancement parameters of the first large model using the loss function to generate a speech enhancement model. Existing hearing aids typically employ an omnidirectional, non-discriminatory sound pickup method. This results in a large amount of irrelevant environmental noise being mixed into the received audio data. This not only increases the burden on signal processing but also makes it difficult to accurately focus on the target speaker in noisy scenes. Furthermore, the use of fixed parameters or a one-size-fits-all approach to noise reduction prevents the dynamic adjustment of noise reduction intensity and strategy based on the real-time audio characteristics of the audio data. Consequently, hearing aids are unable to effectively suppress environmental noise in audio data in complex and rapidly changing scenarios, thus reducing the noise reduction effect of the audio data.

[0004] In view of this, the present invention proposes a noise reduction and audio enhancement system for wearable hearing aids to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a noise reduction and audio enhancement system for wearable hearing aids, applied to a voice processor, comprising: The region recognition module is used to divide the space into sub-regions based on the reference direction line of the hearing aid device, perform audio trigger detection on the sub-regions in hearing aid mode, and identify the target region from the sub-regions. The segmentation and filtering module is used to segment the audio data of the target region into audio samples according to audio segmentation criteria, perform void analysis on the audio samples, and filter out the target samples from the audio samples. The noise reduction adjustment module is used to index the parameter adjustment coefficients of the noise reduction parameters from the database based on the audio features of the target sample, dynamically adjust the noise reduction model based on the parameter adjustment coefficients, and convert the target sample into a noise-reduced sample through the noise reduction model. The enhancement and splicing module is used to semantically enhance the key semantics of the noise reduction samples, generate enhanced samples, and splice and combine the enhanced samples into the final audio.

[0006] Furthermore, the method for dividing sub-regions is as follows: Draw lines along the edges of the video screen on both sides of the camera on the hearing aid to generate boundary lines, and record the intersection of the two boundary lines at adjacent positions on the two video screens as the reference point; The built-in positioning chip retrieves the coordinates of the two hearing aids, imports the coordinates of the two points and the reference point into three-dimensional space, and connects the coordinates of the two points to generate a point line. Draw an auxiliary line through the reference point along a direction perpendicular to the point line, and extend the auxiliary line to the point line to generate the reference direction line; Using the baseline direction line as the first region boundary line and the preset region angle as the standard, draw a ring-shaped distribution of region boundary lines, and record the spatial region between two adjacent region boundary lines as a sub-region, thus obtaining A sub-regions.

[0007] Furthermore, the hearing aid modes include indoor mode, noisy mode, outdoor mode, and leisure mode; The method for identifying the target region is as follows: The lower and upper frequency limits of the hearing aid are preset, and audio signals with frequency values ​​between the lower and upper frequency limits are collected simultaneously in A sub-regions to obtain A audio sets; Human voices are detected one by one in the audio signals of A audio sets using voice activity detection technology. Audio signals with positive correlation are recorded as human voice audio, and audio signals with negative correlation are recorded as environmental audio. When human voice audio exists in a sub-region, the standard audio that matches the hearing aid mode is retrieved from the database, and the human voice audio and environmental audio are compared with the human voice part and environmental part of the standard audio respectively. When the human voice audio in a sub-region overlaps with the human voice portion, and the environmental audio overlaps with the environmental portion, the sub-region is recorded as the target region, resulting in B target regions.

[0008] Furthermore, the audio segmentation criterion is: to dynamically segment audio samples using a frequency band span as the frequency band interval.

[0009] Furthermore, the audio sample segmentation method is as follows: Within the audio data acquisition period, randomly mark C non-repeating time points, and query the frequency values ​​of the audio data at each of the C time points; Remove the maximum and minimum frequency values, sum the remaining C-2 frequency values ​​and average them to calculate the frequency mean. Divide the upper and lower frequency limits to generate the frequency span value, and divide the frequency span value by the frequency mean to calculate the frequency multiple. When the frequency multiple is an integer, the average frequency is used as the frequency range spanning one frequency band. When the frequency multiple is not an integer, the average frequency is continuously reduced and the frequency multiple after reduction is calculated until the frequency multiple after reduction is an integer. Then, the average frequency after reduction is used as a frequency range spanning a frequency band. Using a frequency band span as a standard, the frequency between the upper and lower limits of the frequency band is divided into D frequency band intervals, and the audio data with frequency values ​​in the D frequency band intervals are segmented to obtain D audio samples.

[0010] Furthermore, the method for screening target samples is as follows: Based on a preset frame duration, the audio samples are divided into E consecutive frame samples, and windowing is applied to each of the E frame samples to generate E windowed frame samples. The short-time energy of each of the E window frame samples is calculated one by one using the short-time energy algorithm, and the number of window frame samples with short-time energy less than the standard energy is counted and recorded as the short-time value. The zero-crossing rate of each of the E window frame samples is calculated one by one using the zero-crossing rate algorithm, and the number of window frame samples whose zero-crossing rate is not within the standard zero-crossing range is counted and recorded as the zero-crossing value. The short-time value is compared with the number of window frame samples to calculate the short-time percentage, and the zero-crossing value is compared with the number of window frame samples to calculate the zero-crossing percentage. When the proportion of short-time frames is greater than or equal to the preset short-time ratio threshold, and the proportion of zero-crossing frames is greater than or equal to the preset zero-crossing ratio threshold, the window frame sample is recorded as a valid sample. The number of valid samples in each of the D audio samples is counted one by one. When the number of valid samples accounts for more than 85% of the number of window frame samples, the audio sample is recorded as the target sample, and F target samples are obtained.

[0011] Furthermore, noise reduction parameters include frequency band gain, noise reduction depth, and compression ratio; audio features include temporal features, frequency features, prosodic features, and spatial features.

[0012] Furthermore, the method for dynamically adjusting the noise reduction model is as follows: Based on time-domain features, frequency features, prosodic features, and spatial features, respectively, and using positive matching of features and coefficients as the indexing rule, the corresponding time-domain coefficient set, frequency-domain coefficient set, prosodic coefficient set, and spatial coefficient set are indexed from the database. The time-domain coefficients, frequency-domain coefficients, prosodic coefficients, and spatial coefficients in the time-domain coefficient set, frequency-domain coefficient set, prosodic coefficient set, and spatial coefficient set are summed and averaged to calculate the time-domain mean, frequency-domain mean, prosodic mean, and spatial mean. The mean of the time domain, frequency domain, prosody, and space is added together and averaged to calculate the mean of the coefficients. The mean of the time domain, frequency domain, prosody, and space is then subtracted from the mean of the coefficients, and the minimum value of the difference is recorded as the parameter adjustment coefficient. Initialize the frequency band gain, noise reduction depth, and compression ratio of the noise reduction model. Using the increase of one parameter adjustment coefficient as a standard, dynamically adjust the frequency band gain, noise reduction depth, and compression ratio until the signal-to-noise ratio is greater than the standard signal-to-noise ratio, the segmented signal-to-noise ratio is greater than the standard segmented signal-to-noise ratio, and the speech spectrum distortion is less than the standard speech spectrum distortion. This will give you the noise reduction model.

[0013] Furthermore, when generating enhanced samples, H key semantics in the denoised samples are identified through automatic speech recognition technology. Enhancement contour lines are configured at the beginning and end of each of the H key semantics. F denoised samples are then imported one by one into the enhancement engine equipped with an active enhancement algorithm for semantic enhancement, generating H enhanced samples.

[0014] Furthermore, the final audio generation method is as follows: Establish a basic queue with D blank spaces, and use a frequency band span as a standard to annotate the frequency values ​​in the D blank spaces in sequence, so that the blank spaces are converted into splicing spaces; Based on the standard that the frequency values ​​overlap or are consistent, H enhanced samples are sequentially imported into the corresponding splicing positions, and splicing positions without enhanced samples are recorded as transition positions. The average value of the transition is calculated by adding the maximum value of the frequency value of the previous splicing bit above the transition bit and the minimum value of the frequency value of the next splicing bit below the transition bit. The transition mean is reduced by two-thirds to form the spliced ​​value, and the spliced ​​value is noted on the corresponding transition bit, which causes the base queue to be converted into the final audio.

[0015] The technical effects of the noise reduction and audio enhancement system of the wearable hearing aid device of this invention: (1): By dividing the spatial ring into sub-regions, the present invention can break down the audio reception range of the overall structure into smaller parts, thereby achieving localized audio signal reception. This avoids the high burden caused by the indiscriminate processing of audio signals throughout the entire space, which not only reduces the workload of hearing aids but also greatly improves the targeting and accuracy of audio signal reception.

[0016] (2): This invention uses the audio features of the target sample to index the parameter adjustment coefficients and dynamically adjusts the noise reduction model. This allows for adaptive and dynamic adjustment of the noise reduction intensity based on different audio features, ensuring that the audio data achieves the optimal balance in key indicators such as signal-to-noise ratio, segmented signal-to-noise ratio, and speech spectrum distortion. This significantly improves the credibility and authenticity of the audio, thereby avoiding the problem of insufficient targeting caused by the traditional one-size-fits-all noise reduction method and achieving adaptive noise reduction effect for audio data of different frequencies.

[0017] (3): By employing a transition bit smoothing mechanism in the final audio synthesis stage, the present invention can orderly splice discrete enhancement samples according to frequency bands and perform smooth interpolation processing on missing frequency bands, thereby ensuring that the output final audio is smooth and natural in sound, effectively avoiding the phenomenon of frequency band breakage or abrupt jump in audio data, and improving the use effect of hearing aids. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the noise reduction and audio enhancement system of the wearable hearing aid device provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of a voice processor module provided in Embodiment 1 of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: Please refer to Figure 1-2 As shown, the noise reduction and audio enhancement system of the wearable hearing aid device described in this embodiment is applied to a voice processor and includes: The region recognition module determines the reference direction line of the hearing aid device, divides the surrounding space into sub-regions, performs audio trigger detection on the sub-regions in the selected hearing aid mode, and identifies the target region from the sub-regions. The reference direction line refers to the direction line of the intersection of the field of view of the real-time captured images of the cameras on the two hearing aids when the hearing aids are receiving audio signals normally at the current moment. It can be used as the initial position basis for dividing the audio signal receiving range of the two hearing aids. In this embodiment, when using wearable hearing aids, the user needs to wear the hearing aids on both ears simultaneously, and each hearing aid is integrated with a miniature camera and a positioning chip to obtain real-time images and coordinate information in front of the user's line of sight.

[0021] After obtaining the reference direction line, the user's surrounding space can be divided into a ring shape with equal angles, using the reference direction line as the initial position. This allows the surrounding space to be divided into A continuously distributed sub-regions, with each sub-region having the same corresponding angle. This process breaks down the audio signal reception range of the hearing aid into smaller parts, achieving a localized audio signal reception effect. Specifically, the method for dividing sub-regions is as follows: At the current moment, video images from the cameras on the two hearing aids are captured, lines are drawn along the edges of the video images to generate boundary lines, and the intersection of the two boundary lines at adjacent positions on the two video images is recorded as the reference point. The built-in positioning chip retrieves the coordinates of the two hearing aids, imports the coordinates of the two points and the reference point into three-dimensional space, and connects the coordinates of the two points to generate a point line. Draw an auxiliary line through the reference point along a direction perpendicular to the point line, and extend the auxiliary line to the point line to generate the reference direction line; Using the baseline direction line as the first region boundary line and the preset region angle as the standard, draw a ring-shaped distribution of region boundary lines, and record the spatial region between two adjacent region boundary lines as a sub-region, thus obtaining A sub-regions.

[0022] It should be noted that the sub-regions are not actually enclosed spaces in physical structure, but rather spaces with virtual boundary lines as the outer contour and point coordinates as the location basis. This allows the surrounding open space to be divided into several relatively independent and scattered areas, so as to facilitate the subsequent detailed acquisition and analysis of audio signals within each sub-region.

[0023] In this embodiment, the sub-region is a fan-shaped area with the intersection of the auxiliary line and the point line as its vertex. This fan-shaped area only needs to limit the coverage angle, not the coverage radius. Any audio that can be received by the hearing aid device within the sub-region is considered a potential audio signal.

[0024] The divided sub-regions serve as the location and spatial limits for subsequent audio signal acquisition. Under normal circumstances, each sub-region is in a real-time standby state for audio signal reception. However, when users wear hearing aids for daily life and work, external audio signals do not appear at all times, and the source of audio signals does not fill the space corresponding to each sub-region. Therefore, it is necessary to determine and analyze whether there are audio signals in the sub-regions that need to be acquired, denoised, and enhanced. When identifying the target region from the sub-region, it is necessary to perform audio trigger detection on the sub-region so that the audio trigger detection can detect and analyze all audio signals present in the sub-region and use this as a direct basis for identifying the target region. When identifying the target area, it is necessary to first select the appropriate hearing aid mode. The hearing aid mode is a mode used to limit the working state of the hearing aid device, which can provide the corresponding basis for noise reduction and enhancement of audio signals in different usage scenarios. In this embodiment, the hearing aid modes include indoor mode, noisy mode, outdoor mode, and leisure mode; indoor mode refers to the mode in which the user is in a relatively quiet indoor environment, noisy mode refers to the mode in which the user is in a noisy environment with a lot of background noise, outdoor mode refers to the mode in which the user is in an outdoor everyday environment, and leisure mode refers to the mode in which the user is in a comfortable entertainment and leisure environment.

[0025] In this embodiment, when selecting a hearing aid mode, the user needs to set and select it on the mobile phone, and only one hearing aid mode can be selected at any given time.

[0026] Specifically, the method for identifying the target region is as follows: The lower and upper frequency limits of the hearing aid are preset, and audio signals with frequency values ​​between the lower and upper frequency limits are collected simultaneously in A sub-regions to obtain A audio sets; the lower and upper frequency limits are the minimum and maximum signal frequencies when the hearing aid is collecting audio signals normally. Human voice detection is performed on each of the audio signals in the A audio sets one by one using speech activity detection technology. The audio signals with positive correlation are recorded as human voice audio, and the audio signals with negative correlation are recorded as ambient audio. Positive and negative correlation are used to specifically represent whether the detection results of human voice detection contain audio signals of human speech. If there is no human voice audio in a sub-region, it means that there is no human speaking sound in that sub-region, and therefore no audio trigger detection will be performed on that sub-region. When human voice audio is present in a sub-region, it means that there is human speaking in that sub-region, and audio trigger detection is performed on that sub-region. The database is used to retrieve standard audio that matches the hearing aid mode. The human voice audio and environmental audio are compared with the human voice and environmental audio in the standard audio respectively. The standard audio is the human voice audio and environmental audio that are pre-stored and set in the database and correspond to the hearing aid mode. When the human voice audio in a sub-region overlaps with the human voice portion and the environmental audio overlaps with the environmental portion, it indicates that there is a strong correlation between the audio signal in that sub-region and the user. In this case, the sub-region is recorded as the target region, and B target regions are obtained. When the audio of the human voice in a sub-region does not overlap with the human voice portion, it indicates that there is no strong correlation between the audio signal in that sub-region and the user, and therefore that sub-region is not recorded as the target region.

[0027] It should be noted that the number of target regions may be any number between 0 and A. Since the user's position changes in real time when wearing the hearing aid, the spatial position of the divided sub-regions also changes in real time. Therefore, the target regions may change at any given moment, so it is necessary to identify and process the target regions in real time.

[0028] The segmentation and filtering module, based on audio segmentation criteria, segments the audio data of the target region into audio samples, performs void analysis on the audio samples, and filters out the target samples from the audio samples. The audio data in the target area is usually composed of a large number of audio signals in different frequency bands, which makes the frequency range covered in the audio data also wide. Since the amplitude and methods corresponding to different frequencies are not the same when performing noise reduction and enhancement, and the audio data with a large frequency span is not conducive to precise analysis and processing, it is necessary to segment the audio data to divide the overall audio data into scattered and continuous audio samples. It should be noted that although the audio data is collected in real time, the collection time of each audio data is not equal to 0. In this embodiment, real-time collection refers to the collection of audio data with a relatively short collection time. For example, the collection time is 50ms, and the audio data at this time is the sum of audio signals collected within 50ms.

[0029] In this embodiment, the frequency ranges corresponding to each audio sample are non-overlapping, the frequencies corresponding to two adjacent audio samples are continuous, and the frequency coverage range of all audio samples is between the lower frequency limit and the upper frequency limit.

[0030] When segmenting audio data into audio samples, in order to ensure the accuracy and reliability of audio sample segmentation, it is necessary to do so within the constraints of audio segmentation criteria. Specifically, the audio segmentation criterion is as follows: use a frequency band span as the frequency band interval to dynamically segment the audio samples; this ensures that the segmented audio samples can maintain a dynamic correlation with the frequency of the collected audio data, thereby avoiding the phenomenon of poor fit when segmenting frequency bands with fixed spans.

[0031] The method for segmenting audio samples is as follows: Within the audio data acquisition period, randomly mark C non-repeating time points, and query the frequency values ​​of the audio data at each of the C time points; Remove the maximum and minimum frequency values, sum the remaining C-2 frequency values, and calculate the average frequency value. The frequency span value is generated by subtracting the upper frequency limit from the lower frequency limit value. The frequency span value is then divided by the frequency mean value to calculate the frequency multiple. When the frequency multiple is an integer, the average frequency is used as the frequency range spanning one frequency band. When the frequency multiple is not an integer, the average frequency is continuously reduced and the frequency multiple after reduction is calculated until the frequency multiple after reduction is an integer. Then, the average frequency after reduction is used as a frequency range spanning a frequency band. Using a frequency band span as a standard, the frequency between the upper and lower limits of the frequency band is divided into D frequency band intervals, and the audio data with frequency values ​​in the D frequency band intervals are segmented to obtain D audio samples.

[0032] After obtaining the audio samples, the frequencies corresponding to the same audio sample are all close and adjacent, and audio data with similar frequencies can be summarized and represented.

[0033] After the audio data is segmented into audio samples, each audio sample may not necessarily contain an audio signal. There may be some audio samples with no actual audio signal. Therefore, it is necessary to detect and analyze the audio samples with gaps, and identify the target sample from the audio samples based on the results of the gap analysis. In this embodiment, the target sample refers to an audio sample that does not exhibit audio signal voids.

[0034] Specifically, the method for selecting target samples is as follows: Using a preset frame duration as a standard, the audio samples are divided into E consecutive frame samples, and windowing is performed on each of the E frame samples to generate E windowed frame samples. The specific windowing type can be a Hamming window. The preset frame duration is the duration used to divide the audio samples equally to ensure that the duration corresponding to each frame sample is consistent. For example, when the acquisition duration is 50ms, the preset frame duration can be 5ms or 10ms. The short-time energy of E window frame samples is calculated one by one using the short-time energy algorithm, and the number of window frame samples with short-time energy less than the standard energy is counted and recorded as the short-time value; the short-time energy algorithm is the prior art in this field; The zero-crossing rate of each of the E window frame samples is calculated one by one using the zero-crossing rate algorithm, and the number of window frame samples whose zero-crossing rate is not within the standard zero-crossing range is counted and recorded as the zero-crossing value; the zero-crossing rate algorithm is the prior art in this field. The short-time value is compared with the number of window frame samples to calculate the short-time percentage, and the zero-crossing value is compared with the number of window frame samples to calculate the zero-crossing percentage. When the short-time ratio is greater than or equal to the preset short-time ratio threshold and the zero-crossing ratio is greater than or equal to the preset zero-crossing ratio threshold, it indicates that there is no audio signal hole in the window frame sample, and the window frame sample is recorded as a valid sample. The number of valid samples in each of the D audio samples is counted one by one. When the number of valid samples accounts for more than 85% of the number of window frame samples, the audio sample is recorded as the target sample, and F target samples are obtained.

[0035] It should be noted that short-time energy measures the energy of a speech signal within a short time window. Speech segments usually have higher energy, while silence or noise segments have lower energy. Zero-crossing rate refers to the number of times the signal crosses the zero axis per unit time, reflecting the high-frequency components or the intensity of oscillations of the signal. The higher the short-time energy or zero-crossing rate, the lower the probability of holes appearing in the audio sample. The preset short-time ratio threshold and the preset zero-crossing ratio threshold refer to the minimum values ​​of the short-time ratio and the zero-crossing ratio when a sample is identified as a target sample, respectively. These values ​​provide numerical limits on the magnitude of the short-time energy and zero-crossing rate in the target sample, thereby improving the accuracy of target sample screening.

[0036] The noise reduction adjustment module extracts the audio features of the target sample, matches the parameter adjustment coefficients of the noise reduction parameters, dynamically adjusts the noise reduction model, and converts the target sample into a noise-reduced sample through the noise reduction model. After obtaining the target sample, the target sample can be used as the object to perform subsequent noise reduction processing. In this embodiment, when denoising the target sample, it is necessary to use a noise reduction model to denoise the target sample, so that the target sample generates a denoised sample after noise reduction.

[0037] When denoising target samples using a denoising model, the intensity and requirements of denoising vary depending on the frequency of the target sample. Therefore, it is necessary to adjust the magnitude of the denoising parameters. In order to improve the accuracy of the dynamic adjustment of the denoising parameters, it is necessary to match the parameter adjustment coefficient of the denoising parameters. Specifically, the parameter adjustment coefficient is a proportional value used to dynamically adjust the magnitude of the noise reduction parameters, thereby achieving dynamic changes in the noise reduction effect of the noise reduction model.

[0038] In this embodiment, the noise reduction parameters include frequency band gain, noise reduction depth, and compression ratio. Frequency band gain refers to the amount of amplification or attenuation applied to the signal within a specific frequency sub-band (such as low frequency, mid frequency, and high frequency). Noise reduction depth refers to the maximum attenuation that the system can apply to the estimated background noise components. Compression ratio refers to the ratio of the output increment to the input increment after the input signal exceeds the threshold.

[0039] When calculating the parameter adjustment coefficient, the audio features of the target sample must first be extracted, and the required parameter adjustment coefficient can only be obtained after analysis and calculation based on the audio features. Audio features refer to numerical information extracted from the original audio signal of a target sample that can characterize its acoustic properties. Audio features are widely used in tasks such as speech recognition, speaker recognition, music information retrieval, hearing aid noise reduction, emotion analysis, and environmental sound classification. Specifically, audio features include temporal features, frequency features, prosodic features, and spatial features; Temporal characteristics refer to the changes of audio signals on the time axis. Temporal characteristics include, but are not limited to, short-time average amplitude and maximum short-time energy. Temporal characteristics are obtained through temporal analysis.

[0040] Frequency characteristics refer to the frequency distribution characteristics of a signal after Fourier transform. Frequency domain characteristics include, but are not limited to, spectral bandwidth and spectral flatness. Frequency characteristics are obtained through analysis after Fourier transform.

[0041] Prosodic features refer to the features that reflect the rhythm and vocal characteristics of speech. Prosodic features include, but are not limited to, fundamental frequency peak and intensity. Prosodic features are obtained through autonomous detection by hearing aids.

[0042] Spatial features refer to features in spatial location, including but not limited to direction of arrival and estimated time delay; spatial features are obtained through autonomous detection by hearing aids.

[0043] After obtaining the audio features, the parameter adjustment coefficients of the noise reduction parameters corresponding to the audio features can be matched from the database, and the noise reduction model can be dynamically adjusted based on the parameter adjustment coefficients. Specifically, the method for dynamically adjusting the noise reduction model is as follows: Based on time-domain features, frequency features, prosodic features, and spatial features, respectively, and using positive matching of features and coefficients as the indexing rule, the corresponding time-domain coefficient set, frequency-domain coefficient set, prosodic coefficient set, and spatial coefficient set are indexed from the database. The time-domain coefficients, frequency-domain coefficients, prosodic coefficients, and spatial coefficients in the time-domain coefficient set, frequency-domain coefficient set, prosodic coefficient set, and spatial coefficient set are summed and averaged to calculate the time-domain mean, frequency-domain mean, prosodic mean, and spatial mean. The mean of the time domain, frequency domain, prosody, and space is added together and averaged to calculate the mean of the coefficients. The mean of the time domain, frequency domain, prosody, and space is then subtracted from the mean of the coefficients, and the minimum value of the difference is recorded as the parameter adjustment coefficient. Initialize the frequency band gain, noise reduction depth, and compression ratio of the noise reduction model. Based on the increase of one parameter adjustment coefficient, dynamically adjust the frequency band gain, noise reduction depth, and compression ratio, and query the signal-to-noise ratio, segmented signal-to-noise ratio, and speech spectrum distortion. The signal-to-noise ratio (SNR), segmented SNR, and speech spectral distortion are compared with the standard SNR, standard segmented SNR, and standard speech spectral distortion in the hearing aid mode, respectively. The dynamic adjustment of the noise reduction parameters is stopped when the SNR is greater than the standard SNR, the segmented SNR is greater than the standard segmented SNR, and the speech spectral distortion is less than the standard speech spectral distortion, thus obtaining the noise reduction model.

[0044] In this embodiment, the noise reduction model is an artificial intelligence model that takes the target sample and noise reduction parameters as input data and the noise reduction sample as output data. This enables the noise reduction model to automatically perform noise reduction processing on the target sample based on the specific situation of the input target sample and combined with the dynamically adjusted noise reduction parameters, and output the required noise reduction sample. It should be noted that the noise reduction model is based on the speech noise reduction model and is obtained by integrating it with deep learning technology for training. When training the noise reduction model, a large number of different types of target samples, noise reduction parameters and corresponding noise reduction samples are collected in advance. The target samples and noise reduction parameters are used as input data and the noise reduction samples are used as output data. The noise reduction model is continuously iterated and optimized until the noise reduction rate of the noise reduction model reaches the preset threshold. Then the corresponding noise reduction model can be obtained.

[0045] In this embodiment, after dynamically adjusting the denoising parameters of the denoising model, F target samples can be sequentially input into the denoising model to output the corresponding F denoised samples.

[0046] The enhancement and splicing module performs semantic enhancement on the key semantics of the noise reduction samples, generates enhanced samples, and splices and combines the enhanced samples to generate the final audio. After obtaining the noise-reduced sample, the noise-reduced sample has not been enhanced at this time, so the key audio signals in the noise-reduced sample have not been enhanced accordingly, which may result in some key contents being blurred or even lost. Therefore, it is necessary to enhance the noise-reduced sample. When enhancing denoised samples, it is necessary to first identify the key semantics within the denoised samples and then perform targeted semantic enhancement on the parts corresponding to the key semantics. Specifically, key semantics are the audio signals corresponding to human speech or environmental sounds in the denoised samples that need to be promptly transmitted to the user. In this embodiment, the number of key semantics is not fixed, and the conversion between denoised samples and enhanced samples can be achieved by semantically enhancing the key semantics.

[0047] Specifically, when the enhanced sample is obtained, H key semantics in the noise reduction sample are identified by automatic speech recognition technology, and enhancement contour lines are configured at the beginning and end of each of the H key semantics. Then, F noise reduction samples are imported one by one into the enhancement engine equipped with an active enhancement algorithm, which can enhance the key semantics of the noise reduction sample and generate enhanced samples.

[0048] It should be noted that the enhancement engine is a processor used to enhance key semantics. It has active enhancement algorithms (including spectral subtraction, nonparametric methods, and model-based methods) that can actively and directly enhance the audio signal corresponding to the key semantics in order to restore the quality of the audio data.

[0049] The obtained enhanced samples are in an independent and discrete structure, which cannot represent the audio data of the target area in a holistic and comprehensive way. Therefore, it is necessary to splice and combine the discrete enhanced samples so that the audio data in the target area can be effectively summarized and the final audio output by the hearing aid device can be obtained. Specifically, the method for generating the final audio is as follows: Establish a basic queue with D blank spaces, and use a frequency band span as a standard to annotate the frequency values ​​in the D blank spaces in sequence, so that the blank spaces are converted into splicing spaces; Based on the standard that the frequency values ​​overlap or are consistent, H enhanced samples are sequentially imported into the corresponding splicing positions, and splicing positions without enhanced samples are recorded as transition positions. The average value of the transition is calculated by adding the maximum value of the frequency value of the previous splicing bit above the transition bit and the minimum value of the frequency value of the next splicing bit below the transition bit. The transition mean is reduced by two-thirds to form the spliced ​​value, and the spliced ​​value is noted on the corresponding transition bit, which causes the base queue to be converted into the final audio.

[0050] It should be noted that by noting the splicing value on the transition bit, the connection and transition between the splicing bit and the transition bit are made smoother and more stable, avoiding abnormal situations such as local silence, no sound, or disjointedness in the final audio, and improving the overall orderliness and smoothness of the final audio.

[0051] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A noise reduction and audio enhancement system for wearable hearing aids, applied to a speech processor, characterized in that, include: The region recognition module is used to divide the space into sub-regions based on the reference direction line of the hearing aid device, perform audio trigger detection on the sub-regions in hearing aid mode, and identify the target region from the sub-regions. The segmentation and filtering module is used to segment the audio data of the target region into audio samples according to audio segmentation criteria, perform void analysis on the audio samples, and filter out the target samples from the audio samples. The noise reduction adjustment module is used to index the parameter adjustment coefficients of the noise reduction parameters from the database based on the audio features of the target sample, dynamically adjust the noise reduction model based on the parameter adjustment coefficients, and convert the target sample into a noise-reduced sample through the noise reduction model. The enhancement and splicing module is used to semantically enhance the key semantics of the noise reduction samples, generate enhanced samples, and splice and combine the enhanced samples into the final audio.

2. The noise reduction and audio enhancement system for wearable hearing aids according to claim 1, characterized in that, The method for dividing sub-regions is as follows: Draw lines along the edges of the video screen on both sides of the camera on the hearing aid to generate boundary lines, and record the intersection of the two boundary lines at adjacent positions on the two video screens as the reference point; The built-in positioning chip retrieves the coordinates of the two hearing aids, imports the coordinates of the two points and the reference point into three-dimensional space, and connects the coordinates of the two points to generate a point line. Draw an auxiliary line through the reference point along a direction perpendicular to the point line, and extend the auxiliary line to the point line to generate the reference direction line; Using the baseline direction line as the first region boundary line and the preset region angle as the standard, draw a ring-shaped distribution of region boundary lines, and record the spatial region between two adjacent region boundary lines as a sub-region, thus obtaining A sub-regions.

3. The noise reduction and audio enhancement system for wearable hearing aids according to claim 2, characterized in that, Hearing aid modes include indoor mode, noisy mode, outdoor mode, and leisure mode; The method for identifying the target region is as follows: The lower and upper frequency limits of the hearing aid are preset, and audio signals with frequency values ​​between the lower and upper frequency limits are collected simultaneously in A sub-regions to obtain A audio sets; Human voice detection is performed on the audio signals in A audio sets one by one using voice activity detection technology. The audio signals with positive correlation are recorded as human voice audio, and the audio signals with negative correlation are recorded as environmental audio. When human voice audio exists in a sub-region, the standard audio that matches the hearing aid mode is retrieved from the database, and the human voice audio and environmental audio are compared with the human voice part and environmental part of the standard audio respectively. When the human voice audio in a sub-region overlaps with the human voice portion, and the environmental audio overlaps with the environmental portion, the sub-region is recorded as the target region, resulting in B target regions.

4. The noise reduction and audio enhancement system for wearable hearing aids according to claim 3, characterized in that, The audio segmentation criterion is: use a frequency band span as the frequency band interval to dynamically segment audio samples.

5. The noise reduction and audio enhancement system for wearable hearing aids according to claim 4, characterized in that, The method for segmenting audio samples is as follows: Within the audio data acquisition period, randomly mark C non-repeating time points, and query the frequency values ​​of the audio data at each of the C time points; Remove the maximum and minimum frequency values, sum the remaining C-2 frequency values ​​and average them to calculate the frequency mean. Divide the upper and lower frequency limits to generate the frequency span value, and divide the frequency span value by the frequency mean to calculate the frequency multiple. When the frequency multiple is an integer, the average frequency is used as the frequency range spanning one frequency band. When the frequency multiple is not an integer, the average frequency is continuously reduced and the frequency multiple after reduction is calculated until the frequency multiple after reduction is an integer. Then, the average frequency after reduction is used as a frequency range spanning a frequency band. Using a frequency band span as a standard, the frequency between the upper and lower limits of the frequency band is divided into D frequency band intervals, and the audio data with frequency values ​​in the D frequency band intervals are segmented to obtain D audio samples.

6. The noise reduction and audio enhancement system for wearable hearing aids according to claim 5, characterized in that, The method for selecting target samples is as follows: Based on a preset frame duration, the audio samples are divided into E consecutive frame samples, and windowing is applied to each of the E frame samples to generate E windowed frame samples. The short-time energy of each of the E window frame samples is calculated one by one using the short-time energy algorithm, and the number of window frame samples with short-time energy less than the standard energy is counted and recorded as the short-time value. The zero-crossing rate of each of the E window frame samples is calculated one by one using the zero-crossing rate algorithm, and the number of window frame samples whose zero-crossing rate is not within the standard zero-crossing range is counted and recorded as the zero-crossing value. The short-time value is compared with the number of window frame samples to calculate the short-time percentage, and the zero-crossing value is compared with the number of window frame samples to calculate the zero-crossing percentage. When the proportion of short-time frames is greater than or equal to the preset short-time ratio threshold, and the proportion of zero-crossing frames is greater than or equal to the preset zero-crossing ratio threshold, the window frame sample is recorded as a valid sample. The number of valid samples in each of the D audio samples is counted one by one. When the number of valid samples accounts for more than 85% of the number of window frame samples, the audio sample is recorded as the target sample, and F target samples are obtained.

7. The noise reduction and audio enhancement system for wearable hearing aids according to claim 6, characterized in that, Noise reduction parameters include frequency band gain, noise reduction depth, and compression ratio; Audio features include temporal features, frequency features, prosodic features, and spatial features.

8. The noise reduction and audio enhancement system for wearable hearing aids according to claim 7, characterized in that, The method for dynamically adjusting the noise reduction model is as follows: Based on time-domain features, frequency features, prosodic features, and spatial features, respectively, and using positive matching of features and coefficients as the indexing rule, the corresponding time-domain coefficient set, frequency-domain coefficient set, prosodic coefficient set, and spatial coefficient set are indexed from the database. The time-domain coefficients, frequency-domain coefficients, prosodic coefficients, and spatial coefficients in the time-domain coefficient set, frequency-domain coefficient set, prosodic coefficient set, and spatial coefficient set are summed and averaged to calculate the time-domain mean, frequency-domain mean, prosodic mean, and spatial mean. The mean of the time domain, frequency domain, prosody, and space is added together and averaged to calculate the mean of the coefficients. The mean of the time domain, frequency domain, prosody, and space is then subtracted from the mean of the coefficients, and the minimum value of the difference is recorded as the parameter adjustment coefficient. Initialize the frequency band gain, noise reduction depth, and compression ratio of the noise reduction model. Using the increase of one parameter adjustment coefficient as a standard, dynamically adjust the frequency band gain, noise reduction depth, and compression ratio until the signal-to-noise ratio is greater than the standard signal-to-noise ratio, the segmented signal-to-noise ratio is greater than the standard segmented signal-to-noise ratio, and the speech spectrum distortion is less than the standard speech spectrum distortion. This will give you the noise reduction model.

9. The noise reduction and audio enhancement system for wearable hearing aids according to claim 8, characterized in that, When generating enhanced samples, H key semantics in the denoised samples are identified by automatic speech recognition technology. Enhancement contour lines are configured at the beginning and end of each of the H key semantics. F denoised samples are then imported into the enhancement engine equipped with an active enhancement algorithm for semantic enhancement to generate H enhanced samples.

10. The noise reduction and audio enhancement system for wearable hearing aids according to claim 9, characterized in that, The final audio generation method is as follows: Establish a basic queue with D blank spaces, and use a frequency band span as a standard to annotate the frequency values ​​in the D blank spaces in sequence, so that the blank spaces are converted into splicing spaces; Based on the standard that the frequency values ​​overlap or are consistent, H enhanced samples are sequentially imported into the corresponding splicing positions, and splicing positions without enhanced samples are recorded as transition positions. The average value of the transition is calculated by adding the maximum value of the frequency value of the previous splicing bit above the transition bit and the minimum value of the frequency value of the next splicing bit below the transition bit. The transition mean is reduced by two-thirds to form the spliced ​​value, and the spliced ​​value is noted on the corresponding transition bit, which causes the base queue to be converted into the final audio.

Images