A method of simulating a hearing aid algorithm

Abstract

The application discloses a hearing aid algorithm simulation method, which comprises the following steps: a MATLAB simulation step: simulation conversion is performed according to operator simulation categories, data types and operator clock cycles to obtain MATLAB simulation code; a floating-point simulation step: the MATLAB simulation code is converted into a floating-point processing version; and a fixed-point simulation step: the floating-point processing version is converted into a fixed-point processing version; the MATLAB simulation step, the floating-point simulation step and the fixed-point simulation step are repeatedly executed until the MATLAB simulation result and the floating-point version result are consistent, and the floating-point version result and the fixed-point version result remain consistent. According to the application, the operation of a hearing aid algorithm on a PC can be accurately simulated.

Description

Technical Field

[0001] This invention relates to the field of DSP algorithm simulation technology, and in particular to a method for simulating hearing aid algorithms. Background Technology

[0002] Hearing aid chips typically have multiple processor cores (usually DSP cores), and each core requires a separate project. Algorithm development involves compiling projects for different processor cores and then running the programs for each core individually, a cumbersome and time-consuming process. Furthermore, many DSP cores require licenses for compilation and simulation, and these licenses are generally quite expensive.

[0003] Therefore, developing DSP algorithms directly on the hearing aid chip has several drawbacks: firstly, it is inefficient, requiring numerous steps for creating hearing aid versions and downloading programs; secondly, it only allows real-time listening and cannot capture data for detailed analysis using tools; and thirdly, hearing aid chip DSP licenses are expensive, leading to high development costs for multiple developers. Summary of the Invention

[0004] This invention provides a method for simulating hearing aid algorithms, which accurately simulates the operation of algorithms on a hearing aid on a PC.

[0005] To achieve the above objectives, embodiments of this application provide a hearing aid algorithm simulation method, comprising:

[0006] Acquire hearing aid speech data;

[0007] The operator analog type, data type, and operator clock cycle are determined based on the operating function of the hearing aid DSP.

[0008] MATLAB simulation steps: Perform simulation conversion according to the operator simulation type, the data type, and the operator clock period to obtain MATLAB simulation code, and process the hearing aid speech data through the MATLAB simulation code to obtain MATLAB simulation results;

[0009] Floating-point simulation steps: Convert the MATLAB simulation code into a floating-point processing version, and process the hearing aid speech data using the floating-point processing version to obtain the floating-point version result;

[0010] Fixed-point simulation steps: Convert the floating-point processing version to a fixed-point processing version, and process the hearing aid speech data using the fixed-point processing version to obtain the fixed-point version result;

[0011] Repeat the MATLAB simulation steps, the floating-point simulation steps, and the fixed-point simulation steps until the MATLAB simulation results and the floating-point version results are consistent, and the floating-point version results and the fixed-point version results are consistent.

[0012] Save and play back the audio data of the fixed-point version result.

[0013] In one possible implementation, the repeated execution of the MATLAB simulation steps, the floating-point simulation steps, and the fixed-point simulation steps until the MATLAB simulation result and the floating-point version result are consistent, specifically includes:

[0014] Compare the MATLAB simulation results with the floating-point version results to see if they are consistent. If they are inconsistent, modify the parameter information of the algorithm in the floating-point processing version until the MATLAB simulation results and the floating-point version results are consistent. Then output the final floating-point processing version that is consistent with the comparison.

[0015] Compare the results of the fixed-point version and the floating-point version to see if they are consistent. If they are inconsistent, modify the parameter information of the algorithm in the fixed-point processing version until the results of the fixed-point version and the floating-point version are consistent, and output the final consistent fixed-point version.

[0016] In one possible implementation, the operator simulation types include basic mathematical operation operators, commonly used digital signal processing operators, real number vector operation operators, and complex number vector operation operators.

[0017] In one possible implementation, the data type is determined by the DSP data type used during the execution of the DSP's running function.

[0018] In one possible implementation, acquiring the hearing aid speech data specifically involves:

[0019] Capture speech data from hearing aid microphones and receivers in a real-world environment over a period of time.

[0020] In one possible implementation, acquiring the hearing aid speech data specifically involves:

[0021] It enables the hearing aid chip to maintain a communication connection with the PC, and captures real-time voice data from the hearing aid microphone and receiver in the real environment.

[0022] In one possible implementation, maintaining a communication connection between the hearing aid chip and the PC specifically includes:

[0023] This enables the hearing aid chip to maintain a communication connection with the PC via the SPI / I2S protocol.

[0024] One possible implementation also includes:

[0025] If an SPI interruption is detected between the hearing aid chip and the PC, the data acquisition process will be temporarily suspended until the SPI data transmission and reception are completed, at which point the data acquisition interruption will be resumed.

[0026] If a data acquisition interruption is detected between the hearing aid chip and the PC, the SPI clock rate is reduced, and the data acquisition interruption is processed when the data in the SPI transmit / receive buffer is not in an underflow state or is in an overflow state.

[0027] One possible implementation also includes:

[0028] Set the PC's CPU clock speed to the highest level, and handle all interrupts on the first processor core, while other applications are handled on other processor cores.

[0029] In one possible implementation, after saving and playing the audio data of the fixed-point version result, the method further includes:

[0030] Send a localized processing version to the hearing aid, and cause the hearing aid to run the localized processing version;

[0031] Real-time capture of speech data after the hearing aid runs the aforementioned point-processing version.

[0032] Compared to existing technologies, the hearing aid algorithm simulation method provided in this invention simulates some basic mathematical operation interfaces provided by the DSP core, as well as some commonly used signal processing methods in hearing aids, such as FIR / IIR filters, FFT transforms, and WOLA (Weighted Overlap Add) processing. These operators allow for both non-real-time and real-time processing of the algorithm on a PC. Performing the algorithm on a PC ensures consistency between the MATLAB simulation results and the floating-point version results, and between the floating-point version results and the fixed-point version results, enabling the hearing aid to largely reproduce the original algorithm effect when running the simulation algorithm.

[0033] During real-time processing, a communication connection is maintained between the hearing aid chip and the PC to capture real-time voice data from the hearing aid microphone and receiver in a real-world environment. On the receiver side, the SPI interrupt priority and SPI clock rate are adjusted to reduce jitter on the hearing aid side; on the PC side, the PC operating mode and interrupt application priority are adjusted to reduce PC-side jitter. Combining these PC-side and receiver-side adjustments achieves low-latency transmission and low-jitter processing, ensuring the normal operation of the real-time simulation algorithm. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a hearing aid algorithm simulation method according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of a real-time simulation of a hearing aid algorithm using a single PC, provided by an embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of an embodiment of the present invention that provides a real-time simulation of a hearing aid algorithm using two PCs. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Please see Figure 1 An embodiment of the present invention provides a hearing aid algorithm simulation method, comprising:

[0039] S10. Acquire hearing aid voice data.

[0040] S11. Based on the hearing aid DSP's operating function, confirm the operator simulation type, data type, and operator clock cycle.

[0041] S12. MATLAB simulation steps: Perform simulation conversion according to the operator simulation type, the data type, and the operator clock period to obtain MATLAB simulation code, and process the hearing aid speech data through the MATLAB simulation code to obtain MATLAB simulation results.

[0042] S13. Floating-point simulation step: Convert the MATLAB simulation code into a floating-point processing version, and process the hearing aid speech data through the floating-point processing version to obtain the floating-point version result.

[0043] S14. Fixed-point simulation step: Convert the floating-point processing version to a fixed-point processing version, and process the hearing aid speech data through the fixed-point processing version to obtain the fixed-point version result.

[0044] S15. Repeat the MATLAB simulation steps, the floating-point simulation steps, and the fixed-point simulation steps until the MATLAB simulation results and the floating-point version results are consistent, and the floating-point version results and the fixed-point version results are consistent.

[0045] S16. Save and play the audio data of the fixed-point version result.

[0046] This invention can be directly implemented on a PC. In this embodiment, three simulation versions with different expressions were obtained during algorithm simulation: a MATLAB version written in MATLAB, and floating-point and fixed-point versions written in C. The floating-point version uses floating-point representations for parameters, while the fixed-point version uses fixed-point integer representations. All three versions achieve the same functionality, but the final device should run the fixed-point version. During conversion, each step of the algorithm and its parameters must correspond one-to-one, ensuring the error remains within a controllable range. However, the three versions have different advantages. In the first stage of simulation, when making extensive modifications to the simulation algorithm, the MATLAB version is preferred because it consists of MATLAB code, allowing technical personnel to intuitively understand it. In the simulation verification stage, the floating-point version is preferred because it is easier to implement the algorithm's functionality and verify the results. In the simulation verification stage, the fixed-point version is preferred because its simulation algorithm can be directly run on the hearing aid.

[0047] For example, S15 specifically includes:

[0048] S150. Compare the MATLAB simulation results with the floating-point version results to see if they are consistent. If they are inconsistent, modify the parameter information of the algorithm in the floating-point processing version until the MATLAB simulation results and the floating-point version results are consistent. Output the final consistent floating-point processing version.

[0049] S151. Compare the fixed-point version result with the floating-point version result to see if they are consistent. If they are inconsistent, modify the parameter information of the algorithm in the fixed-point processing version until the fixed-point version result and the floating-point version result are consistent. Output the final consistent fixed-point version.

[0050] This embodiment first ensures that the floating-point processing version and the MATLAB version are consistent. If the fixed-point processing version and the floating-point processing version are inconsistent, it means that there is a problem with the results of the fixed-point processing version. The implementation of the fixed-point algorithm needs to be modified, such as redetermining the Q value of some parameters and adjusting the precision of the parameters. The fixed-point processing version is the one that will be run on the hearing aid.

[0051] For example, the types of operator simulations include basic mathematical operation operators, commonly used digital signal processing operators, real number vector operation operators, and complex number vector operation operators.

[0052] Operator simulation refers to writing functions on x86 to simulate mathematical operations on a DSP (usually using the VS compiler). The inputs and outputs of these functions are consistent with those on the DSP, and their data types are also consistent. For example, some DSPs have 24-bit integers and 48-bit long integers, which can be simulated on x86 using 32-bit integers and 64-bit long integers respectively.

[0053] The data types and operators that need to be written mainly include:

[0054] DSP data types: integer, long integer, single-precision floating-point, double-precision floating-point, etc.

[0055] Basic mathematical operations: addition, subtraction, multiplication, division, shifting, AND, OR, NOT, logarithms, exponentiation, etc.

[0056] Commonly used digital signal processing techniques include: FIR / IIR filters, FFT, WOLA, mean, variance, covariance, etc.

[0057] Real number vector operations: addition, subtraction, multiplication, division, shifting, dot product, maximum value, minimum value, etc.;

[0058] Complex vector operations: addition, subtraction, multiplication, division, shifting, dot product, maximum value, minimum value, etc.;

[0059] Operator clock cycle: Obtain the number of clock cycles for operator processing on the DSP under different compilation optimization options, which makes it convenient to count the total operator overhead on the PC and simulate the clock cycle statistics of algorithm operation on the DSP.

[0060] For example, the data type is determined by the DSP data type used when the DSP's running function is executed.

[0061] The S10 acquires hearing aid speech data in two ways: real-time and non-real-time, corresponding to two different simulation methods. Non-real-time simulation allows for testing on a PC, independent of the hearing aid device, after acquiring the input speech data. Some algorithm testing scenarios do not require real-time testing (e.g., when the output of consecutive frames does not interfere with each other, or when static simulation is ineffective, or when environmental reflections have a significant impact). Generally, real-time simulation can be performed, but it requires a shorter processing time.

[0062] As an example of non-real-time simulation, S10 is specifically as follows:

[0063] Capture speech data from hearing aid microphones and receivers in a real-world environment over a period of time.

[0064] As an example of real-time simulation, S10 is specifically as follows:

[0065] It enables the hearing aid chip to maintain a communication connection with the PC, and captures real-time voice data from the hearing aid microphone and receiver in the real environment.

[0066] Real-time simulation of hearing aids refers to transmitting data collected by the hearing aid microphone to a PC in real time, running signal processing algorithms on the PC, and finally transmitting the processed data back to the hearing aid in real time, which then plays the data through the receiver.

[0067] The microphone and receiver are placed on the hearing aid to acquire data collected in the actual hearing aid structure, i.e., speech data from the real environment. This actual sound field environment is very complex and difficult to simulate in real time.

[0068] The most critical aspects of real-time simulation are achieving extremely low latency transmission and extremely low jitter in thread processing.

[0069] For example, maintaining a communication connection between the hearing aid chip and the PC specifically includes:

[0070] This enables the hearing aid chip to maintain a communication connection with the PC via the SPI / I2S protocol.

[0071] To minimize the total latency from microphone to receiver, extremely low latency transmission must be achieved, with a transmission latency of less than 0.2ms and a total latency of less than 0.6ms.

[0072] The hearing aid chip and the PC can transmit voice data through standard transmission protocols, such as SPI and I2S (PCM). The hearing aid chip collects microphone data, and immediately transmits it to the PC via SPI / I2S after collecting one or two samples. The PC receives the data, processes it using an algorithm, and then transmits the same amount of data (one or two samples) back to the hearing aid to ensure data synchronization.

[0073] The voltages on the PC and hearing aid pins are generally different, requiring level conversion. Furthermore, they cannot share a common ground and must be isolated to prevent power frequency interference from the PC from affecting the hearing aid's microphone and receiver.

[0074] For example, in addition to maintaining a communication connection between the hearing aid chip and the PC, it also includes:

[0075] If an SPI interruption is detected between the hearing aid chip and the PC, the data acquisition process will be temporarily suspended until the SPI data transmission and reception are completed, at which point the data acquisition interruption will be resumed.

[0076] If a data acquisition interruption is detected between the hearing aid chip and the PC, the SPI clock rate is reduced, and the data acquisition interruption is processed when the data in the SPI transmit / receive buffer is not in an underflow state or is in an overflow state.

[0077] Set the PC's CPU clock speed to the highest level, and handle all interrupts on the first processor core, while other applications are handled on other processor cores.

[0078] Low jitter processing refers to the fact that the acquisition and transmission of data on the hearing aid, as well as the transmission and transmission of data and algorithm processing on the PC, all require extremely low jitter; otherwise, the total latency (the latency from the microphone to the receiver) will increase.

[0079] The vibration in hearing aids is generally small because they are simple systems without the interference of multi-threaded scheduling. They only need to handle the mutual influence between data acquisition interrupts and SPI / I2S transmit / receive interrupts.

[0080] On PCs, jitter is generally greater because their operating systems are complex multitasking systems. To achieve extremely low jitter, this embodiment uses a headless Linux system and sets the processor to high-performance mode. These two operations can greatly reduce data transmission and reception jitter and ensure the real-time performance of the entire processing flow.

[0081] It should be noted that other operating systems can be used on the PC, but the PC must have at least a kernel and commonly used transmission protocol drivers. When emulating the PC, the CPU frequency needs to be set to the highest level, and all interrupts should be placed on the first processor core, while other applications run on other processor cores.

[0082] After achieving low-latency transmission and low-jitter processing, the algorithm processing program was written on a PC for algorithm simulation. The input data of the algorithm is the real-time received microphone data (AFC (Adaptive Acoustic Feedback Cancellation) uses all the processed data). The algorithm processing can be a floating-point version or a fixed-point version based on analog operators. The output data of the algorithm is transmitted to the hearing aid in real time, and the developers can wear the hearing aid to hear the processed effect in real time.

[0083] There are two main methods for real-time simulation of hearing aids, see [link / reference] Figure 2 One approach is to directly compile and debug the algorithm on a PC connected to the hearing aid. See [link / reference]. Figure 3 Another approach is to run the program on PC1, which is connected to the hearing aid and another PC2. The program on PC1 can be remotely debugged from PC2. PC1 is a dedicated PC, while PC2 is the developer's own office PC. Developing on one's own office PC is more convenient and less constrained by space.

[0084] It's important to note that to achieve low power consumption and high performance, many hearing aid chips integrate multiple DSP processor cores. This allows multiple algorithms to be processed in parallel on various processor cores at a relatively low system clock speed. With powerful PC processors, some simple algorithms can be processed serially, as long as the total processing latency is lower than the latency on the hearing aid itself. However, to more realistically simulate the parallel operation of the various DSP cores on a hearing aid, multi-threaded processing programs need to be developed on the PC. x86 processors generally support multi-core, multi-threading, such as 4 cores with 4 threads or 4 cores with 8 threads. One thread can simulate the processing of one DSP core, and multiple threads can simulate multiple DSP cores respectively.

[0085] The PC can save the data transmitted from the hearing aid and the data processed by the PC algorithm. However, data saving needs to be done in a separate thread to prevent file read and write operations from affecting the operation of the algorithm or data transmission and reception.

[0086] For example, after saving and playing the audio data of the fixed-point version result, the method further includes:

[0087] Send a localized processing version to the hearing aid, and cause the hearing aid to run the localized processing version;

[0088] Real-time capture of speech data after the hearing aid runs the aforementioned point-processing version.

[0089] Algorithm developers can wear hearing aids during real-time simulations to hear the effects of the algorithm's processing. This real-time simulation platform also offers another method for testing the algorithm's processing effect on the hearing aid chip in real time: reading a speech file on a PC, transmitting it to the hearing aid in real-time, running the algorithm on the hearing aid, and then transmitting the data back to the PC in real-time after execution. The PC receives the data, saves it as a file, and then compares the processing effect on the hearing aid with the effect processed on the PC.

[0090] This method can test certain algorithms that do not require feedback signals. In this case, the hearing aid acts like a loudspeaker, and the sound it plays is picked up by the microphone. If left unprocessed, feedback will occur. All feedback algorithms require a feedback signal as a reference (the feedback signal is the signal to be played), and then the playback signal picked up by the microphone is removed.

[0091] Compared to existing technologies, the hearing aid algorithm simulation method provided in this invention simulates some basic mathematical operation interfaces provided by the DSP core, as well as some commonly used signal processing methods in hearing aids, such as FIR / IIR filters, FFT transforms, and WOLA (Weighted Overlap Add) processing. These operators allow for both non-real-time and real-time processing of the algorithm on a PC. Performing the algorithm on a PC ensures consistency between the MATLAB simulation results and the floating-point version results, and between the floating-point version results and the fixed-point version results, enabling the hearing aid to largely reproduce the original algorithm effect when running the simulation algorithm.

[0092] During real-time processing, a communication connection is maintained between the hearing aid chip and the PC to capture real-time voice data from the hearing aid microphone and receiver in a real-world environment. On the receiver side, the SPI interrupt priority and SPI clock rate are adjusted to reduce jitter on the hearing aid side; on the PC side, the PC operating mode and interrupt application priority are adjusted to reduce PC-side jitter. Combining these PC-side and receiver-side adjustments achieves low-latency transmission and low-jitter processing, ensuring the normal operation of the real-time simulation algorithm.

[0093] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A hearing aid algorithm simulation method, characterized in that, include: Acquire hearing aid speech data; The operator analog type, data type, and operator clock cycle are determined based on the operating function of the hearing aid DSP. MATLAB simulation steps: Perform simulation conversion according to the operator simulation type, the data type, and the operator clock period to obtain MATLAB simulation code, and process the hearing aid speech data through the MATLAB simulation code to obtain MATLAB simulation results; Floating-point simulation steps: Convert the MATLAB simulation code into a floating-point processing version, and process the hearing aid speech data using the floating-point processing version to obtain the floating-point version result; Fixed-point simulation steps: Convert the floating-point processing version to a fixed-point processing version, and process the hearing aid speech data using the fixed-point processing version to obtain the fixed-point version result; Repeat the MATLAB simulation steps, the floating-point simulation steps, and the fixed-point simulation steps until the MATLAB simulation results and the floating-point version results are consistent, and the floating-point version results and the fixed-point version results are consistent. Specifically, the repeated execution of the MATLAB simulation step, the floating-point simulation step, and the fixed-point simulation step until the MATLAB simulation result and the floating-point version result are consistent includes: comparing whether the MATLAB simulation result and the floating-point version result are consistent; if they are inconsistent, modifying the parameter information of the algorithm in the floating-point processing version until the MATLAB simulation result and the floating-point version result are consistent, and outputting the final consistent floating-point processing version; comparing whether the fixed-point version result and the floating-point version result are consistent; if they are inconsistent, modifying the parameter information of the algorithm in the fixed-point processing version until the fixed-point version result and the floating-point version result are consistent, and outputting the final consistent fixed-point version. Save and play back the audio data of the fixed-point version result.

2. The hearing aid algorithm simulation method as described in claim 1, characterized in that, The types of operators simulated include basic mathematical operation operators, commonly used digital signal processing operators, real number vector operation operators, and complex number vector operation operators.

3. The hearing aid algorithm simulation method as described in claim 1, characterized in that, The data type is determined by the DSP data type used during the execution of the DSP's running functions.

4. The hearing aid algorithm simulation method as described in claim 1, characterized in that, The acquisition of hearing aid speech data specifically includes: Capture speech data from hearing aid microphones and receivers in a real-world environment over a period of time.

5. The hearing aid algorithm simulation method as described in claim 1, characterized in that, The acquisition of hearing aid speech data specifically includes: It enables the hearing aid chip to maintain a communication connection with the PC, and captures real-time voice data from the hearing aid microphone and receiver in the real environment.

6. The hearing aid algorithm simulation method as described in claim 5, characterized in that, Maintaining a communication connection between the hearing aid chip and the PC specifically includes: This enables the hearing aid chip to maintain a communication connection with the PC via the SPI / I2S protocol.

7. The hearing aid algorithm simulation method as described in claim 6, characterized in that, Also includes: If an SPI interruption is detected between the hearing aid chip and the PC, the data acquisition process will be temporarily suspended until the SPI data transmission and reception are completed, at which point the data acquisition interruption will be resumed. If a data acquisition interruption is detected between the hearing aid chip and the PC, the SPI clock rate is reduced, and the data acquisition interruption is processed when the data in the SPI transmit / receive buffer is not in an underflow state or is in an overflow state.

8. The hearing aid algorithm simulation method as described in claim 5, characterized in that, Also includes: Set the PC's CPU clock speed to the highest level, and handle all interrupts on the first processor core, while other applications are handled on other processor cores.

9. The hearing aid algorithm simulation method as described in claim 5, characterized in that, After saving and playing the audio data of the fixed-point version result, the method further includes: Send a localized processing version to the hearing aid, and cause the hearing aid to run the localized processing version; Real-time capture of speech data after the hearing aid runs the aforementioned point-processing version.

Images