A sound coordination method and system based on tinnitus conditions

Abstract

The present invention provides a sound coordination method and system based on tinnitus conditions, which relates to the technical field of tinnitus treatment. The method establishes a tinnitus feature vector of a patient through broadband hearing tests and three-dimensional sound field modeling, generates a dual-mode treatment audio based on the tinnitus feature vector, and the dual-mode treatment audio includes a cancellation sound wave with opposite phases and a masking sound wave with frequency band matching; establishes a dynamic mapping relationship between game interaction events and audio parameters of the dual-mode treatment audio; and dynamically adjusts treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data. The present invention accurately establishes a tinnitus feature vector of a patient by comprehensively applying broadband hearing tests, three-dimensional sound field modeling, and binaural sound field matching, and dynamically adjusts treatment audio according to the personalized characteristics of tinnitus. This personalized sound coordination method can effectively relieve tinnitus symptoms and improve the treatment compliance of patients according to the tinnitus symptoms of each patient.

Description

Technical Field

[0001] The present invention relates to the technical field of tinnitus treatment and can automatically perform inspection tasks at the engineering site. Specifically, it relates to a sound coordination method and system based on the tinnitus situation. Background Art

[0002] Tinnitus is a common auditory disease, manifested as continuous ringing or buzzing sounds in the ears. Its severity varies from person to person and often affects the patient's quality of life, sleep quality, and mental health. Currently, there are various treatment methods for tinnitus, among which the most common ones are drug treatment, sound therapy, and psychotherapy. However, these methods often have certain limitations. For example, drugs may have side effects, the compliance of ordinary sound therapy for patients is relatively low, and the effect of psychotherapy is unstable, etc.

[0003] Sound therapy (including white noise, customized music, etc.), as a non-invasive treatment method, has obtained certain recognition in relieving tinnitus. However, traditional sound therapy has not fully solved the problems of insufficient personalization and patient compliance. The current sound therapy is generally a unified standard sound and fails to make personalized adjustments according to the tinnitus type, frequency, and intensity of different patients. Conventional sound therapy requires patients to wear headphones or contact sound devices for a long time, which easily leads to low patient compliance and thus affects the treatment effect.

[0004] Therefore, how to improve the treatment effect and increase the patient's compliance through sound coordination according to the specific tinnitus situation of the patient has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to propose a sound coordination method and system based on the tinnitus situation. By adjusting parameters such as the frequency and loudness of the treatment audio in real time, it can better match the tinnitus situation of the patient, thereby effectively relieving the tinnitus symptoms and improving the patient's treatment compliance.

[0006] To achieve the above purpose, the present invention provides the following technical solutions:

[0007] Based on the above purpose, in the first aspect, the present invention provides a sound coordination method based on the tinnitus situation, including the following steps:

[0008] Establish a tinnitus feature vector of the patient through broadband hearing test and three-dimensional sound field modeling, wherein the tinnitus feature vector includes main frequency band energy ratio, time domain modulation depth, and spatial perception dispersion parameters;

[0009] Generate a dual-mode treatment audio based on the tinnitus feature vector, and the dual-mode treatment audio includes a cancellation sound wave with opposite phases and a masking sound wave with frequency band matching;

[0010] Establish a dynamic mapping relationship between game interaction events and the audio parameters of the dual-mode therapeutic audio;

[0011] Dynamically adjust the treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data, where the real-time physiological feedback data includes heart rate variability and skin conductance index.

[0012] As a further solution of the present invention, before establishing the tinnitus feature vector of the patient through broadband hearing test and three-dimensional sound field modeling, a three-dimensional tinnitus sound image model is constructed by binaural sound field matching, the auditory cortex response data is obtained through a 128-channel EEG acquisition module, and the frequency-specific audiometric threshold curve in the range of 0.125 - 16 kHz is calculated.

[0013] As a further solution of the present invention, generating dual-mode therapeutic audio based on the tinnitus feature vector includes the following steps:

[0014] Mix white noise and pink noise through a base noise generation module, and the mixing ratio is dynamically adjusted according to the main frequency band energy ratio;

[0015] Insert dynamic harmonic components in the target frequency band using the FFT analysis result;

[0016] Perform spatial sound field modulation using the HRTF personalized adaptation algorithm.

[0017] As a further solution of the present invention, when performing spatial sound field modulation using the HRTF personalized adaptation algorithm, an improved LMS adaptive filter is used, and the iteration formula is:

[0018] W(n + 1)=W(n)+μ·e(n)·X(n) / (||X(n)||²+δ)

[0019] Where μ = 0.2 is the step size parameter, δ = 0.01 is the stability factor, and the number of convergence iterations does not exceed 50 times.

[0020] As a further solution of the present invention, the mapping relationship includes operation response delay triggering phase inversion and game progress threshold controlling bandwidth expansion.

[0021] As a further solution of the present invention, the established dynamic mapping relationship satisfies:

[0022] The correlation function between the visual stimulation frequency Fv and the sound stimulation bandwidth Bw is Bw = K·log(1 + Fv), where K is the patient tolerance coefficient;

[0023] The negative correlation relationship between the operation accuracy rate Ac and the treatment intensity It is It = Imax·(1 - 0.7Ac), where Imax is the maximum treatment intensity.

[0024] As a further aspect of the present invention, when dynamically adjusting the treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data, when the standard deviation of heart rate variability is lower than 15% of the baseline value, a game difficulty downgrading mechanism is triggered; when the rising slope of skin conductance exceeds 0.05 μS / s, an environmental sound fusion mode is activated.

[0025] In a second aspect, the present invention also provides a sound coordination system based on tinnitus conditions, comprising the following components:

[0026] A feature analysis terminal, configured with binaural in-ear audiometry probes and a 128-channel EEG acquisition module, for collecting the patient's auditory data and electroencephalogram activity data;

[0027] A dynamic synthesis server, built-in with a real-time convolutional processor accelerated by GPU, supporting audio processing at a sampling rate of 192 kHz, and configured with:

[0028] A parameterized sound engine, having a configurable base noise mixer, a dynamic harmonic inserter, and a spatial sound field modulator;

[0029] A cross-modal coordination control module, realizing triangular closed-loop control of game events, audio parameters, and physiological indicators;

[0030] An interactive treatment terminal, integrating a six-degree-of-freedom haptic feedback handle and a bone conduction audio output device, for real-time interaction with the patient to provide a personalized treatment experience;

[0031] A physiological monitoring array, including a wrist-type PPG sensor and an auricle temperature detection module, for real-time monitoring of the patient's physiological feedback data.

[0032] As a further aspect of the present invention, the dynamic synthesis server further includes:

[0033] A personalized sound generation module, supporting dynamic adjustment, capable of generating adapted dual-mode treatment audio according to the personalized characteristics of the patient's tinnitus;

[0034] A real-time audio processing module, for processing multi-channel spatial sound effects and frequency response, enhancing the masking effect of the treatment audio.

[0035] As a further aspect of the present invention, the interactive treatment terminal is configured with:

[0036] A three-dimensional soundscape game scene built based on the Unreal Engine, including a frequency matching level module, for dynamically adjusting the audio elements of the game scene according to tinnitus characteristics;

[0037] Supporting 256 groups of personalized HRTF databases, for dynamic coupling of sound image positions and visual elements.

[0038] As a further aspect of the present invention, the system further includes:

[0039] A deep belief network (DBN) model, which is composed of three stacked restricted Boltzmann machines (RBMs). The input layer receives 12-dimensional tinnitus features and 5-dimensional physiological baseline data, and the output layer generates a treatment parameter combination including 6 adjustment factors.

[0040] As a further aspect of the present invention, the physiological monitoring array further includes:

[0041] An eye movement tracking device, which is used to monitor the patient's visual focus and attention changes in real time, and provide additional feedback data for game difficulty adjustment and dynamic adjustment of treatment audio.

[0042] In another aspect of the present invention, there is also provided a computer device, including a memory and a processor. A computer program is stored in the memory, and when the computer program is executed by the processor, it executes any one of the above-mentioned sound coordination methods based on tinnitus conditions according to the present invention.

[0043] In still another aspect of the present invention, there is also provided a computer-readable storage medium storing computer program instructions, and when the computer program instructions are executed, they implement any one of the above-mentioned sound coordination methods based on tinnitus conditions according to the present invention.

[0044] Compared with the prior art, a sound coordination method and system based on tinnitus conditions proposed by the present invention have the following beneficial effects:

[0045] 1. By comprehensively applying technologies such as broadband hearing tests, three-dimensional sound field modeling, and binaural sound field matching, the present invention accurately establishes the tinnitus feature vector of the patient and dynamically adjusts the treatment audio according to the personalized characteristics of tinnitus. This personalized sound coordination method can provide targeted treatment according to the tinnitus symptoms of each patient, significantly improving the treatment effect.

[0046] 2. Through base noise mixing, dynamic harmonic insertion, and HRTF personalized adaptation algorithms, the present invention can generate a dual-mode treatment audio with a good masking effect. This audio includes cancellation sound waves and masking sound waves, which can effectively reduce the annoyance of tinnitus to the patient. At the same time, the treatment effect is enhanced through spatial sound field modulation, meeting the different needs of different patients for treatment audio.

[0047] 3. Through the dynamic mapping of game interaction events and therapeutic audio parameters, combined with real-time physiological feedback data (such as heart rate variability, skin conductance, eye movement tracking, etc.), the present invention can adjust the parameters of treatment in real time. For example, when the physiological state of the patient is poor, the system can trigger a mechanism according to the change of the standard deviation of heart rate variability or skin conductance, automatically adjust the game difficulty or activate the environmental sound fusion mode, so as to avoid excessive stimulation and ensure that the treatment process is more comfortable and effective.

[0048] 4. The sound coordination system of the present invention includes various technical components such as a dynamic synthesis server, a personalized sound generation module, and a real-time audio processing module. These modules can dynamically generate suitable dual-mode therapeutic audio according to the tinnitus characteristics and physiological feedback of the patient, while optimizing the spatial sound effect and enhancing the sound masking effect. Through the cross-modal coordination control module, the system can achieve triangular closed-loop control of game events, audio parameters, and physiological feedback, enabling the patient to not only receive sound treatment during the treatment process, but also obtain positive psychological feedback through interactive games, improving the overall treatment experience.

[0049] 5. The present invention adopts a deep belief network (DBN) model, inputs tinnitus feature data and physiological baseline data into the model, and outputs a combination of treatment parameters. Through machine learning technology, the system can learn from the historical data of the patient, further optimize the treatment plan, gradually improve the treatment effect, and can continuously adjust the treatment strategy according to the feedback of the patient.

[0050] 6. The interactive treatment terminal in the system of the present invention includes a six-degree-of-freedom haptic feedback handle and a bone conduction audio output device, providing a multi-sensory interaction experience for the patient not only through audio treatment, but also through haptic feedback. In addition, the three-dimensional soundscape game scene built based on the Unreal Engine can adjust the audio elements in the game in real time, and create an immersive treatment environment for the patient by dynamically coupling the visual and audio image positions, helping to distract the patient's attention from tinnitus, thereby improving the enthusiasm and effect of treatment.

[0051] 7. By integrating a wrist-mounted PPG sensor, an auricle temperature detection module, and an eye movement tracking device, the system of the present invention can monitor the physiological feedback data of the patient in real time. These data can provide the psychological and physiological state of the patient, providing a basis for real-time adjustment during the treatment process and ensuring that the treatment is always in the best comfortable state of the patient.

[0052] In summary, the present invention not only provides a personalized and intelligent sound coordination method and system based on tinnitus conditions, but also significantly improves the treatment effect and the patient's treatment experience through technical means such as multi-modal interaction, deep learning optimization, and real-time physiological feedback regulation, ensuring that tinnitus patients obtain more accurate and comfortable treatment during the treatment process. These innovative features give the present invention significant technical advantages and broad application prospects in the field of tinnitus treatment.

[0053] These aspects or other aspects of the present application will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the related art, the following will briefly introduce the accompanying drawings required for the description of the exemplary embodiments or the related art. The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification, together with the embodiments of the present invention, to explain the present invention, and do not constitute a limitation to the present invention. In the accompanying drawings:

[0055] Figure 1 It is a flowchart of a sound coordination method based on tinnitus conditions according to an embodiment of the present invention.

[0056] Figure 2 It is a flowchart of generating a dual-mode treatment audio in a sound coordination method based on tinnitus conditions according to an embodiment of the present invention.

[0057] Figure 3 It is a structural block diagram of a sound coordination system based on tinnitus conditions according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] Next, in combination with the accompanying drawings and the specific embodiments, the present application will be further described. It should be noted that, on the premise of no conflict, the following described embodiments or technical features can be arbitrarily combined to form new embodiments.

[0059] To make the purpose, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the specific embodiments and the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] It should be noted that in the embodiments of the present invention, all expressions using "first" and "second" are for distinguishing two non-identical entities or non-identical parameters with the same name. It can be seen that "first" and "second" are only for the convenience of expression and should not be construed as a limitation on the embodiments of the present invention. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units inherently includes other steps or units.

[0061] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0062] The flowchart shown in the accompanying drawings is only an example illustration, and does not necessarily include all contents and operations / steps, nor does it necessarily execute in the described order. For example, some operations / steps can also be decomposed, combined or partially merged. Therefore, the actual execution order may change according to the actual situation.

[0063] Next, some embodiments of the present application will be described in detail with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0064] See Figure 1 As shown, an embodiment of the present invention provides a sound coordination method based on tinnitus conditions. The method includes the following steps:

[0065] Step S10: Establish a tinnitus feature vector of the patient through broadband hearing test and three-dimensional sound field modeling, where the tinnitus feature vector includes the main frequency band energy ratio, time domain modulation depth, and spatial perception dispersion parameter.

[0066] In this step, before establishing the tinnitus feature vector of the patient through broadband hearing test and three-dimensional sound field modeling, a three-dimensional tinnitus sound image model is constructed by binaural sound field matching, the auditory cortex response data is acquired through a 128-channel EEG acquisition module, and the frequency-specific audiogram in the range of 0.125 - 16 kHz is calculated.

[0067] When establishing the tinnitus feature vector of the patient through broadband hearing test and three-dimensional sound field modeling, first perform binaural sound field matching. By conducting a binaural hearing assessment for the patient on a standard hearing test device, the binaural sound field matching technology is used to measure the auditory responses in each frequency band. This test helps construct the patient's binaural auditory model by separately inputting sound sources with different frequencies into the patient's left and right ears and measuring the response differences of the ears to the sound sources.

[0068] Then, a three-dimensional tinnitus sound image model is constructed. Using three-dimensional sound field modeling technology, the test results are combined with the tinnitus characteristics of the patient, and a complex sound field modeling algorithm is used to generate a tinnitus sound image model. This tinnitus sound image model takes into account the spatial perception attributes of tinnitus, namely the sound source position of tinnitus, the sound field propagation characteristics, etc. Then, EEG acquisition and frequency-specific auditory threshold calculation are carried out. Using a 128-channel EEG (electroencephalogram) acquisition module, the auditory cortex response data of the patient is recorded in real time. The EEG data can analyze the auditory cortex response of the patient at different frequencies, so as to calculate the frequency-specific auditory threshold curve (in the range of 0.125 - 16 kHz). This curve reflects the frequency characteristics and sensitivity of tinnitus, and further precisely locates the tinnitus frequency band and its interference degree.

[0069] Finally, tinnitus feature vector extraction is carried out. According to the aforementioned data, it is converted into a tinnitus feature vector, which includes the following parameters:

[0070] Main frequency band energy ratio: The relative proportion of energy between different frequency bands, reflecting the frequency components of tinnitus.

[0071] Time-domain modulation depth: The amplitude of the time-domain fluctuation of the tinnitus sound, indicating the dynamic changes of tinnitus.

[0072] Spatial perception dispersion: The spatial distribution characteristics of the tinnitus sound source, helping to determine the spatial perception differences of the tinnitus sound source.

[0073] Step S20: Generate a dual-mode treatment audio based on the tinnitus feature vector. The dual-mode treatment audio includes a cancellation sound wave with opposite phases and a masking sound wave with frequency band matching.

[0074] In this step, as shown in Figure 2 generating a dual-mode treatment audio based on the tinnitus feature vector includes the following steps:

[0075] Step S201: Mix white noise and pink noise through a base noise generation module. The mixing ratio is dynamically adjusted according to the main frequency band energy ratio.

[0076] When generating and mixing the base noise in this step, first, the base noise generation module generates a mixture of white noise and pink noise. The noise mixing ratio is dynamically adjusted based on the main frequency band energy ratio in the tinnitus feature vector. For example, if tinnitus mainly concentrates in the frequency band of 2 - 4 kHz, the system will increase the noise component in this frequency band to enhance the treatment effect.

[0077] During the generation process, the ratio of white noise and pink noise will be dynamically adjusted according to spectral analysis to ensure that the noise components in different frequency bands match the tinnitus characteristics of the patient.

[0078] Step S202: Insert dynamic harmonic components in the target frequency band using the FFT analysis results.

[0079] When performing dynamic harmonic insertion in this step, the frequency-domain analysis of the generated noise signal is carried out by FFT (Fast Fourier Transform), and dynamic harmonic components are inserted in a specific frequency band. For example, if the tinnitus frequency is around 3 kHz, the system will insert harmonics in this frequency band to enhance the audio coverage in this band and help reduce the interference of the tinnitus frequency. This method of inserting harmonics ensures a high degree of matching between the audio frequency band and the tinnitus characteristics, improving the treatment effect.

[0080] Step S203: Perform spatial sound field modulation using the HRTF personalized adaptation algorithm.

[0081] Among them, when performing spatial sound field modulation using the HRTF personalized adaptation algorithm, an improved LMS adaptive filter is used, and the iterative formula is:

[0082] W(n + 1)=W(n)+μ·e(n)·X(n) / (||X(n)||²+δ)

[0083] In the formula, W(n) is the weight vector of the filter, e(n) is the error signal, X(n) is the input signal, μ = 0.2 is the step size parameter, δ = 0.01 is the stability factor, and the number of convergence iterations does not exceed 50 times.

[0084] In this step, the HRTF (Head-Related Transfer Function) personalized adaptation algorithm is used to perform spatial sound field modulation on the audio. The purpose of this step is to simulate the natural auditory spatial perception so that the patient can perceive the directionality and spatial sense of the treatment audio. Among them, the improved LMS adaptive filter adjusts the spatial perception effect of the audio signal through iterative calculation, enabling it to match the patient's tinnitus characteristics and ear shape, enhancing the effect of audio treatment, and can realize the adjustment of the spatial sense of the audio signal to match the patient's auditory space, enhancing the comfort and masking effect of the audio.

[0085] Through base noise mixing, dynamic harmonic insertion, and the HRTF personalized adaptation algorithm, the present invention can generate a dual-mode treatment audio with a good masking effect. This audio contains cancellation sound waves and masking sound waves, which can effectively reduce the trouble of tinnitus for patients. At the same time, the treatment effect is enhanced through spatial sound field modulation, meeting the different needs of different patients for treatment audio.

[0086] Step S30: Establish a dynamic mapping relationship between the game interaction events and the audio parameters of the dual-mode treatment audio.

[0087] In this step, the mapping relationship includes operation response delay triggering phase inversion and game progress threshold controlling bandwidth expansion. The established dynamic mapping relationship satisfies:

[0088] The correlation function between the visual stimulus frequency Fv and the acoustic stimulus bandwidth Bw is Bw = K·log(1 + Fv), where K is the patient tolerance coefficient (which can be adjusted individually according to the patient's tolerance), Bw is the bandwidth of the therapeutic audio, and Fv is the frequency of the visual stimulus (such as the update frequency of the game screen, the task trigger frequency, etc.);

[0089] The negative correlation between the operation accuracy rate Ac and the treatment intensity It is It = Imax·(1 - 0.7Ac), where Imax is the maximum treatment intensity, Ac is the operation accuracy rate, and It is the current treatment intensity.

[0090] Among them, when the operation response time delay triggers phase inversion, during the treatment process, the patient controls the audio output of the treatment process through an interactive game, for example, controls certain elements in the game (such as virtual object movement, task completion, etc.) through a touch screen, buttons or other means. Whenever the player performs an operation, the system will detect the input operation and trigger the phase inversion of the audio according to the operation response time delay (that is, the time delay between the player's completion of the operation and the audio reaction).

[0091] Exemplarily, the player needs to hit the "tinnitus sound source" in a specific frequency band in a virtual environment for treatment. When the player hits the target, the phase of the treatment audio will immediately invert to counteract the specific frequency of tinnitus. The design of this operation response time delay helps to ensure the synchronization of the treatment audio with the player's actions, thereby enhancing the interactivity of the treatment.

[0092] When the game progress threshold controls the bandwidth expansion, during the game process, as the patient completes the treatment tasks, the system will monitor the game progress and dynamically adjust the bandwidth of the treatment audio according to the progress. Specifically, the frequency band (or frequency range) of the treatment audio will expand or contract as the game progresses to adapt to the tinnitus characteristics of the patient at different stages.

[0093] For example: At the initial stage of the game, the patient may face relatively mild tinnitus symptoms, and the system will set a relatively narrow treatment bandwidth; when the game progresses to a certain extent and the patient's tinnitus symptoms are alleviated, the system will increase the bandwidth to cover a wider range of tinnitus frequencies, thereby providing a stronger treatment effect.

[0094] When there is a negative correlation between the operation accuracy rate and the treatment intensity, the accuracy of the game interaction (such as the operation accuracy of the player in the game) directly affects the intensity of the treatment audio. The operation accuracy rate (Ac) and the treatment intensity (It) are negatively correlated, which means that when the operation accuracy is high, the system will reduce the intensity of the treatment audio. For example, if the player operates very precisely in the game, the system will automatically reduce the intensity of the treatment audio to avoid overstimulation; conversely, if the operation accuracy rate is low, the system will increase the treatment intensity to enhance the treatment effect.

[0095] Step S40: Dynamically adjust the treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data, where the real-time physiological feedback data includes heart rate variability and skin conductance metrics.

[0096] In this step, when dynamically adjusting the treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data, when the standard deviation of heart rate variability is lower than 15% of the baseline value, a game difficulty downgrading mechanism is triggered; when the rising slope of skin conductance exceeds 0.05 μS / s, an environmental sound fusion mode is activated.

[0097] When obtaining the real-time physiological feedback data, the real-time physiological feedback data of the patient is continuously monitored, including heart rate variability (HRV) and skin conductance (EDA) metrics. These physiological feedbacks reflect the physiological state and psychological reactions of the patient during the treatment process and can provide a strong basis for the adjustment of treatment parameters. Exemplarily, physiological data such as the patient's heart rate and skin conductance are collected through wearable devices or physiological monitoring instruments and fed back to the system in real time.

[0098] When the standard deviation of heart rate variability is lower than the baseline value, a game difficulty downgrading mechanism is triggered. Among them, heart rate variability (HRV) is a key indicator for measuring heart health and the function of the autonomic nervous system. Lower HRV values are usually associated with negative emotions such as stress and anxiety. During the treatment process, when the standard deviation of heart rate variability is lower than 15% of the patient's baseline value, the system will identify that the patient is in a state of excessive tension or anxiety, thereby triggering the game difficulty downgrading mechanism. For example, if the standard deviation of heart rate variability (HRV) is low, the system will automatically adjust the game difficulty, reduce the complexity of the task or reduce the intensity of the treatment audio to avoid excessive psychological pressure on the patient, thereby ensuring the comfort and effectiveness of the treatment process.

[0099] When the rising slope of skin conductance exceeds the threshold, an environmental sound fusion mode is activated. Skin conductance reflects the emotional changes and sympathetic nerve activities of the human body. When the rising slope of skin conductance exceeds a certain threshold (such as 0.05 μS / s), it means that the patient may be in a state of tension, anxiety, etc. At this time, the system will activate the environmental sound fusion mode to help the patient relax.

[0100] Exemplarily, when the skin conductance rises rapidly (usually indicating an increase in anxiety or stress), the system will automatically introduce environmental sounds into the treatment audio (such as natural environmental sounds, white noise, etc.), and through gentle environmental sound fusion, help the patient relieve stress and enhance the treatment experience.

[0101] When dynamically adjusting the treatment parameters in real time, according to the dynamic mapping relationship and real-time physiological feedback data, the system will adjust the following treatment parameters:

[0102] Therapy intensity (It): Adjust the intensity of the therapeutic audio according to the operation accuracy rate and heart rate variability.

[0103] Bandwidth (Bw): Adjust the bandwidth of the therapeutic audio according to the relationship between the game progress and the visual stimulation frequency.

[0104] Spatial perception adjustment of the therapeutic audio: Adjust the spatial perception effect of the audio according to the change of skin conductance, enhance or weaken the stereoscopic effect of the sound to promote relaxation.

[0105] Through the dynamic mapping of game interaction events and therapeutic audio parameters, combined with real-time physiological feedback data (such as heart rate variability, skin conductance, eye movement tracking, etc.), the present invention can adjust the parameters of the therapy in real time. For example, when the physiological state of the patient is poor, the system can trigger a mechanism according to the change of the standard deviation of heart rate variability or skin conductance, automatically adjust the game difficulty or start the environmental sound fusion mode, so as to avoid excessive stimulation and ensure that the treatment process is more comfortable and effective.

[0106] By comprehensively applying technologies such as broadband hearing test, three-dimensional sound field modeling, and binaural sound field matching, the present invention accurately establishes the tinnitus feature vector of the patient and dynamically adjusts the therapeutic audio according to the personalized characteristics of tinnitus. This personalized sound coordination method can provide targeted treatment according to the tinnitus symptoms of each patient, significantly improving the treatment effect.

[0107] It should be noted that the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0108] It should be understood that although the above is described in a certain order, these steps are not necessarily executed in the above order. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, a part of the steps in this embodiment may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0109] In the second aspect of the embodiment of the present invention, see Figure 3 As shown, the present invention also provides a sound coordination system based on the tinnitus situation, including the following components:

[0110] The feature analysis terminal 101 is configured with binaural in-ear audiometry probes and a 128-channel EEG acquisition module for collecting the patient's auditory data and electroencephalogram activity data;

[0111] The dynamic synthesis server 201 is built-in with a GPU-accelerated real-time convolution processor, supports audio processing at a sampling rate of 192 kHz, and is configured with:

[0112] A parametric sound engine with a configurable base noise mixer, dynamic harmonic inserter, and spatial sound field modulator;

[0113] A cross-modal coordination control module to achieve triangular closed-loop control of game events, audio parameters, and physiological indicators;

[0114] The interactive treatment terminal 301 integrates a six-degree-of-freedom haptic feedback handle and a bone conduction audio output device for real-time interaction with the patient to provide a personalized treatment experience;

[0115] The physiological monitoring array 401 includes a wrist-mounted PPG sensor and an auricle temperature detection module for real-time monitoring of the patient's physiological feedback data.

[0116] Among them, the dynamic synthesis server 201 further includes:

[0117] A personalized sound generation module that supports dynamic adjustment and can generate adapted dual-mode treatment audio according to the personalized characteristics of the patient's tinnitus;

[0118] A real-time audio processing module for processing multi-channel spatial sound effects and frequency response to enhance the masking effect of the treatment audio.

[0119] In this embodiment, the interactive treatment terminal 301 is configured with:

[0120] A three-dimensional soundscape game scene built based on the Unreal Engine, including a frequency matching level module for dynamically adjusting the audio elements of the game scene according to the tinnitus characteristics;

[0121] Supporting 256 groups of personalized HRTF databases for dynamic coupling of sound image positions and visual elements.

[0122] In this embodiment, the physiological monitoring array 401 further includes:

[0123] An eye movement tracking device for real-time monitoring of the patient's visual focus and attention changes to provide additional feedback data for game difficulty adjustment and dynamic adjustment of treatment audio.

[0124] The system further includes:

[0125] The Deep Belief Network (DBN) model is composed of three stacked Restricted Boltzmann Machines (RBMs). The input layer receives 12-dimensional tinnitus features and 5-dimensional physiological baseline data, and the output layer generates a combination of treatment parameters containing 6 regulatory factors.

[0126] The present invention adopts a Deep Belief Network (DBN) model, inputs tinnitus feature data and physiological baseline data into the model, and outputs a combination of treatment parameters. Through machine learning techniques, the system can learn from the patient's historical data, further optimize the treatment plan, gradually improve the treatment effect, and can continuously adjust the treatment strategy according to the patient's feedback.

[0127] The sound coordination system of the present invention includes various technical components such as a dynamic synthesis server, a personalized sound generation module, and a real-time audio processing module. These modules can dynamically generate suitable dual-mode treatment audio according to the patient's tinnitus characteristics and physiological feedback, while optimizing the spatial sound effect and enhancing the sound masking effect. Through the cross-modal coordination control module, the system can achieve triangular closed-loop control of game events, audio parameters, and physiological feedback, enabling the patient to not only receive sound treatment during the treatment process, but also obtain positive psychological feedback through interactive games, improving the overall treatment experience.

[0128] The interactive treatment terminal in the system of the present invention includes a six-degree-of-freedom haptic feedback handle and a bone conduction audio output device, providing a multi-sensory interaction experience for the patient not only through audio treatment but also through haptic feedback. In addition, the three-dimensional soundscape game scene built based on the Unreal Engine can adjust the audio elements in the game in real time, and create an immersive treatment environment for the patient by dynamically coupling the visual and sound image positions, helping to distract the patient's attention from tinnitus, thereby improving the enthusiasm and effect of treatment.

[0129] The sound coordination system based on the tinnitus condition of the present invention can real-time monitor the patient's physiological feedback data by integrating a wrist PPG sensor, an auricle temperature detection module, and an eye movement tracking device. These data can provide the patient's psychological and physiological states, providing a basis for real-time adjustment during the treatment process to ensure that the treatment is always in the patient's best comfort state.

[0130] Through the above detailed steps, the sound coordination system based on the tinnitus condition of the present invention is used to execute the steps of the sound coordination method based on the tinnitus condition in the above embodiments, which will not be elaborated here.

[0131] The present invention not only provides a personalized and intelligent sound coordination method and system based on tinnitus conditions, but also significantly improves the treatment effect and the patient's treatment experience through technical means such as multi-modal interaction, deep learning optimization, and real-time physiological feedback regulation, ensuring that tinnitus patients obtain more accurate and comfortable treatment during the treatment process. These innovative features give the present invention significant technical advantages and broad application prospects in the field of tinnitus treatment.

[0132] In a third aspect of the embodiments of the present invention, there is also provided a computer device, including a memory and a processor. A computer program is stored in the memory, and when the computer program is executed by the processor, the method of any one of the above embodiments is implemented.

[0133] The computer device includes a processor and a memory, and may further include: an input system and an output system. The processor, the memory, the input system, and the output system can be connected through a bus or other means. The input system can receive input digital or character information, and generate a signal input related to the migration of sound coordination based on tinnitus conditions. The output system may include a display device such as a display screen.

[0134] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the sound coordination system based on tinnitus conditions in the embodiments of the present application. The memory can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created for the use of the sound coordination system based on tinnitus conditions, etc. In addition, the memory can include a high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory can optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the local module through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0135] In some embodiments, the processor can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data. The processors of multiple computer devices in this embodiment of the computer device execute various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory, that is, implement the steps of the sound coordination method based on tinnitus conditions in the above method embodiments.

[0136] It should be understood that, without conflict, all the embodiments, features and advantages described above for the method for sound coordination based on tinnitus according to the present invention are equally applicable to the method for sound coordination based on tinnitus and the storage medium according to the present invention.

[0137] Those skilled in the art will also understand that the various exemplary logical blocks, modules, circuits and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, a general description of the functionality of various illustrative components, blocks, modules, circuits and steps has been presented. Whether this functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. The functionality that can be implemented in various ways for each particular application by those skilled in the art, but such implementation decisions should not be construed as causing a departure from the scope of the disclosure of the embodiments of the present invention.

[0138] Finally, it should be noted that the computer-readable storage medium herein (e.g., memory) can be a volatile memory or a non-volatile memory, or can include both volatile memory and non-volatile memory. By way of example and not limitation, non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which can act as an external cache memory. By way of example and not limitation, RAM can be obtained in various forms, such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to include, but are not limited to, these and other suitable types of memory.

[0139] The various exemplary logical blocks, modules, and circuits described in connection with the disclosure herein can be implemented or performed using the following components designed to perform the functions herein: a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP and / or any other such configuration.

[0140] The foregoing are exemplary embodiments of the present invention disclosure, but it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of the present invention disclosed as defined by the claims. The functions, steps, and / or actions of the method claims according to the disclosed embodiments herein need not be performed in any particular order. In addition, although the elements of the embodiments of the present invention disclosure may be described or claimed in individual form, they can also be understood as plural unless explicitly limited to the singular.

[0141] It should be understood that, as used herein, unless the context clearly supports the exception, the singular form "a" is also intended to include the plural form. It should also be understood that the "and / or" used herein refers to any and all possible combinations of one or more of the associated listed items. The foregoing serial numbers of the disclosed embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0142] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the present invention disclosure (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features between the above embodiments or different embodiments can also be combined, and there are many other variations in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present invention shall be included within the protection scope of the embodiments of the present invention.

Claims

1. A sound coordination method based on tinnitus conditions, characterized in that, The method includes the following steps: Establish a tinnitus feature vector of the patient through broadband hearing tests and three-dimensional sound field modeling, where the tinnitus feature vector includes the main frequency band energy ratio, time-domain modulation depth, and spatial perception dispersion parameters; Generate a dual-mode treatment audio based on the tinnitus feature vector, where the dual-mode treatment audio includes a cancellation sound wave with opposite phases and a masking sound wave with frequency band matching; Establish a dynamic mapping relationship between game interaction events and the audio parameters of the dual-mode treatment audio; Dynamically adjust the treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data, where the real-time physiological feedback data includes heart rate variability and skin conductance indicators; Among them, before establishing the tinnitus feature vector of the patient through broadband hearing tests and three-dimensional sound field modeling, a three-dimensional tinnitus sound image model is constructed by binaural sound field matching, auditory cortex response data is acquired through a 128-channel EEG acquisition module, and a frequency-specific audiogram curve in the range of 0.125 - 16 kHz is calculated; generating the dual-mode treatment audio based on the tinnitus feature vector includes the following steps: Mix white noise and pink noise through a base noise generation module, and the mixing ratio is dynamically adjusted according to the main frequency band energy ratio; Insert dynamic harmonic components in the target frequency band using the FFT analysis result; Perform spatial sound field modulation using the HRTF personalized adaptation algorithm; Among them, when dynamically adjusting the treatment parameters according to the dynamic mapping relationship and real-time physiological feedback data, when the standard deviation of heart rate variability is lower than 15% of the baseline value, a game difficulty downgrading mechanism is triggered; when the rising slope of skin conductance exceeds 0.05 μS / s, an environmental sound fusion mode is activated.

2. The method for sound coordination based on tinnitus conditions according to claim 1, wherein, When performing spatial sound field modulation using the HRTF personalized adaptation algorithm, an improved LMS adaptive filter is used, and the iteration formula is: W(n + 1)=W(n)+μ·e(n)·X(n) / (||X(n)||²+δ) Where, W(n) is the weight vector of the filter, e(n) is the error signal, X(n) is the input signal, μ = 0.2 is the step size parameter, δ = 0.01 is the stability factor, and the number of convergence iterations does not exceed 50 times.

3. The method for sound coordination based on tinnitus conditions according to claim 1, wherein, The mapping relationship includes operation response delay triggering phase inversion and game progress threshold controlling bandwidth expansion.

4. The method for sound coordination based on tinnitus conditions according to claim 3, wherein The established dynamic mapping relationship satisfies: The correlation function between the visual stimulus frequency Fv and the sound stimulus bandwidth Bw is Bw = K·log(1 + Fv), where K is the patient tolerance coefficient; The negative correlation relationship between the operation accuracy rate Ac and the treatment intensity It is It = Imax·(1 - 0.7Ac), where Imax is the maximum treatment intensity.

5. A sound coordination system based on tinnitus conditions, characterized in that, For implementing the sound coordination method based on tinnitus conditions as described in any one of claims 1 - 4, the system includes the following components: A feature analysis terminal (101), configured with binaural in-ear audiometry probes and a 128-channel EEG acquisition module, for collecting the patient's auditory data and electroencephalogram activity data; A dynamic synthesis server (201), with a real-time convolution processor accelerated by GPU built-in, supporting audio processing at a sampling rate of 192 kHz, and configured with: Parametric sound engine with configurable base noise mixer, dynamic harmonic inserter, and spatial sound field modulator; Cross-modal coordination control module to achieve triangular closed-loop control of game events, audio parameters, and physiological indicators; Interactive treatment terminal (301), integrated with a six-degree-of-freedom haptic feedback handle and bone conduction audio output device, for real-time interaction with patients to provide a personalized treatment experience; Physiological monitoring array (401), including a wrist PPG sensor and auricle temperature detection module, for real-time monitoring of patients' physiological feedback data.

6. The sound coordination system based on tinnitus conditions according to claim 5, wherein The dynamic synthesis server (201) further includes: Personalized sound generation module, supporting dynamic adjustment, capable of generating adapted dual-mode treatment audio according to the personalized characteristics of patients' tinnitus; Real-time audio processing module for processing multi-channel spatial sound effects and frequency response.

7. The sound coordination system based on tinnitus conditions according to claim 6, wherein, The interactive treatment terminal (301) is configured with: A three-dimensional soundscape game scene built based on the Unreal Engine, including a frequency matching level module for dynamically adjusting the audio elements of the game scene according to tinnitus characteristics; Support for a 256-group personalized HRTF database for dynamic coupling of sound image positions and visual elements.

Images