Intelligent full-automatic debugging system and method for artificial cochlea

Abstract

The invention discloses an intelligent full-automatic debugging system and method for an artificial cochlea. A user side is used for receiving and executing a debugging instruction; response information fed back by the user is sent to the full-intelligent debugging terminal; the user side is also used for collecting audio and video information of a user and sending the audio and video information to the full-intelligent debugging side; the full-intelligent debugging terminal comprises an intelligent debugging module and a user analysis module; the intelligent debugging module sends the debugging instruction to the user side and receives response information and audio and video information of the user side; the user analysis module receives and analyzes the response information and the audio and video information, and sends an analysis result to the intelligent debugging module; the intelligent debugging module adjusts a next debugging instruction according to the response information, the audio and video information and the analysis result; according to the system and the method, through joint operation of the face analysis module, the voice analysis module and the character analysis module, the original debugging work which can be carried out by a debugging worker is replaced, and the whole debugging process can be completed without manual participation.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical devices, and in particular relates to an intelligent fully automatic adjustment system and method for a cochlear implant. Background Art

[0002] Debugging of cochlear implants Cochlear implants are the only medical device on the market that can effectively restore hearing to patients with severe or profound hearing loss. Cochlear implants are generally composed of two parts: a sound processor and an implant. 7-14 days after the implant is implanted, a tuner can be found to perform the first debugging of the sound processor so that it can work with the implant, which is usually called "starting up". Cochlear implants generally require one-on-one manual debugging before they can be used, and are a very important part of the speech rehabilitation of cochlear users. Considering the postoperative rehabilitation period, the changes in cochlear impedance and hearing perception take 1 year to stabilize. Therefore, the polyphonic sound processor needs to be adjusted 1 month, 3 months, 6 months, and 12 months after startup to enable users to get the best possible hearing effect. In addition, if the equipment is replaced, it also needs to be debugged before it can be worn.

[0003] At present, the debugging of a sound processor takes as short as 1 hour and as long as more than 3 hours. A tuner can only debug one user at a time. The repetitive and tedious tuning process is very labor-intensive and time-consuming. Due to the limited number of tuners and working hours, each tuning requires advance reservations, coordination of time, and waiting in line. Although the auxiliary tuning system can alleviate problems 1 and 2, there are still some complex situations, such as face-to-face draws during the tuning process, or some subjective feedback, which are very specific and personalized. General auxiliary tuning systems cannot solve this problem alone. It is necessary to completely liberate the tuners through a more intelligent system. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an intelligent fully automatic adjustment system and method for a cochlear implant, which converts some tedious, repetitive, and mechanical adjustment processes into automatic machine responses, while retaining manual adjustments for some complex and advanced adjustment functions. The technical solution of the present invention is as follows: the intelligent fully automatic adjustment system for a cochlear implant includes a user end and a fully intelligent adjustment end; the user end is used to receive and execute adjustment instructions; and send the response information fed back by the user according to the adjustment instructions to the fully intelligent adjustment end; the user end is also used to collect the user's audio and video information, and send it to the fully intelligent adjustment end;

[0005] The fully intelligent machine adjustment terminal includes an intelligent machine adjustment module and a user analysis module; the intelligent machine adjustment module is used to send the machine adjustment instruction to the user terminal, and receive the response information and audio and video information of the user terminal; the user analysis module is used to receive and analyze the response information and audio and video information, and send the analysis result to the intelligent machine adjustment module;

[0006] The intelligent machine adjustment module adjusts the next machine adjustment instruction according to the response information, audio and video information and analysis results.

[0007] Preferably, the response information includes text information input by the user; the audio and video information includes voice information and video information collected by the user end; and the analysis result includes text analysis information, voice analysis information and facial image analysis information.

[0008] Preferably, the user analysis module includes a speech analysis module, which receives speech information, analyzes the speech information using an acoustic language model based on a neural network, generates speech analysis information, and sends it to the intelligent machine adjustment module.

[0009] Preferably, the user analysis module includes a text analysis module, which, after receiving text information, uses a semantic recognition technology model based on a generative pre-trained model to analyze the text information, generates text analysis information, and sends it to the intelligent machine adjustment module.

[0010] Preferably, the user analysis module includes a facial analysis module, which receives video information during the machine adjustment process, analyzes the video information using a facial recognition technology model based on a deep neural network, generates facial image analysis information, and sends the facial image analysis information to the intelligent machine adjustment module.

[0011] Preferably, if the facial image analysis information is information that the user has abnormal facial extraction, the intelligent machine adjustment module receives the abnormal information and turns off the electrodes that cause the abnormal facial extraction during the machine adjustment process.

[0012] Preferably, before sending the machine adjustment instruction, the intelligent machine adjustment module is also used to perform impedance measurement and neural response telemetry on the user to obtain measurement results, and initialize the machine adjustment instruction according to the measurement results.

[0013] An intelligent fully automatic adjustment method for a cochlear implant comprises the following steps:

[0014] S1: The intelligent machine adjustment module sends a machine adjustment instruction to the user end, and the user end receives and executes the machine adjustment instruction;

[0015] S2: The user responds to the machine adjustment command and generates response information, and the user terminal collects the user's audio and video information; the user terminal sends the response information and the audio and video information to the intelligent machine adjustment terminal and the user analysis module;

[0016] S3: The user analysis module receives and analyzes the response information and the audio and video information, and sends the analysis results to the intelligent machine adjustment module;

[0017] S4: The intelligent machine adjustment module adjusts the next machine adjustment instruction according to the received response information, audio and video information and analysis results, and sends it to the user end until the machine adjustment is completed.

[0018] Preferably, in step S1, before the intelligent machine adjustment module sends the machine adjustment instruction, it first performs impedance measurement and neural response telemetry to obtain the measurement results and initializes the machine adjustment instruction according to the measurement results.

[0019] Preferably, the initialization adjustment instruction includes stimulation strategy, stimulation pulse width, stimulation frequency, threshold value, and comfort value.

[0020] The beneficial effects of the present invention are:

[0021] 1. The intelligent fully automatic adjustment system for cochlear implants of the present invention, by setting up an intelligent adjustment module and a user analysis module, adjusts the next adjustment action through the user's feedback response information, audio and video information and corresponding analysis results during the adjustment process, thus achieving full intelligence in the adjustment process. Through the joint operation of the facial analysis module, the voice analysis module and the text analysis module, the adjustment work that was originally only performed by the adjuster is replaced, so that the entire adjustment process can be completed without human participation, completely freeing the adjuster.

[0022] 2. The intelligent fully automatic adjustment system for cochlear implants of the present invention collects video information such as facial images during the user adjustment process through the user end, and obtains facial image analysis information through analysis by the user analysis module, thereby obtaining more specific and personalized feedback content during the user adjustment process and achieving the best adjustment effect for the user.

[0023] 3. In the cochlear implant intelligent fully automatic adjustment system of the present invention, the user analysis module uses appropriate analysis models to analyze the received text information, voice information and video information, effectively ensuring the accuracy of the information analysis and further improving the user adjustment effect.

[0024] 4. The intelligent fully automatic tuning system for cochlear implants of the present invention first performs targeted tuning initialization configuration before formal tuning, which effectively improves the tuning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is an overall schematic diagram of an intelligent fully automatic adjustment system for a cochlear implant according to a specific embodiment of the present invention;

[0026] Figure 2It is a schematic diagram of an intelligent fully automatic adjustment system for a cochlear implant according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0027] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0028] See also Figure 1 and Figure 2 The intelligent fully automatic adjustment system of the cochlear implant includes a user terminal 1 and a fully intelligent adjustment terminal 2; the user terminal 1 is used to receive and execute the adjustment command; and send the response information fed back by the user according to the adjustment command to the fully intelligent adjustment terminal 2; the user terminal 1 is also used to collect the user's audio and video information and send it to the fully intelligent adjustment terminal 2. Further, the response information includes text information input by the user, and the audio and video information includes voice information and video information collected by the user terminal 1.

[0029] The user terminal 1 is a software program that can be deployed on a smartphone, tablet computer, or PC, or a specially designed electronic device that includes audio and video information interaction. The fully intelligent tuning terminal 1 can be a program software running on a server, tablet computer, or PC, or a specially designed electronic device, which will not be described in detail here.

[0030] The fully intelligent machine adjustment terminal 2 includes an intelligent machine adjustment module 21 and a user analysis module (not shown in the figure); the intelligent machine adjustment module 21 is used to send the adjustment instruction to the user terminal 1, and receive the response information and audio and video information of the user terminal 1; the user analysis module is used to receive and analyze the response information and audio and video information, and send the analysis results to the intelligent machine adjustment module 21. The intelligent machine adjustment module 21 adjusts the next step of the adjustment instruction according to the response information, audio and video information and the analysis results. Specifically, the analysis results include text analysis information, voice analysis information and facial image analysis information.

[0031] Specifically, the user analysis module includes a speech analysis module 22, which receives speech information, and after analyzing the speech information using an acoustic language model based on a neural network, generates speech analysis information and sends it to the intelligent machine adjustment module 21. Speech analysis includes two parts, semantics and pronunciation quality, wherein semantics is to identify whether the patient's expression is consistent with the test semantics through a hidden Markov model; pronunciation quality is judged through a deep neural network fitting model, the input of which includes fundamental frequency and Mel cepstral coefficients, and the fitting result of the fundamental frequency is used to determine the accuracy of the four tones of Chinese; Mel cepstral is suitable for determining the spectral quality of speech.

[0032] The user analysis module also includes a text analysis module 23 , which receives text information, analyzes the text information using a semantic recognition technology model based on a generative pre-trained model, generates text analysis information, and sends it to the intelligent adjustment module 21 .

[0033] The user analysis module also includes a facial analysis module 24. During the machine adjustment process, the facial analysis module 24 receives video information, analyzes the video information using a facial recognition technology model based on a deep convolutional neural network, generates facial image analysis information, and sends it to the intelligent machine adjustment module 21.

[0034] Specifically, if the facial image analysis information indicates that the user has abnormal facial extraction, the intelligent machine adjustment module receives the abnormal information and turns off the electrodes that cause the abnormal facial extraction during the machine adjustment process.

[0035] The intelligent fully automatic adjustment system for cochlear implants of the present invention, by setting up an intelligent adjustment module and a user analysis module, adjusts the next adjustment action through the user's feedback response information, audio and video information and the corresponding analysis results during the adjustment process, thereby achieving full intelligence in the adjustment process. Through the joint operation of the facial analysis module, the voice analysis module and the text analysis module, appropriate analysis models are used to analyze text information, voice information and video information, respectively, replacing the adjustment work that was originally performed by the adjuster, so that the entire adjustment process can be completed without human participation; it also effectively guarantees the accuracy of information analysis and further improves the user's adjustment effect.

[0036] Furthermore, before sending the tuning command, the intelligent tuning module 21 is also used to perform impedance measurement and neural response telemetry on the user to obtain test results, and initialize the tuning parameters according to the measurement results, thereby effectively improving the tuning effect.

[0037] The sound processor is connected to the user terminal 1 and is used to receive information sent by the user terminal 1. The information includes the tuning instructions of the sound processor.

[0038] An intelligent fully automatic adjustment method for a cochlear implant comprises the following steps:

[0039] S1: The intelligent machine adjustment module 21 sends a machine adjustment instruction to the user terminal 1, and the user terminal 1 receives and executes the machine adjustment instruction;

[0040] S2: The user responds to the machine adjustment command and generates response information, and the user terminal 1 collects the user's audio and video information; the user terminal 1 sends the response information and the audio and video information to the intelligent machine adjustment terminal 21 and the user analysis module;

[0041] S3: The user analysis module receives and analyzes the response information and the audio and video information, and sends the analysis results to the intelligent machine adjustment module 21;

[0042] S4: The intelligent machine adjustment module 21 adjusts the next machine adjustment instruction according to the received response information, audio and video information and analysis results, and sends it to the user terminal 1 until the machine adjustment is completed.

[0043] Specifically, in step S1, before the intelligent machine adjustment module 21 sends the machine adjustment instruction, it first performs impedance measurement and neural response telemetry, and initializes the machine adjustment parameters according to the measurement results, so as to facilitate the subsequent machine adjustment and improve the machine adjustment efficiency. The initialized machine adjustment parameters include stimulation strategy, stimulation pulse width, stimulation frequency, threshold, comfort value, etc.

[0044] The intelligent fully automatic tuning system and method for the cochlear implant of the present invention replaces the tuning work that was originally performed by a tuner through the joint operation of a facial analysis module, a speech analysis module, and a text analysis module, so that the entire tuning process can be completed without human intervention, completely freeing up the tuner and effectively improving the tuning efficiency and effect.

[0045] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. An intelligent fully automatic adjustment system for cochlear implants, characterized in that: It includes a user terminal and a fully intelligent machine adjustment terminal; the user terminal is used to receive and execute the machine adjustment instruction; and send the response information fed back by the user according to the machine adjustment instruction to the fully intelligent machine adjustment terminal; the user terminal is also used to collect the user's audio and video information and send it to the fully intelligent machine adjustment terminal; The fully intelligent machine adjustment terminal includes an intelligent machine adjustment module and a user analysis module; the intelligent machine adjustment module is used to send the machine adjustment instruction to the user terminal, and receive the response information and audio and video information of the user terminal; The user analysis module is used to receive and analyze the response information and the audio and video information, and send the analysis results to the intelligent machine adjustment module; The intelligent machine adjustment module adjusts the next machine adjustment instruction according to the response information, audio and video information and analysis results.

2. The intelligent fully automatic adjustment system for cochlear implants according to claim 1, characterized in that: The response information includes text information input by the user; the audio and video information includes voice information and video information collected by the user end; the analysis result includes text analysis information, voice analysis information and facial image analysis information.

3. The intelligent fully automatic adjustment system for cochlear implants according to claim 2, characterized in that: The user analysis module includes a speech analysis module, which receives speech information, analyzes the speech information using an acoustic language model based on a neural network, generates speech analysis information, and sends it to the intelligent machine adjustment module.

4. The intelligent fully automatic adjustment system for cochlear implants according to claim 2, characterized in that: The user analysis module includes a text analysis module. After receiving text information, the text analysis module uses a semantic recognition technology model based on a generative pre-trained model to analyze the text information, generates text analysis information, and sends it to the intelligent adjustment module.

5. The intelligent fully automatic adjustment system for cochlear implants according to claim 2, characterized in that: The user analysis module includes a facial analysis module. During the machine adjustment process, the facial analysis module receives video information, analyzes the video information using a facial recognition technology model based on a deep neural network, generates facial image analysis information, and sends the facial image analysis information to the intelligent machine adjustment module.

6. The intelligent fully automatic adjustment system for cochlear implants according to claim 5, characterized in that: If the facial image analysis information is that the user has a facial extraction abnormality, the intelligent machine adjustment module receives the abnormal information and turns off the electrodes that cause the facial extraction abnormality during the machine adjustment process.

7. The intelligent fully automatic adjustment system for cochlear implants according to claim 5, characterized in that: Before sending the machine adjustment instruction, the intelligent machine adjustment module is also used to perform impedance measurement and neural response telemetry on the user, obtain measurement results, and initialize the machine adjustment parameters according to the measurement results.

8. An intelligent fully automatic adjustment method for a cochlear implant using the system according to claims 1-7, characterized in that: The following steps are involved: S1: The intelligent machine adjustment module sends a machine adjustment instruction to the user end, and the user end receives and executes the machine adjustment instruction; S2: The user responds to the machine adjustment command and generates response information, and the user terminal collects the user's audio and video information; the user terminal sends the response information and the audio and video information to the intelligent machine adjustment terminal and the user analysis module; S3: The user analysis module receives and analyzes the response information and the audio and video information, and sends the analysis results to the intelligent machine adjustment module; S4: The intelligent machine adjustment module adjusts the next machine adjustment instruction according to the received response information, audio and video information and analysis results, and sends it to the user end until the machine adjustment is completed.

9. The intelligent fully automatic adjustment method for a cochlear implant according to claim 8, characterized in that: In step S1, before the intelligent machine adjustment module sends a machine adjustment instruction, it first performs impedance measurement and neural response telemetry to obtain measurement results and initializes the machine adjustment parameters according to the measurement results.

10. The intelligent fully automatic adjustment method of a cochlear implant according to claim 9, characterized in that: The initialization adjustment parameters include stimulation strategy, stimulation pulse width, stimulation frequency, threshold value, and comfort value.

Images