Method and system for cochlear implant to recognize sound scene by mobile device

Abstract

The present invention discloses a method and system for a cochlear implant to perform sound scene recognition with the aid of a mobile device, which mainly includes the following steps: after a central processing unit receives a scene recognition abnormality signal, a DSP module transmits the received voice signal to the central processing unit via low-power Bluetooth technology; the central processing unit starts to perform sound scene recognition depth calculation according to the voice signal, and after calculating several frames of sound signals, the central processing unit obtains new scene recognition information and sends it to the DSP module; the DSP module compares the new scene recognition information with the current scene recognition information stored in a register, and if the scene is judged to be different, the sound signal processing strategy corresponding to the new scene is changed, and the characteristic value calculation is restored. The present invention utilizes the low-power wireless voice transmission technology of the speech processor and the stronger computing power of the mobile device to improve the reliability of sound scene recognition, and reduces the device resource usage time and power consumption through distributed computing.

Description

Technical Field

[0001] The present invention belongs to the field of medical devices, and in particular relates to a method and system for using a cochlear implant to perform sound scene recognition with the aid of a mobile device. Background Art

[0002] With the continuous development of cochlear implant technology, using different strategies to process sound signals in different scenarios to enable users to obtain the best hearing experience and further improve users' speech recognition rates in different scenarios has gradually become a mainstream application.

[0003] In the process of actual application, how to stably and reliably perform scene recognition has become a bottleneck problem that restricts the development of this technology. For cochlear implant speech processors, due to power consumption limitations, their processing speed is limited, but sound signals are always transmitted in real time, and it is necessary to complete sound signal frequency domain operations, sound signal processing in different scenes, sound scene recognition, stimulus coding output and routine work monitoring functions at the same time within a fixed time. Therefore, the amount of calculation required for sound scene recognition is a heavy burden for cochlear implant speech processors. On the other hand, the accuracy of sound scene recognition is closely related to the amount of calculation. Current research points to complex machine learning or neural network algorithms for sound scene recognition. In the actual design process, the amount of calculation and accuracy must be repeatedly weighed, and it is difficult to find an algorithm that takes both aspects into consideration. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method and system for sound scene recognition by a cochlear implant with the help of a mobile device. The cochlear implant speech processor and the mobile device independently perform distributed computing, the cochlear implant speech processor detects whether the sound scene changes, the mobile device determines the current sound scene, and finally the cochlear implant speech processor adjusts the corresponding sound processing strategy according to the judgment of the mobile device.

[0005] To achieve the above object, the present invention provides a method for finding a lost cochlear implant speech processor, comprising the following steps:

[0006] When the cochlear implant speech processor works normally, the DSP module calculates several frequency domain eigenvalues ​​for each frame of sound signal input by the sound input module;

[0007] The frequency domain eigenvalue change judgment is performed once every several frames. If no eigenvalues ​​change beyond the threshold, the DSP module continues to calculate the eigenvalues. If some eigenvalues ​​change beyond the threshold, it jumps to the next step.

[0008] The DSP module stops the eigenvalue calculation, activates the front-end Bluetooth module of the cochlear implant speech processor, and sends the scene recognition abnormality signal to the front-end Bluetooth module. The front-end Bluetooth module establishes a wireless connection, sends the signal to the back-end Bluetooth module, and finally transmits it to the central processing unit of the mobile device;

[0009] After the central processing unit receives the abnormal scene recognition signal, the front-end Bluetooth module establishes a voice stream connection with the back-end Bluetooth module, and the DSP module transmits the received voice signal to the central processing unit through low-power Bluetooth technology;

[0010] The central processing unit starts to perform sound scene recognition depth calculation based on the voice signal. After calculating several frames of sound signals, the central processing unit obtains new scene recognition information.

[0011] The central processing unit sends the new scene recognition information to the DSP module through the Bluetooth connection established by the front-end Bluetooth module and the back-end Bluetooth module;

[0012] The DSP module compares the new scene recognition information with the current scene recognition information stored in the register. If the scenes are judged to be the same, the DSP module resumes the feature value calculation and does not perform other operations. If the scenes are judged to be different, it proceeds to the next step.

[0013] The DSP module will be changed to the sound signal processing strategy corresponding to the new scene and the eigenvalue calculation will be restored.

[0014] Preferably, the DSP module and the central processing unit independently complete scene anomaly calculation and sound scene recognition depth calculation respectively.

[0015] Preferably, all calculations are based on one frame of voice data as a basic unit, and the time length of one frame of voice data ranges from 1 to 16 milliseconds.

[0016] Preferably, the cochlear implant speech processor performs abnormal scene recognition and judgment through frequency domain eigenvalues, and the number of eigenvalues ​​ranges from 1 to 16.

[0017] Based on the above purpose, the present invention also provides a cochlear implant sound scene recognition system with the aid of a mobile device, comprising a cochlear implant speech processor and a mobile device, wherein:

[0018] The cochlear implant speech processor includes a sound input module, a DSP module, a front-end Bluetooth module and a register, wherein:

[0019] The sound input module is used to receive sound signals and convert them into electrical signals;

[0020] The DSP module is connected to the sound input module, digitizes the electrical signal input by the sound input module, calculates the frequency domain eigenvalues, and processes the sound signal according to the strategy corresponding to the scene;

[0021] The front-end Bluetooth module is connected to the DSP module and is used to send and receive commands and voice signals using low-power Bluetooth technology and to achieve data communication with the DSP module;

[0022] The register is connected to the DSP module and is used to store feature value information and scene information;

[0023] The mobile device comprises a back-end Bluetooth module and a central processing unit, wherein:

[0024] The back-end Bluetooth module is paired with the front-end Bluetooth module, and two-way data communication is performed between them via low-power Bluetooth technology;

[0025] The central processing unit is connected to the back-end Bluetooth module, performs scene recognition depth calculation on the voice signal sent by the back-end Bluetooth module, and sends the calculation result to the back-end Bluetooth module for subsequent judgment and operation of the DSP module.

[0026] Preferably, the front-end Bluetooth module and the back-end Bluetooth module are connected via low-power Bluetooth and perform voice and data transmission simultaneously, with a transmission distance of 0 to 10 meters.

[0027] Preferably, the mobile device uses the sound signal input by the cochlear implant speech processor to perform scene recognition operations.

[0028] The beneficial effects of the present invention are: using the existing computing power of the cochlear implant speech processor to perform relatively simple scene change judgments, and then relying on the more powerful computing power of the mobile device to calculate the specific scene. At the same time, due to the use of the voice signal of the cochlear implant speech processor, there is no need to place the mobile device in an open environment, avoiding the limitation of the location of the mobile device, and being closer to the patient's actual listening scene. This two-step calculation also avoids the mobile device from performing real-time sound scene recognition, saving power consumption. The method of the present invention has the advantages of fully automatic control, easy engineering implementation and automatic intelligent judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration:

[0030] Figure 1 A flowchart of the steps of a specific application example of the method for performing sound scene recognition with a cochlear implant by means of a mobile device according to an embodiment of the present invention;

[0031] Figure 2The present invention is a block diagram of a specific application example of a system for performing sound scene recognition with the help of a mobile device in a cochlear implant according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0033] See also Figure 1 , which is a flowchart of a specific application example of a method for a cochlear implant to recognize a sound scene by using a mobile device according to an embodiment of the present invention. For an embodiment of a cochlear implant to recognize a sound scene by using a mobile device, see Figure 2 , the method comprises the following steps:

[0034] S101, when the cochlear implant speech processor 10 works normally, the DSP module 120 calculates several frequency domain eigenvalues ​​for each frame of the sound signal input by the sound input module 110;

[0035] S102, performing frequency domain eigenvalue change judgment every several frames, if no eigenvalues ​​change beyond the threshold, the DSP module 120 continues to calculate the eigenvalues, if some eigenvalues ​​change beyond the threshold, jump to the next step;

[0036] S103, the DSP module 12 stops the feature value calculation, activates the front-end Bluetooth module 130 of the cochlear implant speech processor, and sends the scene recognition abnormality signal to the front-end Bluetooth module 130, the front-end Bluetooth module 130 establishes a wireless connection, sends the signal to the back-end Bluetooth module 210, and finally transmits it to the central processing unit 220 of the mobile device;

[0037] S104, after the central processing unit 220 receives the scene recognition abnormality signal, the front-end Bluetooth module 130 establishes a voice stream connection with the back-end Bluetooth module 140, and the DSP module 120 transmits the received voice signal to the central processing unit 220 via the low-power Bluetooth technology;

[0038] S105, the central processing unit 220 starts to perform sound scene recognition depth calculation according to the voice signal. After calculating several frames of sound signals, the central processing unit 220 obtains new scene recognition information;

[0039] S106, the central processing unit 220 sends the new scene recognition information to the DSP module 120 through the Bluetooth connection established between the front-end Bluetooth module 130 and the back-end Bluetooth module 210;

[0040] S107, the DSP module 120 compares the new scene recognition information with the current scene recognition information stored in the register 140. If the scenes are determined to be the same, the DSP module 120 resumes the feature value calculation S101 and does not perform other operations. If the scenes are determined to be different, the next step is entered;

[0041] S108, the DSP module 120 changes to the sound signal processing strategy corresponding to the new scene and resumes the eigenvalue calculation S101.

[0042] Furthermore, the DSP module 120 and the central processing unit 220 independently complete scene anomaly calculation and sound scene recognition depth calculation respectively.

[0043] Furthermore, all calculations are based on one frame of voice data as a basic unit, and the time length of one frame of voice data ranges from 1 to 16 milliseconds.

[0044] Furthermore, all operations are completed automatically in the background without any manual operation.

[0045] Furthermore, the cochlear implant speech processor 10 performs abnormal scene recognition and judgment through frequency domain eigenvalues, and the number of eigenvalues ​​ranges from 1 to 16.

[0046] Based on the above purpose, the present invention also provides a cochlear implant sound scene recognition system 100 with the aid of a mobile device, comprising a cochlear implant speech processor 10 and a mobile device 20, wherein:

[0047] The cochlear implant speech processor 10 includes a sound input module 110, a DSP module 120, a front-end Bluetooth module 130 and a register 140, wherein:

[0048] The sound input module 110 is used to receive sound signals and convert them into electrical signals;

[0049] The DSP module 120 is connected to the sound input module 110, digitizes the electrical signal input by the sound input module, calculates the frequency domain eigenvalues, and processes the sound signal according to the strategy corresponding to the scene;

[0050] The front-end Bluetooth module 130 is connected to the DSP module 120 and is used to send and receive commands and voice signals using low-power Bluetooth technology and to achieve data communication with the DSP module 120;

[0051] The register 140 is connected to the DSP module 120 and is used to store feature value information and scene information;

[0052] The mobile device 20 includes a back-end Bluetooth module 210 and a central processing unit 220, wherein:

[0053] The back-end Bluetooth module 210 is paired with the front-end Bluetooth module 130, and two-way data communication is performed between them via low-power Bluetooth technology;

[0054] The central processing unit 220 is connected to the back-end Bluetooth module 210, performs scene recognition depth calculation on the voice signal sent by the back-end Bluetooth module 210, and sends the calculation result to the back-end Bluetooth module 210 for subsequent judgment and operation of the DSP module 120.

[0055] Furthermore, the front-end Bluetooth module 120 and the back-end Bluetooth module 210 are connected via Bluetooth Low Energy (BLE) technology and simultaneously transmit voice and general data, with a transmission distance of 0 to 10 meters.

[0056] Furthermore, the mobile device 20 uses the sound signal input by the cochlear implant speech processor 10 to perform scene recognition operations.

[0057] The present invention simultaneously utilizes the simple computing power of the cochlear implant speech processor and the complex computing power of the mobile device, optimizes computing efficiency through distributed computing, and in view of the fact that mobile devices are often in closed environments such as trouser pockets, through low-power wireless voice transmission, the mobile device uses the sound received by the cochlear implant speech processor to perform signal processing, thereby avoiding misjudgment and restrictions on the location of the mobile device. The method of the present invention does not require any human intervention and has the advantages of easy engineering implementation and high judgment accuracy.

[0058] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A method for using a cochlear implant to recognize sound scenes with the aid of a mobile device, characterized in that: The following steps are involved: When the cochlear implant speech processor works normally, the DSP module calculates several frequency domain eigenvalues ​​for each frame of sound signal input by the sound input module; The frequency domain eigenvalue change judgment is performed once every several frames. If no eigenvalues ​​change beyond the threshold, the DSP module continues to calculate the eigenvalues. If some eigenvalues ​​change beyond the threshold, it jumps to the next step. The DSP module stops the eigenvalue calculation, activates the front-end Bluetooth module of the cochlear implant speech processor, and sends the scene recognition abnormality signal to the front-end Bluetooth module. The front-end Bluetooth module establishes a wireless connection, sends the signal to the back-end Bluetooth module, and finally transmits it to the central processing unit of the mobile device; After the central processing unit receives the abnormal scene recognition signal, the front-end Bluetooth module establishes a voice stream connection with the back-end Bluetooth module, and the DSP module transmits the received voice signal to the central processing unit through low-power Bluetooth technology; The central processing unit starts to perform sound scene recognition depth calculation based on the voice signal. After calculating several frames of sound signals, the central processing unit obtains new scene recognition information. The central processing unit sends the new scene recognition information to the DSP module through the Bluetooth connection established by the front-end Bluetooth module and the back-end Bluetooth module; The DSP module compares the new scene recognition information with the current scene recognition information stored in the register. If the scenes are judged to be the same, the DSP module resumes the feature value calculation and does not perform other operations. If the scenes are judged to be different, it proceeds to the next step. The DSP module will be changed to the sound signal processing strategy corresponding to the new scene and the eigenvalue calculation will be restored.

2. The method according to claim 1, characterized in that The DSP module and the central processing unit independently complete scene anomaly calculation and sound scene recognition depth calculation respectively.

3. The method according to claim 1, characterized in that All calculations are based on one frame of speech data as the basic unit, and the time length of one frame of speech data ranges from 1 to 16 milliseconds.

4. The method according to claim 1, characterized in that: The cochlear implant speech processor performs abnormal scene recognition and judgment through frequency domain eigenvalues, and the number of eigenvalues ​​ranges from 1 to 16.

5. A cochlear implant for sound scene recognition using a mobile device, characterized in that: Includes cochlear implant speech processors and mobile devices, wherein: The cochlear implant speech processor includes a sound input module, a DSP module, a front-end Bluetooth module and a register, wherein: The sound input module is used to receive sound signals and convert them into electrical signals; The DSP module is connected to the sound input module, digitizes the electrical signal input by the sound input module, calculates the frequency domain eigenvalues, and processes the sound signal according to the strategy corresponding to the scene; the DSP module judges the change of the frequency domain eigenvalues ​​once every several frames, and if no several eigenvalues ​​change beyond the threshold, the DSP module continues to calculate the eigenvalues, and if some eigenvalues ​​change beyond the threshold, jumps to the next step; The front-end Bluetooth module is connected to the DSP module, and is used to send and receive commands and voice signals using low-power Bluetooth technology, and to achieve data communication with the DSP module; the DSP module sends the scene recognition abnormality signal to the front-end Bluetooth module; The register is connected to the DSP module and is used to store feature value information and scene information; The mobile device comprises a back-end Bluetooth module and a central processing unit, wherein: The back-end Bluetooth module is paired with the front-end Bluetooth module, and two-way data communication is performed between them through low-power Bluetooth technology; the front-end Bluetooth module establishes a wireless connection and sends the scene recognition abnormality signal to the back-end Bluetooth module; The central processing unit is connected to the back-end Bluetooth module, and the back-end Bluetooth module sends the scene recognition abnormality signal to the central processing unit; after the central processing unit receives the scene abnormality signal, the front-end Bluetooth module establishes a voice stream connection with the back-end Bluetooth module, and the DSP module transmits the received voice signal to the back-end Bluetooth module through the low-power Bluetooth technology; the central processing unit performs scene recognition depth calculation on the voice signal sent by the back-end Bluetooth module, and sends the calculation result, that is, the new scene recognition information, to the back-end Bluetooth module for subsequent judgment and operation of the DSP module; The DSP module compares the new scene recognition information with the current scene recognition information stored in the register. If the scenes are judged to be the same, the DSP module resumes the feature value calculation and does not perform other operations. If the scenes are judged to be different, it proceeds to the next step. The DSP module will be changed to the sound signal processing strategy corresponding to the new scene and the eigenvalue calculation will be restored.

6. The system according to claim 5, characterized in that The front-end Bluetooth module and the back-end Bluetooth module are connected via low-power Bluetooth and perform voice and data transmission simultaneously, with a transmission distance of 0 to 10 meters.

7. The system according to claim 5, characterized in that The mobile device uses the sound signal input by the cochlear implant speech processor to perform scene recognition operations.

Images