Visual prosthesis device, system, control method and storage medium thereof

Abstract

The embodiment of the present invention discloses a visual prosthesis device, system and control method thereof, and storage medium, wherein the main control module of the visual prosthesis device controls the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction received by the first communication module, and the electrode array can not only receive the stimulation pulse signal to release electrical stimulation to the visual neuron tissue, but also receive the analog neuron signal generated by the visual nerve tissue, and then combine the signal amplification and analog-to-digital conversion module to amplify and convert the analog neuron signal to obtain a digital neuron signal, and the main control module can control the first communication module to transmit the digital neuron signal externally. The visual prosthesis system including the external control module, the second communication module and the visual prosthesis device and the control method thereof can receive and process and analyze the digital neuron signal, find the optimal stimulation parameters of each electrode of the visual prosthesis device, and improve the efficiency and accuracy of obtaining the optimal stimulation parameters of the electrode.

Description

Technical Field

[0001] The present invention relates to the technical field of visual prosthesis, and in particular to a visual prosthesis device, a visual prosthesis system, a control method for the visual prosthesis system and a computer storage medium. Background Art

[0002] Visual prosthesis technology is a type of functional electrical stimulation. It takes advantage of the fact that most blind people only have a part of their visual pathway diseased, while the structure and function of the rest of the nerve tissue are intact. It applies specific artificial electrical stimulation to the intact part of the visual pathway to excite nerve cells, simulate the effect of natural light stimulation, and make the blind feel visually. The visual sensation produced by the fixed-point electrical stimulation of a single electrode is called phosphene. The visual prosthesis system includes a video acquisition device located outside the human body (usually a small camera), a video processing module, an electrical stimulation encoding module, and a multi-electrode array implanted in a specific part of the visual pathway in the body. The latter two constitute the visual prosthesis in-body chip. The working principle of the visual prosthesis is: the real-time video image collected by the video acquisition device is processed and converted into a signal to drive the multi-electrode array. The multi-electrode array applies current stimulation of a certain amplitude, waveform and frequency to the visual nerve tissue to excite the visual neurons, thereby making the patient feel visually.

[0003] Since the distances between different sub-electrodes and visual neurons in the multi-electrode array are different, the impedances of different sub-electrodes are different, resulting in different minimum stimulation intensities required for different sub-electrodes to activate their neighboring visual neurons. Therefore, the stimulation parameters of each sub-electrode need to be optimized individually.

[0004] In the prior art, the optimization process of the electrical stimulation parameters of the electrodes of the visual prosthesis is roughly the following process: the medical staff transmits a set of continuously enhanced stimulation parameters to the visual prosthesis in-body chip, and tests which electrodes in the electrode array can make the subject have a 50% probability of perceiving light hallucinations under which parameters. The medical staff selects from these parameters and sets the optimal stimulation parameters for each sub-electrode in the electrode array. However, relying on the patient's verbal description to optimize the electrode stimulation parameters will consume a lot of time and energy of medical workers and patients because the visual prosthesis chip contains a large number of electrodes. In addition, the verbal description cannot accurately reflect the effective range of electrode stimulation, which will significantly reduce the spatial resolution of electrode stimulation. This also means that electrical stimulation is often in an overly high state, which will cause heat accumulation and increase the risk of thermal damage to neural tissue. Moreover, over time, the contact between the in-body chip and the neural tissue changes, causing the electrode impedance to change, resulting in the failure of the previously determined electrical stimulation parameters. Patients need to seek help from medical staff regularly, which further increases the burden on medical workers and patients. Finally, when the subjects are animals, they are unable to verbally describe the phosphenes they perceive, limiting the research and development of optimized electrode stimulation parameters. Summary of the invention

[0005] The embodiments of the present invention provide a visual prosthesis device, a system, a control method thereof, and a storage medium, which can improve the accuracy and acquisition efficiency of the optimal stimulation parameters of the stimulation electrodes.

[0006] On the one hand, an embodiment of the present invention provides a visual prosthesis device, including a first communication module, a main control module, a stimulation pulse generation module, an electrode array, and a signal amplification and analog-to-digital conversion module, wherein the first communication module is connected to the main control module, the output end of the main control module is connected to the input end of the stimulation pulse generation module, the output end of the stimulation pulse generation module is connected to the input end of the electrode in the electrode array, the output end of the electrode is connected to the input end of the signal amplification and analog-to-digital conversion module, and the output end of the signal amplification and analog-to-digital conversion module is connected to the input end of the main control module; wherein:

[0007] The first communication module is used to receive electrode control instructions and transmit digital neuron signals;

[0008] The main control module is used to control the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction;

[0009] The stimulation pulse generating module is used to generate the stimulation pulse signal according to the electrode control instruction;

[0010] The electrodes in the electrode array are used to receive the stimulation pulse signal, release electrical stimulation to the visual nerve tissue to generate phosphenes, and receive simulated neuron signals generated by the visual nerve tissue;

[0011] The signal amplification and analog-to-digital conversion module is used to amplify the analog neuron signal and perform digital processing to obtain the digital neuron signal.

[0012] Optionally, the main control module includes a first processor, a second processor and a data register module including multiple data registers, the output end of the signal amplification and analog-to-digital conversion module is connected to the input end of the first processor, the output end of the first processor is connected to the input end of the data register module, the data register module is connected to the second processor, the second processor is connected to the first communication module, the second processor is used to take out the digital neuron signal stored in the data register and send it to the first communication module, and the first processor is used to store the new digital neuron signal in the vacated data register.

[0013] Optionally, the stimulation pulse generating module includes multiple sub-stimulation pulse generating modules, the number of the sub-stimulation pulse generating modules is the same as the number of electrodes of the electrode array, and the output end of one of the sub-stimulation pulse generating modules is connected to the input end of an electrode of the electrode array.

[0014] Optionally, the device also includes a temperature sensor for detecting the temperature of the visual nerve tissue and / or an impedance sensor for detecting the electrode impedance of the electrode array, and the output end of the temperature sensor and the output end of the impedance sensor are respectively connected to the input end of the main control module.

[0015] On the other hand, an embodiment of the present invention provides a visual prosthesis system, comprising an extracorporeal control module, a second communication module and the visual prosthesis device, wherein the extracorporeal control module is connected to the second communication module, the second communication module is connected to the first communication module, and the extracorporeal control module is used to send the electrode control instructions to the first communication module, and to receive and process the digital neuron signals.

[0016] Optionally, the visual prosthesis system further comprises an alarm module for outputting an alarm signal, and the output end of the external control module is connected to the input end of the alarm module.

[0017] On the other hand, an embodiment of the present invention provides a control method of a visual prosthesis system, which is applied to the visual prosthesis system, and includes:

[0018] Sending the electrode control instruction to the visual prosthesis device, the electrode control instruction including the electrode number of the stimulation electrode, the initial stimulation parameter of the stimulation electrode, and the electrode number of the recording electrode group, the electrode control instruction is used to instruct the visual prosthesis device to control the electrodes with corresponding numbers in the electrode array to emit electrical stimulation to the visual nerve tissue according to the electrode number of the stimulation electrode and the initial stimulation parameter;

[0019] receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

[0020] Whether a preset condition is met is determined based on the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is determined that the preset condition is met, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrode; if it is determined that the preset condition is not met, the initial stimulation parameters are modified until the preset condition is met.

[0021] Optionally, the electrode numbers of the recording electrode group include the electrode numbers of the first recording electrode group and the electrode numbers of the second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the second recording electrode group includes the stimulation electrode;

[0022] The step of judging whether a preset condition is satisfied according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset condition is satisfied, taking the initial stimulation parameter as the optimal stimulation parameter of the stimulation electrode; and if the preset condition is not satisfied, modifying the initial stimulation parameter until the preset condition is satisfied, comprises:

[0023] According to the first digital neuron signal of the first recording electrode group and the second digital neuron signal of each electrode in the first recording electrode group in a resting state, obtaining the number of first electrodes in the first recording electrode group whose first digital neuron signal reaches a first preset threshold, wherein the first preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a first preset multiple;

[0024] According to the first digital neuron signal of the second recording electrode group and the second digital neuron signal of each electrode in the second recording electrode group in a resting state, obtaining the number of second electrodes in the second recording electrode group whose first digital neuron signal reaches a second preset threshold, wherein the second preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a second preset multiple;

[0025] An effective stimulation range is obtained by adding the number of the first electrodes and the number of the second electrodes, and a judgment is made based on the effective stimulation range and a preset value. When the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, the initial stimulation parameters are modified until the effective stimulation range is equal to the preset value.

[0026] Optionally, the method further comprises:

[0027] Sending a test instruction to the visual prosthesis device, the test instruction including an optimal stimulation parameter of a stimulation electrode and a standard test image, the standard test image designating the stimulation electrode, the test instruction being used to instruct the visual prosthesis device to project the standard test image onto the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode;

[0028] receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

[0029] Reconstructing an image according to the third digital neuron signal to obtain a perceived image;

[0030] Calculating image similarity between the standard test image and the perception image;

[0031] Whether the optimal stimulation parameters of the stimulation electrode meet the requirements is judged according to the image similarity and the preset similarity; if the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is less than the preset similarity, the optimal stimulation parameters of the stimulation electrode do not meet the requirements.

[0032] On the other hand, an embodiment of the present invention provides a computer storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the control method of the visual prosthesis system is executed.

[0033] In the visual prosthesis device in the embodiment of the present invention, the main control module controls the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction received by the first communication module, and the electrode array can not only receive the stimulation pulse signal to release electrical stimulation to the visual neuron tissue, but also receive the analog neuron signal generated by the visual nerve tissue, and then combine the signal amplification and analog-to-digital conversion module to amplify and convert the analog neuron signal to obtain a digital neuron signal, and the main control module can control the first communication module to transmit the digital neuron signal to the outside. The visual prosthesis system including the external control module, the second communication module and the visual prosthesis device can receive and process and analyze the digital neuron signal, and find the optimal stimulation parameters of each electrode of the visual prosthesis device through closed-loop feedback, thereby improving the efficiency and accuracy of obtaining the optimal stimulation parameters of the electrode. In addition, in the control method of the visual prosthesis system, the electrode control instruction instructs the visual prosthesis device to control the stimulation electrode to emit electrical stimulation according to the initial stimulation parameters, and then receives the first digital neuron signal of the electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group, and judges whether the preset conditions are met according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is judged that the preset conditions are met, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrode; if it is judged that the preset conditions are not met, the initial stimulation parameters are modified until the preset conditions are met; the optimal stimulation parameters of the stimulation electrode are obtained, the accuracy and acquisition efficiency of the optimal stimulation parameters of the stimulation electrode are effectively improved, and the development cost of the visual prosthesis device is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0035] Figure 1 is a structural schematic diagram of a visual prosthesis system provided by an embodiment of the present invention;

[0036] Figure 2 is a schematic structural diagram of a visual prosthesis device provided by an embodiment of the present invention;

[0037] Figure 3 is a structural schematic diagram of a visual prosthesis system provided by an embodiment of the present invention;

[0038] Figure 4 is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention;

[0039] Figure 5is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention;

[0040] Figure 6 is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention;

[0041] Figure 7 is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention;

[0042] Figure 8 is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention;

[0043] Fig. 9 is a schematic structural diagram of a control device for a visual prosthesis system provided by an embodiment of the present invention;

[0044] Fig.10 is a schematic structural diagram of a control device for a visual prosthesis system provided by an embodiment of the present invention;

[0045] Fig.11 is a schematic structural diagram of a control device for a visual prosthesis system provided by an embodiment of the present invention;

[0046] Fig.12 It is a structural diagram of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0048] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0049] See also Figure 1 and Figure 2 , Figure 1 is a structural schematic diagram of a visual prosthesis system provided by an embodiment of the present invention, Figure 2: is a structural schematic diagram of a visual prosthesis device provided by an embodiment of the present invention; the visual prosthesis system includes an external control module 13, a second communication module (not shown) and a visual prosthesis device 12, the visual prosthesis device 12 includes a first communication module 201, a main control module 203, a stimulation pulse generation module 206, an electrode array 207 and a signal amplification and analog-to-digital conversion module 205, wherein:

[0050] The in vitro control module 13 is connected to the second communication module, and the second communication module is connected to the first communication module 201. The in vitro control module 13 is used to send electrode control instructions to the first communication module 201, and receive and process digital neuron signals; the first communication module 201 is connected to the main control module 203, and the output end of the main control module 203 is connected to the input end of the stimulation pulse generation module 206, and the output end of the stimulation pulse generation module 206 is connected to the input end of the electrode in the electrode array 207, and the output end of the electrode is connected to the input end of the signal amplification and analog-to-digital conversion module 205, and the output end of the signal amplification and analog-to-digital conversion module 205 is connected to the input end of the main control module 203.

[0051] Specifically, the first communication module is used to receive the electrode control instruction sent by the in vitro control module 13 through the second communication module and transmit the digital neuron signal to the in vitro control module 13 (through the second communication module), wherein the electrode control instruction includes the stimulation parameters of the electrode;

[0052] The main control module is used to control the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction, which may be a stimulation pulse voltage signal or a stimulation pulse current signal;

[0053] The stimulation pulse generating module is used to generate the stimulation pulse signal according to the electrode control instruction;

[0054] The electrodes in the electrode array are used to receive the stimulation pulse signal, release electrical stimulation to the visual nerve tissue to generate phosphenes, and receive simulated neuron signals generated by the visual nerve tissue;

[0055] The signal amplification and analog-to-digital conversion module is used to amplify the analog neuron signal and perform digital processing to obtain the digital neuron signal.

[0056] The visual prosthesis system in the embodiment of the present invention includes an implantable visual prosthesis device and an external control module. The external control module can send electrode control instructions to the visual prosthesis device, and the main control module in the visual prosthesis device controls the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction received by the first communication module. The electrode array can not only receive the stimulation pulse signal to release electrical stimulation to the visual neuron tissue, but also receive the analog neuron signal generated by the visual nerve tissue, and then combine the signal amplification and analog-to-digital conversion module to amplify and convert the analog neuron signal to obtain a digital neuron signal. The main control module can control the first communication module to transmit the digital neuron signal externally. In this embodiment, the simulated neuron signal detected by the electrode array is transmitted to the in vitro control module (through the second communication module); after processing and analyzing the digital neuron signal, the in vitro control module can adjust the electrode control instruction, and finally realize the closed-loop in vivo visual prosthesis device stimulation parameter optimization by the in vitro control module under weak artificial supervision conditions, thereby improving the optimization efficiency and accuracy of the optimal stimulation parameters of the electrode; according to the digital neuron signal recorded by the visual prosthesis device, the stimulation parameter of any electrode in the in vivo electrode array is accurately adjusted, thereby improving the rationality and effectiveness of the electrical stimulation of a single electrode, as well as the accuracy of the electrical stimulation, and reducing the risk of thermal damage to the neural tissue by the electrical stimulation. Since it does not rely on the oral report of the experimental subject, but relies on the objective indicator of the digital neuron signal, large-scale animal in vivo experiments can be carried out on the visual prosthesis device (chip), reducing the development cost of the visual prosthesis device and improving its safety and effectiveness before entering clinical testing.

[0057] Further, refer to Figure 3 , Figure 3 3 is a schematic diagram of the structure of a visual prosthesis system provided by an embodiment of the present invention. The first communication module can be a wired communication module or a wireless communication module. The wireless communication module can be, for example, a WiFi module, a Bluetooth module, a ZigBee module, etc., as long as it can realize wireless or wired communication between the external control module 301 and the visual prosthesis device, and is not particularly limited here. Preferably, the first communication module and the second communication module are wireless communication modules, and wireless transmission is closer to clinical needs.

[0058] Furthermore, refer to Figure 3The main control module includes a first processor 307, a second processor 303 and a data register module 305 including multiple data registers. The output end of the signal amplification and analog-to-digital conversion module 310 is connected to the input end of the first processor 307, the output end of the first processor 307 is connected to the input end of the data register module 305, the data register module 305 is connected to the second processor 303, the second processor 303 is connected to the first communication module, the second processor 303 is used to take out the digital neuron signal stored in the data register 305 and send it to the first communication module, and the first processor 307 is used to store the new digital neuron signal in the vacated data register. Among them, the first processor 307 and the second processor 303 use the ping-pong mechanism to transmit the digital neuron signal to the extracorporeal control module, that is, the second processor 303 transmits the digital neuron signal data in part of the data registers (k2 data registers) in the data register module 305 to the extracorporeal control module, and the first processor 307 stores the newly received digital neuron signal data in the data register that has just been vacated (k1 data register). The ping-pong mechanism is used to realize data access, which can realize storing data in the visual prosthesis device while transmitting the data to the extracorporeal control module. It can not only realize continuous recording and transmission of data and avoid incomplete recording of data, but also reduce the storage capacity of the data register module and reduce the volume of the visual prosthesis device.

[0059] Further, refer to Figure 3The main control module also includes a third processor 304, a parameter register module 302, and a fourth processor 309. The first communication module is connected to the parameter register module 302, the parameter register module 302 is connected to the third processor 304, the third processor 304 is connected to the stimulation pulse generation module 306, the electrodes of the electrode array 308 are connected to the fourth processor 309, the parameter register module 302 is connected to the fourth processor 309, and the fourth processor 309 is connected to the signal amplification and analog-to-digital conversion module 310; wherein the parameter register module 302 is used to store electrode control instructions, electrode control instructions It includes the electrode number of the stimulation electrode (i.e., the electrode of the electrode array that specifies the electrical stimulation to the visual nerve tissue), the stimulation parameters of the stimulation electrode (amplitude, frequency, pulse width, duration), the recording mode parameters (such as delay time and sampling frequency), the parameter register clearing flag, the electrode number of the designated recording electrode group (i.e., the designated signal amplification and analog-to-digital conversion module processes the simulated neuron signal of the electrode with the corresponding electrode number), the delay time refers to the time interval between the stimulation electrode giving the electrical stimulation and the electrode with the designated electrode number starting to record the simulated neuron signal, and the sampling frequency refers to the time interval between the signal amplification and analog-to-digital conversion module processing the simulated neuron signal of the electrode with the designated electrode number. The third processor 304 controls the electrode array 308 to send electrical stimulation to the corresponding number of electrodes according to the electrode number of the stimulation electrode and the corresponding stimulation parameters, and the fourth processor 309 controls the signal amplification and analog-to-digital conversion module 310 to process the simulated neuron signal of the corresponding number of electrodes according to the sampling frequency and the electrode number of the designated recording electrode group according to the designated frequency, and the parameter register clearing flag is used to clear the data stored in the register in the parameter register module 302.

[0060] It is easy to imagine that as long as the processing capability of the adopted processor is strong enough, the main control module can adopt only one processor to implement the functions of the first processor, the second processor, the third processor and the fourth processor.

[0061] Further, refer to Figure 3 The stimulation pulse generation module 306 includes a plurality of sub-stimulation pulse generation modules, the number of which is the same as the number of electrodes of the electrode array 308, and the output end of one of the sub-stimulation pulse generation modules is connected to the input end of one electrode of the electrode array. Since the optimal stimulation parameters of each electrode are different, a sub-stimulation pulse generation module is provided to generate a stimulation pulse signal for one electrode, which can meet the different requirements of electrical stimulation parameters of different electrodes. The sub-stimulation pulse generation module can be implemented by a micro-current stimulator chip, but is not limited to this implementation method.

[0062] Furthermore, refer to Figure 2 and Figure 3 The device also includes a sensor module 311, which includes a temperature sensor 204 for detecting the temperature of the visual nerve tissue 312 and / or an impedance sensor 202 for detecting the electrode impedance of the electrode array 308. The output end of the temperature sensor 204 and the output end of the impedance sensor 202 are respectively connected to the input end of the main control module 203; specifically, the output end of the temperature sensor 204 and the output end of the impedance sensor 202 are respectively connected to the input end of the first processor 307. Similarly, the first processor 307 and the second processor 303 implement storage and transmission of sensor parameters through a ping-pong mechanism, and send the sensor parameters to the in vitro control module 301.

[0063] Furthermore, the visual prosthesis system may also include a display module for displaying information such as digital neuron signals, sensor parameters and optimal stimulation parameters of electrodes, and an information input module for inputting information. The output end of the extracorporeal control module is connected to the input end of the display module, and the output end of the information input module is connected to the input end of the extracorporeal control module. The information input module may be a keyboard, a touch screen, etc., for inputting information such as electrical stimulation parameters.

[0064] In actual applications, when optimizing the electrical stimulation parameters of the electrode, there are three working modes. The first is a weak manual supervision optimization method, in which the extracorporeal control module autonomously completes the electrical stimulation parameter optimization according to a pre-stored program, that is, the extracorporeal control module is used to generate and send electrode control instructions to the first communication module, and receive and process digital neuron signals. The extracorporeal control module processes the digital neuron signals according to a preset program to determine whether the electrical stimulation parameters are the optimized electrical stimulation parameters, and continuously adjusts the electrical stimulation parameters and then makes a judgment to complete the electrical stimulation parameter optimization.

[0065] The second is a scheme in which electrical stimulation parameter optimization is performed completely manually. Medical staff manually input stimulation parameters or images they deem appropriate into the extracorporeal control module based on the digital neuron signals and other information recorded by the designated electrode (group) under any stimulation parameters displayed on the display module. The stimulation parameters or images are then transmitted to the visual prosthesis device through the extracorporeal control module. The medical staff then determines whether the requirements are met based on the digital neuron signals returned by the visual prosthesis device, thereby optimizing the stimulation parameters of the visual prosthesis device under strong supervision.

[0066] The third type is between the first and the second type. Medical staff or experimenters supervise the optimization process of the extracorporeal control module. They can intervene or take over the execution process of the extracorporeal control module at any time according to the digital neuronal signals sent back by the visual prosthesis device. After confirming the status of the visual prosthesis device and the neural tissue, they can choose to perform manual parameter optimization, continue automatic optimization, or start a new automatic optimization.

[0067] It can be seen that the present invention can be implemented by an extracorporeal control module combined with human intervention under conditions of any intensity of human supervision, and has the ability to greatly reduce the burden on medical staff and patients.

[0068] Further, refer to Figure 1 The visual prosthesis system also includes an alarm module 14 for outputting an alarm signal. The output end of the external control module 13 is connected to the input end of the alarm module 14. The alarm module 14 can be an audio-visual alarm module, an optical alarm module, or an acoustic alarm module, which respectively sends out audio-visual alarm signals, optical alarm signals, or acoustic alarm signals. When the external control module 13 determines that the received temperature exceeds the preset temperature (for example, 37 degrees Celsius), it indicates that the temperature of the visual nerve tissue is abnormal; or when the received electrode impedance exceeds the preset impedance, it indicates that the sub-stimulation pulse generation module is abnormal or the electrode is abnormal. At this time, the external control module 13 controls the alarm module 14 to work to prompt relevant personnel that an abnormality has occurred, to avoid permanent damage to the nerve tissue caused by electrical stimulation and heating, and to facilitate timely troubleshooting of the sub-stimulation pulse generation module and electrode.

[0069] It is worth pointing out that the external control module can be a single-chip microcomputer, a computer or other device with processing and control capabilities, and is not specifically limited. For the visual prosthesis device, an external image acquisition device can be configured, and the main control module can process the image or video acquired by the image acquisition device to obtain the stimulation parameters of the control electrode array to stimulate the visual nerve tissue and enable the blind patient to have visual perception.

[0070] Based on the above examples, see Figure 1 , Figure 2 and Figure 4 , Figure 4 : is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention; the control method of the visual prosthesis system is applied to the above-mentioned visual prosthesis system, and is used to determine the optimal stimulation parameters of each electrode in the electrode array, and specifically includes:

[0071] Step S401, sending the electrode control instruction to the visual prosthesis device, the electrode control instruction including the electrode number of the stimulation electrode, the initial stimulation parameter of the stimulation electrode, and the electrode number of the recording electrode group, the electrode control instruction is used to instruct the visual prosthesis device to control the electrodes with corresponding numbers in the electrode array to emit electrical stimulation to the visual nerve tissue according to the electrode number of the stimulation electrode and the initial stimulation parameter;

[0072] Specifically, the stimulation electrode refers to an electrode that emits electrical stimulation to the visual nerve tissue, and the stimulation parameters are parameters used to control the stimulation pulse generation module to generate stimulation pulse signals, including the amplitude, frequency, pulse width, duration, etc. of the stimulation pulse signal; the recording electrode group includes multiple electrodes for detecting the simulated neuron signals generated by the visual nerve tissue, that is, the signal amplification and analog-to-digital conversion module processes the simulated neuron signals of the electrodes with corresponding numbers according to the electrode numbers of the recording electrode group; in addition, the electrode control instructions also include recording mode parameters (such as delay time and sampling frequency) for controlling the time interval for processing the simulated neuron signals, and the specific size of the recording mode parameters can be determined in advance according to the animal in vitro experiment. In this step, the initial stimulation parameters can be automatically generated by the in vitro control module according to the preset program, or the initial stimulation parameters can be input to the in vitro control module by the experimenter or other personnel using the input module.

[0073] Step S402, receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

[0074] Step S403, judging whether a preset condition is met according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is judged that the preset condition is met, taking the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; if it is judged that the preset condition is not met, modifying the initial stimulation parameters until the preset condition is met.

[0075] Specifically, it is determined whether the preset conditions are met according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state, so as to determine the optimal stimulation parameters of the stimulation electrode. In step S403, the initial stimulation parameters can be automatically modified by the in vitro control module until the preset conditions are met, or the initial stimulation parameters can be modified by the experimenter based on the digital neuron signal and other information displayed on the display module and based on experience to meet the preset conditions, or the experimenter can manually supervise and judge the in vitro control module, and when the initial stimulation parameters automatically modified by the in vitro control module are inappropriate, the experimenter can modify the initial stimulation parameters.

[0076] By using the method of the embodiment of the present invention, the optimal stimulation parameters of the stimulation electrode can be obtained, the accuracy and acquisition efficiency of the optimal stimulation parameters of the stimulation electrode can be effectively improved, and the electrical stimulation parameters of any electrode in the in-body electrode array can be accurately adjusted according to the neural electrical signals recorded by the visual prosthesis device, thereby improving the rationality and effectiveness of electrical stimulation of a single electrode, as well as the accuracy of electrical stimulation, and reducing the risk of thermal damage to neural tissue caused by electrical stimulation. Since it does not rely on the oral report of the experimental subject, but relies on the objective indicator of digital neuronal signals, large-scale in-vivo animal experiments can be carried out on the visual prosthesis device, reducing the development cost of the visual prosthesis device and improving its safety and effectiveness before entering clinical testing. The scheme of the present invention can be implemented by an in vitro control module combined with human intervention under artificial supervision of any intensity, and has the ability to greatly reduce the burden on medical staff and patients.

[0077] Furthermore, the electrode numbers of the recording electrode group include the electrode numbers of the first recording electrode group and the electrode numbers of the second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the number of electrodes of the first recording electrode group is a first preset number; the second recording electrode group includes the stimulation electrode, and the number of electrodes of the second recording electrode group is a second preset number. Figure 5 , Figure 5 4 is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention; the step S403 includes:

[0078] Step S501, according to the first digital neuron signal of the first recording electrode group and the second digital neuron signal of each electrode in the first recording electrode group in a resting state, obtaining the number of first electrodes in the first recording electrode group whose first digital neuron signal reaches a first preset threshold, wherein the first preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a first preset multiple;

[0079] Specifically, the first preset number and the second preset number may be the same or different, and may be set as needed. For example, if the first preset number is 5 and the second preset number is 6, then 5 electrodes are selected from the remaining electrodes in the electrode array except the stimulation electrode as the first recording electrode group, assuming that the numbers are A1, A2, A3, A4, and A5, and 6 electrodes are selected from the electrode array including the stimulation electrode as the second recording electrode group, assuming that the numbers are B1, B2, B3, B4, B5, and B6 (one of B1 to B6 is a stimulation electrode). When obtaining the first number of electrodes, a judgment is made based on the first digital neuron signal S1 of A1 and the second digital neuron signal S2 of A1 in a resting state to determine whether S1 reaches a first preset threshold value, which is the product of S2 and a first preset multiple. For A2 to A5, the same judgment as A1 is made, and the number of electrodes whose S1 is greater than or equal to the first preset threshold value is counted, that is, the first number of electrodes is obtained. Among them, the first digital neuron signal may be the number of action potentials or the frequency of the signal.

[0080] Step S502, according to the first digital neuron signal of the second recording electrode group and the second digital neuron signal of each electrode in the second recording electrode group in a resting state, obtaining the number of second electrodes in the second recording electrode group whose first digital neuron signal reaches a second preset threshold, wherein the second preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a second preset multiple;

[0081] Specifically, similar to the first preset number, when obtaining the second number of electrodes, a judgment is made based on the first digital neuron signal S3 of B1 and the second digital neuron signal S4 of B1 in a resting state to determine whether S3 reaches a second preset threshold value, which is the product of S3 and a second preset multiple. For B2 to B6, the same judgment as B1 is made, and the number of electrodes whose S3 is greater than or equal to the first preset threshold value is counted, and the second number of electrodes is obtained. The first preset multiple and the second preset multiple can be the same or different, and can be modified as needed.

[0082] Step S503, obtaining an effective stimulation range by adding the first number of electrodes and the second number of electrodes, and making a judgment based on the effective stimulation range and a preset value. When the effective stimulation range is equal to the preset value, using the initial stimulation parameters as the optimal stimulation parameters of the stimulation electrode; if the effective stimulation range is greater than or less than the preset value, modifying the initial stimulation parameters until the effective stimulation range is equal to the preset value.

[0083] Specifically, if the effective stimulation range exceeds the preset value, the stimulation intensity needs to be reduced (modify the reduction amplitude, frequency, pulse width, and duration of the initial stimulation parameters) until the effective stimulation range is equal to the preset value. If the effective stimulation range does not reach the preset value, the stimulation intensity needs to be increased until the effective stimulation range is equal to the preset value.

[0084] Further, refer to Figure 6 , Figure 6 is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention, the method further comprising:

[0085] Step S601, sending a test instruction to the visual prosthesis device, wherein the test instruction includes an optimal stimulation parameter of a stimulation electrode and a standard test image, wherein the standard test image specifies the stimulation electrode, and the test instruction is used to instruct the visual prosthesis device to project the standard test image onto the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode;

[0086] Specifically, one or more stimulation electrodes are specified in the standard test image, and these stimulation electrodes are controlled to emit electrical stimulation to the visual nerve tissue according to the optimal stimulation parameters of these specified stimulation electrodes. The standard test image includes static and dynamic images. Figure 7 , Figure 7 It is a flow chart of a control method for a visual prosthesis system provided by an embodiment of the present invention; in this embodiment, the standard test image is only black and white, and only the light and dark contrast lines are white. The standard test image is projected into the electrode array space to determine whether the pixel corresponding to each electrode is white or black, wherein the electrode corresponding to the white pixel in the image is calibrated as the stimulation electrode.

[0087] Step S602, receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

[0088] Step S603, performing image reconstruction according to the third digital neuron signal to obtain a perceived image;

[0089] Specifically, the image can be reconstructed by an interpolation reconstruction method to obtain a perceived image.

[0090] Step S604, calculating the image similarity between the standard test image and the perception image;

[0091] Specifically, an image hash algorithm may be used to calculate image similarity, or other methods for calculating image similarity may be used, which are not particularly limited herein.

[0092] Step S605, judging whether the optimal stimulation parameters of the stimulation electrode meet the requirements according to the image similarity and the preset similarity; if the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is less than the preset similarity, the optimal stimulation parameters of the stimulation electrode do not meet the requirements.

[0093] pass Figure 6 The method can verify the optimal stimulation parameters of the stimulation electrode obtained previously to determine whether the optimal stimulation parameters meet the requirements. In order to improve the accuracy of the verification, the verification will be performed based on multiple standard test images. If the image similarity of a preset ratio (e.g., 50%) of the standard test images and the corresponding perception images reaches the preset similarity, it is considered that the optimal stimulation parameters of the stimulation electrode meet the requirements and the electrical stimulation parameters of the in-vivo visual prosthesis device are optimized successfully; otherwise, the optimal stimulation parameters of the stimulation electrode do not meet the requirements, the parameter optimization fails, and new optimal stimulation parameters of the stimulation electrode need to be obtained.

[0094] It is worth noting that in the embodiment of the present invention, after obtaining the optimal stimulation parameters of the stimulation electrode, further fine-tuning can be done on this basis to find the optimal stimulation parameter set of the stimulation electrode. The principle of finding is to ensure that the effective stimulation range is always maintained at the set value. Multiple optimal stimulation parameters corresponding to a certain stimulation electrode can be obtained to form its optimal stimulation parameter set. Then, a table can be established and stored according to the corresponding relationship between the number of the stimulation electrode and the corresponding optimal stimulation parameter set. The control method of the present invention is actually the working process of the in vitro control module, and the above-established table will be retained in the in vitro control module.

[0095] refer to Figure 8 , Figure 8 : is a flow chart of a control method of a visual prosthesis system provided by an embodiment of the present invention; the process of fine-tuning the optimal stimulation parameters is described below:

[0096] use Figure 4After obtaining the effective stimulation range of a stimulation electrode by the method, determine whether the requirements are met according to the effective stimulation range and the preset value. If the requirements are met, the initial stimulation parameters corresponding to the effective stimulation range are saved as the optimal stimulation parameters of the stimulation electrode. Otherwise, new stimulation parameters are set as the initial stimulation parameters, and the effective stimulation range is continued to be obtained for judgment. Fine-tuning is performed based on the saved optimal stimulation parameters, which can be fine-tuning in the direction of increasing the parameters or decreasing the parameters. When the effective stimulation range of the fine-tuned stimulation parameters is equal to the preset value, the fine-tuned stimulation parameters are saved as the optimal stimulation parameters, and the parameter fine-tuning is continued. When the effective stimulation range of the fine-tuned stimulation parameters is not equal to the preset value, the parameter fine-tuning is stopped, and the process of determining the optimal stimulation parameters for the next stimulation electrode is entered. After fine-tuning the optimal stimulation parameters, multiple optimal stimulation parameters corresponding to a stimulation electrode can be obtained to constitute its optimal stimulation parameter set.

[0097] Furthermore, after obtaining the optimal stimulation parameter set corresponding to each electrode in the electrode array, Figure 6 During parameter verification in the corresponding embodiment, an optimal stimulation parameter is taken out from the optimal stimulation parameter set of each electrode for verification. When the verification fails, a new optimal stimulation parameter is taken out from the set for verification. The above-mentioned entire verification process can be constructed as a machine learning method, which runs under weak manual supervision conditions, effectively improving the efficiency of obtaining the optimal stimulation parameters of the stimulation electrode.

[0098] Based on the description of the control method embodiment of the visual prosthesis system, the present invention also discloses a control device for the visual prosthesis system. Fig. 9 , Fig. 9 is a schematic diagram of the structure of a control device for a visual prosthesis system provided by an embodiment of the present invention, Fig. 9 That is, the functional division inside the external control module, the control device of the visual prosthesis system includes a control instruction sending module 901, a signal receiving module 902, and a judgment module 903; wherein:

[0099] A control instruction sending module 901 is used to send the electrode control instruction to the visual prosthesis device, wherein the electrode control instruction includes the electrode number of the stimulation electrode, the initial stimulation parameter of the stimulation electrode, and the electrode number of the recording electrode group, and the electrode control instruction is used to instruct the visual prosthesis device to control the electrodes with corresponding numbers in the electrode array to emit electrical stimulation to the visual nerve tissue according to the electrode number and the initial stimulation parameter of the stimulation electrode;

[0100] A signal receiving module 902 is used to receive a first digital neuron signal of the electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

[0101] The judgment module 903 is used to judge whether the preset conditions are met based on the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is judged that the preset conditions are met, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrode; if it is judged that the preset conditions are not met, the initial stimulation parameters are modified until the preset conditions are met.

[0102] The specific functional implementation of the control instruction sending module 901, the signal receiving module 902, and the judgment module 903 can be found in the above Figure 4 Steps S401 to S403 in the corresponding embodiment are not described in detail here.

[0103] Further, the electrode number of the recording electrode group includes the electrode number of the first recording electrode group and the electrode number of the second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the second recording electrode group includes the stimulation electrode; Fig.10 , Fig.10 is a structural diagram of a control device of a visual prosthesis system provided by an embodiment of the present invention; the judgment module 903 includes a first electrode number acquisition submodule 101, a second electrode number acquisition submodule 102, and a processing submodule 103, wherein:

[0104] A first electrode number acquisition submodule 101 is used to acquire the number of first electrodes in the first recording electrode group whose first digital neuron signal reaches a first preset threshold value according to the first digital neuron signal of the first recording electrode group and the second digital neuron signal of each electrode in the first recording electrode group in a resting state, wherein the first preset threshold value is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a first preset multiple;

[0105] A second electrode number acquisition submodule 102 is used to acquire the number of second electrodes in the second recording electrode group whose first digital neuron signal reaches a second preset threshold value according to the first digital neuron signal of the second recording electrode group and the second digital neuron signal of each electrode in the second recording electrode group in a resting state, wherein the second preset threshold value is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a second preset multiple;

[0106] The processing submodule 103 is used to obtain an effective stimulation range by adding the first number of electrodes and the second number of electrodes, and to make a judgment based on the effective stimulation range and a preset value. When the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, the initial stimulation parameters are modified until the effective stimulation range is equal to the preset value.

[0107] The specific functional implementation of the first electrode number acquisition submodule 101, the second electrode number acquisition submodule 102, and the processing submodule 103 can be found in the above Figure 5 Steps S501 to S503 in the corresponding embodiment are not described in detail here.

[0108] Further, refer to Fig. 9 and Fig.11 , Fig.11 1 is a schematic diagram of the structure of a control device for a visual prosthesis system provided by an embodiment of the present invention, wherein the device further comprises a test instruction sending module 111, a reconstruction module 112, a similarity calculation module 113, and a similarity judgment module 114, wherein:

[0109] A test instruction sending module 111 is used to send a test instruction to the visual prosthesis device, wherein the test instruction includes an optimal stimulation parameter of a stimulation electrode and a standard test image, wherein the standard test image specifies the stimulation electrode, and the test instruction is used to instruct the visual prosthesis device to project the standard test image onto the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode;

[0110] The signal receiving module 902 is further used to receive the third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

[0111] A reconstruction module 112, configured to reconstruct an image according to the third digital neuron signal to obtain a perceived image;

[0112] A similarity calculation module 113 is used to calculate the image similarity between the standard test image and the perception image;

[0113] The similarity judgment module 114 is used to judge whether the optimal stimulation parameters of the stimulation electrode meet the requirements based on the image similarity and the preset similarity. If the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is less than the preset similarity, the optimal stimulation parameters of the stimulation electrode do not meet the requirements.

[0114] The specific functional implementation of the test instruction sending module 111, the signal receiving module 902, the reconstruction module 112, the similarity calculation module 113, and the similarity judgment module 114 can be found in the above Figure 6 Steps S601 to S605 in the corresponding embodiment are not described in detail here.

[0115] It is worth pointing out that Fig. 9 , Fig.10 and Fig.11 The various units or modules in the control device of the visual prosthesis system shown can be combined into one or several other units or modules, or some of the units or modules can be further divided into multiple smaller units or modules, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above units or modules are divided based on logical functions. In practical applications, the functions of one unit (or module) can also be implemented by multiple units (or modules), or the functions of multiple units (or modules) can be implemented by one unit (or module).

[0116] Based on the description of the above method embodiment and apparatus embodiment, an embodiment of the present invention further provides a terminal device.

[0117] See also Fig.12 , is a schematic diagram of the structure of a terminal device provided by an embodiment of the present invention. Fig.12 As shown above Figures 9 to 11 The control device of the visual prosthesis system in the embodiment can be applied to the terminal device 120, and the terminal device 120 may include: a processor 121, a network interface 124 and a memory 125. In addition, the terminal device 120 may also include: a user interface 123, and at least one communication bus 122. The communication bus 122 is used to realize the connection and communication between these components. The user interface 123 may include a display screen (Display) and a keyboard (Keyboard), and the user interface 123 may also include a standard wired interface and a wireless interface. The network interface 124 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 125 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 125 may optionally be at least one storage device located away from the aforementioned processor 121. Fig.12 As shown, the memory 125 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0118] exist Fig.12In the terminal device 120 shown, the network interface 124 can provide a network communication function; the user interface 123 is mainly used to provide an input interface for the user; and the processor 121 can be used to call the device control application stored in the memory 125 to achieve:

[0119] Sending the electrode control instruction to the visual prosthesis device, the electrode control instruction including the electrode number of the stimulation electrode, the initial stimulation parameter of the stimulation electrode, and the electrode number of the recording electrode group, the electrode control instruction is used to instruct the visual prosthesis device to control the electrodes with corresponding numbers in the electrode array to emit electrical stimulation to the visual nerve tissue according to the electrode number of the stimulation electrode and the initial stimulation parameter;

[0120] receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group;

[0121] Whether a preset condition is met is determined based on the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is determined that the preset condition is met, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrode; if it is determined that the preset condition is not met, the initial stimulation parameters are modified until the preset condition is met.

[0122] Further, the electrode number of the recording electrode group includes the electrode number of the first recording electrode group and the electrode number of the second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the second recording electrode group includes the stimulation electrode; in one embodiment, the processor 121 performs the following steps when executing the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state to determine whether a preset condition is met; if it is determined that the preset condition is met, the initial stimulation parameter is used as the optimal stimulation parameter of the stimulation electrode; if it is determined that the preset condition is not met, the initial stimulation parameter is modified until the preset condition is met:

[0123] According to the first digital neuron signal of the first recording electrode group and the second digital neuron signal of each electrode in the first recording electrode group in a resting state, obtaining the number of first electrodes in the first recording electrode group whose first digital neuron signal reaches a first preset threshold, wherein the first preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a first preset multiple;

[0124] According to the first digital neuron signal of the second recording electrode group and the second digital neuron signal of each electrode in the second recording electrode group in a resting state, obtaining the number of second electrodes in the second recording electrode group whose first digital neuron signal reaches a second preset threshold, wherein the second preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a second preset multiple;

[0125] An effective stimulation range is obtained by adding the number of the first electrodes and the number of the second electrodes, and a judgment is made based on the effective stimulation range and a preset value. When the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, the initial stimulation parameters are modified until the effective stimulation range is equal to the preset value.

[0126] In one embodiment, the processor 121 is further configured to perform the following steps:

[0127] Sending a test instruction to the visual prosthesis device, the test instruction including an optimal stimulation parameter of a stimulation electrode and a standard test image, the standard test image designating the stimulation electrode, the test instruction being used to instruct the visual prosthesis device to project the standard test image onto the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode;

[0128] receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device;

[0129] Reconstructing an image according to the third digital neuron signal to obtain a perceived image;

[0130] Calculating image similarity between the standard test image and the perception image;

[0131] Whether the optimal stimulation parameters of the stimulation electrode meet the requirements is judged according to the image similarity and the preset similarity; if the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is less than the preset similarity, the optimal stimulation parameters of the stimulation electrode do not meet the requirements.

[0132] It should be understood that the terminal device 120 described in the embodiment of the present invention can execute the above Figures 4 to 8 The description of the control method of the visual prosthesis system in the corresponding embodiment can also be performed as described above. Figures 9 to 11 The description of the control device of the visual prosthesis system in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0133] In addition, it should be pointed out here that: an embodiment of the present invention further provides a computer storage medium, and the computer storage medium stores a computer program executed by the control device of the visual prosthesis system mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the above-mentioned Figures 4 to 8 The description of the control method of the visual prosthesis system in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in the present invention, please refer to the description of the method embodiment of the present invention.

[0134] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0135] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A visual prosthesis device, characterized in that: The device comprises a first communication module, a main control module, a stimulation pulse generation module, an electrode array, and a signal amplification and analog-to-digital conversion module, wherein the first communication module is connected to the main control module, the output end of the main control module is connected to the input end of the stimulation pulse generation module, the output end of the stimulation pulse generation module is connected to the input end of the electrode in the electrode array, the output end of the electrode is connected to the input end of the signal amplification and analog-to-digital conversion module, and the output end of the signal amplification and analog-to-digital conversion module is connected to the input end of the main control module; wherein: The first communication module is used to receive electrode control instructions and transmit digital neuron signals; the electrode control instructions include the electrode number of the stimulation electrode, the initial stimulation parameters of the stimulation electrode, and the electrode number of the recording electrode group, and the electrode control instructions are used to instruct the visual prosthesis device to control the electrodes with corresponding numbers in the electrode array to emit electrical stimulation to the visual nerve tissue according to the electrode number and the initial stimulation parameters of the stimulation electrode; The first communication module is also used to send a first digital neuron signal of the electrode corresponding to the electrode array returned according to the electrode number of the recording electrode group; The main control module is used to control the stimulation pulse generation module to generate a stimulation pulse signal according to the electrode control instruction; the main control module is also used to judge whether a preset condition is met according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is judged that the preset condition is met, the initial stimulation parameter is used as the optimal stimulation parameter of the stimulation electrode; if it is judged that the preset condition is not met, the initial stimulation parameter is modified until the preset condition is met; The electrode numbers of the recording electrode group include the electrode numbers of the first recording electrode group and the electrode numbers of the second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the second recording electrode group includes the stimulation electrode; The step of judging whether a preset condition is satisfied according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset condition is satisfied, taking the initial stimulation parameter as the optimal stimulation parameter of the stimulation electrode; and if the preset condition is not satisfied, modifying the initial stimulation parameter until the preset condition is satisfied, comprises: According to the first digital neuron signal of the first recording electrode group and the second digital neuron signal of each electrode in the first recording electrode group in a resting state, obtaining the number of first electrodes in the first recording electrode group whose first digital neuron signal reaches a first preset threshold, wherein the first preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a first preset multiple; According to the first digital neuron signal of the second recording electrode group and the second digital neuron signal of each electrode in the second recording electrode group in a resting state, obtaining the number of second electrodes in the second recording electrode group whose first digital neuron signal reaches a second preset threshold, wherein the second preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a second preset multiple; An effective stimulation range is obtained by adding the first number of electrodes and the second number of electrodes, and a judgment is made based on the effective stimulation range and a preset value. When the effective stimulation range is equal to the preset value, the initial stimulation parameter is used as the optimal stimulation parameter of the stimulation electrode; if the effective stimulation range is greater than or less than the preset value, the initial stimulation parameter is modified until the effective stimulation range is equal to the preset value; The stimulation pulse generating module is used to generate the stimulation pulse signal according to the electrode control instruction; The electrodes in the electrode array are used to receive the stimulation pulse signal, release electrical stimulation to the visual nerve tissue to generate phosphenes, and receive simulated neuron signals generated by the visual nerve tissue; The signal amplification and analog-to-digital conversion module is used to amplify the analog neuron signal and perform digital processing to obtain the digital neuron signal.

2. The device according to claim 1, characterized in that The main control module includes a first processor, a second processor and a data register module including multiple data registers. The output end of the signal amplification and analog-to-digital conversion module is connected to the input end of the first processor, the output end of the first processor is connected to the input end of the data register module, the data register module is connected to the second processor, and the second processor is connected to the first communication module. The second processor is used to take out the digital neuron signal stored in the data register and send it to the first communication module, and the first processor is used to store the new digital neuron signal in the vacated data register.

3. The device according to claim 1, characterized in that The stimulation pulse generation module includes multiple sub-stimulation pulse generation modules, the number of the sub-stimulation pulse generation modules is the same as the number of electrodes in the electrode array, and the output end of one of the sub-stimulation pulse generation modules is connected to the input end of one electrode of the electrode array.

4. The device according to claim 1, characterized in that The device also includes a temperature sensor for detecting the temperature of the visual nerve tissue and / or an impedance sensor for detecting the electrode impedance of the electrode array, and the output end of the temperature sensor and the output end of the impedance sensor are respectively connected to the input end of the main control module.

5. The device according to any one of claims 1 to 4, characterized in that: The first communication module is further used to receive a test instruction, the test instruction including an optimal stimulation parameter of a stimulation electrode and a standard test image, the standard test image specifies the stimulation electrode, and the test instruction is used to instruct the visual prosthesis device to project the standard test image onto the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode; Sending a third digital neuron signal to each electrode of the electrode array corresponding to the test instruction; The third digital neuron signal is used to perform image reconstruction to obtain a perceived image, so as to calculate the image similarity between the standard test image and the perceived image; and based on the image similarity and the preset similarity, it is judged whether the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is less than the preset similarity, the optimal stimulation parameters of the stimulation electrode do not meet the requirements.

6. A visual prosthesis system, characterized in that: It comprises an extracorporeal control module, a second communication module and the visual prosthesis device according to any one of claims 1 to 5, wherein the extracorporeal control module is connected to the second communication module, the second communication module is connected to the first communication module, and the extracorporeal control module is used to send the electrode control instruction to the first communication module, and receive and process the digital neuron signal.

7. The visual prosthesis system according to claim 6, characterized in that: The visual prosthesis system further comprises an alarm module for outputting an alarm signal, and an output end of the external control module is connected to an input end of the alarm module.

8. A computer storage medium, characterized in that The computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, a control method of a visual prosthesis system is executed, wherein the control method of the visual prosthesis system includes: Sending an electrode control instruction to the visual prosthesis device, wherein the electrode control instruction includes the electrode number of the stimulation electrode, the initial stimulation parameter of the stimulation electrode, and the electrode number of the recording electrode group, and the electrode control instruction is used to instruct the visual prosthesis device to control the electrodes with corresponding numbers in the electrode array to emit electrical stimulation to the visual nerve tissue according to the electrode number of the stimulation electrode and the initial stimulation parameter; receiving a first digital neuron signal of an electrode corresponding to the electrode array returned by the visual prosthesis device according to the electrode number of the recording electrode group; Determining whether a preset condition is met according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if it is determined that the preset condition is met, using the initial stimulation parameter as the optimal stimulation parameter of the stimulation electrode; if it is determined that the preset condition is not met, modifying the initial stimulation parameter until the preset condition is met; The electrode numbers of the recording electrode group include the electrode numbers of the first recording electrode group and the electrode numbers of the second recording electrode group, the first recording electrode group does not include the stimulation electrode, and the second recording electrode group includes the stimulation electrode; The step of judging whether a preset condition is satisfied according to the first digital neuron signal of the recording electrode group and the second digital neuron signal of each electrode in the recording electrode group in a resting state; if the preset condition is satisfied, taking the initial stimulation parameter as the optimal stimulation parameter of the stimulation electrode; and if the preset condition is not satisfied, modifying the initial stimulation parameter until the preset condition is satisfied, comprises: According to the first digital neuron signal of the first recording electrode group and the second digital neuron signal of each electrode in the first recording electrode group in a resting state, obtaining the number of first electrodes in the first recording electrode group whose first digital neuron signal reaches a first preset threshold, wherein the first preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a first preset multiple; According to the first digital neuron signal of the second recording electrode group and the second digital neuron signal of each electrode in the second recording electrode group in a resting state, obtaining the number of second electrodes in the second recording electrode group whose first digital neuron signal reaches a second preset threshold, wherein the second preset threshold is the product of the second digital neuron signal of the electrode corresponding to the first digital neuron signal and a second preset multiple; An effective stimulation range is obtained by adding the number of the first electrodes and the number of the second electrodes, and a judgment is made based on the effective stimulation range and a preset value. When the effective stimulation range is equal to the preset value, the initial stimulation parameters are used as the optimal stimulation parameters of the stimulation electrodes; if the effective stimulation range is greater than or less than the preset value, the initial stimulation parameters are modified until the effective stimulation range is equal to the preset value.

9. The computer storage medium according to claim 8, wherein: The control method of the visual prosthesis system also includes: Sending a test instruction to the visual prosthesis device, the test instruction including an optimal stimulation parameter of a stimulation electrode and a standard test image, the standard test image designating the stimulation electrode, the test instruction being used to instruct the visual prosthesis device to project the standard test image onto the electrode array and drive the stimulation electrode according to the optimal stimulation parameter of the stimulation electrode; receiving a third digital neuron signal of each electrode of the electrode array corresponding to the test instruction returned by the visual prosthesis device; Reconstructing an image according to the third digital neuron signal to obtain a perceived image; Calculating image similarity between the standard test image and the perception image; Whether the optimal stimulation parameters of the stimulation electrode meet the requirements is judged according to the image similarity and the preset similarity; if the image similarity is greater than or equal to the preset similarity, the optimal stimulation parameters of the stimulation electrode meet the requirements; if the image similarity is less than the preset similarity, the optimal stimulation parameters of the stimulation electrode do not meet the requirements.

Images