Implantable retina photon nerve repairing method and device

Abstract

The invention provides an implantable retina photon nerve repairing method and device, and relates to the field of implantable nerve regulation and control. The method comprises the following steps: acquiring a target image in a current visual field of a user based on external wearing equipment; adopting an edge detection algorithm to extract visual key features in the target image; based on the visual key features, time control pulse coding is adopted to generate a target stimulation pattern; performing neural feedback acquisition based on the target stimulation pattern, and acquiring neural electrical activity information through a feedback electrode preset on an implanted chip; based on the neural electrical activity information, the target stimulation pattern is dynamically adjusted through an adaptive optimization algorithm to perform optic nerve repair. The problem that a traditional implantable retina photon nerve repairing method is poor in retina photon nerve repairing effect is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of implantable neuromodulation, and in particular to an implantable retinal photonic nerve repair method and device. BACKGROUND

[0002] Electrical stimulation retinal prosthesis (such as Argus II) directly injects current into retinal neurons using an electrode array, and its spatial resolution is limited by electrode size, electrode spacing and current diffusion effect. Although the number of stimulation points can be increased in physical structure by densifying the electrode array, the spatial diffusion of current propagation in biological tissue inevitably causes blurred activation area, and point-to-point precise stimulation cannot be achieved.

[0003] In addition, most traditional electrical stimulation systems are open-loop control structures, that is, a fixed set of stimulation parameters is preset and output periodically, and a closed-loop coupling path between stimulation and neural response cannot be established.

[0004] Therefore, the traditional implantable retinal photonic nerve repair method has the problems of poor retinal photonic nerve repair effect, on the one hand, it is difficult to balance biological safety at high stimulation resolution, on the other hand, it lacks a real-time feedback mechanism to obtain neural response, limiting the adaptive optimization ability of stimulation effect.

[0005] Therefore, there is an urgent need for an implantable retinal photonic nerve repair method and device. SUMMARY

[0006] The present application provides an implantable retinal photonic nerve repair method and device, which solves the problems of poor retinal photonic nerve repair effect of the traditional implantable retinal photonic nerve repair method, on the one hand, it is difficult to balance biological safety at high stimulation resolution, on the other hand, it lacks a real-time feedback mechanism to obtain neural response, limiting the adaptive optimization ability of stimulation effect.

[0007] In a first aspect of the present application, an implantable retinal photonic nerve repair method is provided, the method comprising: acquiring a target image in the current visual field of a user based on an external wearable device; extracting visual key features in the target image using an edge detection algorithm; generating a target stimulation pattern using time-controlled pulse coding based on the visual key features; collecting neural feedback based on the target stimulation pattern, and acquiring neural electrical activity information through a feedback electrode on a pre-set implant chip; and dynamically adjusting the target stimulation pattern based on the neural electrical activity information to repair the optic nerve through an adaptive optimization algorithm.

[0008] Optionally, the target image in the current visual field of the user is acquired based on the external wearable device: the current visual field of the user is acquired in real time based on the external wearable device, if it is confirmed that the visual perception area exists in the current visual field of the user, a plurality of to-be-confirmed images in the current visual field of the user are acquired; the preliminary visual features corresponding to the plurality of to-be-confirmed images are acquired through image preprocessing operations; the number of preliminary visual features corresponding to each to-be-confirmed image is counted respectively, and the plurality of to-be-confirmed images are sorted in descending order of the number; the to-be-confirmed images greater than a preset ranking threshold in the sorting result are taken as the target image.

[0009] Optionally, the visual key features in the target image are extracted by using an edge detection algorithm, and specifically include: inputting the target image into a convolutional neural network model, and outputting a denoising image and an edge feature image based on the convolutional neural network model; inputting the denoising image and the edge feature image into a U-Net structure model, and outputting a semantic segmentation result based on the U-Net structure model; performing a region marking operation on the target image based on the semantic segmentation result, and generating a spatial saliency mask corresponding to the target image; and outputting the visual key features in the target image by a semantic feature fusion operation based on the spatial saliency mask, the visual key features including contour features, dynamic object features and contrast boundary features.

[0010] Optionally, based on the visual key features, a target stimulation pattern is generated by using time-controlled pulse coding, and specifically includes: based on the visual key features, a target stimulation pattern is generated by using time-controlled pulse coding, the time-controlled pulse coding including spatial coding, intensity regulation and time modulation; the spatial coding is used for selectively activating a specific VCSEL unit; the intensity regulation is used for controlling the light intensity output by each unit; and the time modulation is used for setting the emission pulse width according to the image motion information.

[0011] Optionally, neural feedback collection is performed based on the target stimulation pattern, and neural electrical activity information is acquired through a feedback electrode on a preset implanted chip, and specifically includes: neural feedback collection is performed based on the target stimulation pattern, and original neural signals are acquired through a feedback electrode on a preset implanted chip; the original signals are denoised by using band-pass filtering and wavelet transform, and denoised neural signals are acquired; whether the target stimulation pattern produces an effective neural response is determined by an adaptive feature extraction network based on the denoised neural signals; if the target stimulation pattern produces an effective neural response, the denoised neural signals are taken as the neural electrical activity information.

[0012] Optionally, based on the neural electrical activity information, the target stimulation pattern is dynamically adjusted by an adaptive optimization algorithm for optic nerve repair, specifically comprising: based on the neural electrical activity information, calculating a stimulation effectiveness score corresponding to the target stimulation pattern by a pre-set feature evaluation function; inputting the stimulation effectiveness score into a closed-loop control algorithm model, the closed-loop control algorithm model adjusting the photon array parameters based on the stimulation effectiveness score; based on the adjusted photon array parameters, dynamically adjusting the target stimulation pattern for optic nerve repair.

[0013] Optionally, adjusting the photon array parameters based on the stimulation effectiveness score, specifically comprising: constructing a stimulation-response mapping relationship based on the stimulation effectiveness score and historical neural electrical activity information; the stimulation-response mapping relationship is used to describe the corresponding relationship between the photon stimulation control parameters of each photon emission unit in the target stimulation pattern and the neural response induced by the target stimulation pattern at the corresponding position; based on the stimulation-response mapping relationship, obtaining the photon emission units that fail to induce effective neural response in the target stimulation pattern; based on the spatial position of the photon emission unit and the visual key feature corresponding to the photon emission unit, adjusting the photon array parameters, the photon array parameters including the spatial layout parameters, intensity parameters and timing parameters of photon emission.

[0014] In a second aspect of the present application, an implantable retinal photon neuro-repair device is provided, the device comprising an external intelligent device module, a biological interface structure module, and an implant chip module, wherein, The external intelligent device module is configured to acquire a target image in the current field of view of a user based on an external wearable device.

[0015] The implant chip module is configured to extract visual key features in the target image using an edge detection algorithm, and generate a target stimulation pattern using time-controlled pulse coding based on the visual key features.

[0016] The biological interface structure module is configured to collect neural feedback based on the target stimulation pattern, and acquire neural electrical activity information through a feedback electrode on the pre-set implant chip.

[0017] The implant chip module is further configured to dynamically adjust the target stimulation pattern by an adaptive optimization algorithm for optic nerve repair based on the neural electrical activity information.

[0018] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface, and a network interface, the memory being configured to store instructions, the user interface and the network interface being configured to communicate with other devices, and the processor being configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of the above aspects.

[0019] In a fourth aspect of the present application, a computer-readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to perform the method of any one of the above.

[0020] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Based on an external wearable device, a target image in the current field of view of a user is acquired; an edge detection algorithm is used to extract visual key features in the target image; based on the visual key features, a time-controlled pulse coding is used to generate a target stimulation pattern; based on the target stimulation pattern, a neural feedback collection is performed, and a neural electrical activity information is acquired through a feedback electrode on a preset implant chip; based on the neural electrical activity information, a self-adaptive optimization algorithm is used to dynamically adjust the target stimulation pattern for optic nerve repair, thereby the first aspect uses a time-controlled pulse coding strategy driven by visual key features to achieve light stimulation control with micron-level spatial resolution, which significantly reduces the spatial blur problem caused by current diffusion in traditional electrical stimulation, and the second aspect uses a low-impedance feedback electrode structure on the implant chip and a neural response recognition mechanism based on wavelet transform and adaptive network to acquire neural electrical activity caused by stimulation in real time, realizes periodic reinforcement and spatial structure reconstruction of the stimulation strategy, breaks through the bottleneck that the traditional open-loop scheme cannot adapt to the dynamic changes of the nerve, and significantly improves the stability of the stimulation effect and the adaptability of individual response.

[0021] 2. The target image is input into a convolutional neural network model, and a denoising image and an edge feature image are output based on the convolutional neural network model; the denoising image and the edge feature image are input into a U-Net structure model, and a semantic segmentation result is output based on the U-Net structure model; a region marking operation is performed on the target image based on the semantic segmentation result, and a spatial saliency mask corresponding to the target image is generated; based on the spatial saliency mask, a visual key feature in the target image is output through a semantic feature fusion operation, thereby realizing multi-level joint modeling of contour structure, dynamic objects and contrast boundaries in the target image, and obtaining visual key feature expression with spatial positioning accuracy and semantic discrimination ability.

[0022] 3. The method comprises the following steps: collecting neural feedback based on the target stimulation pattern, and obtaining neural electrical activity information through the feedback electrode on the preset implant chip, specifically including: collecting neural feedback based on the target stimulation pattern, and obtaining original neural signals through the feedback electrode on the preset implant chip; using band-pass filtering and wavelet transform to denoise the original signals and obtain denoised neural signals; based on the denoised neural signals, determining whether the target stimulation pattern produces effective neural response through the adaptive feature extraction network; if the target stimulation pattern produces effective neural response, the denoised neural signals are used as neural electrical activity information, thereby enhancing the adaptability of the stimulation scheme to the individual neural pathway state and improving the long-term stability and response consistency of the optic nerve repair effect. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of an implantable retinal photonic nerve repair method provided by an embodiment of the present application; Figure 2 is a module schematic diagram of an implantable retinal photonic nerve repair device provided by an embodiment of the present application; Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0024] Legend of reference signs: 21, external intelligent device module; 22, biological interface structure module; 23, implant chip module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0025] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0026] The terms used in the following embodiments of the present application are only for the purpose of describing the specific embodiments, and are not intended to be limiting to the present application. As used in the specification of the present application, the singular expression "one", "a", "said", "the above", "the" and "this" are intended to also include the plural expression, unless there is clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application means any or all possible combinations of one or more listed items.

[0027] Hereinafter, the terms "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" are only used for descriptive purpose and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0028] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings.

[0029] Please refer to Figure 1 It shows a flowchart of an implantable retinal photonic nerve repair method provided by the embodiments of the present application, and the flowchart mainly includes the following steps: S101 to S105.

[0030] Step S101, acquiring a target image in the current field of view of the user based on an external wearable device.

[0031] Specifically, in the visual repair scene involved in the present application, the user's own photoreceptor function has been severely degraded or lost, but the bipolar cells, ganglion cells and optic nerve pathways in the retina still have partial physiological plasticity, which has the basis for rebuilding the information transmission path through external stimulation. The present application realizes a complete closed loop of perception-coding-stimulation-feedback-regulation through the cooperation of the implanted photon stimulation chip and the external visual acquisition device, thereby replacing the "incident light-photoreceptor response" stage in the natural visual pathway. In this step, the external wearable device serves as the perception interface between the user and the implanted chip, replacing the degraded or failed natural photoreception mechanism and undertaking the front-end task of artificial visual input. Among them, the external wearable device is a wearable smart glasses integrating a high-resolution camera and an image processing unit, which is worn on the user's head and faces the user's current gaze direction, and is used to continuously acquire visual scene images in the environment where the user is located. The external wearable device integrates an external camera, which can realize an image acquisition frame rate of not less than 60 frames per second to ensure the continuity and real-time of the image under dynamic visual field changes, adapting to the requirements of motion target recognition and visual feature extraction in the present application. The images obtained by the external camera not only include static backgrounds and stationary objects, but also cover moving targets, boundary contours, color contrast regions and other key visual elements within the user's line of sight. After preliminary noise reduction and preprocessing, the images are used as the original input of the subsequent visual coding and neural stimulation process. The external wearable device also integrates a wireless energy transmitter for providing stable remote energy support for the implanted chip. The wireless energy transmitter uses 13.56 MHz radio frequency transmission technology, which can establish a near-field coupling channel between the set position behind the patient's ear and the receiving coil of the implanted chip, and realize continuous energy injection under the premise of ensuring tissue safety; the energy transmitter and the image processing module work together to start the energy transmission link synchronously in the image acquisition and stimulation control process, ensuring that the VCSEL photon array and the feedback electrode in the chip have sustained power supply capability under high-frequency operation, thereby ensuring the running stability of the entire implanted system and the energy closed-loop support of the stimulation response in the high-frame-rate neural regulation period.

[0032] The image sequence obtained in this step will enter the image screening mechanism, and the number of preliminary visual features in the image is calculated to sort a plurality of candidate images, and the images with rich features, clear structure and obvious interest area are preferentially selected as target images. The target image is used to construct a photon stimulation pattern in the subsequent step, to drive the photon array in the implanted chip to realize high-precision activation of retinal neurons, thereby realizing the replacement and reconstruction of the optic nerve function. Therefore, step S101 plays a basic role in the front-end perception and high-value image selection in the entire visual information closed loop, directly affecting the spatial structure and timing quality of the neural stimulation signal. In the embodiments of the present application, In a possible implementation, step S101 further includes: acquiring a current field of view of the user in real time based on the external wearable device, if it is confirmed that there is a visual perception area in the current field of view of the user, acquiring a plurality of to-be-confirmed images in the current field of view of the user, acquiring preliminary visual features corresponding to the plurality of to-be-confirmed images respectively through image preprocessing operations, respectively counting the number of preliminary visual features corresponding to each to-be-confirmed image, and sorting the plurality of to-be-confirmed images in descending order of the number, and taking the to-be-confirmed images greater than a preset ranking threshold in the sorting result as target images.

[0033] Specifically, a continuous current field of view image frame of the user is acquired in real time from the external wearable device. The external wearable device is an integrated smart glasses structure, integrating a high-resolution camera module with a front optical axis, an edge processor unit and an image buffer module, and can collect an environmental image in a direction facing by the user at a sampling frequency of not less than 60 frames per second, and simultaneously perform image frame buffering and preliminary screening. After receiving the current field of view image frame sequence, the system first performs inter-frame motion vector analysis and optical flow sparse detection, for confirming whether there is a visual perception area in the image. The visual perception area includes but is not limited to a moving target area, a contrast mutation boundary area or a structural change aggregation area. If a continuous area block satisfying a feature gradient threshold is detected in the current image frame, it is determined that the field of view corresponding to the frame image contains an effective visual perception area.

[0034] After the above conditions are met, a plurality of to-be-confirmed images are selected from the near time sequence frames to enter an image quality evaluation process. The image quality evaluation process includes performing an image preprocessing operation on each to-be-confirmed image. The image preprocessing operation includes: image denoising based on a nonlinear bilateral filter, Laplace enhancement for improving edge definition, and local contrast optimization based on adaptive histogram equalization, to ensure the stability and robustness of the subsequent feature extraction process. For each preprocessed image, a visual feature calculation operation is further performed to extract the number of edge pixel points, the structural tensor response value, the significant area area ratio and the high-frequency information distribution density, respectively, to form a preliminary visual feature index set.

[0035] Then, the system counts the preliminary visual feature index sets of the plurality of images, takes the number of visual features extracted in each image as a sorting index, constructs a priority sequence of the to-be-confirmed images, sorts all to-be-confirmed images in descending order of the number of visual features, and determines the image frames ranked above a preset threshold in the sorting result as the current optimal target image. The image is taken as a representative image of the current visual input into a subsequent key feature extraction and target stimulation pattern generation module, to ensure that the input image for photon stimulation coding has sufficient structural complexity, edge definition and neural perceptual characteristics, thereby improving the accuracy of stimulation area positioning and the effectiveness of subsequent neural response.

[0036] Step S102, the edge detection algorithm is used to extract the visual key features in the target image.

[0037] Specifically, the target image is input into a preset image feature extraction process, a multi-scale edge detection algorithm is used to identify the regions with significant gray scale changes in the image, and a non-maximum suppression and threshold judgment mechanism are combined to extract contour lines, contrast boundaries and local structure edges, which are used to represent the spatial distribution characteristics of the visual key regions.

[0038] In one possible implementation, step S102 further includes: inputting the target image into a convolutional neural network model, and outputting a denoised image and an edge feature image based on the convolutional neural network model; inputting the denoised image and the edge feature image into a U-Net structure model, and outputting a semantic segmentation result based on the U-Net structure model; performing a region marking operation on the target image based on the semantic segmentation result, and generating a spatial saliency mask corresponding to the target image; and outputting the visual key features in the target image through a semantic feature fusion operation based on the spatial saliency mask, wherein the visual key features include contour features, dynamic object features and contrast boundary features. Specifically, the key visual information in the target image is processed in multiple stages through a multi-level neural network structure driven by deep learning, and image denoising, edge extraction, semantic analysis and saliency modeling operations are fused to generate a structured visual key feature set, so as to improve the spatial resolution of the stimulation pattern construction and the consistency of the biological response.

[0039] First, the target image is input into a pre-trained convolutional neural network (CNN) model , wherein h represents the height dimension of the input image or feature map, w represents the width dimension of the input image or feature map, low-order texture features are extracted through a shallow convolution kernel, shape and edge structure responses are extracted through a deep convolution kernel, and a denoised image and an edge feature image are output. Wherein:

[0040] Each convolution layer in the CNN structure performs the following operations:

[0041] wherein, is the convolution kernel weight of the i-th layer, is the bias term, is the ReLU activation function, denotes the convolution operation, denotes the i-th layer in the convolutional neural network, ​Input feature map of the layer.

[0042] Subsequently, With Concatenate along the channel dimension to the fusion feature map , and input to the U-Net structure model to perform multi-scale context semantic segmentation. U-Net adopts an encoder-decoder architecture, captures multi-scale semantic representation in the encoding stage, restores spatial accuracy in the decoding stage through jump connection, and finally outputs semantic segmentation results , where is the number of semantic categories. The process is:

[0043] Each down-sampling module in U-Net is implemented by convolution, batch normalization and max-pooling:

[0044] where, represents the feature map output of the layer in the U-Net encoder or decoder, and the semantic segmentation result contains the probability distribution of each pixel in the target image belonging to the contour area, dynamic object or contrast boundary, based on which pixel-level region labeling operation is performed to extract the foreground salient region and generate a spatial saliency mask , defined as:

[0045] where, represents the predicted probability value of the row column pixel in the image in the th semantic category in the U-Net semantic segmentation output, on the basis of which semantic feature fusion operation is performed, and a multi-channel attention mechanism is used to fuse with the original feature map to obtain the visual key feature map :

[0046] where, is a channel weighted attention mapping function, for example, a Squeeze-and-Excitation structure can be used, which operates as:

[0047] where, represents the feature map feature maps outputted after applying the squeeze-and-excitation attention mechanism, is a global average pooling operation, , , is a ReLU activation function, is a Sigmoid function, is a compression rate.

[0048] the final output visual key feature As a high-dimensional representation of the fusion of structural edges, semantic saliency, and dynamic perception features, it has clear spatial positioning attributes and semantic pointing properties, and is used for subsequent spatial coding, intensity regulation, and time modulation configuration of target stimulation patterns. In the embodiments of the present application, the micro photonic array (5x5 μm² / unit, 470-630 nm dual band) is the core photonic emission structure of the implanted chip, which is composed of two-dimensional arrangement of vertical cavity surface emitting laser (VCSEL) units. The unit area is small, and high-density distribution can be achieved in a limited retinal implant area; each photonic emission unit has independent spatial activation, light intensity regulation, and wavelength selection capability, supporting on-demand output of 470 nm or 630 nm band laser, which is used to activate ganglion cells and bipolar cells respectively, to realize selective stimulation of neuron types; the array is one-to-one corresponding to the target stimulation pattern matrix in spatial distribution, so that the visual key feature can be accurately mapped to the photonic activation behavior, and then a high-resolution, low-diffusion, and strong-targeted photonic stimulation path is formed. The flexible polyimide substrate is the carrier material of the micro photonic array chip in the present application, which has excellent mechanical compliance and biological compatibility, and can be attached to the surface of the retinal curved structure for a long time without causing foreign body rejection reaction. The bending radius of the polyimide substrate matches the natural curvature (about 12 mm) of the posterior pole of the eyeball of the human eye, which can realize stable fitting after implantation and avoid stimulation deviation caused by micro displacement; at the same time, the substrate has high thermal stability and excellent electrical insulation performance, which can carry densely distributed VCSEL units, electrode traces and passive electronic components, and maintain the integrity and working reliability of the overall structure of the chip. While maintaining the lightness and thinness, the polyimide substrate realizes the layout of the porous structure through the micro processing process, which helps to maintain the exchange of nutrients and the stability of the local tissue physiological environment at the retinal-chip interface, and is the key material basis for realizing the implantable high-resolution photonic stimulation function. The porous silicon nitride packaging layer as a key biological interface structure on the surface of the implanted chip covers the flexible polyimide substrate and the micro photonic array, which is used to ensure the stability of the device function while realizing the physiological environment compatibility between the chip and the retinal tissue. The packaging layer uses a nanoscale silicon nitride material with a thickness of 200 nm and a pore size of about 0.8 μm, which is constructed by low-temperature plasma enhanced chemical vapor deposition process (PECVD), and has excellent chemical inertness, electrical insulation and tissue biocompatibility.

[0049] Step S103, generating the target stimulation pattern based on the visual key features, using time-controlled pulse coding.

[0050] Specifically, generating the target stimulation pattern based on the visual key features, using time-controlled pulse coding, which includes spatial coding, intensity modulation, and time modulation; spatial coding is used to selectively activate specific VCSEL units; intensity modulation is used to control the output light intensity of each unit; time modulation is used to set the emission pulse width according to image motion information.

[0051] Perform spatial coding operation: project the corresponding active region position in the visual key features to the two-dimensional activation matrix corresponding to the VCSEL photon emission array. Let the two-dimensional stimulation matrix be , where is the number of photon arrays in the single-axis direction, and each matrix unit corresponds to a VCSEL unit on the chip; for feature region pixel points that meet the activation threshold, their corresponding positions in are set to 1, indicating that the unit will emit photons in this stimulation period. This operation is equivalent to performing the following mapping:

[0052] , where is the activation region index set determined by the visual key features.

[0053] Perform intensity modulation operation: for the activated VCSEL units that have been marked, calculate the intensity indicator of the visual feature response in the region covered by them, combine pixel gradient, feature saliency score, etc. Parameters to determine the output light intensity value. The output light intensity is controlled between mW / mm², forming a light intensity matrix of the same dimension as , where:

[0054] The function represents the light intensity estimation function based on image local gradient or feature importance mapping, is the projection region center position corresponding to the unit of the original image coordinates.

[0055] Then perform the time modulation operation: construct the pulse control matrix to configure the emission pulse width for each activated unit. The pulse width is determined according to the motion information of the target object in the image, such as the motion speed vector obtained by optical flow estimation or time sequence edge difference , whose amplitude is mapped to the time interval ms, is defined as follows:

[0056] wherein is a mapping function of velocity to pulse duration with nonlinear compression characteristics for maintaining short-time stimulation response of fast targets.

[0057] The above three groups of parameter matrices , , jointly constitute a complete target stimulation pattern control template, which defines the spatial activation state, output light intensity and emission timing of each VCSEL unit in the chip. The template will be transmitted to the implanted chip in the subsequent steps to drive the VCSEL array to perform spatially selective, light intensity adaptive and timing matched photonic emission operations, and finally achieve precise activation of retinal neurons. Through the time-controlled pulse coding strategy, the target stimulation pattern can fully match the spatial structure and dynamic changes of visual key features, ensuring the effectiveness of neural response and the integrity of visual information transmission.

[0058] In step S104, neural feedback acquisition is performed based on the target stimulation pattern, and neural electrical activity information is acquired through feedback electrodes on the preset implanted chip.

[0059] Specifically, the target stimulation pattern is transmitted to a photonic array chip implanted on the surface of the retina, and each photonic emission unit is controlled to emit photons at a preset spatial position, output intensity and pulse timing to stimulate corresponding retinal neurons; at the same time, local neural electrical signals induced by stimulation are collected through platinum-iridium alloy feedback electrodes arranged on the surface of the chip, and original neural response data containing action potentials and local field potentials are output. In this application, the feedback electrode can be a platinum-iridium alloy feedback electrode for collecting local neural electrical activity induced by photonic stimulation. Platinum-iridium alloy has low impedance, high stability and excellent biocompatibility, is suitable for long-term implantation in biological tissue environment, and still maintains signal conductivity and corrosion resistance at microelectrode scale. The feedback electrode can be arranged at the edge region or array gap position of the micro photonic array, and is integrally formed with the chip substrate through micro processing technology, and has a typical impedance of less than 50Ω, which can capture neural response signals such as action potentials and local field potentials induced by stimulation with high sensitivity without introducing electrical noise.

[0060] In a possible implementation, step S104 further includes: performing neural feedback acquisition based on the target stimulation pattern, and obtaining original neural signals through feedback electrodes on the preset implant chip; performing denoising on the original signals using band-pass filtering and wavelet transform, and obtaining denoised neural signals; determining whether the target stimulation pattern produces an effective neural response based on the denoised neural signals through an adaptive feature extraction network; and if the target stimulation pattern produces an effective neural response, taking the denoised neural signals as the neural electrical activity information.

[0061] Specifically, the VCSEL photon emission units in the implant chip are controlled to perform pulse emission operations based on the target stimulation pattern, so as to achieve directional photon stimulation of retinal neurons under the set spatial position, output light intensity, and time modulation parameters. After the photon stimulation acts on the target neurons, potential changes occur on the neural cell membranes, and neural response signals such as local field potentials (LFPs) or action potentials (APs) are triggered.

[0062] The feedback electrodes on the implant chip are composed of low-impedance platinum-iridium alloy materials and are distributed at the boundary of the stimulation array, and are used to synchronously record local neural electrical activities triggered by the photon stimulation. The signals collected by the feedback electrodes are analog voltage signals, which include neural response potentials, background noise, and interference waveforms, and constitute original neural signals .

[0063] Subsequently, the original neural signals are preprocessed. First, a band-pass filter is used to limit the frequency range of the signals, for example, a second-order Butterworth filter with a passband range of 1-300 Hz is used to suppress low-frequency drift and high-frequency interference. Then, a wavelet transform is used to perform multi-scale decomposition on the time-domain signals, to filter out non-transient components and non-structural fluctuations, and to output denoised neural signals optimized in time and frequency .

[0064] The denoised neural signals are input into an adaptive feature extraction network. The network can use a time series modeling structure based on a gated recurrent unit (GRU) and an attention mechanism to extract high-order feature indicators such as the time delay structure, discharge density, and waveform shape of the neural response. The network output is used to determine whether the stimulation has triggered an effective neural response. The determination logic includes but is not limited to the following conditions: the occurrence of an action potential cluster with a specific time sequence structure in the signal, the local discharge amplitude exceeding a physiological threshold, and the response time delay being within a set window.

[0065] If the network confirms that the target stimulation pattern produces an effective neural response, the corresponding denoised neural signals are taken as the neural electrical activity information of this round of stimulation operation, and are transmitted to the subsequent closed-loop control module for stimulation parameter adjustment and effect optimization.

[0066] If the judgment result is that no effective neural response is generated, i.e., there is a lack of structured discharge in the feedback signal, the response intensity is insufficient, or the response time delay exceeds the set threshold, the signal is not used for target stimulation pattern update, but the corresponding position and stimulation parameters are recorded as a "low activation area" and can be used as a negative feedback factor to guide parameter adjustment in subsequent reinforcement learning or stimulation pattern reconstruction.

[0067] In step S105, the target stimulation pattern is dynamically adjusted based on the neural electrical activity information through an adaptive optimization algorithm to repair the optic nerve.

[0068] Specifically, the neural electrical activity information is input into a preset closed-loop control model, a stimulation-response relationship is constructed combining the current stimulation parameters and the feedback response, and a stimulation effectiveness score is calculated. The spatial activation state, output intensity, and pulse timing configuration of the corresponding photonic emission unit in the target stimulation pattern are adjusted according to the score, so as to optimize the subsequent stimulation effect and realize stable activation and functional recovery of the retinal neurons.

[0069] In one possible implementation, step S105 further includes: based on the neural electrical activity information, calculating a stimulation effectiveness score corresponding to the target stimulation pattern through a preset feature evaluation function; inputting the stimulation effectiveness score into a closed-loop control algorithm model, and the closed-loop control algorithm model adjusts the photonic array parameters based on the stimulation effectiveness score, specifically including: constructing a stimulation-response mapping relationship based on the stimulation effectiveness score and historical neural electrical activity information; the stimulation-response mapping relationship is used to describe the corresponding relationship between the photonic stimulation control parameters of each photonic emission unit in the target stimulation pattern and the neural response induced by the target stimulation pattern at the corresponding position; based on the stimulation-response mapping relationship, obtaining the photonic emission units that fail to induce effective neural response in the target stimulation pattern; based on the spatial position of the photonic emission unit and the visual key feature corresponding to the photonic emission unit, adjusting the photonic array parameters, including the spatial layout parameters, intensity parameters, and timing parameters of photonic emission; based on the adjusted photonic array parameters, dynamically adjusting the target stimulation pattern to repair the optic nerve.

[0070] Specifically, based on the neural electrical activity information obtained in step S104 , a feature evaluation function is used to calculate a stimulation effectiveness score corresponding to the target stimulation pattern , which comprehensively considers parameters such as signal amplitude , neural response delay , signal-to-noise ratio , etc.:

[0071] wherein, is a weighting coefficient of each factor, Indicates the normalization process. The higher the score represents the more effective neural response induced by the current stimulation.

[0072] Then, the score Input the closed-loop control algorithm model , combined with historical neural electrical activity data With the corresponding stimulation pattern , construct the stimulation-response mapping relationship . This mapping is used to describe the association between the control parameters (spatial position, light intensity, pulse duration) of each photon emission unit in the target stimulation pattern and the neural response it induces at the corresponding spatial position:

[0073] Where, is the light intensity, is the pulse duration, is the feedback neural signal response intensity index.

[0074] Based on the mapping relationship, identify the set of photon emission units that do not induce effective neural response in the current stimulation , which satisfies:

[0075] Where is the effective response threshold.

[0076] Subsequently, according to the position of each ineffective unit and its corresponding area intensity value in the visual key feature map , perform fine-tuning optimization on its photon array control parameters, adjustment strategies include: spatial layout parameters: if there are high response units in the adjacent area of the ineffective unit, perform local migration and fine-tune the activation center; intensity parameter: increase the emission power of the unit ; timing parameter: fine-tune the pulse width according to the previous response delay .

[0077] The optimization function can take the following form (taking intensity adjustment as an example):

[0078] Where is the step coefficient.

[0079] Finally, apply the updated photon array parameters to construct a new round of target stimulation pattern , and use it for the next round of neural stimulation and feedback collection, forming a closed-loop adaptive iterative process.

[0080] Please refer to Figure 2It shows a module schematic diagram of an implantable retina photon nerve repair device provided by the embodiment of the application, the device comprising an external intelligent device module 21, a biological interface structure module 22 and an implant chip module 23, wherein The external intelligent device module 21 is configured to acquire a target image in a current visual field of a user based on an external wearable device.

[0081] The biological interface structure module 22 is configured to collect neural feedback based on a target stimulation pattern and acquire neural electrical activity information through a feedback electrode on a preset implant chip.

[0082] The implant chip module 23 is configured to extract visual key features in the target image by using an edge detection algorithm, and generate the target stimulation pattern by using time-controlled pulse coding based on the visual key features.

[0083] The implant chip module 23 is further configured to dynamically adjust the target stimulation pattern by using an adaptive optimization algorithm based on the neural electrical activity information to repair the optic nerve.

[0084] In a possible implementation, the external intelligent device module 21 is configured to acquire a target image in a current visual field of a user based on an external wearable device: acquire the current visual field of the user in real time based on the external wearable device, if it is confirmed that there is a visual perception area in the current visual field of the user, acquire a plurality of to-be-confirmed images in the current visual field of the user; acquire preliminary visual features corresponding to the plurality of to-be-confirmed images respectively by image preprocessing operation; respectively count the number of preliminary visual features corresponding to each to-be-confirmed image, and sort the plurality of to-be-confirmed images in descending order of the number; and take the to-be-confirmed images greater than a preset ranking threshold in the sorting result as the target image.

[0085] In a possible implementation, the implant chip module 23 is configured to extract visual key features in the target image by using an edge detection algorithm, specifically including: inputting the target image into a convolutional neural network model, and outputting a denoising image and an edge feature image based on the convolutional neural network model; inputting the denoising image and the edge feature image into a U-Net structure model, and outputting a semantic segmentation result based on the U-Net structure model; performing a region marking operation on the target image based on the semantic segmentation result, and generating a spatial saliency mask corresponding to the target image; and outputting the visual key features in the target image by a semantic feature fusion operation based on the spatial saliency mask, wherein the visual key features include contour features, dynamic object features and contrast boundary features.

[0086] In a possible implementation, the implant chip module 23 is configured to generate a target stimulation pattern based on the visual key features using time-controlled pulse coding, specifically including: generating a target stimulation pattern based on the visual key features using time-controlled pulse coding, the time-controlled pulse coding including spatial coding, intensity modulation, and time modulation; the spatial coding is used to selectively activate specific VCSEL units; the intensity modulation is used to control the light intensity output by each unit; and the time modulation is used to set the pulse width of the emission according to the image motion information.

[0087] In a possible implementation, the biological interface structure module 22 is configured to perform neural feedback collection based on the target stimulation pattern, and obtain neural electrical activity information through the feedback electrodes on the preset implant chip, specifically including: performing neural feedback collection based on the target stimulation pattern, and obtaining original neural signals through the feedback electrodes on the preset implant chip; using band-pass filtering and wavelet transform to denoise the original signals and obtain denoised neural signals; based on the denoised neural signals, determining whether the target stimulation pattern produces an effective neural response through an adaptive feature extraction network; if the target stimulation pattern produces an effective neural response, the denoised neural signals are used as the neural electrical activity information.

[0088] In a possible implementation, the implant chip module 23 is configured to dynamically adjust the target stimulation pattern based on the neural electrical activity information to repair the optic nerve through an adaptive optimization algorithm, specifically including: based on the neural electrical activity information, calculating a stimulation effectiveness score corresponding to the target stimulation pattern through a preset feature evaluation function; inputting the stimulation effectiveness score into a closed-loop control algorithm model, the closed-loop control algorithm model adjusts the photon array parameters based on the stimulation effectiveness score; and based on the adjusted photon array parameters, dynamically adjusting the target stimulation pattern to repair the optic nerve.

[0089] In a possible implementation, the implant chip module 23 is configured to adjust the photon array parameters based on the stimulation effectiveness score, specifically including: constructing a stimulation-response mapping relationship based on the stimulation effectiveness score and historical neural electrical activity information; the stimulation-response mapping relationship is used to describe the correspondence between the photon stimulation control parameters of each photon emission unit in the target stimulation pattern and the neural response induced by the target stimulation pattern at the corresponding position; based on the stimulation-response mapping relationship, obtaining the photon emission units that fail to induce effective neural responses in the target stimulation pattern; and based on the spatial position of the photon emission unit and the visual key features corresponding to the photon emission unit, adjusting the photon array parameters, the photon array parameters including the spatial layout parameters, the intensity parameters, and the timing parameters of the photon emission.

[0090] It should be noted that the apparatus provided in the above examples is only used as an example for the division of the above functional modules in realizing the functions thereof, and in actual applications, the above functions can be completed by different functional modules according to the needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided in the above examples belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0091] The present application also provides an electronic device. Referring to Figure 3 , Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device can include at least one processor 301, at least one communication bus 302, a user interface 303, at least one network interface 304, and a memory 305.

[0092] The communication bus 302 is used to realize the connection and communication between the components.

[0093] The user interface 303 can include a display screen (Display) and a camera (Camera), and the optional user interface 303 can further include a standard wired interface and a wireless interface.

[0094] The network interface 304 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0095] The processor 301 can include one or more processing cores. The processor 301 connects various parts within the server through various interfaces and lines, performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Alternatively, the processor 301 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 301 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 301, but can be realized by a separate chip.

[0096] The memory 305 can include a random access memory (RAM) and a read-only memory (ROM). Alternatively, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 305 can also be at least one storage device located away from the aforementioned processor 301. Referring to Figure 3 The memory 305 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an implanted retinal photonic nerve repair application program.

[0097] In Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 301 can be used to call the implantable retinal photonic neural repair application program stored in the memory 305, and when executed by one or more processors 301, causes the electronic device to perform the method described in one or more of the above embodiments. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0098] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors, causes the electronic device to perform the method described in one or more of the above embodiments.

[0099] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0100] In several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some services interface, device or unit, and can be electrical or other forms.

[0101] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0103] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0104] The above is only exemplary embodiments of the present application, and cannot limit the scope of the present application. That is, any equivalent changes and modifications made in accordance with the teachings of the present application are still within the scope of the present application.

[0105] The present application is intended to cover any variations, uses or adaptive changes of the present application, which follow the general principles of the present application and include common knowledge or conventional technical means in the technical field not disclosed in the present application.

Claims

1. An implantable retinal photonic neurorestorative method, comprising: The method comprises: obtaining a target image in the current visual field of a user based on an external wearable device; extracting visual key features in the target image using an edge detection algorithm; generating a target stimulation pattern using time-controlled pulse coding based on the visual key features; performing neural feedback collection based on the target stimulation pattern, and obtaining neural electrical activity information through feedback electrodes on a pre-set implanted chip; dynamically adjusting the target stimulation pattern for optic nerve repair through an adaptive optimization algorithm based on the neural electrical activity information.

2. The method of claim 1, wherein, The method comprises: real-time acquisition of the current visual field of the user based on the external wearable device, and if it is confirmed that there is a visual perception area in the current visual field of the user, a plurality of to-be-confirmed images in the current visual field of the user are obtained; obtaining a plurality of preliminary visual features corresponding to the plurality of to-be-confirmed images through image preprocessing operations; statistically counting the number of preliminary visual features corresponding to each of the plurality of to-be-confirmed images, and sorting the plurality of to-be-confirmed images in descending order of the number; the to-be-confirmed images greater than a pre-set ranking threshold in the sorting result are taken as the target image.

3. The method of claim 2, wherein, The method comprises: inputting the target image into a convolutional neural network model, and outputting a denoising image and an edge feature image based on the convolutional neural network model; inputting the denoising image and the edge feature image into a U-Net structure model, and outputting a semantic segmentation result based on the U-Net structure model; performing a region marking operation on the target image based on the semantic segmentation result, and generating a spatial saliency mask corresponding to the target image; outputting the visual key features in the target image through a semantic feature fusion operation based on the spatial saliency mask, the visual key features including contour features, dynamic object features, and contrast boundary features.

4. The method of claim 1, wherein, The method comprises: generating a target stimulation pattern using time-controlled pulse coding based on the visual key features, the time-controlled pulse coding including spatial coding, intensity control, and time modulation; the spatial coding is used for selectively activating specific VCSEL units; the intensity control is used for controlling the output light intensity of each unit; and the time modulation is used for setting the emission pulse width according to image motion information.

5. The method of claim 1, wherein, The method comprises: performing neural feedback collection based on the target stimulation pattern, and obtaining an original neural signal through feedback electrodes on a pre-set implanted chip; using band-pass filtering and wavelet transform to denoise the original signal and obtain a denoised neural signal; judging whether the target stimulation pattern produces an effective neural response through an adaptive feature extraction network based on the denoised neural signal; if the target stimulation pattern produces an effective neural response, taking the denoised neural signal as the neural electrical activity information.

6. The method of claim 1, wherein, The target stimulation pattern is dynamically adjusted based on the neural electrical activity information through an adaptive optimization algorithm to repair the optic nerve, and specifically includes: Based on the neural electrical activity information, the stimulation effectiveness score corresponding to the target stimulation pattern is calculated through a pre-set feature evaluation function; The stimulation effectiveness score is input into a closed-loop control algorithm model, which adjusts the photon array parameters based on the stimulation effectiveness score; Based on the adjusted photon array parameters, the target stimulation pattern is dynamically adjusted to repair the optic nerve.

7. The method of claim 6, wherein, The stimulation effectiveness score is input into a closed-loop control algorithm model, which adjusts the photon array parameters based on the stimulation effectiveness score; Based on the stimulation effectiveness score and historical neural electrical activity information, a stimulation-response mapping relationship is constructed; the stimulation-response mapping relationship is used to describe the corresponding relationship between the photon stimulation control parameters of each photon emission unit in the target stimulation pattern and the neural response induced by the target stimulation pattern at the corresponding position; Based on the stimulation-response mapping relationship, the photon emission units that fail to induce effective neural response in the target stimulation pattern are obtained; Based on the spatial position of the photon emission unit and the visual key feature corresponding to the photon emission unit, the photon array parameters are adjusted, including the spatial layout parameters, intensity parameters and timing parameters of photon emission.

8. An implantable retinal photonic neuroprosthetic device, comprising: The device includes an external intelligent device module, a biological interface structure module and an implanted chip module, wherein, The external intelligent device module is configured to acquire a target image in the current visual field of a user based on an external wearable device; The implanted chip module is configured to extract visual key features in the target image using an edge detection algorithm, and generate a target stimulation pattern using time-controlled pulse coding based on the visual key features; The biological interface structure module is configured to collect neural feedback based on the target stimulation pattern, and acquire neural electrical activity information through feedback electrodes on a pre-set implanted chip; The implanted chip module is further configured to dynamically adjust the target stimulation pattern based on the neural electrical activity information through an adaptive optimization algorithm to repair the optic nerve.

9. An electronic device, comprising: The electronic device includes a processor, a communication bus, a user interface, a network interface and a memory, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method of any one of claims 1 to 7.

Images