A UAV control system and method based on intelligent biological eye complex

Abstract

This invention discloses a drone control system and method based on an intelligent biological eye complex. The system includes: an isolated retinal stimulation and signal acquisition module for preparing an isolated retina into an intelligent biological eye complex, applying light stimulation to the intelligent biological eye complex, and acquiring its neural response signals; a signal processing module for preprocessing and decoding the acquired neural response signals, converting the decoded signals into control commands recognizable by the drone; and a drone flight control module for receiving control commands and controlling the drone's flight state. This invention utilizes an isolated retina as the core sensing component and, through neural signal encoding and decoding technology, can convert the neural response signals of the retina into control commands in real time, thereby achieving autonomous control of the drone. This not only improves the drone's response speed but also frees the operator's hands, allowing them to perform other tasks while controlling the drone's flight state.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) control technology, and particularly relates to a UAV control system and method based on an intelligent biological eye complex. Background Technology

[0002] Traditional vision systems primarily rely on electronic chips composed of transistors, typically employing CMOS or CCD optical sensors for image acquisition. These sensors convert images into digital electrical signals and transmit them to specialized image processing systems to extract target features. Image processing systems usually integrate machine vision perception and recognition algorithms, with convolutional neural networks being the most classic and widely used algorithm. Convolutional layers first extract local features, then pooling layers further compress the feature maps while retaining key feature information, and finally fully connected layers summarize all features, providing a classifier for prediction and recognition. Furthermore, existing biomimetic vision technologies are constantly evolving, attempting to improve the performance of vision systems by mimicking the structure and function of biological eyes. These technologies typically involve multiple fields such as microelectronics, biomaterials, optics, and neuroscience, aiming to process and convert visual signals using complex microelectronic components.

[0003] However, existing vision systems have many limitations. Traditional vision sensors are prone to overexposure or loss of detail under strong light, low light, or complex lighting conditions (such as high dynamic range scenes). Their performance degrades significantly in low-visibility environments such as nighttime or haze, requiring additional infrared illumination or other auxiliary equipment. While existing bionic vision technologies simulate the function of biological eyes to some extent, they offer lower visual resolution and a much narrower field of view than normal vision, and struggle to simulate the complex details of biological eyes, such as the density of photoreceptor cells in the retina. Furthermore, traditional image sensors sample at a fixed frame rate, which may lead to missed critical information and visual delays during rapid movement or sudden events. These problems stem from the limitations of current chip processing speeds and algorithms. Simultaneously, the development of bionic vision involves multiple fields, resulting in high initial investment costs, and the special materials used also lead to high usage and maintenance costs. In addition, traditional drone control relies primarily on handheld remote controls, joysticks, or touchscreens, requiring high user skills and demanding continuous use of both hands, preventing the operator from performing other tasks simultaneously. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a drone control system and method based on an intelligent biological eye complex, thereby resolving the issues present in the prior art.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a drone control system based on an intelligent biological eye complex, comprising:

[0006] An isolated retinal stimulation and signal acquisition module is used to prepare an isolated retina into an intelligent biological eye complex, and to apply light stimulation to the intelligent biological eye complex and acquire its neural response signals.

[0007] The signal processing module is used to preprocess and decode the acquired neural response signals, and convert the decoded signals into control commands that the UAV can recognize.

[0008] The drone flight control module is used to receive control commands and control the drone's flight status.

[0009] Preferably, the isolated retinal stimulation and signal acquisition module includes:

[0010] An ex vivo retinal acquisition unit is used to acquire the eyeball of an experimental animal and place it on a multi-electrode array;

[0011] A retinal stimulation unit is a stimulator used to apply light stimulation to the retina.

[0012] The signal acquisition unit is used to acquire the neural response signal generated after the isolated retina is stimulated by the signal acquisition device; wherein the signal acquisition device includes an acquisition device equipped with a multi-electrode array chip, an amplifier for amplifying and preprocessing weak electrical signals, and a USB connection for transmitting the amplified signal to a computer.

[0013] Preferably, the signal processing module includes:

[0014] The data preprocessing unit is used to process the acquired neural response signals and extract the effective neural action potential (spike) signals as neural response data.

[0015] The signal decoding unit is used to train the model using the preprocessed spike signal as the model training input, and to train the model to express the correspondence between the input stimulus signal and the output neural response data.

[0016] Preferably, the UAV flight control module sends the output of the signal processing module to the UAV ground station control computer via TCP / IP. The ground station control computer then maps the received results into control commands required for UAV control and sends them to the UAV.

[0017] Preferably, the UAV flight control module adopts incremental fixed-point control, and each time a new control command is received, the UAV only makes a small displacement in the corresponding direction.

[0018] Preferably, the fixed-point control adopts a multi-level closed-loop control architecture, and the pose adjustment is achieved through hierarchical mapping from the coordinate space to the power system.

[0019] Preferably, the UAV flight control module includes an action control unit, which transmits the captured content to the onboard computer in real time via the UAV camera. The onboard computer then sends the captured content to the photostimulation module, and the intelligent biological eye complex controls the next action of the UAV based on the image projected by the photostimulation module.

[0020] Secondly, this invention also discloses a method for controlling unmanned aerial vehicles (UAVs) based on an intelligent biological eye complex, comprising the following steps:

[0021] Mouse retinas were obtained and placed on MEAs signal acquisition devices to construct an intelligent biological eye complex;

[0022] The mouse retina was light-stimulated to obtain the mapping relationship between the stimulation paradigm and the neural response of the intelligent biological eye complex.

[0023] Based on the mapping relationship, a correspondence between the neural response data of the intelligent biological eye complex and the control commands of the UAV is constructed. The correspondence is written into the signal decoding unit of the signal processing module. The signal decoding part converts the neural response data of the intelligent biological eye complex into control commands for the UAV.

[0024] A drone control and feedback model is constructed. The drone transmits the content captured by its camera to an onboard computer. The onboard computer receives the content captured by the camera and transmits it to a photostimulation module, which is used to stimulate the intelligent biological eye complex. The neural response of the intelligent biological eye complex to the photostimulation signal is used to continue controlling the drone.

[0025] Preferably, the process of obtaining the mapping relationship between the stimulus paradigm and the neural response of the intelligent biological eye complex includes:

[0026] A light stimulation image composed of different colors and shapes is applied to an isolated retina using a light stimulation device;

[0027] Collect the spike response electrical signals of an isolated retina to different image stimuli.

[0028] Thirdly, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0029] Compared with the prior art, the present invention has the following advantages and technical effects:

[0030] This invention provides a drone control system based on an intelligent biological eye complex, comprising: an isolated retina stimulation and signal acquisition module for preparing an isolated retina into an intelligent biological eye complex, applying light stimulation to the intelligent biological eye complex and acquiring its neural response signals; a signal processing module for preprocessing and decoding the acquired neural response signals, and converting the decoded signals into control commands recognizable by the drone; and a drone flight control module for receiving control commands and controlling the flight state of the drone.

[0031] This invention utilizes an isolated retina as the core sensing component, directly reading the neural response signals of the retina through MEAs (multi-electrode arrays) and converting them into control commands for a drone. Compared to traditional vision systems, the intelligent bio-eye complex has the following advantages: Utilizing a real biological retina, it can maintain stable sensing capabilities in both strong and low light environments, much like a biological vision system, thus solving the problem of overexposure or detail loss that traditional vision sensors easily experience under varying lighting conditions. Furthermore, the intelligent bio-eye complex requires no external power supply, consuming significantly less power than traditional devices, while also being more sensitive to changes in light intensity and color, addressing the issue of significant performance degradation in low-visibility environments for traditional vision sensors.

[0032] This invention utilizes neural signal encoding and decoding technology to read neural response signals from an isolated retina in real time and convert them into control commands via a computer, thereby achieving autonomous control of the drone. This not only improves the drone's response speed but also frees the operator's hands, allowing them to perform other tasks while controlling the drone's flight status. It solves the problems of traditional drone control methods, which require high operator skills and cannot perform multitasking.

[0033] The intelligent biological eye complex in this invention utilizes a real biological retina to reconstruct and study the complex cell networks and signal processing mechanisms in biological vision systems in vitro. It provides a more natural and efficient visual information processing solution, solving the problems of low resolution, narrow field of view, and difficulty in simulating the complex details of biological eyes in existing bionic vision technologies. Attached Figure Description

[0034] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0035] Figure 1 This is a system schematic diagram according to an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of the isolated retinal stimulation and signal acquisition module implemented in this invention.

[0037] Figure 3 This is a schematic diagram of the signal processing module implementation of the present invention;

[0038] Figure 4 This is a schematic diagram of the flight control module of the UAV implemented in this invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0041] The technical terms used in the following embodiments will be explained first.

[0042] The stimulation paradigm of the intelligent biological eye complex refers to a standardized stimulation design pattern that relies on a high-precision light stimulator (such as a small display screen) to present patterns composed of different colors and shapes. Through systematic control of the color, geometric characteristics and spatiotemporal arrangement of the light source, the intelligent biological eye complex can produce a specific photosensitive response and conduct functional evaluation.

[0043] Example 1

[0044] like Figure 1 As shown, this embodiment provides a drone control system based on an intelligent biological eye complex, including:

[0045] An isolated retinal stimulation and signal acquisition module is used to prepare an isolated retina into an intelligent biological eye complex, and to apply light stimulation to the intelligent biological eye complex and acquire its neural response signals.

[0046] Furthermore, the isolated retinal stimulation and signal acquisition module includes:

[0047] An ex vivo retinal acquisition unit is used to acquire the eyeball of an experimental animal and place it on a multi-electrode array;

[0048] A retinal stimulation unit is a stimulator used to apply light stimulation to the retina.

[0049] Specifically, to meet the need for maintaining retinal activity and recording electrophysiological activity, it is necessary to construct intelligent biological eye complexes. MEAs systems allow for the non-invasive simultaneous recording and stimulation of electrophysiological activity at multiple sites.

[0050] The signal acquisition unit is used to acquire the neural response signal generated after the isolated retina is stimulated by the signal acquisition device; wherein the signal acquisition device includes an acquisition device equipped with a multi-electrode array chip, an amplifier for amplifying and preprocessing weak electrical signals, and a USB connection for transmitting the amplified signal to a computer.

[0051] Specifically, the method for preparing an isolated retina is as follows:

[0052] The eyeballs of the experimental animals were obtained and placed in a culture dish containing retinal maintenance solution (Ames solution), ensuring that the Ames solution completely submerged the eyeballs. The culture medium temperature was 37°C, and the eyes were continuously perfused using maintenance medium filled with 95% oxygen and 5% carbon dioxide to maintain eyeball activity.

[0053] The preparation method of Ames solution is as follows: Take 900 ml of pure water at 15-20℃, and bubble 100% CO2 into the water to prevent precipitation. Add 1.9 g of sodium bicarbonate powder and stir until completely dissolved. Add 8.8 g of Ames powder and gently stir until completely dissolved. Do not heat during the above process, and adjust the pH value to be 0.1-0.3 lower than the expected value. Make up to 1000 ml, filter the culture medium through a 0.22 μm or smaller filter membrane, and then dispense into sterile containers. Store in a refrigerator at 2-8℃, protected from light.

[0054] Using an anatomical microscope, a red filter membrane was attached to the microscope light source to ensure it emitted only a faint red light. Qualitative filter paper was placed at the bottom of the petri dish to increase the eyeball's adhesion and facilitate fixation. Under the microscope, the eyeball was gently held and fixed with straight forceps. A small hole was made 0.2 mm posterior to the corneal limbus to allow Ames solution to seep into the eye. A circular cut was made along the cornea and sclera using canthoplasty scissors. The lens, vitreous humor, and other components of the eyeball were then carefully detached using straight forceps without significant contact. Subsequently, the pigment epithelium and retina were slowly separated using straight forceps, avoiding rapid detachment that could cause retinal fragmentation. The separated retina was bowl-shaped with some pigment spots inside. These pigment spots were carefully removed using straight forceps to clear away any remaining vitreous humor. The bowl-shaped edges of the detached retina were trimmed with canthoplasty scissors to prevent curling and concavity.

[0055] After trimming the retina, it was aspirated using a Pasteur pipette and transferred to a multi-electrode array (MEAs). The retina was flipped so that the photoreceptor cells completely covered the MEAs electrode chip. Excess solution was aspirated using a pipette to ensure the retina was completely attached to the MEAs electrode chip, ensuring that the retina did not shift and remained at the center of the MEAs electrode array throughout the process. A microporous filter membrane was then placed over the retina. The pressure mesh was picked up with tweezers, and the mesh side was gently placed over the filter membrane to ensure the retina lay flat and was evenly stressed. The RGC side was placed downwards to connect the neural synapses to the electrode points. Ames solution was added using a pipette until it was above the pressure mesh, forming a biological eye complex. This complex was then placed on a signal detection and acquisition device so that its electrical activity could be recorded by the MEAs.

[0056] In this embodiment, the signal acquisition system MCS consists of three parts: the MEAs 2100-Mini-System, which carries MEAs electrode chips to acquire neuronal response signals; an amplifier for amplifying and preprocessing weak electrical signals; and finally, the amplified signal is transmitted to a computer via USB for further processing and visualization. This structure not only allows for the simultaneous recording of large-scale neuronal activity but also enables stimulation of isolated retina mounted on MEAs multi-electrode arrays to study their response characteristics. The MEAs 2100-Mini-System has a central perforated hole in the area where the MEAs are placed, corresponding to all the electrodes on the MEAs. This allows light stimulation to directly illuminate the retina through the MEAs electrodes, forming a stimulus-response-acquisition experimental loop. The MEAs chip with the isolated retina placed on it is placed on the MEAs2100-Mini-System, and the light stimulation program is run. Different colors and shapes of light stimulation images are applied to the isolated retina, and the instrument simultaneously acquires the spike response electrical signals of the isolated retina to different image stimuli, such as… Figure 2 As shown.

[0057] As an innovative implementation method, this embodiment achieves a photostimulation paradigm through an on-chip driven active-matrix color OLED display based on single-crystal silicon transistors. This allows the stimulation paradigm to be directly projected onto the on-chip retina. The stimulation paradigm in this embodiment is based on independent programming using Psytoolbox and Matlab, possessing powerful visual stimulation capabilities. It can flexibly adjust key photostimulation parameters such as light intensity, frequency, wavelength, shape, color, and duration, presenting diverse stimulation forms. It also possesses high-precision time control capabilities, ensuring the temporal consistency of the stimulation process and the synchronization of the recording system, comprehensively supporting in-depth research on the photosensitive response characteristics and visual information processing capabilities of the intelligent biological eye complex. Furthermore, to ensure the direct projection of the stimulation paradigm onto the retina, this embodiment designs and 3D prints a stimulator shell that can be attached to the shell of the acquisition device, with the stimulation screen facing the retina. The stimulation paradigm designed in this embodiment has a total duration of 12 minutes, encompassing stimuli such as light intensity, color, shape, and motion, and can acquire the firing of neural potential spikes in approximately more than half of the channels.

[0058] The signal processing module is used to preprocess and decode the acquired neural response signals, and convert the decoded signals into control commands that the UAV can recognize.

[0059] Furthermore, such as Figure 3 As shown, the signal processing module includes:

[0060] The data preprocessing unit is used to process the acquired neural response signals and extract the effective spike signals as neural response data.

[0061] Specifically, the data preprocessing section processes the acquired raw neural response signals. The raw neural response signals are first processed by a 300-3000Hz bandpass filter. This frequency band is selected based on the typical spectral characteristics of spikes, effectively suppressing low-frequency biological noise and high-frequency environmental interference. Targeting the amplitude difference between spikes and background noise, the system employs a dynamic threshold detection algorithm: calculating the standard deviation of each signal channel using a sliding window approach, and setting 5-6 times the standard deviation as the spike identification threshold. When the signal amplitude exceeds the channel-specific threshold, the corresponding time stamp is recorded. Setting an appropriate detection threshold is crucial for accurate spike detection. Too low a threshold easily leads to false positives due to noise, while too high a threshold may cause small spikes to be missed. Experimental verification has shown that 5-6 times the standard deviation strikes a balance between detection sensitivity and specificity.

[0062] The signal decoding unit is used to train the model by using the preprocessed spike signal as the model training input and supervising the model to express the correspondence between the stimulus paradigm and the neural response signal of the intelligent biological eye complex.

[0063] Specifically, after completing spike detection, the system establishes a precise temporal correlation based on a preset visual stimulus sequence. Specifically, the system sets a specific time window after each visual stimulus, typically ranging from 50 to 200 milliseconds, to capture the neuronal firing activity induced by the stimulus. Within this time window, the system counts the frequency of spike events in each electrode channel and calculates the average firing rate parameter for each channel. These parameters reflect the neural response intensity of different retinal regions under specific stimuli, helping to characterize the functional differentiation features of cell populations.

[0064] It should be noted that the retina contains various types of neurons, such as photoreceptor cells, bipolar cells, horizontal cells, amacrine cells, and retinal ganglion cells (RGCs). Different types of cells exhibit varying response preferences to visual features such as color, shape, brightness, and direction of motion. In experiments, when patterns of different color and shape combinations are presented, these cells respond to stimuli with specific firing patterns, thus forming neural codes with characteristic selectivity.

[0065] To decode these complex neural response patterns, the system employs a Convolutional Neural Network (CNN) to construct a stimulus-response mapping model. First, the neural spike signals, after preprocessing steps such as bandpass filtering and threshold detection, are converted into a spatiotemporally encoded firing rate matrix, which serves as the input to the CNN model. The input layer receives this three-dimensional matrix (e.g., number of channels × time step × spatial location), and the hidden layers consist of multiple convolutional layers, pooling layers, and nonlinear activation functions (such as ReLU) to extract spatiotemporal feature structures from the neural signals. Through a local receptive field mechanism, the model can identify local patterns in neural firing activity and capture the collaborative response relationships between neural groups at a deeper level.

[0066] In the output layer, the CNN maps the high-dimensional features extracted from the hidden layers to specific stimulus labels, such as color types, geometric shapes, or combinations thereof in an image. During training, the model uses a supervised learning strategy, taking the error between the input true labels and the model's predictions as input and optimizing the network weights through backpropagation. To improve the model's generalization performance, the system introduces a cross-validation mechanism, rotating training and validation across multiple data subsets to select the optimal network structure and hyperparameter configurations (such as kernel size, number of layers, learning rate, dropout rate, etc.).

[0067] Ultimately, the trained CNN model is able to accurately decode new neural response data and automatically identify the visual stimulus category corresponding to the current neural activity.

[0068] In this embodiment, the signal decoding unit in the signal processing module is used to convert the pre-processed retinal neuron response data into control commands that can be recognized by the UAV. That is, the neural response signal of the intelligent biological eye complex is used as a converter to establish the correspondence between stimulus paradigms and control commands. Examples of the correspondence between stimulus paradigm patterns and UAV flight control commands are shown in Table 1 below.

[0069] Table 1

[0070] Stimulus Paradigm Unmanned Aerial Vehicle Control Commands All blue take off Black background with white triangle Fly forward Black background with green triangle flying backward Black background with white circles Increase altitude Black background with green circles Reduce height Black background with white square Fly to the left Black background with green square Fly to the right Black background with blue circle Circular trajectory All green landing

[0071] The drone flight control module is used to receive control commands and control the drone's flight status.

[0072] Furthermore, the UAV flight control module transmits the results output by the signal processing module to the UAV ground station control computer via the TCP / IP protocol to achieve end-to-end data transmission. The ground station control computer then maps the received results into control commands required for UAV control and sends them to the UAV.

[0073] Furthermore, to ensure the robustness of UAV flight control and prevent flight accuracy from being affected by inaccurate classification results of the decoder in one or more instances, an incremental fixed-point control UAV flight strategy is adopted. After receiving a new control command each time, the UAV only makes a small displacement in the direction corresponding to the control command.

[0074] The UAV's fixed-point control employs a multi-level closed-loop control architecture, achieving precise attitude adjustment through a hierarchical mapping from coordinate space to the power system. Specifically, this includes: First, the position loop PID controller generates a velocity reference value based on the deviation between the target coordinates and the actual coordinates; this value is input to the angular velocity loop PID controller, which, after calculation, outputs the desired attitude angle value; the attitude loop calculates the angular velocity command in the body coordinate system based on the current Euler angle deviation; finally, the angular velocity loop PID controller calculates the total thrust required to maintain the target attitude, and through a power distribution matrix (a quadcopter hybrid matrix constructed based on the UAV's structural parameters), maps it to the desired speeds of the four motors and transmits it to the ESC (Electronic Speed ​​Controller), driving the brushless motors through closed-loop speed control to achieve precise thrust output. For example... Figure 4 As shown.

[0075] Furthermore, the drone flight control module includes an action control unit, which transmits the captured content to the onboard computer in real time via the drone camera. The onboard computer then sends the captured content to the photostimulation module, and the intelligent biological eye complex controls the drone's next action based on the image projected by the photostimulation module.

[0076] Beneficial effects of this embodiment:

[0077] This embodiment innovatively proposes using the photosensitive function of an isolated retina as a research object for controlling external hardware; it establishes a signal transmission pathway between the isolated retina and the UAV, enabling real-time acquisition, decoding, and control output of retinal discharge signals, and studying the impact of different light stimuli on retinal nerve discharge patterns. The retina can sense changes in light intensity and color within a certain range, and exhibits different optic nerve responses to light stimuli of different shapes. This research method ensures the authenticity of the biological network, provides a realistic and controllable platform for the encoding and decoding research of real biological visual networks, and optimizes the biocompatibility of UAV flight control systems.

[0078] Compared to traditional biomimetic vision technologies, the intelligent bio-eye complex utilizes an isolated retina as its core sensing component. As a living organism, the intelligent bio-eye complex possesses a certain dynamic adjustment mechanism. This approach not only ensures that the intelligent bio-eye complex relies on a real biological network, but also uses relatively readily available materials, thus providing a more natural and sustainable approach to visual research. Furthermore, the intelligent bio-eye complex is designed without an external power supply, resulting in a significant reduction in power consumption compared to traditional devices.

[0079] This embodiment addresses the issue of operators' hands being continuously occupied during drone flight, enhancing their ability to handle multiple events in parallel during drone operation. Simultaneously, the intelligent biological eye complex provides a research platform for exploring and simulating the working principles of neural networks, particularly the complex mechanisms of visual information processing. By observing and intervening in the responses of retinal cells in an experimental environment, a more detailed analysis can be conducted on how neurons interact and transmit information naturally without human intervention.

[0080] Furthermore, the use of intelligent bio-eye complexes can facilitate the development of medical solutions tailored to the needs of patients with eye diseases. In addition, this technology reduces the dependence of visual assistive devices on external energy sources while also promoting the development of sustainable medical technologies. This study not only demonstrates the potential of ex vivo retinal technology in clinical applications but also emphasizes the importance of interdisciplinary collaboration in advancing medical technology innovation.

[0081] Example 2

[0082] Based on the same inventive concept, this embodiment also provides a drone control method based on an intelligent biological eye complex, including the following steps:

[0083] Mouse retinas were obtained and placed on MEAs signal acquisition devices to construct an intelligent biological eye complex;

[0084] The mouse retina was light-stimulated to obtain the mapping relationship between the stimulation paradigm and the neural response of the intelligent biological eye complex.

[0085] Based on the mapping relationship, a correspondence between the neural response data of the intelligent biological eye complex and the control commands of the UAV is constructed. That is, the neural response signal of the intelligent biological eye complex is used as a converter of the correspondence between the stimulus paradigm and the control command. The correspondence between the stimulus paradigm and the UAV flight control command is shown in Table 1 above. The correspondence is written into the signal decoding unit of the signal processing module. The neural response data of the intelligent biological eye complex is converted into control commands for the UAV through the signal decoding part.

[0086] A drone control and feedback model is constructed. The drone transmits the content captured by its camera to an onboard computer. The onboard computer receives the content captured by the camera and transmits it to a photostimulation module, which is used to stimulate the intelligent biological eye complex. The neural response of the intelligent biological eye complex to the photostimulation signal is used to continue controlling the drone.

[0087] Furthermore, the process of obtaining the mapping relationship between stimulus paradigms and neural responses of the intelligent biological eye complex includes:

[0088] A light stimulation image composed of different colors and shapes is applied to an isolated retina using a light stimulation device;

[0089] Collect the spike response electrical signals of an isolated retina to different image stimuli;

[0090] By using the spike response electrical signal as input for training the deep learning model and comparing it with the actual stimulus paradigm, the deep learning model is trained to obtain the mapping relationship between the stimulus paradigm and the neural response of the intelligent biological eye complex.

[0091] The UAV control method based on the intelligent biological eye complex provided in this embodiment has all the advantages of the UAV control system based on the intelligent biological eye complex provided in Embodiment 1.

[0092] Example 3

[0093] This embodiment also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0094] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A drone control system based on an intelligent biological eye complex, characterized in that, include: An isolated retinal stimulation and signal acquisition module is used to prepare an isolated retina into an intelligent biological eye complex, and to apply light stimulation to the intelligent biological eye complex and acquire its neural response signals. The signal processing module is used to preprocess and decode the acquired neural response signals, and convert the decoded signals into control commands that the UAV can recognize. The drone flight control module is used to receive control commands and control the drone's flight status. The drone flight control module includes a motion control unit, which transmits the captured content to the onboard computer in real time via the drone's camera. The onboard computer then sends the captured content to the photostimulation module, and the intelligent bio-eye complex controls the drone's next action based on the image projected by the photostimulation module.

2. The system according to claim 1, characterized in that, The isolated retinal stimulation and signal acquisition module includes: An ex vivo retina acquisition unit is used to acquire a trimmed retina and place it on a multi-electrode array; A retinal stimulation unit is a stimulator used to apply light or electrical stimulation to the retina. The signal acquisition unit is used to acquire the neural response signal generated by the retina after stimulation in vitro through a signal acquisition device; wherein the signal acquisition device includes an acquisition device equipped with a multi-electrode array chip, an amplifier for amplifying and preprocessing weak electrical signals, and a USB connection for transmitting the amplified signal to a computer.

3. The system according to claim 1, characterized in that, The signal processing module includes: The data preprocessing unit is used to process the acquired neural response signals and extract the effective neural action potential signals as neural response data. The signal decoding unit is used to use the preprocessed spike signal as the model training input and to train the model to express the correspondence between the input stimulus signal and the output neural response data.

4. The system according to claim 1, characterized in that, The UAV flight control module sends the output of the signal processing module to the UAV ground station control computer via TCP / IP. The ground station control computer maps the received results into control commands required for UAV control and then sends them to the UAV.

5. The system according to claim 4, characterized in that, The UAV flight control module uses incremental fixed-point control. Each time a new control command is received, the UAV only moves slightly in the corresponding direction.

6. The system according to claim 5, characterized in that, The fixed-point control adopts a multi-level closed-loop control architecture, which realizes pose adjustment through hierarchical mapping from coordinate space to the power system.

7. A method for controlling unmanned aerial vehicles (UAVs) based on an intelligent biological eye complex, characterized in that, Includes the following steps: An ex vivo retina is placed on a MEAs signal acquisition device to construct an intelligent biological eye complex; By applying light stimulation to the intelligent biological eye complex, the mapping relationship between the stimulation paradigm and the neural response of the intelligent biological eye complex was obtained. Based on the mapping relationship, a correspondence between the neural response data of the intelligent biological eye complex and the control commands of the UAV is constructed. The correspondence is written into the signal decoding unit of the signal processing module. The signal decoding part converts the neural response data of the intelligent biological eye complex into control commands for the UAV. A drone control and feedback model is constructed. The drone transmits the content captured by its camera to an onboard computer. The onboard computer receives the content captured by the camera and transmits it to a photostimulation module, which is used to stimulate the intelligent biological eye complex. The neural response of the intelligent biological eye complex to the photostimulation signal is used to continue controlling the drone.

8. The method according to claim 7, characterized in that, The process of obtaining the mapping relationship between stimulus paradigms and neural responses of intelligent biological eye complexes includes: A light stimulation image composed of different colors and shapes is applied to an isolated retina using a light stimulation device; Collect neural action potential signals from the retina in response to different image stimuli. By using neural action potential signals as input for training a deep learning model and comparing them with actual stimulus paradigms, the deep learning model is trained to obtain the mapping relationship between stimulus paradigms and the neural responses of the intelligent biological eye complex.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program performs the steps of the method according to any one of claims 7-8.

Images