Method and system for cognitive computing based on biological neural networks

Abstract

To provide cognitive computing systems and methods to perform a diversity of high-level, complex cognitive tasks mimicking and extending the biological brain functions.SOLUTION: A BNN core unit comprising a neural cell culture, an input stimulation unit, and an output readout unit may be controlled through various life cycles to provide data processing functionality. An automation system comprising an environmental and chemical controller unit adapted to operate the BNN stimulation and readout data interfaces facilitates the monitoring and adaptation of the BNN core unit parameters. The proposed system provides a BNN operating system as a core component for a wetware server to receive, process and transmit data for different client applications without exposing the BNN core unit components to the client user while requiring significantly less energy than conventional silicon-based hardware and software information processing for high-level cognitive computing tasks.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to cognitive computing systems, methods, and processes that mimic and extend biological brain function to perform a variety of high-level, complex cognitive tasks. The proposed cognitive computing systems, methods, and processes employ multiple neural cells as the core processing elements of biotechnology, combined with brain-on-a-chip interfaces and controllers to complement and interact with traditional information technology network and computing architectures. [Background technology]

[0002] The remarkable development of information technology (IT) over the past few decades has enabled the availability of methods and systems for performing a wide variety of computational tasks, including calculations, data optimization, data classification, natural language processing and translation, and image and video processing and recognition. Recent advances in cognitive computing have included machine learning, deep learning, and artificial neurocomputing (AI), which mimics biological neural networks. Furthermore, the latter aims to provide AI to assist humans with a wide range of tasks, such as object and face recognition, natural language processing (NLP), and sentiment analysis, in the workplace, at home, on social media, and even in hostile environments such as the deep sea, space, and nuclear reactors. High-performance computing, high-throughput computing, and highly available systems and infrastructure are provided as cloud computing services by major companies such as Google (Google Cloud Platform), Amazon (Amazon Web Services AWS), and Microsoft (Azure). Additionally, Facebook and Apple operate their own private data centers in select locations around the world known as "data farms" with affordable and reliable electricity. However, one of the major limitations of current silicon-based computing environments, including software and hardware, is the extremely high power required to perform complex cognitive tasks. Silicon-based systems consume an order of magnitude more power than biological systems with comparable performance, such as the brain. Another limitation is that they require explicit logic programming and structured data representations, which are currently inadequate for implementing concepts not yet understood and modeled by the human sciences, such as higher-order cognitive processes, creative thinking, and consciousness.

[0003] Neural networks can be viewed as a means of creating a spatiotemporal mapping between two spaces of different dimensions. That is, in configurations where the number of outputs is fewer than the number of inputs, neural networks create a concise representation of a problem in a space of limited dimensionality. For example, well-known spatial transformations include the Hough transform, which maps a two-dimensional space into a two-parameter voting space; the Fourier transform, which maps a time-periodic signal into its frequency representation; and the wavelet class of transforms, which maps an image into a scale-space representation. Of course, these transformations are well-described mathematically and can be implemented directly, and even if neural networks could be used for this purpose, they would likely not be the most computationally efficient choice. However, when, for example, a 1000-dimensional multivariate input space produces only 10 relevant variables as outputs, explicit mathematical solutions are usually too complex. Neural networks are good candidates for solving such problems. However, the required dimensionality of the internal state space to solve such problems would be enormous, and current hardware technology may require too many computational resources.

[0004] At the end of the 20th century, wetware solutions based on biological components such as cultured cells instead of transistors were being explored, primarily academically, as an alternative to traditional software and hardware information technologies (https: / / www.technologyreview.com / s / 400707 / biological-computing / ). Most scientists have focused on how to achieve basic functions similar to the fundamental core processes of IT, such as logic gates, computation, and memory storage, as in DNA computing (https: / / www.nature.com / subjects / dna-computing). While this approach holds promise, particularly with recent advances in synthetic biology and DNA editing, the engineering path to construct higher-order cognitive processes from such basic functions remains as challenging as that of traditional silicon-based logic.

[0005] An example of a higher-order cognitive function is the function required for a machine of general intelligence to pass the Turing test. Current research using artificial neural networks (ANNs) for this purpose has at least two major limitations. State-of-the-art multi-layer networks used in deep learning lack temporal flexibility compared to biological neural networks (BNNs). The human brain is fundamentally a biological multi-core system for which there is no mathematical modeling. Most of the powerful mathematics used in great scientific models like quantum mechanics and general relativity is useless in this case. Previously developed recurrent spiking neural networks (SNNs) have been unable to learn complex tasks or perform large-scale simulations, primarily due to the lack of computational efficiency of digital processing. Because all neurons operate essentially in parallel, they essentially represent up to 100 billion processors (assuming, for example, that the average human brain contains approximately 100 billion neurons). While it might be possible to replicate this processing power if each neuron were equivalent to one flop of computational power, the accuracy of neuron simulations required to achieve the goals of the Turing Test is unclear. Even current technology makes it impossible to implement a system with a computational power on the order of 100 megaflops per neuron. In fact, the highest computational power is around 1E16 flops, or 100E9 x 100E6 = 1E11 x 1E8 = 1E19, which is approximately 1000 times the computational power available in the world's best high-performance computing systems. Even non-real-time simulations are too slow to produce meaningful results in a reasonable time. The computational efficiency of biological brains is also several orders of magnitude higher than that of digital computers: for example, a brain typically consumes 20 W for 100 billion neurons (5 billion neurons / W), whereas digital simulations require several orders of magnitude more power for even simple neuron models such as "integrate and fire" spiking neurons.

[0006] Instead of using silicon-based digital computers to replicate advanced cognitive processes in AI, an alternative approach has emerged: biological neural networks. Recent advances in biotechnology have made it possible to culture and construct biological neural networks from embryonic stem cells, such as rat embryonic stem cells, or differentiated human induced pluripotent stem cells (IPSc). Cultured BNNs can then be stimulated and read using multielectrode arrays (MEAs). The primary use of MEAs to date has been to develop neuron and brain models for pharmacology, drug testing, and toxicity studies, as well as to deepen our understanding of common brain diseases such as Alzheimer's and Parkinson's. In recent years, other industrial applications have also been proposed. For example, the application of BNNs to aircraft control was proposed by DeMarse et al. in "Adaptive Flight Control With Living Neuronal Networks on Microelectrode Arrays" (Proceedings of the IEEE International Joint Conference on Neural Networks, 2005). U.S. Navy Patent 7,947,626 discloses a neural network MEA (mechanical array) derived from passaged progenitor cells, which can be used to detect and / or quantify various biological or chemical toxins. Koniku (www.koniku.com), the first company to develop a "wet chip" as a computing chip based on cultured neural cells, has been commercializing highly specialized products since 2017 for the detection of odorous compounds in the security, military, and agricultural / food markets, following the same procedure. As described in their patent application WO2018 / O81657, neuronal cells can be modified by gene editing, methylation editing, and various other biotechnological processes to express unique odorant receptor profiles using cell surface receptors, as known from the cutting edge of biology. Neuronal cells can also be coupled to a computer using cutting-edge neurophysiological interfaces, such as the electrodes of a multi-electrode array (MEA).A computer may measure the electrical signals generated by neural cells when exposed to odorants in specialized chambers and detect the presence of specific odorants using traditional signal processing methods, possibly including artificial neural networks (ANNs) or machine learning classifiers for learning and classification. ANNs are particularly useful for discriminating between multiple signals when networks of neurons with different odor receptor properties are employed in real-world environments that may contain complex combinations of multiple detectable compounds. However, one major limitation of such techniques is that they are limited to the application of very specific sensors.

[0007] Baker Hughes' U.S. patent application US20140279772 discloses the use of cultivated biological neural networks in a downhole processing system for signals conveyed downhole through a borehole in a container. The biological neural network is coupled to electrodes in an MEA, receiving input signals from sensors and outputting measurements from the neural network. The proposed BNN system also includes environmental control components such as a nutrient dispenser. While this disclosure mentions the advantages of BNNs over conventional computing systems in challenging environments, such as their parallel computing capabilities, robustness against vibration and electrical noise, and self-healing capabilities, it does not explicitly describe how the system is managed over time, particularly the dynamic learning process. Rather, because it is highly specialized for a specific application, pre-processing training can be applied as an offline preparation process rather than in a challenging runtime environment.

[0008] In his 2013 doctoral thesis, titled "Computing with simulated and cultured neuronal networks," Ju Han of the National University of Singapore explored the capabilities of dissociated neuronal cultures of 18-day-old rat embryonic cortical cells from the perspective of a state-dependent computational paradigm. He demonstrated that random networks formed by living neurons can process complex spatiotemporal information and are suitable for implementing prototype neurocomputer devices based on the Liquid State Machine (LSM) paradigm. To verify the ability of neuronal cultures to classify temporal and complex spatiotemporal patterns, he designed two stimuli: a jittered spike train template classification task, a benchmark test for LSM, and classification of randomly composed piano music. Processing temporal inputs relies on memory fading, and Ju Han observed that the short-term memory of dissociated neuronal cultures in his setup was longer than 4 seconds. To control neuronal cultures, Ju Han used optogenetics (a method of stimulating modified neurons with light) combined with multielectrode arrays (MEAs) to both stimulate neuronal inputs and record neuronal output. While MEAs can both electrically stimulate (BNN inputs) and measure (BNN outputs) with low spatial and temporal resolution, optogenetics allows for more precise stimulation control, non-contact manipulation, and repeated interrogation of neurons, making this field a more intensive area of ​​investigation over the past five years.Examples of recent exciting advances in optogenetics can be found in the review by Agus and Janovjak on "Optogenetic methods in drug screening: technologies and applications" in Curr. Opin. Biotechnol, December 2017, and in "Optogenetic stimulation and recording of primary cultured neurons with spatiotemporal control" by Barral and Reyes in Bio Protoc, August 2017, which describes a high-speed video projector based on the operation of a Digital Micromirror Device (DMD) that can spatially focus light stimuli down to single neurons while enabling temporal display patterns of 1.44 kHz or higher.

[0009] To optimize the overall functionality of the BNN system, Ju Han proposed using standard machine learning approaches (especially genetic algorithms) to further control the biological culture units and ultimately define the electrical stimuli and processing to achieve higher-level functionality. Instead of single-neuron readouts, as is often considered in neuroscience experiments, the output of the network layers can be read out and processed as multivariate signals (e.g., using MEA electrophysiology probes in combination with signal processing software such as Matlab).

[0010] Furthermore, recent research in neuroscience and neurocomputing has emphasized that biological neural systems, like the brain, require a learning process to develop their computing capabilities. In his doctoral thesis published in 2013, Ju Han pointed out that biological neural systems can learn and be manipulated through optical stimulation, and stated that combining this network's learning ability with drug manipulation could be a step toward optimizing neural circuits for computation. BNN systems can be trained to generate behaviors specified by a reference model through reinforcement learning, such as emitting global reward or punishment signals based on feedback from behavioral outcomes. Ju Han has proposed using drug treatment with an NDMA receptor antagonist for reinforcement learning.

[0011] More generally, in addition to the open-loop systems widely used in neuroscience experiments, closed-loop systems can be designed, as in typical automated systems engineering. Current state-of-the-art BNN systems are small-scale and tailored to very specific applications, and therefore suffer from two major limitations in terms of their use as central components of an overall general-purpose wetware computing system with high-level cognitive capabilities that are adaptive (to different needs) and evolvable (over time): their architecture is limited to a single input (stimulation of a single neuron or layer with a given configuration and protocol) and output layer design (measurement of a single neuron or layer with a given biological or signal processing method), and their inherent network connectivity limits them to processing only a single function at a time, regardless of whether they have been previously trained.

[0012] Therefore, new solutions and architectures are needed that harness the inherent advanced cognitive processing capabilities of biological neural networks as low-power alternatives or collaborative processing complements to traditional silicon-based information technology processing systems, devices, software, and electronic chips. Summary of the Invention

[0013] An automated processing system for converting spatiotemporal input data signals into spatiotemporal output data signals is described, the system comprising an in vitro biological nervous system cell culture (BNN core unit), an input stimulation unit (SU) adapted to apply input spatiotemporal stimulation signals to a first set of nervous system cells, an output readout unit (RU) adapted to capture output spatiotemporal readout signals from a second set of nervous system cells, one or more nutrient tanks connected to one or more nutrient dispensers to infuse one or more nutrients into the biological nervous system cell culture, one or more additive tanks connected to one or more additive dispensers, respectively, to infuse one or more additives into the BNN culture, one or more nutrient waste collectors for filtering and discharging nutrient waste from the BNN culture, and one or more additive waste collectors for filtering and discharging additive waste from the BNN culture. one or more angiogenic networks connecting the nutrient dispensers, the additive dispensers, the nutrient waste collectors, and the additive waste collectors to the BNN culture; one or more sensors for measuring at least one environmental parameter of the BNN culture; and an automated controller configured to adapt stimulus signals to input data signals and output data signals to readout signals and to control at least one of the environmental parameters of the BNN core unit, the nutrient supply of the BNN neural system cell culture, the additive supply of the BNN neural system cell culture, the nutrient waste collection of the BNN neural system cell culture, and the additive waste collection of the BNN neural system cell culture, so as to maintain homeostasis of the BNN neural system cell culture over time, such that the spatiotemporal input data signals are continuously converted into spatiotemporal output data signals.

[0014] The automatic controller may comprise a pre-processing unit for converting the input data signal into a stimulus signal using at least one of a spatiotemporal signal filter, a spatiotemporal signal classifier, a machine learning algorithm based on a mathematical or statistical model, an artificial neural network, a convolutional neural network, a support vector machine classifier, a random forest classifier, a genetic algorithm, a genetic programming algorithm, or a reservoir computing method.

[0015] The automatic controller may further comprise a post-processing unit for converting the readout signal into an output data signal using at least one of a spatiotemporal signal filter, a spatiotemporal signal classifier, a machine learning algorithm based on a mathematical or statistical model, an artificial neural network, a convolutional neural network, a support vector machine classifier, a random forest classifier, a genetic algorithm, a genetic programming algorithm, or a reservoir computing method.

[0016] Also described is a method for converting spatiotemporal input data signals into spatiotemporal output data signals using an automatic controller and a core unit of a biological neural network (BNN), the BNN core unit comprising at least an adapted in vitro neural system cell culture, adapted to provide spatiotemporal stimulation signals to a first set of neural system cells using an input stimulation unit (SU), and adapted to read out spatiotemporal signals from a second set of neural system cells using an output readout unit (RU). The method comprises the steps of pre-processing, with an automatic controller, spatiotemporal input data signals to form spatiotemporal stimulation signals; post-processing, with the automatic controller, spatiotemporal readout signals to form spatiotemporal output data signals; and maintaining homeostasis of the BNN neural system cell culture over time by controlling at least one of environmental parameters of the BNN core unit, nutrient supply of the BNN neural system cell culture, additive supply of the BNN neural system cell culture, nutrient waste collection of the BNN neural system cell culture, additive waste collection of the BNN neural system cell culture, pre-processing parameters, and post-processing parameters such that the BNN core unit continuously converts the spatiotemporal input data signals into spatiotemporal output data signals.

[0017] The method may also include minimizing an error between the output data signal and the subject output data signal by adapting at least one of the environmental parameters of the BNN core unit, the nutrient supply of the BNN neural system cell culture, the additive supply of the BNN neural system cell culture, the nutrient waste collection of the BNN neural system cell culture, the additive waste collection of the BNN neural system cell culture, the pre-processing parameters, and the post-processing parameters. [Brief explanation of the drawings]

[0018] [Figure 1] Figure 1 shows the core BNN unit as the building block of a biological computing server. [Figure 2]Figure 2a)b)c)d)e) shows various configurations and fabrication options for assembling the BNN core unit. [Figure 3] FIG. 3 is an exemplary schematic diagram of the automated angiogenesis system (AVS) operating with the BNN core unit. [Figure 4] FIG. 4 shows a side cutaway view of an exemplary sponge-like structure as a potential host for neural cells with the support of an intrinsic angiogenic network for growing neural cells in 3D BNN cultures. [Figure 5] FIG. 5 shows a side cutaway view of an exemplary sponge-like structure as a potential host for neural cells with the support of an intrinsic angiogenic network for growing neural cells in 3D BNN cultures. [Figure 6] Figure 6 is a schematic diagram showing a possible system for automated growth, maintenance, and control of BNNs. [Figure 7] FIG. 7 is another schematic diagram of the automated growth, maintenance, and control of BNNs, possible as a Biological Operating System (BOS). [Figure 8] FIG. 8 illustrates a learning process that can be implemented by real-time processing software. [Figure 9] Figures 9a) and 9b) show possible embodiments of the BNN Biological Computing Stack (BCS). [Figure 10] Figures 10a) and 10b) show a possible embodiment of a BNN Biological Computing Stack (BCS) suitable for training (T-BCS). [Figure 11] Figure 11 shows an example of implementing T-BCS using ANN. [Figure 12] Figure 12 shows an example of the maintenance process for a T-BCS. [Figure 13] Figure 13 shows the host server running the T-BCS wetware architecture. [Figure 14] FIG. 14 shows the functions that may be operated by the host server to manage user clients. [Figure 15] FIG. 15 further illustrates a general architecture with load balancing for serving multiple clients from the same T-BCS server host. [Figure 16] Figure 16 compares the possible applications of the proposed BNN server for image processing with legacy applications with deep learning architectures. [Figure 17] FIG. 17 shows ~200 μm wide neurospheres after 4 days of maturation of rat cortical stem cells. [Figure 18] Figure 18 shows a schematic example of 12 electrodes regularly arranged along the virtual circumference of a neurosphere of cortical neural stem cells at three different developmental stages. [Figure 19] Figure 19 shows a schematic example of 12 electrodes regularly arranged along the virtual circumference of a neurosphere of cortical neural stem cells at three different developmental stages. [Figure 20] Figure 20 shows a schematic example of 12 electrodes regularly arranged along the virtual circumference of a neurosphere of cortical neural stem cells at three different growth stages. [Figure 21] FIG. 21 shows a photograph of a biological neural network on an MEA circuit. [Figure 22] FIG. 22 shows a microscopic image of adherent cells and Matrigel, including neural cells, grown on the MEA surface. DETAILED DESCRIPTION OF THE INVENTION

[0019] <BNNコアユニット> Figure 1 illustrates a core BNN unit 100 as a building block of a biological computing server. The biological material of the BNN unit typically consists of an active biological culture 120, a collection of multiple living neural and glial cells, but is not limited to this. The cells may be assembled through a variety of different processes, including, but not limited to, cell culture or approaches such as organogenesis. In this disclosure, the term cell culture is used interchangeably to distinguish between growing cells in vitro and maintaining them for life in their natural in vivo environment. BNN cells may be arranged in two or three dimensions. The stimulation unit 110 (SU) represents the input interface between a digital data input signal 105 and the biological culture 120. It is used to selectively stimulate different neurons, dendrites, or axons. The stimulation unit 110 can control the transfer of the digital data input signal to the biological culture spatially (addressing different neurons), temporally (stimulating with variable signals over time), and / or by varying it spatiotemporally. Implementations of the SU 110 include, but are not limited to, light-induced stimulation such as multi-electrode arrays (MEAs), patch clamps, and optogenetic systems; magnetic fields, electric fields, ionic stimulation; focused laser light; optical tweezers; and mechanical stimulation via gravity or pressure changes. The readout unit 130 represents an output interface between the biological culture 120 and a digital data output signal 135, which is used to selectively measure the activity of different neurons, dendrites, or axons. The readout unit 130 can control the conversion of biological culture activity into a digital data output signal by sampling spatially (capturing the individual activity of different neurons), temporally (capturing signals that vary over time), and / or spatiotemporally, potentially in multiple dimensions.Implementations of RU130 include, but are not limited to, multi-electrode arrays (MEAs), patch clamps, imaging systems, ion-sensitive sensors, electrical-sensitive sensors, magnetic-sensitive sensors, chemical sensors, and other sensors suitable for neuronal culture.

[0020] A subset of neurons is excited by an input signal via a BNN input interface, such as an electrical signal from a multi-electrode array (MEA). Alternatively, some neurons can be genetically modified to receive optical stimulation from an optogenetics system as an input signal. The electrical activity of neuronal cells can be monitored at multiple locations in the biomaterial as an output of the BNN unit using a measurement system, such as a multi-electrode array (MEA), that receives the electrical signal. Alternatively, some neurons can be genetically modified to express fluorescence as an output signal from the BNN to an imaging sensor system. However, there are many other systems that can be integrated with BNNs. For example, processes that use concepts similar to those in the human body, such as converting electrochemical stimulation into observable mechanical movements, such as muscle movement or sound, can be implemented.

[0021] In a possible embodiment, a MaxOne MEA from Maxwell Biosystems (https: / / www.mxwbio.com) may be used as a host platform for the BNN core unit 100. Dissociated cell culture BNNs 120 may be plated and grown on MaxOne microsensors in CMOS technology following a protocol such as that suggested below from https: / / www.mxwbio.com / applications / neuronal-networks / applications / neuronal-networks / . Sample cell culture plating procedure A thin layer of polyethyleneimine (PEI) (Sigma, Missouri, USA) diluted to 0.05 wt % in pH 8.5 borate buffer (Chemie Brunschwig, Basel, Switzerland) is precoated on the surface of the electrode array. For cell attachment, add 10 μl of 0.02 mg / ml laminin (Sigma) in Neurobasal medium (Invitrogen, California, USA) dropwise. 〇After 20 to 30 minutes, add 1 ml of plating solution. After 24 hours, replace the plating medium with 1-2 ml of growth medium and maintain the cultures in an incubator under controlled environmental conditions (37°C, 65% humidity, 5% CO2). Change 50% of the growth medium twice a week.

[0022] In the protocol proposed by Maxwell Biosystems, the plating medium consists of 850 ml of Neurobasal supplemented with 10% horse serum (HyClone, Utah, USA), 0.5 mM GlutaMAX (Invitrogen, California, USA), and 2% B27 (Invitrogen, California, USA), but other formulations are possible for those familiar with cell culture techniques.

[0023] In the protocol proposed by Maxwell Biosystems, the growth medium consists of 850 ml of DMEM (Invitrogen, California, USA) supplemented with 10% horse serum, 0.5 mM GlutaMAX, and 1 mM sodium pyruvate, but other formulations are possible for those familiar with cell culture techniques.

[0024] In a possible embodiment, Maxwell MEA microsensors may operate as the stimulation unit 110. These allow for stimulation of BNN activity from input digital data patterns using a subset of active stimulation electrode sites. The input digital data patterns 105 may be prepared by various data processing methods and software. In a possible embodiment, a Maxwell stimulation module may be used as the SU 110 to provide 32 stimulation channels. Each stimulation channel may provide up to ±1.6 V voltage or ±1.5 mA current amplitude with 2 nA amplitude resolution and 2 μs time resolution. The MaxLab Live software component may generate a variety of digital data stimulation patterns appropriate for these resolutions, including monophasic, biphasic, and triphasic pulses, ramp waveforms, and other custom pulse shapes.

[0025] In a possible embodiment, Maxwell's MEA microsensors can act as readout units 130. These sensors can output digital data readouts of BNN activity, which can be recorded simultaneously using multiple active electrode sites, with configurable timescales ranging from microseconds to months. The digital data readouts 135 can then be processed by various signal processing methods. The readouts can also be visualized with an imaging system, for example, as a raster plot. The current Maxwell Biosystems MEA technology, as described by Ballini et al. in "A 1024-Channel CMOS Microelectrode Array With 26,400 Electrodes for Recording and Stimulation of Electrogenic Cells In Vitro," IEEE Journal of Solid-State Circuits, vol. 49, no. 11, pp. 2705-2719, 2014, features a high-resolution CMOS-based microelectrode array embedded as a two-dimensional plating, featuring 1024 low-noise readout channels, 26,400 electrodes, and a density of 3265 electrodes / mm², includes an on-chip 10-bit ADC, consumes only 75mW of power, and other configurations are possible.

[0026] While the above possible embodiments have been described using Maxwell Biosystems' MEA solution as an exemplary implementation of a high-density, high-throughput core BNN unit based on recent technological advances, it will be apparent to those skilled in the art that other neurotechnology, electrophysiology and / or optogenetics systems, circuits, devices, probes, components, software, protocols and methods may similarly be employed individually or in combination with one another to provide a functional core BNN unit 100, such as those developed by Multichannel Systems, Inc. (), a division of Harvard Bioscience Inc. (www.multichannelsystems.com), 3Brain (www.3brain.com), Nuvectra subsidiary NeuroNexus (www.neuronexus.com), Axion Biosystems (https: / / www.axionbiosystems.com / ), Charles River Laboratories (www.criver.com), Plexon (www.plexon.com), Koniku (www.koniku.com), mesh electronics for chronic recording at the single neuron level by the Potter Lab at Georgia Tech (https: / / sites.google.com / site / neurorighter / ), and the Lieber lab at Harvard University (http: / / cml.harvard.edu).

[0027] As will be apparent to those skilled in the art, different types of neuronal cells may be employed as the biological substrate of the BNN 120. Furthermore, the biological substrate may, of course, comprise a single neural cell type, a defined combination of different neural cell types, or even other cells. Furthermore, the composition of cell types may vary throughout the two-dimensional or three-dimensional structure of the BNN 120. Possible embodiments may employ rat fetal neural stem cells (NSCs), such as those offered by Invitrogen, catalog nos. N7744-100 and N7744-200, or Gibco® cell lines from ThermoFisher Scientific. Alternative embodiments may employ human neural stem cells (hmNPCs), such as Lonza Poietics™ Neural Progenitor Cells (NHNP), MilliporeSigma ReNcell® VM, or ReNcell® CX. Other examples include StemPro™ neural stem cells from ThermoFisher Scientific. Neuronal cells can be maintained in vitro using biological media such as MEM (Modified Eagle Medium - Gibco) or DMEM (Dulbecco's modified Eagle's medium - Gibco, Invitrogen, ThermoFisher).

[0028] In possible embodiments, BNNs 120 may be configured in neurosphere or neurovolume systems, adherent monolayer systems, or other specialized configurations and constructs to anchor BNN cells and / or other cells, such as neural stem cells (NSCs). As will be apparent to those skilled in the art of biomaterials, BNNs 120 may be configured in two dimensions, and preferably three dimensions, using various types of scaffolds to provide the cells with a suitable living environment. This includes, but is not limited to, processes for growing them into cerebral organoids or simply maintaining the position of living cells. The resulting in vitro cerebral organoids can function sustainably in a manner as close as possible to the mammalian brain environment. Examples of state-of-the-art scaffold formats and materials, such as hydrogels, that can be employed for the 3D culture and differentiation of a variety of rat, mouse, and human neural cell types are listed in Table 1 of "Scaffolds for 3D in vitro culture of neural lineage cells," A. Murphy et al., Acta Biomaterialia 54 (2017) 1-20. Hybrid hydrogels, such as those recently described in "Evaluation of RGD functionalization in hybrid hydrogels as 3D neural stem cell culture systems," Mauri et al., Biomater Sci. 2018 Feb 27;6(3):501-510, may also be employed. Commercially available hydrogel scaffolds, such as those from Corning, Lonza, Qgel, and Ibidi, may also be employed. In further embodiments, the scaffold itself may be composed of a biological material that allows for growth along with neural stem cell replication and differentiation.

[0029] The biocompatible material may also be specifically adapted for the microelectronic components of the stimulation unit SU110 and / or the readout unit RU130. As a possible embodiment, FIG. 2a) illustrates the possible application of mask etching to create a biocompatible layer 251 that closely matches the underlying multi-electrode array structure 250. Indeed, without precise positioning, the readout and stimulation electrodes would simultaneously read or inject signals into multiple neurons. Etching may be performed using a mask 252 on the side opposite the MEA array structure 250 to create a biocompatible layer 251 that guides neuronal positioning more precisely by aligning the biocompatible material 251 with the underlying MEA 250, so that the stimulation unit 110 can stimulate the BNN 120 at the neuronal level and / or the readout unit 130 can read out the BNN 120 at the neuronal level.

[0030] In a possible embodiment, a special adhesion layer may be coated where the RU sensor and / or SU probe are located to ensure that neural cells preferentially attach and / or grow in those areas, thereby facilitating control via the RU and SU interfaces. As a possible alternative embodiment, Figure 2b shows a schematic diagram of an assembly support 270 for a BNN core unit adapted to ensure maximum efficiency of communication between the RU and SU interfaces and neural cells. The RU and SU interface locations 271 may be coated with an adhesion layer 272 in a conventional approach, and an additional layer 274 with masking properties may be further deposited outside the RU and SU interface locations to prevent cells 273 from attaching and spreading outside the RU and SU interface locations. Alternatively, the interface layer may be composed of a special membrane that allows optimal attachment and exchange of molecules and atoms for different purposes, such as a Matrigel® matrix or any similar material that simulates growth and connectivity.

[0031] Figure 2c) shows a side view of a possible embodiment of a three-dimensional layered stack of neurospheres 280. A liquid solution of nutrients and additives useful for the growth and maintenance of BNN cultures may optimally flow throughout the neurosphere stack joints. That flow may be facilitated and controlled by various means, such as natural gravity, centrifugation, electrical or magnetic forces, and / or pumping.

[0032] Figure 2d) shows a possible embodiment of an optogenetic stimulation unit SU interface for inducing spikes in a genetically modified light-sensitive neuron 293. Multiple light beams 290, 291, 292 may be directed at the same neuron, with all beams intersecting at the location of the neuronal cell 293 to induce spikes. In this way, spikes can be generated in a three-dimensional collection of neuronal cells. Here, the intensity threshold of the light stimulation at which the illuminated neuron spikes is T. Then, the intensity value I of p beams p is the time when the individual intensity of each beam is below the spike threshold I p <Tでも、sum(I p ) ≧T. As will be apparent to those familiar with optogenetics technology, various illumination means can be used depending on the characteristics of the genetically modified light-sensitive nervous system cells. In a possible embodiment, an optogenetic laser may be used as the illumination beam. To excite specific neurons at a predetermined depth within a three-dimensional neuronal aggregate, a laser (or multispectral) beam may be focused to reach the maximum light energy precisely at the neuronal depth. Furthermore, the target position can be adjusted by changing the position of the light-emitting element or by changing the orientation of a mirror to change the beam direction.

[0033] As a possible alternative, Figure 2e) shows an alternative embodiment in which neurons 260 are deposited on a digital display support 261. The digital display support can be, for example, an LCD or OLED screen. Furthermore, because neurons range in size from 4 to 100 µm, compared to 5 µm or less for other technologies such as LCoS (Liquid Crystal on Silicon), DLP, or DMD micromirror approaches, it is possible to selectively control pixel illumination either outside the neuron 262, at the neuron's border 263, or at the neuron's periphery 264. Furthermore, color pixels can be selected to achieve specific effects on neuronal activity, tailored to the characteristics of genetically engineered light-sensitive neural cells.

[0034] <Building, growing, and maintaining automated BNNs> To acquire advanced cognitive abilities, BNN core units must be maintained in a stable, sustainable, and safe environment long enough to optimally grow and train their internal biological networks according to their biological needs, just as they would in vivo. As those skilled in the biotechnology field will appreciate, various automated systems have been developed for efficient biomanufacturing using well-established cell lines for many applications. However, mammalian nervous system cells are particularly vulnerable to in vitro manipulation, away from their natural brain environment, which is particularly well protected from potential disturbances such as mechanical strain and impact, contamination by pathogens and chemicals, and light exposure. Another issue is that while traditional cell line cultures typically operate at the cellular level (e.g., cells are the fundamental "factory" component for producing proteins of interest), for nervous system cells to operate as high-level cognitive function systems, they must operate at a higher-dimensional level, typically through the interconnection of multidimensional networks. Even more problematic is that it takes many years for the human brain to develop to the point where it can exhibit high-level cognitive function in vivo, and there is currently no research evidence that this process could be accelerated in vitro (Pasca, “The rise of three-dimensional human brain cultures”, Nature Vol. 553, Jan 2018).

[0035] Therefore, there is a need for a dedicated biological neural network automation method and system for producing, maintaining, and controlling neural cell cultures in a sustainable, reproducible process that can be performed automatically and in parallel. The production process consists of both the assembly of BNN core unit elements and the growth of BNN cultures into functional BNN core units, including the automated learning process. The maintenance and control process consists of various activities such as feeding, stimulation, measurement, and even curing and cleaning. Possible embodiments of such an automation method and system are now described in more detail.

[0036] To scale up the development of BNN core units beyond the current manual laboratory setup, the manufacturing process may include automated assembly of BNN core unit elements. In possible embodiments, 3D bioprinting of neural cells, along with the scaffold and the necessary components for growing neural cells, may be employed. Examples of 3D biocompatible tissue engineering include products developed by Sichuan Revotek Co., Ltd., Biosynsphere, Organovo, and Aspect Biosystems. In possible embodiments, the BNN core unit assembly may employ neural progenitor-derived stem cells, as described, for example, in "3D printed stem-cell derived neural progenitors generate spinal cord scaffolds," Adv. Funct. Mater. 2018. This may facilitate optimal cell localization directly on the scaffold. Clusters of cells in a bioink, such as a cell-laden hydrogel, may be deposited in successive layers of channels with a resolution of approximately 200 μm. As the authors report in their study of tissue models and future implants for treating spinal cord injury, this method facilitates both axonal propagation and the maintenance of cell viability and mechanical stability during the in vitro CNS tissue construction process.

[0037] To automatically maintain the pre-assembled BNN core units in operation beyond the current manual laboratory maintenance tasks, a second step in the automation process could involve automated vascularization via microfluidic circuits to biologically supply the BNN core unit's cells and collect their biological waste. Only limited solutions have been proposed for in vitro vascularization of 3D brain organoids, and they remain size-limited due to central necrosis of cells that are no longer accessible by peripheral nutrients after the organoids reach a certain depth and density. Therefore, research biologists have recently proposed transplanting them into adult mouse brains (Mansour et al. "An in vivo model of functional and vascularized human brain organoids," Nature Biotechnology, Vol. 36, No. 5, May 2018 and Lancaster, "Brain organoids get vascularized," Nature Biotechnology, Vol. 36, No. 5, May 2018). The technical feasibility of vascularizing brain organoids with the patient's own endothelial cells was also recently demonstrated by "Generation of human vascularized brain organoids," NeuroReport, Vol. 29, Issue 7, pp. 588-593, May 2018. However, the latter approach is essentially limited to natural fluid distribution and can only be controlled externally in a "black box" manner, limiting automation capabilities to very simple models. To overcome this limitation of previous BNN cultures, various possible alternative embodiments can be considered, either separately or in combination, as will now be described in more detail.

[0038] FIG. 3 is a schematic diagram illustrating a first possible embodiment of an Automated Angioplasty System (AVS) 300 operating with the BNN core unit 100, comprising: One or more nutrient tanks 319 connected to one or more nutrient dispensers 320 for injecting one or more nutrients into the BNN culture 120. The nutrient dispensers 320 may include a mixer-injector active module responsible for delivering the correct dose of nutrients according to the specific nutritional needs of the BNN culture using mechanical devices 351 such as valves, injectors, or pumps. One or more additive tanks 321, 323 connected to one or more additive dispensers 322, 324, respectively, that inject one or more additives into the BNN culture 120. The additive dispensers 322, 324 may include a mixer-injector active module that is responsible for delivering the correct dose of additive depending on the growth needs of the BNN culture using mechanical devices 352, 353 such as valves, injectors, pumps, etc. · One or more nutrient waste collectors 325, including mechanical devices 331 such as valves, injectors, pumps, etc., for filtering and discharging the nutrient waste into a tank 335 or returning it to the BNN. · One or more additive waste collectors 326, 327 equipped with mechanical devices 332, 333 such as valves, injectors, pumps, etc., for filtering and discharging additive waste into tanks 336, 337 or returning it to the BNN. One or more angiogenic networks for connecting nutrient and additive dispensers 320, 322, 324 and nutrient and additive waste collectors 325, 326, 327 to the BNN culture 120. Different angiogenic flows can also be re-injected into the system to minimize losses.

[0039] Nutrients include amino acids, carbohydrates, vitamins, minerals, etc., which may be in the same solution or in separate solutions.

[0040] Additives may consist of chemicals, drugs, or other elements, such as dopaminergic stimulation enhancers that increase the dopaminergic response of BNNs or dopaminergic stimulation inhibitors that decrease the dopaminergic response of BNNs. As will be apparent to those skilled in the art of automation, this allows for the reproduction and control of the dopaminergic / anti-dopaminergic system. More generally, additives may include chemicals known to affect central nervous system neurotransmitters, such as Botox, nicotine, curare, amphetamine, cocaine, MDMA, strychnine, THC, caffeine, benzodiazepines, barbiturates, alcohol, and opium. Additives may also include growth factors, hormones, and gases (e.g., CO2, O2).

[0041] Each nutrient dispenser 320 is interconnected to the BNN culture 120 via a nutrient angiogenic network 328 that delivers nutrients to the BNN cells. Each additive dispenser 322, 324 is interconnected to the BNN culture 120 via an additive angiogenic network 329, 330 that delivers additives to the BNN cells. The nutrient angiogenic network 328 and the additive angiogenic network 329, 330 may be the same network or different networks.

[0042] Each nutrient waste collector is interconnected to the BNN cells 120 via a nutrient waste angiogenic network that conveys nutrient waste from the BNN cells. Each additive waste collector is interconnected to the BNN culture 120 via an additive waste angiogenic network that conveys additive waste from the BNN cells.

[0043] The nutrient waste angiogenic network may be the same as or a different network from the nutrient angiogenic network 328. The additive waste angiogenic network may be the same as or a different network from the additive angiogenic networks 329, 330. The angiogenic networks may be fabricated using three-dimensional bioprinting using biocompatible materials or may be grown from stem cells on a culture support for BNNs 120.

[0044] It has been noted that angiogenesis may consider a similar system in the human brain or an entirely different type, such as a porous material structure. Figure 4 shows a side-cut view of an exemplary embodiment of a sponge-like structure as a possible host for neural cells 440, providing support for an intrinsic angiogenic network for growing neural cells 440 in a three-dimensional BNN 120. The sponge material 439 is preferably soft and compressible to mechanically host the BNN cells 440 within its pores 442 (as shown in 410) while accommodating subsequent growth and development as a three-dimensional BNN culture. The sponge material 439 is also preferably porous and absorbent, delivering nutrients and / or additives and / or waste products throughout the BNN culture. In a possible embodiment, a solution containing neural cells 440 may be soaked into the sponge-like structure 439, resulting in an orderly distribution of the neural cells throughout its pores and channels. In a possible embodiment, polyethyleneimine (PEI) polymer may be used to enhance adhesion factors. In a further possible embodiment, neural stem cells 440 may be submerged in a solution containing growth factors, such as fibroblast growth factor FGF2, to prevent cell specialization until the stem cells are distributed onto the sponge-like structure 439. Once the distributed cells attach to the sponge-like structure 439, the BNN culture 120 may be assembled, and the sponge-like structure 439 may further be used as its angiogenic system. The adhesion and growth factor solution may be washed away and replaced with a further nutrient and additive solution 441 suitable for the specialization, growth, and maintenance of the BNN culture cells. In another embodiment (not shown), induced pluripotent stem cells may be mechanically inserted and cultured directly onto the sponge-like structure 339 as their own angiogenic network system.

[0045] 5 shows a top cutaway view of another exemplary embodiment of a sponge-like structure as a possible host for neural cells 544 with a native angiogenic network support for growing neural cells 544 in 3D BNN cultures while facilitating the interfacing of electrical and / or optical components of stimulation and / or readout units to the 3D BNN cultures. To this end, the sponge-like structure may be formed as a sphere, and electrophysiological clamps or electrodes may traverse the sponge-like structure from various locations distributed around the sphere and penetrate it at different depths.

[0046] In a further possible exemplary embodiment (not shown), 3D BNNs may be cultured in a sandwich-like configuration along a biocompatible filament suspended between two flat surfaces. Neural cells may be attached to the biocompatible filament in a variety of ways. In a possible embodiment, the filament may be coated with an attachment factor such as PEI, although other embodiments are possible.

[0047] <BNNコントローラ> FIG. 6 is a schematic diagram illustrating an example of an automated BNN growth, maintenance, and control system, including a BNN core unit 120, a stimulation module 110, a readout module 130, and an automated angiogenesis system (AVS) 300, which may include nutrient and additive reservoirs, dispensers, and waste collectors. In a preferred embodiment, the operation of the AVS is directed by a BNN automated controller 600, which is responsible for calculating in real time the selection and amounts of nutrients and additives to be infused into the BNN culture by the AVS, as well as the amount of waste collected from the BNN culture by the AVS. The BNN automated controller may be built with electronic hardware and a computer processor adapted to execute software algorithms. In a simple embodiment, the BNN automated controller may operate in an open loop, identifying and quantifying required nutrients and additives and deriving resulting waste values ​​based on scientific expertise in neurophysiological parameterization. In further embodiments, the BNN automated controller may also continuously monitor readout information 635 from the BNN readout module 130 to determine the current state of the BNN culture and adjust the parameters of the AVS accordingly.

[0048] In yet another embodiment, the BNN automatic controller may control the stimulation signal 605 for the stimulation unit to operate in synchronization with the state of the BNN. For example, the nutrients and additives infused into the BNN culture medium by the AVS and the waste products collected vary depending on the maturation stage of the BNN cell along its life cycle, such as the differentiation stage (assembly), growth stage (learning), operation stage (stable function), and death stage (excessive waste production). The rate of reinfusion and disposal of nutrients and drugs can be controlled by the nutrient dispenser, additive dispenser, and respective waste collector. These feedback loops allow the amount of nutrients and drugs to be dynamically adjusted to: 1) optimize lifespan, 2) minimize manual care, and 3) adjust BNN responsiveness and stabilize performance. In this way, BNN homeostasis can be maintained over time.

[0049] In a possible embodiment, the BNN auto-controller 600 may directly provide the raw data input signal 105 as the stimulus signal 605. In an alternative embodiment, the BNN auto-controller may further include a pre-processing unit 610 responsible for converting the data input signal 105 into the stimulus signal 605. This allows the BNN auto-controller to better adapt the raw input to the capabilities of the actual stimulus unit SU and the capabilities of the BNN, so that the end-to-end BNN processing task is optimally performed without the end user having to specifically adapt their input signal for each possible configuration. This also facilitates the learning task, as the capabilities of the BNN can evolve over time and the signal pre-processing 610 can be adapted accordingly.

[0050] In a possible embodiment, the BNN automation controller may directly output the raw BNN readout signal 635 as a signal resulting from end-to-end BNN system processing. In an alternative embodiment, the BNN automation controller 600 may further include a post-processing unit 630 responsible for converting the raw BNN readout signal 635 into the output signal 135. This allows the BNN automation controller to better adapt raw readouts, which may be too noisy to easily interpret, to the needs of the actual application. An exemplary application is the extraction of relevant signals from a series of spikes using a spike-sorting signal processing algorithm, but other approaches, such as statistical models, classifiers, or machine learning methods, may also be used. This also facilitates the learning task, as the end-to-end BNN system can be trained to associate a generic output signal of interest 135 with a predefined input signal 105, without the end user having to specifically interpret the output signal measured according to each possible internal BNN biological culture or readout unit configuration. This also facilitates the learning task, as the BNN's capabilities can evolve over time, and the signal post-processing 630 can be adapted accordingly.

[0051] Figure 7 shows another schematic diagram of automated BNN growth, maintenance, and control, possibly as a biological operating system (BOS, a computer operating system analogy). Such a BOS could host real-time functional control software for BNN processing unit functions by abstracting the BNN automation process transparently to end users, similar to how traditional computer operating systems or cloud computing services abstract underlying hardware-specific devices and operations. The proposed BOS operates in conjunction with the previously described BNN core unit culture 120, stimulation and readout modules SU / RU 110 / 130, and a BNN health control unit 710 for the automated angiogenesis system (AVS). The BNN health control unit 710 may include nutrient and additive tanks and dispensers, a chemical control unit with a waste collector, and an environmental control unit responsible for controlling other environmental parameters of the BNN culture, such as temperature, pressure, humidity, and O2 or CO2 ratios. The environmental control unit may include, for example, sensors for temperature, humidity, pH, and CO2, as well as digitally controlled micropumps for delivering chemicals. The environmental control unit may also modify other environmental parameters, including, for example, sound or light waves of any frequency band. The BNN health control unit is responsible for monitoring the health of the BNN cells under real-time monitoring by the BNN health control software 700. This health monitoring system acts as a homeostatic system, transparently adjusting for natural changes in BNN performance over time. This system ensures proper functionality of the BNN processing unit over time by adjusting environmental parameters (e.g., chemicals, nutrients, temperature, CO2). This may include periodically checking that a training set of SU input signals still results in the expected RU output signals. The BNN health control software 700 may be managed by a system user administrator via the administrator interface 740.

[0052] The real-time function control software 701 is further responsible for managing the function of the BNN as a processing unit via a BNN function interface 730. The BNN interface 730 may be used to process the inputs and outputs of the BNN in real time, which are interfaced with the SU / RU system described above. The BNN interface 730 may, for example, apply any necessary stimulus signal pre-processing tasks (610) and / or readout signal post-processing tasks (630).

[0053] An end user can interact with the real-time control software 701 through a user interface 741. The user interface 741 may be the same or different user interfaces for the end user and / or the administrative user, but if the user is the same, the administrative user may have access to more functionality. Parameters for the real-time function control software 701 as well as the BNN health control software and / or BNN function interface may be stored in a database 720.

[0054] For higher level BNN cognitive processing unit functions that may require very low latency and high data throughput bandwidth, the BNN function interface 730 may apply additional data pre-processing 610 and / or data post-processing algorithms 630 under control of real-time function control software 701. In possible embodiments, the real-time control software does not operate over the internet but operates locally. Thus, function control software 701 may be uploaded by an administrator using administrator interface 740 or directly by an end user using user interface 741.

[0055] The BNN controller computer system (also referred to herein as the "system" or "automated system") is programmed or otherwise configured to perform different BNN processing methods, such as receiving and / or combining stimulus input signals, processing them, and generating and / or combining readout output signals, depending on a given application.

[0056] The BNN controller may be a computer system or part of a computer system that includes a central processing unit (CPU, herein "processor" or "computer processor"), memory such as RAM, a storage unit such as a hard disk, and a communications interface for communicating with other computer systems over a communications network, e.g., the Internet or a local network. Examples of computing systems, environments, and / or configurations include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, etc. In some embodiments, the computer system may comprise one or more computer servers, which may be capable of operating with numerous other general-purpose or special-purpose computing systems, enabling distributed computing such as cloud computing, e.g., in a BNN data farm. In some embodiments, the BNN controller may be integrated into a massively parallel system.

[0057] The BNN controller system may be applied in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform particular tasks or implement particular abstract data types. Program modules may be used in native operating system or file system functions; in standalone applications, such as browser or application plug-ins or applets; in commercial or open source libraries and / or library tools programmed in Python, Biotython, C / C++, or other programming languages; in custom scripts, such as Perl or BioPerl scripts; or in highly specialized languages ​​suitable for linear genetic programming or cognitive computing, such as SlashA, machine code, or other data structures that describe computational steps (such as those used in Push Genetic Programming, Cartesian Genetic Programming, and Tree-Based Genetic Programming), as is well known to those skilled in the art.

[0058] The instructions may also be executed in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

[0059] In this way, the entire system may operate as a biological operating system 750, as it brings together all the systems necessary to ensure the operation of the BNN computing system.

[0060] In possible embodiments, the BNN function interface 730 may include various input and / or output signal processing algorithms, such as pre-processing or post-processing filters, classifiers, and machine learning algorithms based on mathematical or statistical models, and the function control software 701 may control these algorithms and their parameters according to the needs of the end user. Signal pre-processing may include converting the signals to be trained by the BNN to optimally adapt to the SU stimulus capabilities and format. Signal post-processing may include converting the signals output from the BNN into a more comprehensive format to provide the target function. In possible embodiments, the BNN may be modeled as a nonlinear system, and signal pre-processing may include applying nonlinear gains to the input signals. In possible embodiments, the BNN function interface may include one or more artificial neural networks as pre-processors and / or post-processors, and parameters may include weight values, activation function selection, and other parameters to achieve signal learning and / or classification. For example, a pre-processor may be useful for repeating the input application signal over a period of time or slightly changing the signal for more robust training or reinforcing the learning process.

[0061] In general, the goal of automation is to build a BNN processing system that generates the correct spatiotemporal output signal Oi(t) for a given spatiotemporal input signal Sj(t) (i is the stimulating input electrode, j is the detecting output electrode, and t is the sampling time). Typically, a BNN can be trained on a set of k different functions {O(t), S(t)}(k), called the "training set," where O(t) and S(t) represent vectors of components Oi(t) and S(t), respectively. After successful training, the BNN can be used to predict the correct output for inputs S(t) that do not belong to the training set. Successful training and prediction can be measured as minimizing the difference between the target signal O(t) and the measured BNN output signal. Various metrics can be used for this purpose, such as mean squared error (MSE) or, more generally, n-norm distances such as Euclidean distance or 1-norm distance. Formally, this means that it can output the correct value for a set of p different functions {O(t), S(t)}(p), called the "test set," which do not belong to the training set.

[0062] In general, the goal of automation is to build a BNN processing system that generates the correct spatiotemporal output signal Oi(t) for a given spatiotemporal input signal Sj(t) (i is the stimulating input electrode, j is the detecting output electrode, and t is the sampling time). Typically, a BNN can be trained on a set of k different functions {O(t), S(t)}(k), called the "training set," where O(t) and S(t) represent vectors of components Oi(t) and S(t), respectively. After successful training, the BNN can be used to predict the correct output for inputs S(t) that do not belong to the training set. Successful training and prediction can be measured as minimizing the difference between the target signal O(t) and the measured BNN output signal. Various metrics can be used for this purpose, such as mean squared error (MSE) or, more generally, n-norm distances such as Euclidean distance or 1-norm distance. Formally, this means that it can output the correct value for a set of p different functions {O(t), S(t)}(p), called the "test set," which do not belong to the training set.

[0063] In some embodiments, to achieve this successful learning, machine learning algorithms (particularly artificial neural networks (ANN), convolutional neural networks (CNN), support vector machines (SVM), deep learning, including any ML approach such as random forests, genetic algorithms, genetic programming, reservoir computing, etc.) may be used to optimally present inputs and measure outputs. For example, - {O(t), S(t)}(k) - some of the k functions may need to be repeated more frequently than others, Some of the inputs may be mapped to different electrodes over time. - Simultaneous reading of multiple electrodes may be required. - And many more... is.

[0064] More generally, raw signal input 135 may be provided to a first pre-processing subunit 610 of the BNN interface 730 for processing before being provided to the BNN 120 with the SU 110, while the output signal 635 of the BNN 120 read from the RU 130 may itself be further transformed by a second post-processing subunit 630 of the BNN interface 730 to generate a more suitable output signal 135 depending on the needs of the application. In an open-loop architecture, the pre-processing and post-processing subunits may operate independently of each other. In a closed-loop architecture, they may operate jointly, e.g., some outputs from the post-processing unit may be fed back to the stimulus pre-processing unit.

[0065] Preferably, the pre-processing unit may perform spatio-temporal processing of the raw input Sr(t) to generate a processed signal S(t) that is fed to the BNN together with the SU. The BNN then processes the raw signal O rThe pre-processing unit 610 outputs a time domain signal O(t), which is processed by a post-processing unit to generate a processed output signal O(t). These signals may have different dimensionalities. For example, for a BNN to output a single spike in response to a simple raw one-dimensional sinusoidal signal with a frequency of 1 Hz, it may be necessary to sequentially excite 100 different electrodes with different time domain signals defined with a resolution of 100 Hz. The pre-processing unit 610 may be adapted accordingly. Machine learning may also be used for the pre-processing unit 610 as well as the post-processing unit 630. In the latter embodiment, the three subsystems may be trained to achieve the goal of globally minimizing the difference between the target signal and the measured final output signal 135 for a pre-defined input signal 105. It should be noted that as BNN technology develops, it is conceivable that the pre-processing unit 610 and / or the post-processing unit 630 may be implemented as wetware rather than software or hardware, e.g., as a simple BNN, to gradually build more complex systems suitable for learning higher cognitive functions.

[0066] In this way, the BOS 750 integrates both health monitoring and functional control of the BNN culture 120 to provide the end user with real-time, sustainable, and reliable BNN computing operation (including learning) by realizing the following functions: Homeostasis function, which works by optimizing the BNN functional interface 730 and ECCU parameters 710 to ensure a constant level of performance. A learning function that works by optimizing the BNN interface (which may include input signal pre-processing and output signal post-processing algorithms) in collaboration with the ECCU parameters necessary to learn new input-output relationships. In particular, the function control software 701 has the ability to adjust the supply of additives and / or the values ​​of environmental parameters for BNN health control in real time to promote optimal learning on the training set in closed-loop coordination with the stimulus and readout signals processed by the BNN function interface. · A maintenance function that may require cloning a culture of an aged BNN to operate directly on a training set already learned by an existing BNN. At least some of the BNN interface and ECCU parameters of the existing BNN are retrieved from the database 720 and can be replicated to shorten the learning time in the new BNN.

[0067] <BNN Automatic Learning> The essential dynamic interconnectivity of neuron cells is considered a means of creating (re)programmable functions based on data feeds and constitutes the building blocks necessary for machine learning. As is clear to those proficient in neuroscience techniques, the BNN system can first use the input data signal 105 to provide the desired output data signal 135 and can be learned until it reaches a stable state for inputs not supplied during learning. When the BNN can associate the output data signal of the target with the input data signal as expected, the system can be said to have learned and generalized and can generate a deterministic response when a given input is supplied.

[0068] To learn the BNN, the automatic controller may adapt the input spatio-temporal signal 605 for the stimulation unit SU to both the input data signal 105 and the output spatio-temporal data signal 135 until the output spatio-temporal data signal matches the target output data signal. The automatic controller - Environmental parameters of the BNN core unit - Nutrient supply for the BNN nervous system cell culture - Additive supply for the BNN nervous system cell culture - Collection of nutrient waste from the BNN nervous system cell culture - Collection of additive waste from the BNN nervous system cell culture - Parameters of the preprocessing algorithm for BNN interface signals - Parameters of the postprocessing algorithm for BNN interface signals of at least one or more may be further adapted until the output spatio-temporal data signal matches the expected output data signal.

[0069] FIG. 8 illustrates a learning process that may be implemented by real-time processing software, with a feedback loop H800 for adapting the stimulation signal 605 to the measured readout signal 635. In a possible embodiment, the real-time processing software may trigger periodic repetition of the stimulation signal 605 to the BNN to promote long-term potentiation. More generally, environmental and chemical parameters controlled by a health monitoring system may also be part of the closed-loop learning system. In a possible embodiment, the real-time processing software may trigger the administration of a drug known to strengthen the BNN as a reward when the measured readout signal 635 matches the expected signal of the training set for a given stimulation signal input.

[0070] In a further possible embodiment (not shown), the BNN may be interfaced with external information sources, such as internet web databases, through an interface in the BNN automation controller, and by accessing and utilizing this external information, the BNN may learn more, connect new concepts, and acquire more advanced cognitive abilities over time.

[0071] <Biological Computing Stack> For the feedback loop to function, careful control of the BNN core units is necessary. The proposed automated angiogenesis system may be limited to growing and maintaining relatively small BNN cultures, e.g., on the order of 10,000 interconnected neurons, compared to mammalian brains. Therefore, integrating a single BNN core unit into a BNN computing system may not be sufficient for learning and performing high-level cognitive functions. This limitation can be overcome by a BNN biological computing stack (BCS), as shown in Figures 9a and 9b. In a possible embodiment, one or more BNN core units may be arranged in series. The readout unit (RU) of the first BNN core unit may be connected to the stimulation unit (SU) of the next BNN core unit, or they may be integrated into a single piece of hardware. Each interface between blocks may also accept an external stimulus signal input (ES) 900, as shown in Figure 9b.

[0072] In one possible embodiment, two-dimensional BNN core units may be stacked vertically, and in another embodiment, BNN core units fabricated as stacks of neurospheres or layers of three-dimensional bioprinted material may be mechanically arranged in a series stack with electrophysiological probes inserted at the interface between adjacent units.

[0073] As shown in Figure 9b), the learning process can be controlled end-to-end using a feedback loop operating between the last readout unit and the first stimulation unit, in an arrangement similar to the layered architecture of deep learning. In this configuration, the inherent nature of biological neural networks allows the feedback elements to be used as stabilizers, changing their internal state even in the absence of external stimuli.

[0074] 10a) and 10b) show two further embodiments of an adaptive closed-loop system controlling a BNN computing stack with an additional nonlinear gain P1000, with and without an additional post-processing block O1010, respectively. As will be apparent to those familiar with deep learning techniques, the pre-processing block P may facilitate the learning process, while the post-processing block O may represent a topology for reservoir computing. Such a configuration realizes a Trainable Bio-Computing Stack (T-BCS).

[0075] FIG. 11 shows a specific implementation example related to FIG. 10b) where the output signal post-processing block 1010 of O is implemented as an artificial neural network (ANN). The biological neural network (BNN) 120 is connected to a digital interface, the readout unit (RU) and the stimulation unit (SU), via a multi-electrode array (MEA) 1135. The feedback function (H1) 1137 corresponds to the ANN's learning process, such as, but not limited to, backpropagation. The outer feedback function (H2) 1136 corresponds to a means for imposing long-term potentiation on the BNN. The outer feedback function H2 can be achieved using genetic programming or machine learning to determine the correct spatiotemporal sequence that produces the desired spikes in the output of the BNN culture 120. Another way to impose long-term potentiation is to use the RU as a stimulation unit with the desired spike sequence to strengthen the internal connections within the BNN. Using this approach, the ANN may be omitted in some embodiments.

[0076] S(t) is the number of different stimulation units S of the MEA. i is a time (t)-dependent periodic data input function for the different readout units of the MEA. i Let the data output function for a given S(t) be O(t).

[0077] And H2 is a function or algorithm (discovered with the help of genetic programming or machine learning as an example) that minimizes (or maximizes) the scalar metric L, such that the BNN always produces the same spike time function O(t) for a given input S(t) when L is minimum (or maximum) for any number of periods P. L is known as the loss (or reward) function.

[0078] For example, L can be:

number

[0079] FIG. 12 illustrates a further possible embodiment of the implementation of FIG. 11 to facilitate maintaining T-BCS operation over time. Indeed, the BNN culture 120 may evolve over time from its initial (learned) structure, thus potentially causing drift in the T-BCS's data output. This can be monitored by periodically testing T-BCS operation using a validation set that includes one or more of the initial signal training set as SU inputs. The output of the T-BCS's readout unit RU may then be checked against the expected prediction set; if they differ too much, the T-BCS must reapply the training process. Because the training process takes time and disrupts the T-BCS's functionality, a possible embodiment involves replacing the T-BCS system with a new, pre-trained one.

[0080] <BNNサーバ> The proposed automated BNN system, BNN operating system (BOS), and trainable BNN computing stack (T-BCS) can provide the core architecture of a wetware-based computing server. Such a BNN server may be more suitable for providing a higher level of cognitive processing more efficiently than conventional software- or hardware-based server architectures.

[0081] As shown in FIG. 13, the wetware architecture of the T-BCS can operate a host that provides different services to user clients. FIG. 14 further shows functions that may be operated by a host server to manage user clients, such as request processing, job scheduling, log reporting and recording, dashboard management, invoice issuance, etc. FIG. 15 further shows a general architecture with load balancing for providing services to multiple clients from the same T-BCS server host.

[0082] Preferably, the BNN server supports redundant T-BCS operation such that at least one other T-BCS continues to operate to respond to client requests even if one T-BCS is removed for maintenance (such as retraining).

[0083] <Maintenance and Update of BNN> [Exemplary Application] Figure 16b) illustrates a potential application of the T-BCS server compared to a traditional deep learning server 16a), namely, a reverse image search service. In the legacy application of Figure 16a), a reverse image search service, such as that implemented by Google, accepts an image as input, tells where the image is located online, provides a description of the image, and potentially returns a collection of similar images. The prior art reverse image search backend in Figure 16a uses several different image processing or image vision systems 1613 to obtain a robust, compact representation of the image, typically as a hash value 1614. The proposed application replaces the complete backend processing with the T-BCS 1615, achieving significant power consumption reductions for the same computing power. In particular, when properly trained, the Biological Computing Stack (BCS) can advantageously replace both the feature extraction portion, consisting of a machine learning AI network, a feature descriptor extraction algorithm, a perceptual hash function, additional image processing tasks, or any combination of these tasks, and the so-called post-hashing process required to obtain a compact representation of the image to be stored in a database.

[0084] [Other experiments, implementations, and applications] Figure 17 shows a ~200 μm wide neurosphere formed after four days of maturation of rat cortical stem cells in DMEM-F12 medium supplemented with Glutamax™, Fibroblast Growth Factor, Epidermal Growth Factor, and Stempro®. In a possible embodiment, three-dimensional electrodes may be distributed over the surface of the neurosphere (or other types of neuronal aggregates). After growth stimulation, the electrode tips gradually become embedded within the growing neurosphere. Figures 18-20 show a schematic example of 12 electrodes regularly arranged around the virtual circumference of a cortical neural stem cell neurosphere that grew from 400 μm to over 1 mm over 15 days, with the tip of each electrode naturally penetrating deep into the neurosphere. In this particular case, growth was stimulated by a Matrigel® matrix extracted from Engelbreth-Holm-Swarm (EHS) mouse sarcoma.

[0085] Matrices can also be used to stimulate three-dimensional growth of neural cells directly on MEAs. This is shown in Figure 21 where adherent cells 2120 are grown to extend neurites 2121, 2122 or migrate through the Matrigel® matrix 2100. This allows for thicker networks that may extend several millimeters or more from the MEA surface 210. Figure 22 shows a microscopic image of adherent cells 2201 and Matrigel® 2202, including neural cells, grown on the MEA surface. As will be apparent to those skilled in the art, the same principles apply to neurospheres or any collection of neural cells.

[0086] While various embodiments have been described above, it should be understood that they are illustrative and not limiting. It will be apparent to those skilled in the relevant art that various changes in form and detail can be made therein without departing from the spirit and scope thereof. Indeed, after reading the above description, it will be apparent to those skilled in the relevant art how to implement alternative embodiments.

[0087] As will be apparent to those skilled in the art of digital data communications, the methods described herein are equally applicable to a variety of data structures, such as data files, data streams, etc. Accordingly, the terms "data," "data structure," "data field," "file," or "stream" may be used interchangeably herein.

[0088] Although the above detailed description contains many specific details, these should not be construed as limiting the scope of the embodiments, but merely as exemplifying some of the embodiments.

[0089] While various embodiments have been described above, it should be understood that they are illustrative and not limiting. It will be apparent to those skilled in the relevant art that various changes in form and detail can be made therein without departing from the spirit and scope thereof. Indeed, after reading the above description, it will be apparent to those skilled in the relevant art how to implement alternative embodiments.

[0090] Furthermore, it should be understood that the diagrams emphasizing functionality and advantages are presented for illustrative purposes only, and the disclosed methods are sufficiently flexible and configurable that they may be utilized in ways other than those illustrated.

[0091] In this specification, claims, and drawings, the term "at least one" is often used, but terms such as "a," "an," "the," and "said" also mean "at least one" or "the at least one" in this specification, claims, and drawings.

[0092] In this specification, multiple instances may implement a component, operation, or structure described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed simultaneously, and the operations need not be performed in the order illustrated. In illustrated configurations, structures and functions presented as separate components may be implemented as combined structures or components. Similarly, structures and functions presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter of this specification.

[0093] Certain embodiments are described herein as including logic or multiple components, modules, units, or mechanisms. A module or unit may constitute either a software module (e.g., code embodied on a machine-readable medium or in a transmission signal) or a hardware module. A "hardware module" is a tangible unit capable of performing specific operations, and may be configured or arranged in a specific physical manner. In various exemplary embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules (e.g., a processor or group of processors) of a computer system may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform specific operations as described herein.

[0094] In some embodiments, a hardware module may be implemented mechanically, electronically, biologically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic permanently configured to perform specific operations. For example, a hardware module may be a special-purpose processor, such as a field-programmable gate array (FPGA) or an ASIC. A hardware module may also include programmable logic or circuitry temporarily configured by software to perform specific operations. For example, a hardware module may include software contained in a general-purpose processor or other programmable processor. It should be understood that whether a hardware module is implemented mechanically, in dedicated, permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be determined by cost and time considerations. With the development of synthetic biology, hardware modules may be fabricated in whole or in part from biological cells (also known as wetware), such as neurospheres or genetically engineered cells.

[0095] Various operations of the example methods described herein may be performed, at least in part, by one or more processors that are temporarily (e.g., by software) or permanently configured to perform the associated operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, a "processor-implemented module" refers to a hardware module implemented using one or more processors.

[0096] Similarly, the methods described herein may be at least partially implemented by a processor, which is an example of hardware. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules.

[0097] Some portions of the material described herein may be presented in terms of algorithms or symbolic representations of operations on data stored as bits or binary digital signals in a machine memory (e.g., a computer memory). Such algorithms or symbolic representations are examples of techniques used by those skilled in the data processing arts to convey the substance of their work to others skilled in the art. As used herein, an "algorithm" refers to a self-consistent sequence of operations or similar processes leading to a desired result. In this context, algorithms and operations involve physical manipulations of physical quantities.

[0098] Although the inventive subject matter has been generally described with reference to certain exemplary embodiments, various modifications and changes can be made to these embodiments without departing from the broad spirit and scope of the inventive embodiments. For example, various embodiments or features thereof may be mixed and matched or optioned by those skilled in the art. Such embodiments of the inventive subject matter may be referred to herein as the "invention," either individually or collectively, when multiple are actually disclosed, for convenience only, and without any intention to limit the scope of this application to any single invention or inventive concept.

[0099] The embodiments illustrated herein are believed to be described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, the detailed description is not to be construed in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.

[0100] Furthermore, for resources, operations, or structures described herein as a single instance, multiple instances may be provided. Moreover, boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and particular operations are described in the context of specific illustrative configurations. Other allocations of functionality are contemplated and may fall within the scope of various embodiments of the invention. In general, structures and functions presented as separate resources in illustrative configurations may be implemented as a combined structure or resource. Similarly, structures and functions presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements are within the scope of embodiments of the invention, as expressed in the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative and not a restrictive sense.

[0101] Finally, it is Applicant's intent that only those claims which expressly contain the phrase "means for" or "step for" be construed under 35 U.S.C. 112, Section 6. Claims which do not expressly contain the phrase "means for" or "step for" should not be construed under 35 U.S.C. 112, Section 6.

Claims

1. A method for converting a spatiotemporal input data signal (105) into a spatiotemporal output data signal (135) using an automated processing system, the automated processing system comprising: an in vitro biological neural cell culture (BNN culture (120)) within the BNN core unit (100); an input stimulation unit SU (110) adapted to apply an input spatiotemporal stimulation signal (605) to said first set of neural cells; an output readout unit RU (130) adapted to capture output spatiotemporal readout signals (635) from said second set of neural cells; one or more nutrient tanks (319) connected to one or more nutrient dispensers (320) for injecting one or more nutrients into said BNN culture (120); one or more additive tanks (321, 323) connected to one or more additive dispensers (322, 324), respectively, for injecting one or more additives into said BNN culture (120); one or more nutrient waste collectors (325) for filtering and discharging nutrient waste from said BNN culture (120); one or more additive waste collectors (326, 327) for filtering and discharging additive waste from said BNN culture (120); one or more angiogenic networks connecting the nutrient dispenser (320), the additive dispensers (322, 324), the nutrient waste collector (325), and the additive waste collectors (326, 327) to the BNN culture (120); one or more sensors for measuring at least one environmental parameter of said BNN culture (120); an automatic controller (600) having a pre-processing unit (610) configured to pre-process the spatiotemporal input data signal (105) into the spatiotemporal stimulation signal (605), and a post-processing unit (630) configured to post-process the spatiotemporal readout signal (635) into the spatiotemporal output data signal (135), Environmental parameters of the BNN core unit; Nutrient supply of the BNN culture; Supplementary feeding of BNN culture; Collection of nutrient waste from the BNN culture; Collection of additive waste from BNN cultures; Preprocessing parameters, Post-processing parameters; to maintain homeostasis of the BNN culture over time such that the spatiotemporal input data signal (105) is continuously converted into the spatiotemporal output data signal (135); Equipped with The method comprises: pre-processing the spatiotemporal input data signal (105) into the spatiotemporal stimulation signal (605) using a pre-processing unit (610) of the automatic controller (600); post-processing the spatiotemporal readout signal (635) into the spatiotemporal output data signal (135) using a post-processing unit (630) of the automatic controller (600); Using the automatic controller (600), Environmental parameters of the BNN core unit; Nutrient supply of the BNN culture; Supplementary feeding of BNN culture; Collection of nutrient waste from the BNN culture; Collection of additive waste from BNN cultures; Preprocessing parameters, Post-processing parameters; maintaining homeostasis of the BNN culture over time by controlling at least one of the following: A method for providing

2. receiving a spatiotemporal output data signal of interest; environmental parameters of the BNN core unit; Nutrient feeding of the BNN culture; Supplementing the BNN culture; collecting nutrient waste from the BNN culture; collecting additive waste from the BNN culture; Preprocessing parameters, Post-processing parameters; to minimize an error between the spatiotemporal output data signal (135) and the target spatiotemporal output data signal; The method of claim 1 further comprising:

3. Pretreatment is spatiotemporal signal filters, spatiotemporal signal classifier, Machine learning algorithms based on mathematical or statistical models, Artificial neural networks, Convolutional neural networks, Support Vector Machine Classifier, Random forest classifier, genetic algorithms, genetic programming algorithms, Reservoir computing methods, converting the spatiotemporal input data signal (105) into a spatiotemporal stimulus signal (605) using at least one of The method according to claim 1 or 2.

4. Post-processing is spatiotemporal signal filters, spatiotemporal signal classifier, Machine learning algorithms based on mathematical or statistical models, Artificial neural networks, Convolutional neural networks, Support Vector Machine Classifier, Random forest classifier, genetic algorithms, genetic programming algorithms, Reservoir computing methods, converting the spatiotemporal readout signal (635) into a spatial output data signal (135) using at least one of The method according to any one of claims 1 to 3.

Images