Methods for analyzing neuronal patterns in Golgi-stained images

Abstract

A first aspect of this disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, comprising the following steps: - acquiring a first dataset having Golgi-stained neuronal structures; - identifying a first auxiliary dataset, AR1, based on a first type of neuronal structure and a second auxiliary dataset, AR2, based on a second type of neuronal structure, based on the first dataset; - analyzing AR1 using a first method to identify information related to a first type of neuronal structure in AR1; - analyzing AR2 using a second method to identify information related to a second type of neuronal structure in AR2; and - generating a second dataset based on the identified information relating to the first and second types of neuronal structures.

Description

Technical Field

[0001] The present disclosure relates to a computer-implemented method for identifying neuronal patterns in images, and also to a device for identifying neuronal patterns in images, such as a microscope.

[0002] Background Art Golgi staining is a classical staining technique widely used in the morphological study of nerve cells in the field of neuroscience. This technique, first published in 1873, has the unique ability to stain a small number of nerve cells (about 1 - 3% of all nerve cells) out of the entire nerve cell population, and can visualize the entire nerve cell structure.

[0003] However, internal experience has shown that when this technique is used in bright-field imaging microscopy, two specific problems become apparent, particularly for downstream image analysis. First, out-of-focus objects appear as blurred objects in multiple planes. Second, due to the dark and opaque nature of Golgi staining, physical shadows are produced from the soma and dendrites of nerve cells, reducing the signal-to-noise ratio of each measurement. Similar problems can occur in related techniques, such as Nissl staining or staining with other fluorescent dyes. Therefore, improvements in these fields are desirable.

[0004] Summary The problem of the present disclosure is to improve the analysis of Golgi staining images.

[0005] This problem is solved, in particular, by the disclosed embodiments defined by the matters recited in the independent claims. The dependent claims provide information about further embodiments. Various aspects and embodiments of these aspects are also disclosed in the following summary and description, which provide additional features and advantages.

[0006] A first aspect of this disclosure is a computer-implemented method for identifying neuronal patterns in an image, the following steps: - A step of obtaining a first dataset having Golgi-stained neuronal cell structures, -The steps include identifying a first auxiliary dataset, AR1, based on a first type of neuronal cell structure, and identifying a second auxiliary dataset, AR2, based on a second type of neuronal cell structure, based on a first dataset. -A step of analyzing AR1 using the first method to identify information related to the first type of neuronal structure in AR1, -A step of analyzing AR2 using a second method to identify information related to a second type of neuronal structure in AR2, The present invention relates to a method comprising the steps of: generating a second dataset by identified information relating to a first type and a second type of neuronal cell structure.

[0007] The first or second dataset may be images or any other format of a 2D dataset. The first or second dataset may optionally be part of a 3D dataset and / or a multidimensional dataset.

[0008] Golgi staining, also known as Golgi-Cahal staining or Golgi-Cahal staining, is a neurohistochemical technique used to visualize and study the complex structures of nerve cells, particularly their dendrites and axons. The Golgi staining process may involve impregnating a small number of nerve cells with a silver chromate or silver nitrate solution. This staining method can be selective for nerve cell structures, such as somas, dendrites, and spines, which are then visualized under a microscope. Stained nerve cells may appear as a dark, complex network of dendrites and axons against a bright background, providing researchers with a three-dimensional view of nerve cell morphology. Despite its importance, Golgi staining has limitations, such as the inability to predict which nerve cells will be stained and the difficulty in staining entire neural networks. Furthermore, Golgi stained datasets can be noisy, making it difficult to distinguish between different nerve cell structures and / or to understand which nerve cell structures are similar. Noisy images may contain blurred structures and / or shadows projected onto other structures by certain structures. Golgi staining in the context of this disclosure may also have other staining techniques, such as Nissl staining, which exhibit similar drawbacks.

[0009] The method according to the first aspect should improve the visibility of different neuronal structures, such as somas, dendrites, and / or spines. The soma, also known as the cell body or pericarion, is the central part of a neuronal cell. The soma may contain the cell nucleus, which houses the genetic material and controls the cell's metabolic functions. The soma integrates input signals from dendrites and initiates the generation of electrical pulses. Dendrites are branched, dendritic extensions from the soma. The function of dendrites may be to receive and transmit input signals from other neuronal cells or sensory receptors. Dendrites may have receptors on their surface that bind to neurotransmitters released by presynaptic neurons at synapses. The received signal is then taken up by the dendrite, and if the bound signal is strong enough, this signal can trigger an action potential in the axon. Spines may be small, protruding structures found on dendrites. Spines may be specialized structures that function as sites for synaptic connections with the axon terminals of other neuronal cells. Dendritic spines can vary in shape and size and may change in response to synaptic activity and learning.

[0010] The first auxiliary dataset may be based on pixel classification, for example, a pixel classification set for somas. In this case, the first auxiliary dataset may have data related to one or more somas, such as confidence maps for somas. The second auxiliary dataset may be based on pixel classification, for example, a pixel classification for dendrites and / or spines. In this case, the second auxiliary dataset may have data related to dendrites and / or spines, such as confidence maps for dendrites and / or spines. The first and / or second auxiliary datasets may also be based on classifications for other types of neuronal structures, in which case they may consequently have data related to these other types of neuronal structures.

[0011] Pixel classification may also be image segmentation. This image segmentation may be based on a deterministic and / or non-deterministic pixel classification method. The pixel classification method may include machine learning algorithms, random forests, support vector machines, and / or (convolutional) neural networks. Additionally or alternatively, the pixel classification method may include semantic segmentation, clustering algorithms (e.g., K-means clustering or mean-shift clustering). Furthermore, the pixel classification method may include thresholding, edge detection, and / or template matching. Transformation-based methods, such as those based on Fourier transforms, may also be part of the pixel classification method.

[0012] Based on the method according to the first embodiment, a second dataset, such as images, can be provided, having neuronal structures identified based on the first dataset, such as somas, dendrites and / or spines.

[0013] One embodiment of a first aspect of the present disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, wherein the first dataset has 2D or 3D information of neuronal structures.

[0014] The first dataset may be images. The first dataset can be stored in a two-dimensional, three-dimensional, or multi-dimensional matrix or another suitable structure. The first dataset may be a dataset acquired by a microscope or any other device operating in visible and / or invisible light. Depending on the algorithm used to generate the first auxiliary dataset and / or the second auxiliary dataset, the first dataset may be labeled to identify specific structures contained within the dataset.

[0015] One embodiment of a first aspect of the present disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, wherein at least one of AR1 and AR2 has one or more confidence maps.

[0016] A confidence map may be a representation of the degree of certainty or confidence associated with information in an image (or a similar dataset). This representation can provide a pixel-by-pixel or region-by-region assessment of how certain or reliable the data is at each location in the image, particularly with respect to specific content, such as somas, dendrites, and / or spines. A confidence map can assign a confidence score to each pixel or region in the image. High-confidence regions typically indicate that the algorithm is very certain about the information in that region, while low-confidence regions suggest uncertainty. Confidence maps can be generated using a variety of techniques, including probabilistic models, machine learning, statistical analysis, or heuristics. In semantic segmentation and / or object detection, a confidence map can accompany segmentation / identification results (with respect to a desired neuronal structure), indicating how certain the technique is about object boundaries and other parameters assigned to each pixel / region in the first dataset. Confidence maps can be visualized in the original first image as a grayscale image or a color-coded overlay.

[0017] One embodiment of a first aspect of the present disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, wherein the first type of neuronal structure is a soma.

[0018] One embodiment of a first aspect of the present disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, wherein a second type of neuronal structure is a dendrite.

[0019] Additionally, the second type of neuronal structure may also have spines. In this case, dendrites and spines may be defined based on the same second auxiliary dataset. This auxiliary dataset may be based on a pixel classification method configured for dendrites and spines.

[0020] One embodiment of a first aspect of the present disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, the method comprising generating AR1 and / or AR2 by a classification machine learning algorithm, in particular a pixel classifier.

[0021] The auxiliary datasets should, in particular, be based on convolutional neural networks.

[0022] Various machine learning algorithms, such as convolutional neural networks (CNNs), can be used to perform image classification, object detection, and / or segmentation tasks. In microscopy, CNNs can be used to identify and classify cells, organelles, and / or other structures of interest in an image, such as nerve cell structures. Additionally or alternatively, semantic segmentation can be used to classify each pixel in an image into a specific class and / or region of interest. Convolutional neural networks with architectures such as U-Net or DeepLab can be used for semantic segmentation tasks in microscopy. Additionally or alternatively, instance segmentation can be used to identify and / or distinguish individual instances of an object in an image, such as a single soma. Region-based convolutional neural networks and / or similar architectures can be used for instance segmentation tasks in microscopy. Additionally or alternatively, object detection algorithms can locate and classify multiple objects of interest in an image.

[0023] One embodiment of a first aspect of the present disclosure is a computer-implemented method for identifying neuronal patterns in an image, Machine learning algorithms -The present invention relates to a method having a convolutional neural network with a first kernel specifically for identifying somas in order to generate AR1.

[0024] The neural network may be a standard neural network, e.g., VGG and / or ResNET. VGG is a convolutional neural network architecture characterized by the use of small 3x3 convolutional filters and deep stacking of convolutional layers. ResNET is a deep convolutional neural network architecture that uses residual connections, enabling training of very deep neural networks up to several hundred layers deep. In particular, convolutional neural networks can be used for efficient image processing. A single fully connected neural network or a multilayer regressive neural network can be used to identify continuous variables for scaling, translation, and rotation. The regressive neural network may be a feedforward neural network, e.g., a multilayer perceptron. The regressive neural network may be configured to take its input from a convolutional neural network. This architecture enables effective identification of neuronal structures, e.g., somas, dendrites, and / or spines.

[0025] A first kernel can be constructed to identify somas. This kernel may be shaped to detect typical somas, such as typical somas (i.e., circular, elliptical). In particular, a CNN may be trained to identify somas based on the first kernel from a confidence map. This allows for the effective identification of somas contained in the first dataset.

[0026] One embodiment of the first aspect of the present disclosure is a computer-implemented method for identifying a neuronal pattern in an image, comprising: a machine learning algorithm, - having a convolutional neural network with a second kernel for generating AR2, particularly for identifying dendrites.

[0027] The second kernel may be smaller than the first kernel. This may be particularly the case when the first kernel, i.e., the kernel for the CNN applied to the first auxiliary dataset, is configured for the soma and the second kernel is configured for the dendrites.

[0028] One embodiment of the first aspect of the present disclosure is a computer-implemented method for identifying a neuronal pattern in an image, comprising: a machine learning algorithm, - having a convolutional neural network with a kernel having line features for generating AR2.

[0029] Based on the kernel with line features, dendrites and / or spines can be effectively identified.

[0030] One embodiment of the first aspect of the present disclosure is a computer-implemented method for identifying a neuronal pattern in an image, comprising analyzing AR1 by the Otsu method.

[0031] The Otsu method, also known as Otsu thresholding or Otsu binarization, is a processing technique for automatic image processing. The Otsu method can be used to automatically identify an optimal threshold for separating an image into two classes, typically foreground and background, based on pixel intensity. The Otsu method is applicable, for example, to a confidence map, particularly a confidence map obtained from a pixel classifier, for detecting a soma.

[0032] An implementation of the Otsu method may have the following steps. First, a histogram of pixel intensity in an image (e.g., a first dataset or an auxiliary dataset) can be calculated. This histogram represents the frequency distribution of pixel values ​​across the entire image. Then, the Otsu method can iteratively search for a threshold that either minimizes the intraclass variance or maximizes the interclass variance between two classes identified in the image (e.g., foreground and background). For each possible threshold between the minimum and maximum pixel intensity, two variances can be compared. The first variance is the intraclass variance (variance within a class). This is a measure of the spread of pixel values ​​within each class (e.g., foreground and background). A small intraclass variance indicates that the classes are more homogeneous. The second variance is the interclass variance (variance between classes). This is a measure of the separation between classes (e.g., foreground and background). A large interclass variance indicates good discrimination between classes. In this case, the Otsu method can select a threshold that maximizes the ratio of intraclass variance to interclass variance. This allows us to find the point or region where the two classes are best separated. Once the optimal threshold is identified, the image is binarized by classifying pixels with intensity below the threshold into one class (e.g., foreground) and pixels with intensity above the threshold into another class (e.g., background).

[0033] One embodiment of a first aspect of the present disclosure relates to a computer-implemented method for identifying neuronal patterns in an image, the method comprising analyzing AR2 by voxel scooping.

[0034] Voxel scooping is a conventional data processing technique in which layers of voxels ("scoops") are repeatedly extracted from the structure and clustered based on connectivity. This data processing technique can be used, in particular, to trace the central line of nerve cells from confocal images. Further explanation can be found, for example, in "Three-dimensional neuron tracing by voxel scooping" by Rodriguez, A et al., J. Neurosci. Methods. (2009), 184 (1): 169-175.

[0035] One embodiment of a first aspect of this disclosure is a computer-implemented method for identifying neuronal patterns in an image, the following steps: -The present invention relates to a method comprising the step of identifying one or more third neuronal structures attached to a dendrite, particularly spine structures.

[0036] The third neuronal structure may, in particular, be based on the second auxiliary dataset. For example, if the second auxiliary dataset is generated for dendrites, and as a result, a confidence map for dendrites is obtained, then the second auxiliary dataset can be further analyzed for spines by a convolutional neural network trained to identify spines. Spine identification based on the second auxiliary dataset can, in particular, be performed in parallel with dendritic identification.

[0037] Spines can be detected using blob detection. Blob detection is a computer vision technique that can be used to identify and / or locate areas of interest or "blobs" within an image. These blobs typically represent objects or areas that have similar characteristics, such as color, intensity, or texture. This process involves analyzing the image to find connected regions of pixels that share common features, which can be useful in various applications such as object tracking, image segmentation, and feature extraction. Blob detection algorithms may be based on, for example, LoG (Laplacian of Gaussian).

[0038] One embodiment of a first aspect of this disclosure is a computer-implemented method for identifying neuronal patterns in an image, the following steps: -The present invention relates to a method comprising the step of assigning an identified first neuronal structure, particularly a soma, to an identified second neuronal structure, particularly a dendrite, and / or a third neuronal structure, particularly a spine.

[0039] Assigning different neuronal structures to each other can also be done using machine learning algorithms, such as CNNs. This makes it easier to understand which neuronal structures are similar, i.e., which complexes of somas, dendrites, and spines form a single neuron.

[0040] One embodiment of a first aspect of the present disclosure is a computer-implemented method for identifying neuronal patterns in an image, The second dataset contains at least one of the following parameters, namely: -size, -resolution, - Regarding the method, which is equivalent to the first dataset in form.

[0041] This makes it easier to further process the second dataset, especially using the same device that was used to obtain the first dataset.

[0042] A second aspect of this disclosure is a device for identifying neuronal patterns in an image, - Perform the method described in any one of the prior claims, and / or - relating to a device, in particular a device configured to connect to an external device in order to interact with the method described in any one of the preceding claims.

[0043] This device may be a microscope, such as a bright-field microscope, or any other imaging device configured for neuronal cell patterns. Additionally or alternatively, this device may be a device that carries out the method and interacts with the microscope, in particular via a communication network, to obtain a first dataset and / or supply a second dataset.

[0044] Other advantages and features are derived from several embodiments described below, some of which refer to the drawings. The drawings do not necessarily show embodiments to scale. Dimensions of various features may be enlarged or reduced, especially for clarity of explanation. For this reason, the drawings are at least partially schematic. [Brief explanation of the drawing]

[0045] [Figure 1] This figure shows a computer-implemented method for identifying neuronal structures according to one embodiment of the present disclosure. [Figure 2] This figure shows a computer-implemented method for identifying neuronal structures according to another embodiment of the present disclosure. [Figure 3] This figure shows a computer-implemented method for identifying neuronal structures according to yet another embodiment of the present disclosure. [Figure 4] This figure shows a computer-implemented method for identifying neuronal structures according to yet another embodiment of the present disclosure. [Figure 5]This figure shows an apparatus for identifying neuronal patterns in an image, according to one embodiment of the present disclosure.

[0046] The following description refers to accompanying drawings that illustrate certain aspects that form part of the disclosure and that enable understanding of the disclosure. The same reference numerals refer to the same features, or features that are at least functionally or structurally similar.

[0047] In general, the disclosure of the described method also applies to the corresponding device (or apparatus) for carrying out the method, or the corresponding system may have one or more devices, and vice versa. For example, if a particular step is described, the corresponding device may include features for performing the described step, even if such features are not explicitly described or represented in the drawings. On the other hand, if a particular device is described based on a functional unit, the corresponding method may include one or more steps for performing the described function, even if such steps are not explicitly described or represented in the drawings. Similarly, a system may be provided with corresponding device features or features for performing a particular step. The various exemplary aspects and features of embodiments described above or below are combinable unless otherwise specified.

[0048] Detailed explanation Figure 1 shows a computer-implemented method 100 for identifying neuronal structures according to one embodiment of the present disclosure. In a first step 110, a first dataset having Golgi-stained neuronal structures is obtained. The first dataset may be images acquired from a Golgi-stained sample by microscopy.

[0049] In the second step 120, a first auxiliary dataset for a first type of neuronal structure is identified based on the first dataset. A second auxiliary dataset for a second type of neuronal structure is also identified from the first dataset. The first type of neuronal structure may be, for example, a soma. The second type of neuronal structure may be, for example, a dendrite. The second type may also have a spine. In particular, if the structures of dendrites and spines are quite similar, both neuronal structures can be identified by the same algorithm and in this case can be included in the same second auxiliary dataset. The method for identifying the first and second auxiliary datasets may be, for example, a trained classification method that is configured for each neuronal structure, i.e., for somas (first auxiliary dataset) and for dendrites / spines (second auxiliary dataset).

[0050] In the third step 130, the first auxiliary dataset is analyzed using the first method to identify information related to the first type of neuronal structure. The first method may be, for example, the Otsu method, which is particularly suitable for detecting soma in the auxiliary dataset (e.g., confidence maps).

[0051] In the fourth step 140, the second auxiliary dataset is analyzed using the second method to identify information related to the second type of neuronal structure. The first method may be, for example, voxel scooping, which may be suitable for detecting dendrites / spines in the auxiliary dataset (e.g., confidence maps).

[0052] Based on the analysis results, an image can be generated as a second dataset containing the identified first and second neuronal structures, namely somas and dendrites / spines. In other words, in a further step, the information generated by the two channels can be integrated for a single result.

[0053] Figure 2 shows a computer-implemented method 200 for identifying neuronal structures according to one embodiment of the present disclosure. Image 1 200a shows a first dataset of Golgi-stained neuronal patterns obtained by microscopy. Neuronal structures such as somas, dendrites, axons, and spines can be seen in the image, but these are degraded by noise, e.g., somas 202 and dendrites 204. Some structures are shadowed by other structures. Other structures are out of focus. More accurate visual processing is needed to identify neuronal structures and / or complete neurons.

[0054] The second image, 200b, shows some of the results of the classification method. Image 200b was identified based on an image processing method configured for soma. For this purpose, image 200a was processed by a pixel classification method to obtain an auxiliary dataset, and subsequently, a thresholding method, such as the Otsu method, was used to separate soma 210,212 from the remaining images.

[0055] The third image, 200c, shows the final results of this method. In the images for the identified somas 210 and 212, the dendrites and spines 220 and 222 belonging to each soma are identified. This identification of dendrites / spines was performed using a different data processing algorithm than that used for soma identification, for example, voxel scooping, but based on the same first dataset 200a. Therefore, dendrite / spine identification can be performed in parallel with or sequentially (i.e., before and after) soma identification.

[0056] Figure 3 shows a computer-implemented method 300 for identifying neuronal structures according to one embodiment of the present disclosure. A Golgi-stained image 302 containing one or more neurons is the input for this process. Each neuron consists of a soma connected to one or more dendritic trees. Each dendritic tree may contain visible and detectable spines. The overall goal is to detect somas, trace the dendritic trees from the center of the somas, and / or detect spines along each of the dendritic segments.

[0057] Because nerve cells appear as dark voxels in images, directly detecting nerve cells from Golgi images can be difficult. This is because Golgi-stained nerve cells are imaged using bright-field microscopy, which uses transmitted light. This presents two challenges when segmenting nerve cells. First, when using bright-field microscopy, the dark staining can create physical shadows that result in noise in the image. Second, out-of-focus objects appear as blurred objects in multiple planes, which further contributes to the noise level.

[0058] To improve detection accuracy, a random forest-based pixel classification method can be used, which involves training a pixel classifier through user teaching and generating a confidence map with high-intensity areas representing the target structure. This may be done to perform pixel classification on somas 304 and on dendrites and / or spines. Separate pixel classifiers 304, 306 are trained to generate a soma confidence map 310 in which somas are highlighted with high intensity values, and a dendritic / spine confidence map 314 in which dendritic trees and spines are highlighted with high intensity values.

[0059] The soma confidence map 310 may be used in soma detection 312 to generate a soma mask 320. The soma detection method 312 can apply an Otsu threshold to the soma confidence map, which may be followed by several mask improvement steps, such as pore filling and boundary smoothing. Other conventional object detection methods or deep learning-based object detection methods may also be applied to detect somas from the soma confidence map.

[0060] The dendritic / spine confidence map 314 is used for dendritic tracing 316 and / or spine detection 318. Dendritic tracing 316 can use a voxel scooping method that can be started from the center of each soma. The roots of the dendritic tree are first identified on the soma surface. From each root, the voxel scooping method is used to trace outward in small steps following bright tubular structures, traversing the branches until the entire dendritic tree 322 is traced. The dendritic diameter is estimated at each small step.

[0061] To perform spine detection 318, blob detection can be performed within a specific distance from the dendritic surface, determined by the estimated diameter. Effective blobs that can be traced back to the dendritic surface will be detected as spines 324. These blobs are identified as spine heads, and the traces are identified as spine necks.

[0062] Figure 4 shows a computer-implemented method 400 for identifying neuronal structures according to one embodiment of the present disclosure. The pixel classifier is a voxel classification algorithm based on random forest classification. It consists of two parts: classifier training and classifier application. In classifier training 400a, one or more training images 404 are used as input. In the feature calculation step 408, the feature images are calculated based on the feature type and kernel size parameters 402 set by the user.

[0063] Feature types represent image processing filters that can be applied to the input image to generate a feature image. Examples of feature types include (a) Gaussian blur, (b) maximum, (c) minimum, (d) mean, (e) variance, (f) median, (g) Hessian, (h) Gaussian difference, (i) Sobel, (j) Laplacian, (k) structure tensor, (l) Gabor, (m) Gabor aggregation, etc. Depending on the size of the object of interest, multiple kernel sizes can be used for all selected filter types.

[0064] One example of feature / filter type selection and kernel size selection is the following combination, namely, the features to be selected. (1) Gaussian blur, (2) Hessian, (3) Laplacian, The selected kernel size can be any combination of 3, 7, or 11. Nine feature images, namely, - A Gaussian blur filter with a kernel size of 3. - A Gaussian blur filter with a kernel size of 7. - A Gaussian blur filter with a kernel size of 11. - A Hessian filter with kernel size 3, - A Hessian filter with kernel size 7, - A Hessian filter with kernel size 11, - Laplacian filter with kernel size 3, - A Laplacian filter with kernel size 7, and - A Laplacian filter with a kernel size of 11 will be created.

[0065] The next step is to train the pixel classifier 410 using the Random Forest machine learning method. The required training data consists of training samples for each class. For pixel classification, any pixel / voxel from the training image may represent a training sample. Sample measurements (pixel / voxel values) are extracted from the feature image at the pixel / voxel location. The ground truth is determined through exemplary learning, where the user draws a class region 406 for each class. The ground truth of a sample is set to the class of the region to which the sample belongs. Only pixels / voxels within the class region are used as training samples to create the Random Forest.

[0066] For classifier application 400b, the feature calculation step 422 is performed first, and the same set of feature images is calculated for the test image 420. Then, a pixel classifier 424 is applied to classify every pixel / voxel in the test image 420 using the measurements (pixel / voxel values) from the feature images. The output is a class confidence map 426, which shows how likely each pixel / voxel is to belong to an individual class.

[0067] Some embodiments relate to a microscope that includes a system such as those described in relation to one or more of Figures 1 to 4. Alternatively, the microscope may be part of a system such as those described in relation to one or more of Figures 1 to 3, or may be connected to a system such as those described in relation to one or more of Figures 1 to 3. Figure 5 shows a schematic diagram of a system 500 configured to carry out the methods described herein. The system 500 includes a microscope 510 and a computer system 520. The microscope 510 is configured to take images and is connected to the computer system 520. The computer system 520 is configured to carry out at least some of the methods described herein. The computer system 520 may be configured to run machine learning algorithms. The computer system 520 and the microscope 510 may be separate entities or may be integrated within a common housing. The computer system 520 may be part of the central processing system of the microscope 510, and / or the computer system 520 may be part of a dependent component of the microscope 510, such as a sensor, actor, camera, or illumination unit of the microscope 510.

[0068] The computer system 520 may be a local computer device (e.g., a personal computer, laptop, tablet computer, or mobile phone) comprising one or more processors and one or more storage devices, or it may be a distributed computer system (e.g., a cloud computing system) comprising one or more processors and one or more storage devices distributed to various locations such as local clients and / or one or more remote server farms and / or data centers. The computer system 520 may include any circuit or combination of circuits. In one embodiment, the computer system 520 may include one or more processors, which can be of any kind. As used herein, the processor may be intended to be any kind of computing circuit, such as a microprocessor for a microscope or microscope component (e.g., a camera), a microcontroller, a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multicore processor, a field-programmable gate array (FPGA), or any other kind of processor or processing circuit. Other types of circuits that may be included in the computer system 520 may be custom circuits, application-specific integrated circuits (ASICs), etc., such as one or more circuits (communication circuits, etc.) used in wireless devices such as mobile phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 520 may also include one or more storage devices that may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives that handle removable media such as compact discs (CDs), flash memory cards, digital video discs (DVDs), etc.The computer system 520 may also include a display device, one or more speakers and a controller which may include a keyboard and / or mouse, trackball, touchscreen, voice recognition device, or any other device which enables a user of the system to input information into and receive information from the computer system 520.

[0069] Some or all of the steps may be performed by a hardware device (or by using a hardware device), such as a processor, microprocessor, programmable computer, or electronic circuit. In some embodiments, one or more of the most critical steps may be performed by such a device.

[0070] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation is feasible using a non-transient recording medium, which is a digital recording medium, etc., that stores electronically readable control signals and cooperates (or can cooperate) with a programmable computer system to carry out each method. Examples include floppy disks, DVDs, Blu-rays, CDs, ROMs, PROMs and EPROMs, EEPROMs, or FLASH memory. Thus, the digital recording medium may be computer-readable.

[0071] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system so as to carry out any of the methods described herein.

[0072] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code, which operates to perform one of the methods when the computer program product is executed on a computer. This program code may be stored, for example, on a machine-readable carrier.

[0073] Another embodiment includes a computer program stored in a machine-readable carrier for carrying out any of the methods described herein.

[0074] Therefore, in other words, embodiments of the present invention are computer programs having program code for carrying out any of the methods described herein when the computer program is executed on a computer.

[0075] Accordingly, another embodiment of the present invention is a recording medium (or data carrier or computer-readable medium) containing a stored computer program for carrying out any of the methods described herein when executed by a processor. The data carrier, digital recording medium, or recording medium is typically tangible and / or non-transient. Another embodiment of the present invention is an apparatus, such as those described herein, comprising a processor and a recording medium.

[0076] Therefore, another embodiment of the present invention is a data stream or signal sequence representing a computer program for carrying out any of the methods described herein. The data stream or signal sequence may be configured to be transmitted, for example, over a data communication connection, such as the Internet.

[0077] Another embodiment includes processing means, for example, a computer or programmable logic device configured or adapted to carry out any of the methods described herein.

[0078] Another embodiment includes a computer having an installed computer program for carrying out any of the methods described herein.

[0079] Another embodiment of the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for carrying out any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0080] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to carry out any of the methods described herein. Generally, the methods are advantageously carried out by any hardware device.

[0081] As used herein, the term "and / or" includes all possible combinations of one or more of the items listed herein and may be abbreviated as " / ".

[0082] While several embodiments have been described in the context of the apparatus, it is clear that these embodiments also represent descriptions of the corresponding methods, where blocks or apparatus correspond to steps or features of steps. Similarly, embodiments described in the context of steps also represent descriptions of the corresponding blocks, items, or features of the corresponding apparatus.

[0083] Embodiments may be based on the use of machine learning models or machine learning algorithms. Instead of relying on models and inference, machine learning may refer to algorithms and statistical models that a computer system can use to perform a particular task without using explicit instructions. For example, machine learning may use data transformations inferred from the analysis of historical data and / or training data instead of rule-based data transformations. For example, image content may be analyzed using a machine learning model or a machine learning algorithm. For a machine learning model to analyze image content, the machine learning model may be trained with training images as input and training content information as output. By training a machine learning model with a large number of training images and / or training sequences (e.g., words or sentences) and associated training content information (e.g., labels or annotations), the machine learning model “learns” to recognize image content so that image content not included in the training data becomes recognizable using the machine learning model. The same principle may be used in the same way for other types of sensor data: by training a machine learning model with training sensor data and a desired output, the machine learning model “learns” the transformation between sensor data and output, which can then be used to provide output based on non-trained sensor data provided to the machine learning model. The provided data (e.g., sensor data, metadata, and / or image data) may be preprocessed to obtain feature vectors that can be used as input to a machine learning model.

[0084] A machine learning model may be trained using training input data. The example above uses a training method called "supervised learning." In supervised learning, a machine learning model is trained using multiple training samples, each of which may contain multiple input data values ​​and multiple desired output values; that is, each training sample is associated with a desired output value. By specifying both the training samples and the desired output values, the machine learning model "learns" during training which output values ​​to provide based on input samples similar to the provided samples. In addition to supervised learning, semi-supervised learning may be used. In semi-supervised learning, some of the training samples lack corresponding desired output values. Supervised learning may be based on a supervised learning algorithm (e.g., a classification algorithm, a regression algorithm, or a similarity learning algorithm). A classification algorithm may be used if the output is limited to a limited set of values ​​(categorical variables), i.e., the input is classified into one of a limited set of values. A regression algorithm may be used if the output may have any numerical value (within a range). Similarity learning algorithms may be similar to both classification and regression algorithms, but are based on learning from examples using a similarity function that measures how similar or related two objects are. In addition to supervised or semi-supervised learning, unsupervised learning may be used to train machine learning models. In unsupervised learning, input data (only) may be provided, and unsupervised learning algorithms may be used to find structure in the input data (for example, by grouping or clustering the input data, or by finding commonalities in the data). Clustering is the process of assigning input data containing multiple input values ​​into multiple subsets (clusters), so that input values ​​within the same cluster are similar according to one or more (predefined) similarity criteria, but are not similar to input values ​​in another cluster.

[0085] Reinforcement learning is a third group of machine learning algorithms. In other words, reinforcement learning may be used to train machine learning models. In reinforcement learning, one or more software actors (referred to as “software agents”) are trained to take actions in their surroundings. A reward is calculated based on the actions taken. Reinforcement learning is based on training one or more software agents to choose actions that result in software agents that perform better on a given task, with cumulative rewards increasing (as revealed by the increase in rewards).

[0086] Furthermore, several techniques may be applied as part of a machine learning algorithm. For example, feature representation learning may be used. In other words, a machine learning model may be trained at least partially using feature representation learning, and / or a machine learning algorithm may include feature representation learning components. A feature representation learning algorithm, which may be called a representation learning algorithm, may not only store information in its own input but may also transform the information to make it useful, often as a preprocessing step before performing classification or prediction. Feature representation learning may be based, for example, on principal component analysis or cluster analysis.

[0087] In some examples, anomaly detection (i.e., outlier detection) may be used, which aims to provide the identification of input values ​​that raise suspicion by being significantly different from the majority of the input or training data. In other words, a machine learning model may be trained with anomaly detection, at least in part, and / or a machine learning algorithm may include anomaly detection components.

[0088] In some examples, a machine learning algorithm may use a decision tree as its predictive model. In other words, a machine learning model may be based on a decision tree. In a decision tree, observations about an item (e.g., a set of input values) may be represented by branches of the decision tree, and the output values ​​corresponding to these items may be represented by leaves of the decision tree. A decision tree may support both discrete and continuous values ​​as output values. When discrete values ​​are used, the decision tree may be represented as a classification tree, and when continuous values ​​are used, the decision tree may be represented as a regression tree.

[0089] Correlation rules are another technique that can be used in machine learning algorithms. In other words, a machine learning model may be based on one or more correlation rules. Correlation rules are created by identifying relationships between variables in a large amount of data. A machine learning algorithm may identify and / or utilize one or more correlational rules that represent knowledge derived from the data. These rules may be used, for example, to store, manipulate, or apply knowledge.

[0090] Machine learning algorithms are typically based on machine learning models. In other words, the term “machine learning algorithm” may refer to a set of instructions that can be used to create, train, or use a machine learning model. The term “machine learning model” may refer to a set of data structures and / or rules that represent learned knowledge (for example, based on training performed by a machine learning algorithm). In embodiments, usage of a machine learning algorithm may mean usage of one underlying machine learning model (or multiple underlying machine learning models). Usage of a machine learning model may mean that a machine learning model and / or a set of data structures / rules that are a machine learning model are trained by a machine learning algorithm.

[0091] For example, a machine learning model may be an artificial neural network (ANN). An ANN is a system influenced by biological neural networks, such as those found in the retina or brain. An ANN consists of multiple interconnected nodes and multiple junctions, or edges, between the nodes. Typically, there are three types of nodes: input nodes that receive input values, hidden nodes that are (simply) connected to other nodes, and output nodes that provide output values. Each node may represent an artificial neuron. Each edge may transmit information from one node to another. The output of a node may be defined as a (nonlinear) function of its input (e.g., the sum of its inputs). The input of a node may be used in a function based on the "weights" of the edges or nodes that provide the input. The weights of nodes and / or edges may be adjusted during the learning process. In other words, training an artificial neural network may involve adjusting the weights of the nodes and / or edges of the artificial neural network to obtain a desired output for a given input.

[0092] Alternatively, a machine learning model may be a support vector machine, a random forest model, or a gradient boosting model. A support vector machine (i.e., a support vector network) is a supervised learning model with a relevant learning algorithm that can be used to analyze data (e.g., in classification or regression analysis). A support vector machine may be trained by providing inputs with multiple training input values ​​belonging to one of two categories. A support vector machine may be trained to assign new input values ​​to one of two categories. Alternatively, a machine learning model may be a Bayesian network, which is a stochastic directed acyclic graphical model. A Bayesian network may use a directed acyclic graph to represent a set of random variables and their conditional dependencies. Alternatively, a machine learning model may be based on a search algorithm and a genetic algorithm, which is a heuristic method that mimics the process of natural selection. [Explanation of Symbols]

[0093] 100 Flowchart of a method for identifying nerve cell structures 110 Acquisition of Golgi-stained images 120 Classification of the first type of nerve cell structure 130 Analysis of the first auxiliary dataset 140 Analysis of the second auxiliary dataset 200 Methods for Identifying Neuronal Cell Structures First dataset from a 200a microscope. 200b dataset containing identified soma 200c A dataset with fully identified structure 202 Blurred Soma 204 Blurred dendrites 210 Identified Soma 212 Identified Soma 220 Identified soma with dendrites and spines 222 Identified soma with dendrites and spines 300 Methods for Identifying Neuronal Structures 302 Golgi images 304 Somapixel Classification 306 Pixel classification of dendrites or spines 310 Soma Reliability Map 312 Soma detection 314 Confidence map for dendrites and spines 316 Dendritic tracing 318 Spine detection 320 Soma Mask 322 Dendrite Tree 324 spines 400 Methods for Identifying Neuronal Structures 400a Classifier Training 400b classifier application 402 Kernel Size Parameters 404 Training Images 406 Class Area 408 Feature Calculation 410-pixel classifier 420 test images 422 Feature Calculation 424-pixel classifier 426 Class Confidence Map 500 Analysis Systems 510 Microscope 520 Computers

Claims

1. A computer-implemented method for identifying neuronal cell patterns in an image, The next step, namely, The steps include obtaining a first dataset (302) having Golgi-stained neuronal cell structures (202, 204), Based on the first dataset, the first auxiliary dataset (310), AR1 is identified based on a first type of neuronal cell structure (202), and the second auxiliary dataset (314), AR2 is identified based on a second type of neuronal cell structure (204). The steps include analyzing AR1 using a first method (312) to identify information (320) related to the first type of neuronal structure in AR1, The steps include analyzing AR2 using a second method (316) to identify information (322) related to the second type of neuronal structure in AR2, A method comprising the step of generating a second dataset (200c) with identified information relating to the first type and the second type of neuronal cell structures.

2. The method according to claim 1, wherein the first dataset (302) has 2D or 3D information of nerve cell structures.

3. The method according to claim 2, wherein at least one of AR1 (310) and AR2 (314) has one or more confidence maps.

4. The method according to any one of claims 1 to 3, wherein the first type of nerve cell structure (202) is a soma.

5. The method according to any one of claims 1 to 4, wherein the second type of nerve cell structure (204) is a dendrite.

6. The method according to any one of claims 1 to 5, comprising generating AR1 (310) and / or AR2 (314) by a classification machine learning algorithm (304, 306), in particular a pixel classifier.

7. The aforementioned machine learning algorithms (304, 306) are: The method according to claim 6, comprising a convolutional neural network having a first kernel for in particular identifying somas in order to generate AR1.

8. The aforementioned machine learning algorithms (304, 306) are: The method according to claim 6 or 7, comprising a convolutional neural network having a second kernel for identifying dendrites in particular, in order to generate AR2.

9. The aforementioned machine learning algorithms (304, 306) are: The method according to any one of claims 6 to 8, comprising a convolutional neural network having a kernel having line features for generating AR2.

10. The method according to any one of claims 1 to 9, wherein AR1 (310) is analyzed by the Otsu method (312).

11. The method according to any one of claims 1 to 10, wherein AR2 (314) is analyzed by voxel scooping (316).

12. The next step, namely, The method according to any one of claims 1 to 11, comprising the step of identifying one or more third nerve cell structures (324), particularly spine structures, attached to a dendritic (322).

13. The next step, namely, The method according to any one of claims 1 to 12, comprising the step of assigning an identified first neuronal structure (320), particularly a soma, to an identified second neuronal structure (322), particularly a dendrite, and / or a third neuronal structure (324), particularly a spine.

14. The second dataset (200c) is defined as having at least one of the following parameters, namely, size, resolution, The method according to any one of claims 1 to 13, wherein the data set is equivalent in form to the first dataset (302).

15. A device for identifying neuronal patterns in images, The method described in any one of claims 1 to 14, and / or A device configured to connect to an external device, in particular, in order to interact with the method according to any one of claims 1 to 14.

Images