A neuron reconstruction method and device based on whole-brain optical imaging data

Abstract

The present invention relates to the field of neural network structure reconstruction, and more specifically, to a method and device for neuron reconstruction based on whole-brain optical imaging data. The present invention proposes a method for neuron reconstruction based on whole-brain optical imaging data, comprising the following steps: S1, preprocessing of original resolution imaging data; S2, based on a semi-automatic reconstruction process that supports search operations, combined with the current position, assisting in traversing neurons and reconstructing neuron models; S3, performing neuron structure analysis. The present invention minimizes user input by supporting an optimized reconstruction process for search operations, and can efficiently perform semi-automatic reconstruction of long-range axonal projections in TB-level whole-brain optical imaging data, providing an effective means for whole-brain network computational analysis at single neuron resolution based on big data.

Description

Technical Field

[0001] The present invention relates to the field of neural network structure reconstruction, and more specifically, to a neuron reconstruction method and device based on whole-brain optical imaging data. Background Art

[0002] The human brain contains a large number of neurons with complex connections. Neurons are the basic units of information processing in the nervous system and are the key objects of neuroscience research.

[0003] Neurons have a tree-like structure, consisting of a cell body, dendrites, and axons. There are large structural differences between different types of neurons. Even for neurons of the same type, the specific projection sites and branching patterns vary.

[0004] At the neuron level, the special structure of neurons is the basis of their signal transmission. At the nervous system level, analyzing the structure and connection relationship of neurons is helpful for studying brain functions. A detailed description of the structure of all neurons in the brain and the connection relationship between neurons is called a connectome. An important part of the construction of a connectome is the description of the structure of neurons.

[0005] Early studies mainly involved drawing neuronal structures observed under a microscope, but this description method was time-consuming and labor-intensive, not accurate enough, and could not reflect three-dimensional structural characteristics.

[0006] With the development of computer technology, the use of digital models to represent the three-dimensional structure of neurons has provided a new method for describing the structure of neurons. Building a digital neuron model based on image data is called neuron reconstruction. According to the type of image data, neuron reconstruction is divided into reconstruction of electron microscope data and reconstruction of light microscope data. The reconstruction of complete neurons is generally based on light microscope data.

[0007] In recent years, the development of new optical imaging technologies has brought higher spatial resolution and larger spatial coverage. For example, Fluorescence Micro-optical Sectioning Tomography (fMOST) combines ultrathin sections with high-resolution imaging, and uses post-imaging slicing to perform three-dimensional complete imaging of the entire plastic-embedded mouse brain. Combined with sparse labeling, the new optical imaging technology makes it possible to reconstruct long-range projections of neurons, providing a new way to construct a single-neuron level connection group across the entire brain.

[0008] However, large-scale high-resolution imaging means extremely large-scale image data: the size of imaging results for a single mouse brain is usually at the TB (Tera-Byte) level, and can even reach tens of TB at the highest resolution.

[0009] How to present and analyze such large-scale imaging data has become a new challenge. Some existing manual reconstruction tools are unable to process TB-level data, and the correctness of some automatic reconstruction tools is difficult to guarantee.

[0010] Manual (or semi-automatic) reconstruction is performed through computer software. The software displays the image data of neurons. Based on the observed data, the user guides the software to reconstruct neurons through input from the computer mouse or keyboard. The specific calculation of the neuron structure is completed by the computer, but the entire reconstruction process is driven by people. The reconstruction result reflects the user's understanding of the neuron structure. The results of manual reconstruction have a higher accuracy rate, but at the same time, the reconstruction process also takes longer. As more and more data are obtained, its slow speed disadvantage is gradually highlighted.

[0011] Fully automatic neuron reconstruction does not require user input, so it can be used to perform faster neuron reconstruction. There are many methods for automatic reconstruction, and the principles and algorithms used in these methods are also diverse. However, even for data with very high neuron labeling sparsity, these fully automatic methods often make mistakes, and the error rate will increase significantly with the increase of data complexity. Manually correcting these automatic reconstruction errors is likely to take more time than direct manual reconstruction. Summary of the invention

[0012] The purpose of the present invention is to provide a neuron reconstruction tool based on terabyte-level whole-brain optical imaging data to solve the problems of slow neuron reconstruction speed and low accuracy in large-scale optical imaging data of terabyte level.

[0013] In order to achieve the above object, the present invention provides a neuron reconstruction method based on whole-brain optical imaging data, comprising the following steps:

[0014] S1, preprocessing of original resolution imaging data;

[0015] S2, based on the semi-automatic reconstruction process supporting the search operation, combined with the current position, assists in traversing neurons and reconstructing the neuron model;

[0016] S3. Perform neuronal structure analysis.

[0017] In one embodiment, the step S1 further comprises the following steps:

[0018] Split the original resolution imaging data and save them in the form of small cubes;

[0019] Downsampling the original resolution imaging data to generate downsampled imaging data;

[0020] The downsampled imaging data and / or the original resolution imaging data in the form of small cubes are compressed.

[0021] In one embodiment, the step S2 further comprises: imaging data loading,

[0022] Automatically load downsampled imaging data and / or original resolution imaging data in the form of small cubes around the current location via the file system or network.

[0023] In one embodiment, in step S2, the neuron model is represented by a tree consisting of edges and vertices, the edge is a node sequence, each node records type, position and radius information, and the vertex is the two end points of the edge.

[0024] In one embodiment, the step S2 further includes visually displaying the original resolution imaging data in two dimensions or three dimensions, and quickly switching between two dimensions and three dimensions according to instructions;

[0025] The step S2 further includes performing three-dimensional visualization of the neuron model.

[0026] In one embodiment, the semi-automatic reconstruction process supporting the search operation in step S2 further includes:

[0027] S21, finding a candidate node sequence according to the current position;

[0028] S22, verifying the node sequence;

[0029] S23. Expand the current neuron model using the node sequence.

[0030] In one embodiment, the step S2 further comprises:

[0031] In the visualization window, the three-dimensional position is quickly selected based on the position and depth of the maximum brightness of the imaging data within a certain range of the mouse click, and the three-dimensional position is accurately selected in combination with the second click at a different angle.

[0032] In one embodiment, the step S21 further comprises:

[0033] Use Dijkstra's algorithm to find a sequence of candidate nodes starting from the current position without a specific target position.

[0034] In one embodiment, the step S2 further includes a step of finding reconstruction errors:

[0035] If two edges in the model partially overlap, the length of the overlapping part exceeds the selection threshold, and the distances between the corresponding nodes at the overlapping part are less than the selection threshold, then the two edges are considered to be in conflict;

[0036] If the distance between the cell bodies of two neurons is less than the selected threshold, the two cell bodies are considered to be in conflict;

[0037] The conflicting edges or duplicate parts of the cell body are deleted and then connected and integrated.

[0038] In one embodiment, the semi-automatic reconstruction process supporting the search operation in step S2 further includes re-reconstructing:

[0039] Based on the current position, the existing reconstruction results are used to find candidate node sequences for re-reconstruction as a proofreading of the existing reconstruction results.

[0040] In one embodiment, the semi-automatic reconstruction process supporting the search operation in step S2 further includes:

[0041] Record each unfinished vertex, and automatically jump to the nearest unfinished vertex when a branch reconstruction is completed to start the reconstruction of the branch.

[0042] In one embodiment, the semi-automatic reconstruction process supporting the search operation in step S2 is implemented through plug-in interface encapsulation.

[0043] In order to achieve the above-mentioned object, the present invention provides a neuron reconstruction device based on whole-brain optical imaging data, including a preprocessing module, a semi-automatic reconstruction module and an analysis module:

[0044] The preprocessing module preprocesses the original resolution imaging data;

[0045] The semi-automatic reconstruction module comprises a reconstruction unit;

[0046] The reconstruction unit, based on a semi-automatic reconstruction process supporting a search operation, assists in traversing neurons and reconstructs a neuron model in combination with a current position;

[0047] The analysis module performs neuron structure analysis.

[0048] In one embodiment, the neuron model is represented by a tree consisting of edges and vertices, wherein the edge is a node sequence, each node records type, position and radius information, and the vertex is the two end points of the edge.

[0049] In one embodiment, the preprocessing module:

[0050] Split the original resolution imaging data and save them in the form of small cubes;

[0051] Downsampling the original resolution imaging data to generate downsampled imaging data;

[0052] The downsampled imaging data and / or the original resolution imaging data in the form of small cubes are compressed.

[0053] In one embodiment, the semi-automatic reconstruction module comprises a loading unit:

[0054] The loading unit loads the imaging data;

[0055] Automatically load downsampled imaging data and / or original resolution imaging data in the form of small cubes around the current location via the file system or network.

[0056] In one embodiment, the semi-automatic reconstruction module further comprises a visualization unit:

[0057] Visualize the original resolution imaging data in 2D or 3D, and quickly switch between 2D and 3D according to instructions;

[0058] The neuron model is visualized in three dimensions.

[0059] In one embodiment, the reconstruction unit:

[0060] Find the candidate node sequence according to the current position;

[0061] Verify the node sequence;

[0062] Extend the current neuron model with this node sequence.

[0063] In one embodiment, the reconstruction unit, in the visualization window, quickly selects the three-dimensional position according to the position and depth of the maximum brightness of the imaging data within a certain range at the mouse click, and accurately selects the three-dimensional position in combination with a second click at a different angle.

[0064] In one embodiment, the reconstruction unit uses the Dijkstra algorithm to find a candidate node sequence starting from the current position without a specific target position.

[0065] In one embodiment, the semi-automatic reconstruction module searches for reconstruction errors:

[0066] If two edges in the model partially overlap, the length of the overlapping part exceeds the selection threshold, and the distances between the corresponding nodes at the overlapping part are less than the selection threshold, then the two edges are considered to be in conflict;

[0067] If the distance between the cell bodies of two neurons is less than the selected threshold, the two cell bodies are considered to be in conflict;

[0068] The conflicting edges or duplicate parts of the cell body are deleted and then connected and integrated.

[0069] In one embodiment, the reconstruction unit uses the existing reconstruction results to find a candidate node sequence based on the current position and reconstructs the node sequence as a check on the existing reconstruction results.

[0070] In one embodiment, the reconstruction unit records each unfinished vertex, and automatically jumps to the nearest unfinished vertex when a branch reconstruction is completed to start the reconstruction of the branch.

[0071] In one embodiment, the reconstruction unit implements a semi-automatic reconstruction process supporting the search operation through a plug-in interface encapsulation.

[0072] In order to achieve the above object, the present invention provides a neuron reconstruction system based on whole-brain optical imaging data, comprising:

[0073] a memory for storing instructions executable by a processor;

[0074] A processor is used to execute the instructions to implement any of the methods described above.

[0075] In order to achieve the above object, the present invention provides a computer-readable medium on which computer instructions are stored, wherein when the computer instructions are executed by a processor, any of the above methods is executed.

[0076] The present invention proposes a neuron reconstruction method, device, system and medium based on whole-brain optical imaging data, which can efficiently perform semi-automatic reconstruction of long-range axon projections in TB-level whole-brain optical imaging data by supporting an optimized reconstruction process for search operations and minimizing user input, thereby providing an effective means for whole-brain network computational analysis with single neuron resolution based on big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The above and other features, properties and advantages of the present invention will become more apparent through the following description in conjunction with the accompanying drawings and embodiments, in which the same reference numerals always represent the same features, wherein:

[0078] Figure 1 A flowchart of a neuron reconstruction method based on whole-brain optical imaging data according to an embodiment of the present invention is disclosed;

[0079] Figure 2 A semi-automatic reconstruction flow chart supporting a search operation according to an embodiment of the present invention is disclosed;

[0080] Figure 3 A schematic diagram of a process of reconstructing an example neuron according to an embodiment of the present invention is disclosed;

[0081] Figure 4 A graph of axonal projection spectrum results according to an embodiment of the present invention is disclosed;

[0082] Figure 5 A neuron clustering data graph according to an embodiment of the present invention is disclosed;

[0083] Figure 6 A neuron clustering effect diagram according to an embodiment of the present invention is disclosed;

[0084] Figure 7 A principle block diagram of a neuron reconstruction device based on whole-brain optical imaging data according to an embodiment of the present invention is disclosed;

[0085] Figure 8 A principle block diagram of a neuron reconstruction system based on whole-brain optical imaging data according to an embodiment of the present invention is disclosed. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the invention and are not used to limit the invention.

[0087] Figure 1 A flowchart of a neuron reconstruction method based on whole-brain optical imaging data according to an embodiment of the present invention is disclosed. Figure 1 As shown, the present invention proposes a neuron reconstruction method based on whole-brain optical imaging data, comprising the following steps:

[0088] S1, preprocessing of original resolution imaging data;

[0089] S2, based on the semi-automatic reconstruction process supporting the search operation, combined with the current position, assists in traversing neurons and reconstructing the neuron model;

[0090] S3. Perform neuronal structure analysis.

[0091] Each step is described in detail below.

[0092] S1. Preprocessing of original resolution imaging data.

[0093] In this embodiment, the original resolution imaging data preprocessing further includes:

[0094] Split the original resolution imaging data and save them in the form of small cubes;

[0095] Downsampling the original resolution imaging data to generate downsampled imaging data;

[0096] Efficient lossy compression can be optionally performed on the downsampled data and / or original resolution imaging data in small cube form to reduce storage requirements and network access latency.

[0097] The result of imaging data is usually an image in TIFF (Tag Image File Format) format. If it is a small-scale imaging, all the TIFF images obtained by imaging can be loaded into memory at one time for visualization and analysis. For large-scale terabyte (TB)-level whole-brain imaging, taking the fMOST imaging data of the entire mouse brain as an example, the result of one imaging contains thousands or even tens of thousands of layers of TIFF images, and each layer of the image contains tens of thousands by tens of thousands of pixels. Obviously, it is not feasible to load all of this data at one time. On the other hand, loading part of the TIFF images at one time is also very limited, because the data loaded at one time is distributed in a cube whose length and width are much larger than the height, and the spatial distribution of information is very scattered, which is not convenient for visualization and analysis.

[0098] In order to make the spatial distribution of the partial data loaded at one time as concentrated as possible, in the embodiment of the present invention, efficient and parallel image splitting is performed, the three-dimensional data composed of all TIFF images are split, and each small cube obtained by splitting is saved separately. At least 8 adjacent small cubes are loaded into the computer memory to form a larger cube, which is the object of subsequent visualization and reconstruction. At the same time, the original resolution imaging data is downsampled, and the downsampled data can provide global information.

[0099] S2. Based on the semi-automatic reconstruction process that supports the search operation, combined with the current position, it assists in traversing neurons and reconstructs the neuron model.

[0100] The neuron model in the prior art is usually represented by a tree consisting of nodes, each node including type, position, radius and index of parent node, and the data structure is designed in nodes to establish the digital neuron model. When reconstructing more neurons in the whole brain, a large number of nodes will be obtained, and the efficiency of calculation and visualization in nodes is low, which is not conducive to the processing of a large number of nodes.

[0101] In the embodiment of the present invention, the neuron model is represented by a tree consisting of edges and vertices.

[0102] The edge is a sequence of nodes, each node records information such as type, position and radius, and corresponds to an axon or dendrite segment of a neuron that does not contain branches.

[0103] The vertices are the two end points of the edge, corresponding to the bifurcation point and the end of the axon or dendrite in the neuron and the cell body of the neuron.

[0104] In the embodiment of the present invention, calculation and visualization are performed in units of edges, which can efficiently process a large number of nodes.

[0105] In this embodiment, step S2 also includes imaging data loading.

[0106] Automatically load downsampled imaging data and original resolution imaging data in the form of small cubes around the current location via the file system or network.

[0107] In this embodiment, step S2 also includes visualization of the imaging data:

[0108] The imaging data can be visualized in two or three dimensions. The two-dimensional and three-dimensional displays can be quickly switched by inputting commands using the mouse or keyboard, which helps users analyze complex neuronal structures. By adjusting the quality of the visualization of the imaging data, better display effects can be obtained, or the GPU load can be reduced to adapt to low-configuration computers.

[0109] In this embodiment, step S2 further includes performing three-dimensional visualization of the neuron model in the reconstruction process.

[0110] The step S2 further includes selecting a three-dimensional position.

[0111] In the visualization window, the three-dimensional position is quickly selected based on the position and depth of the maximum brightness of the imaging data within a certain range of the mouse click, and the three-dimensional position is accurately selected in combination with the second click at a different angle.

[0112] Furthermore, in the visualization window, a point is determined by clicking the mouse, and combined with the depth range of visualization, it corresponds to a line segment in the 3D image space, and the position with the highest image brightness near the line segment is found, that is, the 3D position is determined. A single click can quickly select a 3D position, meeting the point selection requirements in general situations.

[0113] When necessary, the second point selection at a different angle can be combined to calculate the point on the line segment corresponding to the second point selection that is closest to the line segment corresponding to the first point selection, so as to accurately select the three-dimensional position.

[0114] In this embodiment, the novel semi-automatic reconstruction process supporting the search operation further includes:

[0115] S21, finding a candidate node sequence according to the current position;

[0116] S22, verifying the node sequence;

[0117] S23. Expand the current neuron model using the node sequence.

[0118] Figure 2The semi-automatic reconstruction flow chart according to an embodiment of the present invention is disclosed, "○" refers to the operation that requires the mouse to select the position, "□" refers to the shortcut key operation (faster than the mouse), and "◇" refers to the user observing and understanding the image data. Steps S21-S23 are cycled according to the actual operation to complete the reconstruction of the neuron model.

[0119] like Figure 2 As shown, the existing reconstruction method of the connection operation is to select a target position with a computer mouse based on the current position, and obtain a node sequence connecting the two positions by calculating the image. Therefore, in the reconstruction process used by other reconstruction tools, precise mouse operation is unavoidable, which greatly reduces the reconstruction efficiency.

[0120] The reconstruction method of the search operation proposed by the present invention obtains a node sequence without a specific target position starting from the current position by calculating the image, and does not require the selection of the target position. The speed is much faster than the connection operation. Therefore, in the reconstruction process supporting the search operation, precise mouse operation is not necessary, which helps to improve the reconstruction efficiency.

[0121] If the sequence of candidate nodes found by the search operation is wrong, especially when the data complexity is high, the traditional connection operation can be used to obtain the correct reconstruction result through the guidance of the user manually clicking the target point.

[0122] When the accuracy of the search operation is high, the semi-automatic reconstruction process of the present invention does not require user guidance and can greatly improve the reconstruction efficiency.

[0123] If the candidate node sequence passes verification, it can be added to the current neuron model with a single keyboard stroke, thereby extending an existing edge or generating a new edge. If the node sequence is incorrect, it can be ignored and another node sequence can be started. Each extension of the neuron model requires user confirmation, so the correctness of the result is guaranteed.

[0124] In this embodiment, a semi-automatic reconstruction process supporting a search operation records each unfinished vertex and automatically jumps to the nearest unfinished vertex when a branch reconstruction is completed, so as to immediately start the reconstruction of the branch.

[0125] At the branch, you can create a short edge for each branch to initialize the branch, and then select one of the branches to continue refactoring. When a branch is refactored to the end, you can change the state of the corresponding vertex with a single keyboard tap to mark it as refactored.

[0126] Record the terminal state and jump automatically. In this way, the entire neuron can be traversed until the reconstruction is completed.

[0127] Figure 3A schematic diagram of a process of reconstructing an example neuron according to an embodiment of the present invention is disclosed. Figure 3 The neuron reconstruction process shown is as follows: 0, select the starting position; 1, expand to that position through steps S21-S23 (one-step reconstruction operation); 2, expand to that position through a one-step reconstruction operation; 3, add branches through a one-step reconstruction operation and reconstruct to that position; 8, reconstruct to the end and mark the end, and automatically jump to the unfinished end (5); 16, reconstruct to the end and mark the end, there is no unfinished end, the entire neuron is traversed, and the reconstruction is completed.

[0128] The semi-automatic reconstruction process supporting the search operation in step S2, wherein step S21, further includes using the Dijkstra algorithm to search for a candidate node sequence starting from the current position without a specific target position.

[0129] The Dijkstra algorithm is a shortest path algorithm from one vertex to the remaining vertices, which solves the shortest path problem in a weighted graph.

[0130] In this embodiment, the Dijkstra algorithm is used to find a candidate node sequence starting from the current position without a specific target position.

[0131] The distance d(i,j) between neighboring voxels i and j is modified to avoid excessive bending. For the end point i of all partial paths, a vector T(i) is defined.

[0132] The end point of the partial path (s) is s. If s is on a reconstructed edge, the vector T(s) is the unit tangent vector of the edge at the current position, otherwise it is a zero vector.

[0133] For a partial path containing multiple voxels (s,i1,i2,…,i n ), T(i n ) is i n The unit tangent vector for this part of the path.

[0134] The distance d'(i,j) between adjacent voxels i and j is defined as:

[0135]

[0136]

[0137] Among them, P i represents the coordinates of voxel i, and |P| represents the length of vector P.

[0138] The semi-automatic reconstruction process supporting the search operation proposed in step S2 of the present invention is implemented by writing a plug-in interface.

[0139] The interface encapsulates the implementation algorithm of operations such as search in the semi-automatic reconstruction process.

[0140] In this embodiment, step S2 also includes a reconstruction error search step, which searches for edges that do not belong to any neuron, neurons that are not completely reconstructed, and conflicts between edges or cell bodies in the reconstruction result:

[0141] If two edges in the model overlap (partially), the length of the overlapping part exceeds the selection threshold, and the distances between the corresponding nodes at the overlapping part are less than the selection threshold, then the two edges are considered to be in conflict;

[0142] If the distance between the cell bodies of two neurons is less than the selected threshold, the two cell bodies are considered to be in conflict.

[0143] Find all conflicts based on the selected threshold, and then split the parts of the refactoring results with conflicts or other errors into separate files to facilitate manual error correction.

[0144] It should be noted that the selection thresholds corresponding to the node distance on the edge, the length along the edge, and the cell body distance are different, and can be adjusted and selected according to needs.

[0145] The step of finding reconstruction errors mainly involves automatically finding conflicting edges (duplicates), and the correction operation mainly involves deleting duplicates and connecting and integrating the remaining ones.

[0146] In this embodiment, step S2 further includes reconstructing:

[0147] Based on the current position, the existing reconstruction results are used to find candidate node sequences for re-reconstruction as a proofreading of the existing reconstruction results.

[0148] Reconstruction is a special case of the search operation. Reconstruction provides candidate node sequences based on the existing reconstruction results, and can be used to check the existing reconstruction results during the reconstruction process. Since it is not based on images, the checking process is efficient.

[0149] Step S3, performing neuron structure analysis mainly includes cluster analysis based on neuron structure. Cluster analysis is based on the calculation of the distance (structural difference) between two neurons.

[0150] The reconstruction result of a neuron is a tree with the cell body as the root, which contains all the reconstructed nodes. The distance between neurons is calculated based on the node correspondence between the two neurons.

[0151] The node correspondence here means that in two neurons, each node of each neuron is mapped to a node of the other neuron. In this way, for a certain node correspondence, its penalty can be defined as the weighted sum of the Euclidean distances from all nodes to the mapped node.

[0152] The optimal node correspondence between two neurons is the node correspondence with the smallest penalty. Finally, the optimal node correspondence is used to calculate the distance between the two neurons.

[0153] The dynamic programming algorithm can be used to directly calculate the optimal node correspondence between two linear structures (node ​​sequences). By matching sub-branches at the branches, the optimal node correspondence between tree structures can be calculated.

[0154] Figure 4 The axonal projection spectrum result diagram according to one embodiment of the present invention is disclosed. Figure 5 A neuron clustering data graph according to an embodiment of the present invention is disclosed, Figure 6 The neuron clustering effect diagram according to an embodiment of the present invention is disclosed, such as Figures 4 to 6 As shown, the neuronal reconstruction method for whole-brain optical imaging data proposed in the present invention performed large-scale neuronal axon reconstruction on 149 groups of sparsely labeled fMOST imaging data of the mouse prefrontal cortex, and obtained the axon projection spectrum results of 6,101 neurons in total, and performed correlation analysis based on the reconstructed single neuron projection group.

[0155] The results show that the neuron reconstruction method based on whole-brain optical imaging data proposed in the present invention can effectively reconstruct single neuron projection groups on a large scale, and can be used to analyze the reconstructed projection groups, providing an effective means for whole-brain network computational analysis with single neuron resolution based on big data.

[0156] Figure 4 and Figure 6 It is an effect diagram obtained by using a neuron reconstruction method based on whole-brain optical imaging data, which has nothing to do with the technical solution of the present invention, and in order to highlight the display effect, this grayscale perspective method must be used.

[0157] Figure 7 The principle block diagram of a neuron reconstruction device based on whole-brain optical imaging data according to an embodiment of the present invention is disclosed. Figure 7 As shown, the neuron reconstruction device based on whole-brain optical imaging data provided by the present invention includes a preprocessing module 210, a semi-automatic reconstruction module 220 and an analysis module 230:

[0158] The preprocessing module 210 preprocesses the original resolution imaging data;

[0159] The semi-automatic reconstruction module 220 includes a loading unit 221, a reconstruction unit 222 and a visualization unit 223;

[0160] The loading unit 221 loads the imaging data, the input is the processed image data from the file system or the network, and the output is the image data in the memory;

[0161] The reconstruction unit 222 supports the semi-automatic reconstruction process of the search operation, assists in traversing neurons and reconstructing the neuron model in combination with the current position, and the input is image data and user input data;

[0162] The neuron model is represented by a tree consisting of edges and vertices, wherein the edge is a node sequence, each node records the type, position and radius, and the vertex is the two end points of the edge.

[0163] The visualization unit 223 is connected to the loading unit 221 and the reconstruction unit 222, and inputs image data and neuron model data, performs two-dimensional or three-dimensional visualization of the original resolution imaging data, performs three-dimensional visualization of the neuron model in the reconstruction process, and outputs the display to the user.

[0164] The analysis module 230 performs neuron structure analysis.

[0165] In some embodiments, the preprocessing module 210 splits the original resolution imaging data and saves it in the form of small cubes; downsamples the original resolution imaging data to generate downsampled imaging data; and compresses the downsampled imaging data and / or the original resolution imaging data in the form of small cubes.

[0166] In some embodiments, the loading unit 221 automatically loads the downsampled imaging data and the original resolution imaging data in the form of small cubes in a certain area around the current position through a file system or a network.

[0167] In some embodiments, the reconstruction unit 222: finds an alternative node sequence according to the current position; verifies the node sequence; and expands the current neuron model with the node sequence.

[0168] In some embodiments, the reconstruction unit 222 quickly selects a three-dimensional position according to the position and depth of the maximum brightness within a certain range of the selected point, and accurately selects the three-dimensional position in combination with a second selection at a different angle.

[0169] In some embodiments, the reconstruction unit 222 uses the Dijkstra algorithm to find a candidate node sequence starting from the current position without a specific target position.

[0170] In some embodiments, the semi-automatic reconstruction module 220 searches for reconstruction errors:

[0171] If two edges in the model overlap (partially), the length of the overlapping part exceeds the preset threshold, and the distances between the corresponding nodes at the overlapping part are less than the preset threshold, then the two edges are considered to be in conflict;

[0172] If the distance between the cell bodies of two neurons is less than a preset threshold, the two cell bodies are considered to be in conflict; the conflicting edges or repeated parts of the cell bodies are deleted, and the connections are integrated after deletion.

[0173] In some embodiments, the reconstruction unit 222 uses the existing reconstruction results to find a candidate node sequence for re-reconstruction based on the current position, as a check on the existing reconstruction results.

[0174] In some embodiments, the reconstruction unit 222 records each unfinished vertex, and automatically jumps to the nearest unfinished vertex when a branch reconstruction is completed to start the reconstruction of the branch.

[0175] In some embodiments, the reconstruction unit 222 implements a semi-automatic reconstruction process that supports the search operation through a plug-in interface encapsulation.

[0176] The interface encapsulates the implementation algorithm of operations such as search in the semi-automatic reconstruction process.

[0177] Figure 8 The principle block diagram of a neuron reconstruction system based on whole-brain optical imaging data according to an embodiment of the present invention is disclosed. Figure 8 As shown, the neuron reconstruction system based on whole-brain optical imaging data proposed by the present invention may include an internal communication bus 801, a processor (processor) 802, a read-only memory (ROM) 803, a random access memory (RAM) 804, a communication port 805, and a hard disk 807. The internal communication bus 801 can realize data communication between components of the neuron reconstruction system based on whole-brain optical imaging data. The processor 802 can make judgments and issue prompts. In some embodiments, the processor 802 can be composed of one or more processors.

[0178] The communication port 805 can realize data transmission and communication between the neuron reconstruction system based on the whole-brain optical imaging data and the external input / output device. In some embodiments, the neuron reconstruction system based on the whole-brain optical imaging data can send and receive information and data from the network through the communication port 805. In some embodiments, the neuron reconstruction system based on the whole-brain optical imaging data can transmit and communicate data with the external input / output device in a wired form through the input / output terminal 806.

[0179] The neuron reconstruction system based on whole-brain optical imaging data can also include different forms of program storage units and data storage units, such as hard disk 807, read-only memory (ROM) 803 and random access memory (RAM) 804, which can store various data files used for computer processing and / or communication, and possible program instructions executed by processor 802. Processor 802 executes these instructions to implement the main part of the method. The results processed by processor 802 are transmitted to an external output device through communication port 805 and displayed on the user interface of the output device.

[0180] For example, the implementation process file of the above-mentioned neuron reconstruction method based on whole-brain optical imaging data can be a computer program, which is saved in the hard disk 807 and can be recorded in the processor 802 for execution to implement the method of the present application.

[0181] When the implementation process file of the neuron reconstruction method based on whole-brain optical imaging data is a computer program, it can also be stored in a computer-readable storage medium as a product. For example, a computer-readable storage medium may include, but is not limited to, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic strip), an optical disk (e.g., a compact disk (CD), a digital versatile disk (DVD)), a smart card, and a flash memory device (e.g., an electrically erasable programmable read-only memory (EPROM), a card, a stick, a key drive). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain and / or carry code and / or instructions and / or data.

[0182] The present invention proposes a neuron reconstruction method, device, system and medium based on whole-brain optical imaging data, which specifically has the following beneficial effects:

[0183] 1) Support the processing of TB-level imaging data;

[0184] 2) The reconstruction process is semi-automated, supervised by the user, minimizing user input, and achieving high reconstruction efficiency;

[0185] 3) Supports 2D and 3D display, can simultaneously display and analyze multi-channel data, and supports images of different depths;

[0186] 4) It has a wide range of applications. The reconstruction process can be carried out in combination with the user's specific research focus, for example, it can be specifically reconstructed for axons or dendrites;

[0187] 5) Compatible and extensible with other software.

[0188] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0189] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0190] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0191] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0192] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0193] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0194] The above embodiments are provided for persons familiar with the art to implement or use the present invention. Personnel familiar with the art can make various modifications or changes to the above embodiments without departing from the inventive concept of the present invention. Therefore, the protection scope of the present invention is not limited to the above embodiments, but should be the maximum scope of the innovative features mentioned in the claims.

Claims

1. A neuron reconstruction method based on whole-brain optical imaging data, characterized in that: The following steps are involved: S1, preprocessing of original resolution imaging data; S2. Based on the semi-automatic reconstruction process supporting the search operation, combined with the current position, assist in traversing neurons and reconstruct the neuron model, wherein the neuron model is represented by a tree consisting of edges and vertices, wherein the edge is a node sequence, each node records type, position and radius information, and the vertex is the two end points of the edge; S3, perform neuronal structural analysis; Wherein, the step S2 further includes visually displaying the original resolution imaging data in two dimensions or three dimensions, and switching between two dimensions and three dimensions according to the instruction; The step S2 further includes performing a three-dimensional visual display of the neuron model; The semi-automatic reconstruction process supporting the search operation in step S2 further includes: S21, finding a candidate node sequence according to the current position; S22, verifying the node sequence; S23, expanding the current neuron model with the node sequence; The step S2 further comprises: In the visualization window, the three-dimensional position is selected according to the position and depth of the maximum brightness of the imaging data within a certain range of the mouse click, and the three-dimensional position is selected in combination with the second click at a different angle.

2. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The step S1 further comprises the following steps: Split the original resolution imaging data and save them in the form of small cubes; Downsampling the original resolution imaging data to generate downsampled imaging data; The downsampled imaging data and / or the original resolution imaging data in the form of small cubes are compressed.

3. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The step S2 further includes imaging data loading, automatically loading downsampled imaging data and / or original resolution imaging data in the form of small cubes in a certain area around the current position through a file system or a network.

4. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The step S21 further comprises: Use Dijkstra's algorithm to find a sequence of candidate nodes starting from the current position without a specific target position.

5. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The step S2 further includes a step of finding reconstruction errors: If two edges in the model partially overlap, the length of the overlapping part exceeds the selection threshold, and the distances between the corresponding nodes at the overlapping part are less than the selection threshold, then the two edges are considered to be in conflict; If the distance between the cell bodies of two neurons is less than the selected threshold, the two cell bodies are considered to be in conflict; The conflicting edges or duplicate parts of the cell body are deleted and then connected and integrated.

6. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The semi-automatic reconstruction process supporting the search operation in step S2 further includes re-reconstructing: Based on the current position, the existing reconstruction results are used to find candidate node sequences for re-reconstruction as a proofreading of the existing reconstruction results.

7. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The semi-automatic reconstruction process supporting the search operation in step S2 further includes: Record each unfinished vertex, and automatically jump to the nearest unfinished vertex when a branch reconstruction is completed to start the reconstruction of the branch.

8. The neuron reconstruction method based on whole-brain optical imaging data according to claim 1, characterized in that: The semi-automatic reconstruction process supporting the search operation in step S2 is implemented through plug-in interface encapsulation.

9. A neuron reconstruction device based on whole-brain optical imaging data, characterized in that: Including preprocessing module, semi-automatic reconstruction module and analysis module: The preprocessing module preprocesses the original resolution imaging data; The semi-automatic reconstruction module comprises a reconstruction unit; The reconstruction unit, based on a semi-automatic reconstruction process supporting a search operation, assists in traversing neurons and reconstructs a neuron model in combination with a current position; The analysis module performs neuron structure analysis; The neuron model is represented by a tree consisting of edges and vertices, wherein the edge is a node sequence, each node records type, position and radius information, and the vertex is the two end points of the edge; The semi-automatic reconstruction module further includes a visualization unit: Visualize the original resolution imaging data in two or three dimensions, and switch between two and three dimensions according to instructions; Visualize the neuron model in three dimensions; The reconstruction unit: Find the candidate node sequence according to the current position; Verify the node sequence; Extend the current neuron model with this node sequence; The reconstruction unit selects a three-dimensional position in the visualization window according to the position and depth of the maximum brightness of the imaging data within a certain range of the mouse click, and selects the three-dimensional position in combination with the second click at a different angle.

10. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The preprocessing module: Split the original resolution imaging data and save them in the form of small cubes; Downsampling the original resolution imaging data to generate downsampled imaging data; The downsampled imaging data and / or the original resolution imaging data in the form of small cubes are compressed.

11. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The semi-automatic reconstruction module further includes a loading unit: The loading unit loads the imaging data; Automatically load downsampled imaging data and / or original resolution imaging data in the form of small cubes around the current location via the file system or network.

12. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The reconstruction unit uses the Dijkstra algorithm to find a candidate node sequence starting from the current position without a specific target position.

13. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The semi-automatic refactoring module looks for refactoring errors: If two edges in the model partially overlap, the length of the overlapping part exceeds the selection threshold, and the distances between the corresponding nodes at the overlapping part are less than the selection threshold, then the two edges are considered to be in conflict; If the distance between the cell bodies of two neurons is less than the selected threshold, the two cell bodies are considered to be in conflict; The conflicting edges or duplicate parts of the cell body are deleted and then connected and integrated.

14. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The reconstruction unit, based on the current position, uses the existing reconstruction results to find a candidate node sequence for re-reconstruction, as a proofreading of the existing reconstruction results.

15. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The reconstruction unit records each unfinished vertex, and automatically jumps to the nearest unfinished vertex when a branch reconstruction is completed to start the reconstruction of the branch.

16. The neuron reconstruction device based on whole-brain optical imaging data according to claim 9, characterized in that: The reconstruction unit implements a semi-automatic reconstruction process supporting the search operation through plug-in interface encapsulation.

17. A neuron reconstruction system based on whole-brain optical imaging data, comprising: a memory for storing instructions executable by a processor; A processor, configured to execute the instructions to implement the method according to any one of claims 1 to 8.

18. A computer-readable medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is performed.

Images