Neural loop volume data processing method and system based on hierarchical storage and medium

Abstract

The invention discloses a neural loop volume data processing method and system based on hierarchical storage and a medium. According to the data partitioning processing method, input image data are partitioned according to calculation node configuration; clustering and task allocation are carried out on neurons according to the number of calculation nodes; and each computing node independently executes a neural circuit tracking task, reads the data blocks into the memory from the disk array or the local solid-state disk according to a data pre-fetching rule in a computing process, and writes the data blocks into the disk array or the local solid-state disk from the memory according to a data replacement rule until the distributed tasks are completely processed. The method can effectively process ten-TB-magnitude super-large-scale image data, has the characteristics of high processing speed and high expansibility, can be deployed on a distributed computing platform, is flexible to implement, supports parameter configuration, and is easy to transplant and popularize.

Description

technical field [0001] The present invention relates to the large-scale image processing of the analysis method of the fine structure of the neural circuit, in particular to a neural circuit body data processing method, system and medium based on hierarchical storage, which are used to realize the neural circuit body data analysis on the distributed multi-core computing platform Block processing of very large-scale image data in the process. Background technique [0002] Neural circuit information is the key to understanding brain function and the mechanism of brain diseases. How to realize the automatic tracking of neural circuit big data is one of the key scientific problems faced by research in the field of neurology such as brain science. The location of neurons is the key to the analysis of neural circuit data. The accurate location of neuron cell bodies is obtained by analyzing the data of neural circuit bodies, which is the basis for subsequent quantitative analysis. ...

AI Technical Summary

Problems solved by technology

The traditional neural circuit volume data processing method needs to load all the volume data from the disk array into the memory. However, the memory capacity of the existing computing nodes can only reach the order of TB, and cannot process image data of the order of ten TB.

How to effectively process ten terabytes of image data containing tens of thousands of layers is a huge challenge in the image data processing of neural circuit analysis, and has become a bottleneck restricting whether the acquired data can be transformed into knowledge

Images