Adaptive-tree-based spatio-temporal-enhanced clustering method and system for neural electrical pulse signals

Abstract

Disclosed in the present invention are an adaptive-tree-based spatio-temporal-enhanced clustering method and system for neural electrical pulse signals. The method comprises: acquiring an electrophysiological record and preprocessing same, so as to obtain target whitened data, and on the basis of a threshold value, performing detection on the target whitened data, so as to obtain neural electrical pulse signals; performing spatio-temporal enhancement on an action potential and spatio-temporal information of each neural electrical pulse signal, so as to obtain contrasted neural electrical pulse signals; encoding action potentials and spatio-temporal information of embedded neural electrical pulse signals and corresponding contrasted neural electrical pulse signals, so as to obtain encoded signals; and inputting the encoded signals into an adaptive tree structure, and outputting a clustering result for neural electrical pulse signals, thereby realizing adaptive clustering of neural electrical pulse signals. By means of the present invention, an adaptive function and a function for quick clustering are achieved on the basis of a tree structure, and representation learning is performed on pulse features from different neurons by means of additionally providing contrasted neural electrical pulse signals.

Description

Adaptive tree based spatiotemporal enhanced neural spike signal clustering method and system TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to an adaptive tree based spatiotemporal enhanced neural spike signal clustering method, system, terminal and computer readable storage medium. BACKGROUND

[0002] In the past few decades, the study of neurons and their interactions has attracted great interest from most scientists. This trend has given rise to a series of new scientific fields, including cognitive neuroscience, computational neuroscience, and intracortical brain-computer interface (iBCI). These research areas not only improve our understanding of how the brain works, but also provide new possibilities and methods for the treatment of many neurological diseases.

[0003] The study of neurons mainly relies on recording extracellular electrophysiological data of neuron populations through microelectrode arrays. These data can reveal the specific relationship between the recorded neuron activity and its directly perceptible response. For example, researchers can explore and decode how the brain processes and interprets information received from the senses in this way. This specific mapping relationship provides important clues for understanding the complex network of the brain and may help to develop more effective treatment strategies for various neurological diseases. Therefore, this also promotes the progress of silicon technology and the emergence of large-scale integrated circuits.

[0004] Currently, the electrophysiological recordings of hundreds to thousands of neurons can be obtained through multi-electrode arrays and silicon probes. However, how to locate the detected neural spike signals to the assumed neuron from these high-throughput neural electrophysiological recordings is the core content of neural spike signal sorting, and the overlap of spike signals is the key problem of neural spike signal sorting. This technology enables researchers to observe and understand the working mechanism of the brain on a larger scale, and the progress of this technology undoubtedly opens up new possibilities for neuroscience research.

[0005] Currently widely used are some unsupervised learning methods, including deep learning methods based on autoencoders and the Kilosort toolbox. Deep learning methods based on autoencoders first learn the features of neural electrical pulse signals and then use the features for clustering, which requires knowing the number of neurons to be clustered in advance. Kilosort, a software for extracting and classifying neural signals, uses the EM algorithm (Expectation-Maximization algorithm) to perform template matching, which is sensitive to the pre-set threshold for pulse sorting, and this manual intervention has strong subjective uncertainty. In general, these neural electrical pulse signal sorting methods have similar limitations and challenges, requiring a large amount of computing resources, high requirements for data quality and noise level (high sorting of neural electrical pulse signals to some extent depends on data), user-defined thresholds (introducing uncertainty) or specified number of clustered neurons, and no processing of overlapping pulse signals.

[0006] Therefore, the prior art still needs to be improved and developed.

[0007] SUMMARY

[0008] The main purpose of the present application is to provide a kind of adaptive tree based on spatiotemporal enhancement neural electrical pulse signal clustering method, system, terminal and computer readable storage medium, to solve the problems of inaccurate neural electrical pulse signal sorting in prior art, such as unsupervised learning method needing user-defined threshold or specifying the number of clustered neurons, requiring a large amount of computing resources, high requirements for data quality and noise level, and no processing of overlapping pulse signals.

[0009] To achieve the above purpose, the present application provides an adaptive tree based on spatiotemporal enhancement neural electrical pulse signal clustering method, which comprises the following steps:

[0010] Obtain an electrophysiological record, pre-process the electrophysiological record to obtain target whitening data, detect the target whitening data according to a threshold, and obtain neural electrical pulse signals;

[0011] The action potential and spatiotemporal information of each neural electrical pulse signal are spatiotemporally enhanced to obtain a contrast neural electrical pulse signal, and the spatiotemporal information includes the electrode position and probe number when detected, and the time stamp when fired;

[0012] Information is embedded into the neural electrical pulse signal and the action potential and space-time information of the contrast neural electrical pulse signal, the embedded neural electrical pulse signal and the corresponding action potential and space-time information of the contrast neural electrical pulse signal are encoded, and an encoded signal is obtained;

[0013] The encoded signal is input into an adaptive tree structure, and a neural electrical pulse signal clustering result is output to realize adaptive neural electrical pulse signal clustering.

[0014] Optionally, the adaptive tree-based space-time enhanced neural electrical pulse signal clustering method, wherein the preprocessing includes filtering, common mode rejection and whitening;

[0015] The electro-physiological record is obtained, and the electro-physiological record is preprocessed to obtain target whitening data, specifically including:

[0016] The electro-physiological record is obtained, and the electro-physiological record is filtered by a filter to obtain filtered data in a preset frequency range, and the frequency response H(f) of the filter is: H(f) = 1 / sqrt(1+(b / f)(2n))-1 / sqrt(1+(a / f)(2n));

[0017] Wherein, f represents frequency, n represents the order of the filter, determines the steepness of the transition between the passband and the stopband, sqrt represents square root, a and b represent the lower cutoff frequency and the upper cutoff frequency of the filter;

[0018] The filter is applied to the electro-physiological record X(t) to obtain the filtered filtered data Y(t): Y(t) = X(t)*F -1 {H(f)};

[0019] Wherein, F -1 Indicates the inverse Fourier transform, and * indicates convolution operation;

[0020] The common mode rejection is performed on the filtered data Y(t) to remove the common noise component in the filtered data Y(t): Y'(i,t) = Y(i,t)-Median{Y(j,t)|for all j};

[0021] Wherein, i represents the electrode, t represents the sampling time, Median represents the median, j represents the index of all electrodes, Y(i,t) represents the filtered signal at electrode i and sampling time t, Y(j,t) represents the filtered signal at electrode j and sampling time t, the value range of j is [1, total number of electrodes], and Y'(i,t) represents the signal at electrode i after the common mode rejection of Y(i,t);

[0022] The Y'(i, t) is whitened, eigenvalues and eigenvectors of a sample data covariance matrix are calculated, a whitening matrix is constructed, and the Y'(i, t) is multiplied by the whitening matrix to obtain whitened target whitening data Y''(i, t).

[0023] Optionally, the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method, wherein the calculation of the eigenvalues and eigenvectors of the sample data covariance matrix, the construction of the whitening matrix, and the multiplication of the Y'(i, t) by the whitening matrix to obtain the whitened target whitening data Y''(i, t) specifically include:

[0024] The calculation of the eigenvalues and eigenvectors of the sample data covariance matrix: [V, D] = eig(cov(Y'(i, t)));

[0025] Wherein, V represents a matrix containing all eigenvectors, D represents a diagonal matrix, eig represents the calculation of eigenvalues and eigenvectors, and cov represents the calculation of covariance;

[0026] The construction of the whitening matrix W zca : W zca = VD (-1 / 2) V T ;

[0027] Wherein, V T represents a transposition operation on V;

[0028] The whitened target whitening data Y''(i, t) is obtained: Y''(i, t) = Y'(i, t)W zca .

[0029] Optionally, the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method, wherein the detection of the target whitening data according to a threshold value to obtain a neural electric pulse signal specifically includes:

[0030] A threshold value ζ is preset, the threshold value ζ is calculated according to the voltage collected on each electrode, the threshold value ζ is set as a multiple of a median value, and the threshold value ζ is calculated as: ζ = Median(|Y''(i, t)|) / 0.6745.

[0031] The target whitening data is detected according to the threshold value ζ, and when the signal intensity exceeds the threshold value ζ, it is considered that a pulse occurs, and a neural electric pulse signal is obtained.

[0032] Optionally, the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method, wherein the spatiotemporal enhancement of the action potential includes amplitude expansion, lateral displacement, random noise addition, overlap addition, and baseline drift; and the spatiotemporal enhancement of the spatiotemporal information is the transformation of the electrode position where the detection is performed and the probe number and the time stamp when the discharge is performed.

[0033] Optionally, the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method, wherein the action potential and spatiotemporal information of the neural electrical pulse signal and the contrast neural electrical pulse signal are information embedded, the embedded action potential and spatiotemporal information of the neural electrical pulse signal and the corresponding contrast neural electrical pulse signal are encoded to obtain an encoded signal, and the method specifically comprises:

[0034] The action potential, the electrode position at the time of detection, the probe number, and the time stamp at the time of discharge of each neural electrical pulse signal are dimensionally converted through a layer of fixed number of fully connected layers to obtain four embedded information;

[0035] The four embedded information are summed and output to an attention mechanism module for encoding learning to obtain an encoded signal, and the attention mechanism module is composed of stacked layers, each layer containing a multi-head self-attention mechanism and a feedforward fully connected neural network.

[0036] Optionally, the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method, wherein the encoded signal is input to an adaptive tree structure, and a neural electrical pulse signal clustering result is output to realize adaptive neural electrical pulse signal clustering, and the method specifically comprises:

[0037] The encoded signal is pre-clustered, and the mutual information between the current neural electrical pulse signal and the contrast neural electrical pulse signal is calculated using an adaptive tree structure to maximize the mutual information;

[0038] When the mutual information converges and is not less than a preset threshold, pulse clustering is performed, and a neural electrical pulse signal clustering result is output.

[0039] In addition, to achieve the above-mentioned purposes, the application further provides an adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering system, wherein the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering system comprises:

[0040] A data processing module is configured to acquire an electrophysiological record, pre-process the electrophysiological record to obtain target whitening data, detect the target whitening data according to a threshold, and obtain a neural electrical pulse signal;

[0041] A spatiotemporal enhancement module is configured to perform spatiotemporal enhancement on the action potential and spatiotemporal information of each neural electrical pulse signal to obtain a contrast neural electrical pulse signal, and the spatiotemporal information includes the electrode position at the time of detection, the probe number, and the time stamp at the time of discharge;

[0042] An information encoding module is used to embed the action potential and spatiotemporal information of the neural electrical pulse signal and the comparative neural electrical pulse signal, and encode the action potential and spatiotemporal information of the embedded neural electrical pulse signal and the corresponding comparative neural electrical pulse signal to obtain an encoded signal;

[0043] The signal clustering module is used to input the encoded signal into the adaptive tree structure and output the neural electrical pulse signal clustering result to achieve adaptive neural electrical pulse signal clustering.

[0044] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering program stored in the memory and runnable on the processor, wherein the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering program, when executed by the processor, implements the steps of the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method as described above.

[0045] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a spatiotemporal enhanced neural electrical pulse signal clustering program based on an adaptive tree, and when the spatiotemporal enhanced neural electrical pulse signal clustering program based on an adaptive tree is executed by a processor, the steps of the spatiotemporal enhanced neural electrical pulse signal clustering method based on an adaptive tree as described above are implemented.

[0046] In the present invention, an electrophysiological recording is obtained, the electrophysiological recording is preprocessed to obtain target whitened data, the target whitened data is detected according to a threshold value to obtain a neural pulse signal; the action potential and spatiotemporal information of each neural pulse signal are spatiotemporally enhanced to obtain a contrast neural pulse signal, the spatiotemporal information including the electrode position at the time of detection, the probe number, and the timestamp of the release; the action potential and spatiotemporal information of the neural pulse signal and the contrast neural pulse signal are embedded, the action potential and spatiotemporal information of the embedded neural pulse signal and the corresponding contrast neural pulse signal are encoded to obtain an encoded signal; the encoded signal is input into an adaptive tree structure, and a neural pulse signal clustering result is output to achieve adaptive neural pulse signal clustering. The unsupervised clustering scheme proposed in the present invention does not require a user-defined threshold, does not require a known number of neurons for clustering, and solves the problem of pulse signal overlap to a certain extent. It realizes adaptive clustering of neural pulse signal sorting without manual intervention, realizes adaptive and fast clustering based on the tree structure, and represents and learns the pulse features from different neurons by adding contrast neural pulse signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Fig. 1 is a flow chart of a preferred embodiment of the adaptive tree-based spatiotemporal enhanced neural spike signal clustering method of the present application;

[0048] Fig. 2 is a schematic diagram of the principle of neural spike signal clustering in a preferred embodiment of the adaptive tree-based spatiotemporal enhanced neural spike signal clustering method of the present application;

[0049] Fig. 3 is a schematic diagram of an example of adaptive tree-based neural spike signal clustering in a preferred embodiment of the adaptive tree-based spatiotemporal enhanced neural spike signal clustering method of the present application;

[0050] Fig. 4 is a structural diagram of a preferred embodiment of the adaptive tree-based spatiotemporal enhanced neural spike signal clustering system of the present application;

[0051] Fig. 5 is a structural diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0053] Current neural spike signal sorting methods can be roughly divided into two categories: one is supervised learning, and the other is unsupervised learning.

[0054] 1. The method of supervised learning sorts the pulse data by manual annotation and then inputs them into the built model for training. This method needs to consume a lot of human cost, especially in large-scale neural electrophysiological recordings. At the same time, the model based on deep learning also needs to invest a lot of computing cost.

[0055] 2. The method of unsupervised learning is more in line with the needs of advanced technology. In general, unsupervised neural spike signal sorting methods achieve clustering through distance measurement or feature learning. This method needs to know the number of classes in advance.

[0056] 3. Since the number of target neurons in neural spike signal sorting cannot be obtained, in order to cope with the clustering in the above-mentioned case where the number of classes needs to be specified, some advanced unsupervised neural spike signal sorting methods introduce a threshold to regulate the sorting result without explicitly specifying the number of neurons. This makes the result of neural spike signal sorting highly related to the setting of the threshold, which has unstable factors. Moreover, in high-throughput neural electrophysiological recordings, the problem of neural spike signal overlap has not been solved, which affects the performance of neural spike signal sorting.

[0057] For example, the situation of pulse signal overlap occurs when the electrodes around the detected neuron detect the voltage at that time, which involves the consideration of multiple adjacent electrodes, and the existing enhancement method only considers the action potential without data enhancement of the spatiotemporal information. The spatiotemporal information is a key factor for neural electrical pulse sorting. Moreover, the scheme proposed by the prior art adopts a binary tree structure of converting a multi-classification problem into multiple binary classification problems, and cannot perform fast neural electrical pulse signal sorting for large-scale data.

[0058] Therefore, a neural electrical pulse signal sorting method without manual intervention, high efficiency and effectiveness is the goal to be achieved at present.

[0059] The adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method according to the preferred embodiment of the present application, as shown in FIGS. 1 and 2, comprises the following steps:

[0060] In step S10, an electro-physiological record is obtained, the electro-physiological record is pre-processed to obtain target whitening data, the target whitening data is detected according to a threshold value, and a neural electrical pulse signal is obtained.

[0061] Specifically, the electro-physiological record usually contains background noise and neural electrical pulse signals of interest to researchers, so before neural electrical pulse signal clustering (sorting), the neural electrical pulse signal needs to be detected after pre-processing.

[0062] Due to the influence of local field potential and high-frequency noise, the electro-physiological record obtained from the microelectrode cannot be directly operated. The electro-physiological record needs to be pre-processed, and the pre-processing includes filtering, common-mode rejection and whitening. After obtaining the electro-physiological record (i.e. the original signal), the electro-physiological record is filtered by a filter (for example, a Butterworth filter, which is one of electronic filters) to obtain filtered data in a preset frequency range (300 Hz to 3000 Hz), and this part of the signal is likely to be a peak. The frequency response of the Butterworth filter is represented as H(f), the cutoff frequency of the filter is between 300 Hz and 3000 Hz, and the frequency response H(f) of the filter is: H(f) = 1 / sqrt(1+(b / f)(2n))-1 / sqrt(1+(a / f)(2n));

[0063] Wherein, f represents the frequency, n represents the order of the filter, which determines the steepness of the transition between the passband and the stopband, sqrt represents the square root, a and b represent the lower cutoff frequency and the upper cutoff frequency of the filter, a = 300 Hz, b = 3000 Hz; In practice, the order of the filter will be selected according to the specific needs of the application.

[0064] applying a filter to the electrophysiological recording X(t) to obtain filtered data Y(t): Y(t) = X(t) * F -1 {H(f)};

[0065] where F -1 denotes the inverse Fourier transform and * denotes the convolution operation.

[0066] To eliminate the change of electric field caused by electrode movement or current leakage, the filtered data Y(t) is subjected to common mode rejection (Common Mode Rejection, which aims to eliminate or reduce noise introduced by measurement equipment or external environment. Common mode signals usually refer to signals that exist on all channels (electrodes), which often do not contain valid information of interest, but some background noise or interference. For each time point t, first calculate the median of the signal values of all electrodes at this time point, which represents the center position of all electrode signal values at this time point t, which can be regarded as an estimate of background noise, then subtract this median from the signal value of each electrode at time point t. The purpose of this is to remove the common mode in the signal, i.e. the signal component common to all electrodes, which is usually related to noise or other non-specific brain activity. The signal Y'(i,t) after common mode rejection will highlight the brain activity features related to specific tasks more prominently, because these features are differentiated among different electrodes, while the common mode noise is suppressed), remove the common noise component in the filtered data Y(t): Y'(i,t) = Y(i,t) - Median{Y(j,t) | for all j};

[0067] where i represents the electrode, t represents the sampling time, Median represents the median, j represents the index of all electrodes, Y(i,t) represents the filtered signal at electrode i and sampling time t, Y(j,t) represents the filtered signal at electrode j and sampling time t, and j ranges from 1 to the total number of electrodes. Here, because there is a Median function, all electrodes need to be traversed for calculation, and Y'(i,t) represents the signal at electrode i after common mode rejection of Y(i,t).

[0068] This operation removes all the noise added to the electrodes due to movement and electromagnetic interference, among other factors. Then, Y'(i, t) is whitened (ZCA whitening, Zero Component Analysis, used to eliminate redundancies and correlations in the neural voltage signal data, so that the signal of each channel can independently encode the activity of the neurons, thus improving the accuracy and efficiency of the subsequent step of neuron ordering. Specifically, ZCA whitening includes the following steps: centering the data, i.e., subtracting the mean; calculating the covariance matrix of the data; performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and eigenvectors; transforming the original data with the eigenvectors and eigenvalues, so that the covariance matrix of the transformed data is the identity matrix), to reduce the correlation between electrodes.

[0069] First, the eigenvalues and eigenvectors of the sample data covariance matrix are calculated, then a whitening matrix is constructed, and finally Y'(i, t) is multiplied by the whitening matrix to obtain the whitened target whitening data Y"(i, t).

[0070] First, the eigenvalues and eigenvectors of the sample data covariance matrix are calculated: [V, D] = eig(cov(Y'(i, t)));

[0071] where V represents a matrix containing all eigenvectors, D represents a diagonal matrix, eig represents eigenvalues and eigenvectors, and cov represents covariance.

[0072] Then construct the whitening matrix W zca : W zca = VD (-1 / 2) V T ;

[0073] where V T represents transposing V.

[0074] Finally, the whitened target whitening data Y"(i, t) is obtained: Y"(i, t) = Y'(i, t)W zca .

[0075] In addition, as shown in FIG. 2, the neural electrical pulse signal detection generally adopts a threshold crossing method, the basic idea of which is to set a threshold ζ, when the signal strength exceeds the threshold ζ, it is considered that a pulse occurs. The threshold ζ is set according to the voltage collected on each electrode, and the advantage of this method is that it can automatically adapt to the noise level of the signal, so it can still effectively detect the neural electrical pulse signal in the case of noise change. The present application adopts Median Absolute Deviation (MAD) as the threshold setting method. MAD is a robust statistical method for measuring the variability of a data set, which is not sensitive to outliers, so it is often used for outlier detection. In this method, the median of each electrode is calculated, and then the threshold MAD is set as a multiple of the median. The specific calculation formula is: ζ = Median(|Y"(i,t)|) / 0.6745;

[0076] The reason for dividing by 0.6745 is that under the assumption of normal distribution, MAD is standardized to have the same scale as the standard deviation. In the normal distribution, the value of the first quartile is about -0.6745. This value is one of the characteristics of the standard normal distribution (i.e. the normal distribution with a mean of 0 and a standard deviation of 1). However, in practical applications, the absolute value of this value, that is, 0.6745, is often used. Because when calculating the median absolute deviation (MAD), the focus is on the degree of data deviation from the median, not the direction of deviation. Therefore, 0.6745 is often used to standardize MAD, so that when the data follows the standard normal distribution, MAD divided by 0.6745 can obtain an unbiased estimator, and the result is equal to the standard deviation. Therefore, the threshold ζ here is actually an unbiased standard deviation estimate. In this threshold setting method, the threshold ζ is set as a multiple of the median. This is because in the case of relatively stable data distribution, most data points should be close to the median. Therefore, if a data point deviates from the median, it may be an outlier or "peak value of neural electrical pulse signal". Therefore, when the value of the data point exceeds a certain multiple of ζ, it is considered that a neural electrical pulse signal is detected. This multiple is generally set to an integer in [3, 6].

[0077] Step S20, the action potential and space-time information of each neural electrical pulse signal are space-time enhanced to obtain a contrast neural electrical pulse signal, and the space-time information includes the electrode position where the detection is located and the probe number, and the time stamp when the discharge occurs.

[0078] Specifically, the space-time enhancement of the action potential includes amplitude expansion, lateral displacement, random noise addition, overlap addition and baseline drift; and the space-time enhancement of the space-time information is to transform the electrode position where the detection is located and the probe number, and the time stamp when the discharge occurs.

[0079] As shown in FIG. 2, according to the study on the characteristics of the pulses that may occur in a single neuron, a spatiotemporal enhancement mode is designed to enhance the four pieces of information of each neural electrical pulse signal (i.e., the action potential of each neural electrical pulse signal, the electrode position where it is detected and the probe number, and the time stamp when it is fired). The neural electrical pulse signals from the firing of one neuron may be detected on multiple adjacent electrodes, while the neural electrical pulse signals from the firing of different neurons may be detected at different times on one electrode, and the action potential of the firing of a single neuron is relatively stable.

[0080] According to this basic theory, the spatiotemporal enhancement mode can locate the pulse overlap, electrode drift, etc. Since the action potential is relatively stable, the information enhancement of the action potential includes: (1) amplitude expansion (multiply the current action potential by a random number set between 0.9 and 1.1) ; (2) lateral displacement (left and right displacement of the current action potential by a random number generated in [-10, 10]) ; (3) random noise addition (add a random number generated from a normal distribution with a mean of 0 and a standard deviation of 0.08 to the current action potential) ; (4) overlap addition (first add random noise to the current action potential, then multiply it by a random real number drawn from a uniform distribution U(0.2, 0.5), and finally perform lateral displacement) ; (5) baseline drift (add a random number drawn from a uniform distribution U(-0.1, 0.1) to the current action potential).

[0081] The spatiotemporal information enhancement of the current neural electrical pulse signal is a transformation of the electrode position where it is detected and the probe number, and the time stamp when it is fired (each neural electrical pulse signal is composed of an action potential and spatiotemporal information). Since the comparative neural electrical pulse signal is relative to the current neural electrical pulse signal, the physical distance between the probes is much larger than the distance between the neurons, and it is impossible for the neural electrical pulse signals from the same neuron to be recorded by two probes, so the enhancement of the probe information is actually unchanged, while for the electrodes, a random plus or minus 1 is needed for the current electrode, indicating that the neural electrical pulse signals from the same neuron may be detected on adjacent electrodes. In the time stamp, it is possible to detect the neural electrical pulse signals from the same neuron on the current electrode throughout the sampling time, so here the enhancement needs to set a random number in a relatively large range and add it.

[0082] Step S30, information embedding is performed on the action potential and spatiotemporal information of the neural electrical pulse signal and the comparative neural electrical pulse signal, the embedded neural electrical pulse signal and the corresponding comparative neural electrical pulse signal are encoded, and the encoded signal is obtained.

[0083] Specifically, the action potential of each neural electrical pulse signal, the electrode position where it is detected, and the probe number, and the time stamp when it is fired are dimensionally transformed by a layer of the same fixed number of fully connected layers in a set space to obtain four embedding information; the four embedding information is summed and output to an attention mechanism module for encoding learning to obtain an encoded signal, and the attention mechanism module is composed of stacked layers, each layer containing a multi-head self-attention mechanism and a feedforward fully connected neural network.

[0084] When all neural electrical pulse signals are detected, the spatiotemporal information and action potential of the neural electrical pulse signals and the comparative neural electrical pulse signals are information embedded. Specifically, the action potential of each neural electrical pulse signal, the electrode position where it is detected, and the probe number, and the time stamp when it is fired are information embedded in a set space large enough. For example, if the total number of electrodes is 384, then the embedding space of the electrodes is 384, and so on. Therefore, the embedding spaces of the four information are not the same, and they need to be dimensionally transformed by a layer of the same fixed number of fully connected layers, and then the four embedding information is summed and output to an attention mechanism module for encoding learning. The encoding here adopts the encoding structure in the Transformer model (i.e. the attention mechanism module), which is composed of stacked layers, each layer containing two sub-layers: a multi-head self-attention mechanism and a feedforward fully connected neural network. The multi-head self-attention mechanism allows the model to consider other elements when processing each element in the sequence, capturing long-range dependencies. The feedforward fully connected neural network further processes the output of the attention mechanism. Each sub-layer is followed by a residual connection and layer normalization, enhancing the efficiency and stability of model training. This structure makes the Transformer model perform excellently in processing pulse data (time series data) and capturing long-range dependencies.

[0085] Step S40, input the encoded signal to an adaptive tree structure, output the neural electrical pulse signal clustering result, to realize adaptive neural electrical pulse signal clustering.

[0086] Specifically, after encoding by the Transformer model, clustering is performed by a clustering module, which is a fully connected network. Since the number of neurons to be sorted cannot be known in advance, an adaptive tree structure is designed, which can realize parallel computing and fast sorting. As shown in FIG. 3 (which assumes one possible sorting in the case of a total of 6 electrodes), first, the pulses (i.e., the encoded signals) are pre-clustered, and the number of pre-clusters is half of the number of electrodes (the number of output neurons of the clustering module), denoted as G groups. Mutual information maximization is performed until the mutual information converges, and the neural pulse signal data can be divided into G groups. After parallel computing on the G groups of data, a binary tree structure is used for further adaptive division of the model for each group. The information of the neural pulse signal of the current group is embedded, enhanced, encoded, and clustered, and the last layer uses two outputs, one of which is 2 neurons and the other of which is 4 neurons. The two outputs are alternately performed, and are used to alleviate the local optimal value of the calculated mutual information. By calculating the mutual information between the current neural pulse signal and the contrast neural pulse signal, the mutual information is maximized, and the representation space of the data is learned by maximizing the mutual information to achieve the purpose of clustering. Before reaching the preset number of iterations, if the mutual information converges to a value that is not small, clustering is immediately performed, otherwise it is indicated that the correlation between the neural pulse signal and the contrast neural pulse signal cannot be learned, and the neural pulse signal is directly classified into a hypothetical neuron. The calculated mutual information for the latter case is always oscillating around a very small value.

[0087] That is, the mutual information between the current neural pulse signal and the contrast neural pulse signal is calculated and maximized, and when the mutual information converges and is not less than a preset threshold, pulse clustering is performed, and the neural pulse signal clustering result is output.

[0088] The present application realizes the functions of adaptive and fast clustering based on a tree structure, and learns the representation of pulse features from different neurons by adding contrast neural pulse signals. First, a set of spatiotemporal enhancement methods for neural pulse signals is designed (the data after enhancement of each neural pulse signal is called a contrast neural pulse signal), second, the spatiotemporal information and action potential of the embedded neural pulse signal and the corresponding contrast neural pulse signal are encoded, and finally the encoded signal is input to the tree structure to realize adaptive neural pulse signal clustering.

[0089] The action potential of each neural electrical pulse signal, the electrode position where the detection is located, and the probe number, and the time stamp when the discharge occurs are information embedded in a set of sufficient large space, and are encoded based on an attention mechanism-based encoding structure; a contrast neural electrical pulse signal is designed in a spatiotemporal enhancement manner to realize representation learning; and a self-adaptive tree structure and mutual information maximization are used to realize clustering of the neural electrical pulse signal.

[0090] The spatiotemporal encoding manner of the neural electrical pulse signal can maximize the utilization of data characteristics; the spatiotemporal enhancement manner can solve the problem of pulse overlap in high-throughput neural electrophysiological recording to a certain extent; the self-adaptive tree structure can solve the problem in the case that the number of neurons to be clustered is unknown; and the threshold has low sensitivity to the sorting result, and can be fixed as 0.1 to a great extent, which has little effect on the sorting result of the neural electrical pulse signal.

[0091] In in-vivo data (in-vivo data refers to neural electrophysiological recording collected in a biological body (living body)), the output characteristics of the proposed spatiotemporal encoding structure and other traditional characteristics are compared in k-means clustering, and the representation space of the present application carries more information conducive to neural electrical pulse signal sorting. Algorithm testing is performed on the synthesized overlap data set, which can solve this problem to a certain extent. The neural electrical pulse signals detected in the high-throughput neural electrophysiological recording are clustered under the in-vitro and in-vivo experimental conditions, and the experimental results show that the clustering results of the present application are better than those of the kilosort method in the clustering index, and the test results on the in-vitro data set synthesized according to the characteristics of overlap show that the present application can solve the overlap problem to a certain extent.

[0092] Further, the scheme of the present application can be further improved, for example:

[0093] (1) Mutual information maximization can not use the self-adaptive tree structure, but only as an optimization target. Under the premise of known number of neurons, the number of neurons to be sorted out can be directly set artificially, the clustering module is not divided into pre-clustering and then binary tree clustering, but only the model output neuron number of the clustering module is set as the number of neurons to be clustered out as an alternative.

[0094] (2) Other contrast losses are used to realize representation learning to achieve the purpose of sorting as an alternative.

[0095] (3) The threshold is set to other fixed threshold, or changed to dynamic threshold as an alternative.

[0096] (4) Reduce several types of action potential enhancement of the neural electrical pulse signal, or modify the generation of random numbers to other fixed range or distribution as an alternative.

[0097] (5) Different degrees of electrode enhancement of the spatiotemporal information enhancement of the neural electrical pulse signal, for example, a random sampling in a certain fixed range of electrode number to add or subtract the original electrode number of the neural electrical pulse signal, which is an alternative.

[0098] (6) In the alternative mutual information maximization scheme of the output neuron number of 2 or 4 in the binary tree structure of the clustering module, remove the branch with neuron number of 4, or alternately optimize the scheme of every few times using neuron number of 2 and every few times using neuron number of 4 branch optimization as an alternative.

[0099] Further, as shown in FIG. 4, based on the above-mentioned adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method, the present application also correspondingly provides an adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering system, wherein the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering system comprises:

[0100] The data processing module 51 is configured to acquire an electrophysiological record, pre-process the electrophysiological record to obtain target whitening data, detect the target whitening data according to a threshold, and obtain a neural electrical pulse signal.

[0101] The spatiotemporal enhancement module 52 is configured to perform spatiotemporal enhancement on the action potential and spatiotemporal information of each neural electrical pulse signal to obtain a contrast neural electrical pulse signal, wherein the spatiotemporal information includes the electrode position and probe number at the time of detection, and the time stamp at the time of firing.

[0102] The information encoding module 53 is configured to perform information embedding on the action potential and spatiotemporal information of the neural electrical pulse signal and the contrast neural electrical pulse signal, encode the embedded action potential and spatiotemporal information of the neural electrical pulse signal and the corresponding contrast neural electrical pulse signal, and obtain an encoded signal.

[0103] The signal clustering module 54 is configured to input the encoded signal to an adaptive tree structure, output a neural electrical pulse signal clustering result, and realize adaptive neural electrical pulse signal clustering.

[0104] Further, as shown in FIG. 5, based on the above-mentioned adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method and system, the present application also correspondingly provides a terminal, which comprises a processor 10, a memory 20 and a display 30. FIG. 5 only shows part of the components of the terminal, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0105] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores an adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering program 40, which can be executed by the processor 10 to implement the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method in the present application.

[0106] The processor 10 can be a central processing unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method, etc.

[0107] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display a visualized user interface. The components 10-30 of the terminal communicate with each other through a system bus.

[0108] In an embodiment, the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method is implemented when the processor 10 executes the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering program 40 in the memory 20.

[0109] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores an adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering program, which, when executed by a processor, implements the steps of the adaptive tree-based spatiotemporal enhanced neural electric pulse signal clustering method as described above.

[0110] In summary, the application provides a spatiotemporal enhancement neural electric pulse signal clustering method, system, terminal and storage medium based on adaptive tree, the method comprising: acquiring an electro-physiological record, preprocessing the electro-physiological record to obtain target whitening data, detecting the target whitening data according to a threshold to obtain a neural electric pulse signal; performing spatiotemporal enhancement on the action potential and spatiotemporal information of each neural electric pulse signal to obtain a contrast neural electric pulse signal, the spatiotemporal information including the electrode position and probe number when detected and the time stamp when fired; performing information embedding on the action potential and spatiotemporal information of the neural electric pulse signal and the contrast neural electric pulse signal, encoding the embedded action potential and spatiotemporal information of the neural electric pulse signal and the contrast neural electric pulse signal to obtain an encoded signal; inputting the encoded signal into an adaptive tree structure to output a neural electric pulse signal clustering result, so as to realize adaptive neural electric pulse signal clustering. The unsupervised clustering scheme proposed in the application does not require user-defined threshold, and can perform clustering without knowing the number of neurons, and to some extent solves the problem of pulse signal overlap, realizes adaptive and fast clustering based on the tree structure, and represents learning of pulse features from different neurons by adding the contrast neural electric pulse signal.

[0111] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or terminals. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article, or terminal including the element.

[0112] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.) to complete, and the program can be stored in a computer-readable computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a magnetic disc, an optical disc, etc.

[0113] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should be within the protection scope of the appended claims of the application.

Claims

1. A spatiotemporal enhanced neural pulse signal clustering method based on adaptive tree, characterized in that: The adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method includes: Acquiring electrophysiological records, preprocessing the electrophysiological records to obtain target whitening data, and detecting the target whitening data according to a threshold value to obtain a neural electrical pulse signal; Performing spatiotemporal enhancement on the action potential and spatiotemporal information of each neural electrical pulse signal to obtain a comparative neural electrical pulse signal, wherein the spatiotemporal information includes the electrode position at the time of detection, the probe number, and the timestamp of the release; Embedding the action potential and spatiotemporal information of the neural electrical pulse signal and the comparison neural electrical pulse signal, encoding the action potential and spatiotemporal information of the embedded neural electrical pulse signal and the corresponding comparison neural electrical pulse signal to obtain an encoded signal; The encoded signal is input into the adaptive tree structure, and the neural electrical pulse signal clustering result is output to realize adaptive neural electrical pulse signal clustering.

2. The method for clustering spatiotemporal enhanced neural electrical impulse signals based on an adaptive tree according to claim 1, characterized in that: The pre-processing includes filtering, removing common mode and whitening; The obtaining of electrophysiological records and preprocessing of the electrophysiological records to obtain target whitening data specifically includes: Obtain electrophysiological records, filter the electrophysiological records through a filter, and obtain filtered data within a preset frequency range. The frequency response H(f) of the filter is: H(f)=1 / sqrt(1+(b / f)(2n))-1 / sqrt(1+(a / f)(2n)); Where f represents the frequency, n represents the order of the filter, which determines the steepness of the transition between the passband and the stopband, sqrt represents the square root, and a and b represent the lower and upper cutoff frequencies of the filter. Apply the filter to the electrophysiological recording X(t) to obtain the filtered data Y(t): Y(t)=X(t)*F -1 {H(f)}; Among them, F -1 represents the inverse Fourier transform, and * represents the convolution operation; Eliminate the electric field changes caused by electrode movement or current leakage, remove the common mode of the filtered data Y(t), and remove the common noise components in the filtered data Y(t): Y'(i,t)=Y(i,t)-Median{Y(j,t)|for allj}; Where i represents the electrode, t represents the sampling time, Median represents the median, j represents the index of all electrodes, Y(i,t) represents the filtered signal at sampling time t on electrode i, Y(j,t) represents the filtered signal at sampling time t on electrode j, the value range of j is [1, total number of electrodes], and Y'(i,t) represents the signal on electrode i after removing the common mode of Y(i,t); Whiten Y'(i, t), calculate the eigenvalues ​​and eigenvectors of the sample data covariance matrix, construct a whitening matrix, and multiply Y'(i, t) by the whitening matrix to obtain the target whitened data Y"(i, t) after whitening.

3. The method for clustering spatiotemporal enhanced neural electrical impulse signals based on an adaptive tree according to claim 2, characterized in that: The calculation of the eigenvalues ​​and eigenvectors of the sample data covariance matrix, the construction of the whitening matrix, and the multiplication of Y'(i, t) by the whitening matrix to obtain the target whitened data Y"(i, t) after whitening specifically include: Compute the eigenvalues ​​and eigenvectors of the sample data covariance matrix: [V,D]=eig(cov(Y'(i,t))); Where V represents the matrix containing all eigenvectors, D represents the diagonal matrix, eig represents the eigenvalue and eigenvector, and cov represents the covariance. Construct whitening matrix W zca : W zca =VD (-1 / 2) V T ; Among them, V T Indicates that V is transposed; Get the target whitened data Y”(i,t) after whitening: Y”(i,t)=Y'(i,t)W zca 。 4. The method for clustering spatiotemporal enhanced neural electrical impulse signals based on an adaptive tree according to claim 3, characterized in that: The detecting the target whitening data according to the threshold value to obtain the neural electrical pulse signal specifically includes: A threshold ζ is set in advance. The threshold ζ is calculated based on the voltage collected on each electrode. The threshold ζ is set as a multiple of the median. The threshold ζ is calculated as: ζ=Median(|Y”(i,t)|) / 0.6745; The target whitening data is detected according to a threshold value ζ. When the signal strength exceeds the threshold value ζ, it is considered that a pulse occurs, and a neural electrical pulse signal is obtained.

5. The method for clustering spatiotemporal enhanced neural electrical impulse signals based on adaptive trees according to claim 1, characterized in that: The spatiotemporal enhancement of the action potential includes: amplitude expansion, lateral displacement, random noise addition, overlap addition and baseline drift; the spatiotemporal enhancement of the spatiotemporal information is to transform the electrode position at the time of detection, the probe number and the timestamp of the release.

6. The method for spatiotemporal enhanced neural pulse signal clustering based on adaptive tree according to claim 1, characterized in that: The embedding of information of the action potential and spatiotemporal information of the neural electrical pulse signal and the comparison neural electrical pulse signal, encoding the action potential and spatiotemporal information of the embedded neural electrical pulse signal and the corresponding comparison neural electrical pulse signal to obtain the encoded signal specifically includes: The action potential of each neural electrical impulse signal, the electrode position at which it was detected, the probe number, and the timestamp of the release are transformed into four embedded information through a fixed number of fully connected layers in a set space. The four embedded information are summed up and output to the attention mechanism module for encoding learning to obtain the encoded signal. The attention mechanism module consists of stacked layers, each layer of which contains a multi-head self-attention mechanism and a feedforward fully connected neural network.

7. The method for clustering spatiotemporal enhanced neural electrical impulse signals based on an adaptive tree according to claim 1, characterized in that: The method of inputting the encoded signal into the adaptive tree structure and outputting the neural electrical pulse signal clustering result to realize adaptive neural electrical pulse signal clustering specifically includes: The encoded signal is pre-clustered, and the mutual information between the current neural electrical pulse signal and the comparison neural electrical pulse signal is calculated using an adaptive tree structure to maximize it; When the mutual information converges and is not less than the preset threshold, pulse clustering is performed and the neural electrical pulse signal clustering results are output.

8. A spatiotemporal enhanced neural electrical impulse signal clustering system based on an adaptive tree, characterized in that: The adaptive tree-based spatiotemporal enhanced neural electrical impulse signal clustering system includes: a data processing module, configured to obtain electrophysiological records, preprocess the electrophysiological records to obtain target whitening data, and detect the target whitening data according to a threshold value to obtain a neural electrical pulse signal; A spatiotemporal enhancement module is used to perform spatiotemporal enhancement on the action potential and spatiotemporal information of each neural electrical pulse signal to obtain a comparative neural electrical pulse signal. The spatiotemporal information includes the electrode position at the time of detection, the probe number, and the timestamp of the release. An information encoding module is used to embed the action potential and spatiotemporal information of the neural electrical pulse signal and the comparative neural electrical pulse signal, and encode the action potential and spatiotemporal information of the embedded neural electrical pulse signal and the corresponding comparative neural electrical pulse signal to obtain an encoded signal; The signal clustering module is used to input the encoded signal into the adaptive tree structure and output the neural electrical pulse signal clustering result to achieve adaptive neural electrical pulse signal clustering.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and an adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering program stored in the memory and runnable on the processor. When the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering program is executed by the processor, the steps of the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering program, and when the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering program is executed by a processor, the steps of the adaptive tree-based spatiotemporal enhanced neural electrical pulse signal clustering method as described in any one of claims 1 to 7 are implemented.

Images