Adaptive topology decoding test method, device, equipment and medium

Abstract

The invention provides a self-adaptive topology decoding test method, device and equipment and a medium, and the method comprises the steps: obtaining neural data collected by a brain under various stimuli, and taking a group of neural data as a sample to construct a data set; respectively decoding the neural coding data of different brain regions in the neural data, and obtaining the decoding precision of the neural coding data of different brain regions according to the decoded classification result; grouping the neural coding data of the different brain regions based on the decoding precision of the neural coding data of the different brain regions; training and testing a self-adaptive topology decoding model by adopting the grouped data to obtain the decoding precision of each group of data; and based on the decoding precision corresponding to each group of data, analyzing the influence of the data in different brain regions on the decoding precision. The technical problem that in the prior art, it is difficult to establish the corresponding relation between the decoding precision of the brain level and the decoding model architecture can be solved and found.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neuroscience and intelligence science, and in particular to a self-adaptive topological decoding test method, device, equipment and medium. BACKGROUND

[0002] The mammalian visual system completes the perception and processing of external stimuli through the cooperation of multiple brain regions. Its information transmission path starts from the retina, is transmitted to the primary visual cortex (V1) through the lateral geniculate nucleus (LGN), and is further projected to the anterior prefrontal nucleus (APN) and the hippocampus through a double pathway, forming a hierarchical processing structure. Visual stimuli are transmitted in the form of neural pulses between brain regions, and regions such as the primary visual cortex (VISp) are responsible for basic feature extraction (such as edges and directions), while high-order regions (such as the APN) participate in abstract concept coding. Existing research shows that the response delay of high-order regions is longer but the semantic information is richer.

[0003] Current visual decoding techniques mainly focus on the following directions: single-brain-region fine decoding test, based on neural activity data of high information density regions such as VISp, object classification or pixel-level image reconstruction is realized through convolutional neural network (CNN). Multi-brain-region joint decoding test, integrating data from multiple regions such as the visual cortex and thalamus, and modeling the functional connection between brain regions using graph neural network (GNN). The dynamic graph construction method can capture time-dependent relationships, but does not explicitly distinguish between hierarchical information flow. Cross-brain-region topological modeling, the latest research attempts to build a multi-scale decoding framework, but still does not explicitly distinguish between hierarchical information flow.

[0004] Despite the significant progress, the existing solutions still have the following key defects: (1) Insufficient hierarchical information integration, existing cross-brain-region decoding models mostly adopt static topological structures (such as pre-defined brain region connections), and cannot adaptively adjust the information transmission efficiency between different levels. (2) Lack of processing of data heterogeneity, single-brain-region data (such as millisecond-level spike signals of VISp) and cross-brain-region data have significant differences in space-time scale, and existing models cannot construct a unified representation space. (3) Limitations of computational biology interpretability, although traditional deep neural network (DNN) models can achieve high-precision decoding, they cannot reveal the mapping relationship between brain region hierarchy and machine learning performance, hindering the reverse deduction of neural computing principles. SUMMARY

[0005] The present application provides a self-adaptive topological decoding test method, device, equipment and medium, which is used to solve and find the technical problem that it is difficult to establish the correspondence between the decoding accuracy of the brain region hierarchy and the decoding model architecture in the prior art.

[0006] According to an aspect of the present application, a self-adaptive topological decoding test method is provided, comprising: Obtaining neural data collected from the brain under various stimuli, and constructing a data set using a set of neural data as a sample; the neural data includes neural coding data of multiple brain regions; Decoding the neural coding data of different brain regions in the neural data respectively, and obtaining decoding accuracy of the neural coding data of different brain regions according to the classification results after decoding; Based on the decoding accuracy of neural coding data of different brain regions, the neural coding data of different brain regions are grouped; each group of data after grouping includes neural coding data of at least two different brain regions; The adaptive topology decoding model is trained and tested using the grouped data to obtain the decoding accuracy of each group of data; Based on the decoding accuracy corresponding to each set of data, the impact of data in different brain regions on the decoding accuracy is analyzed. Optionally, respectively decoding the neural coding data of different brain regions in the neural data, and obtaining decoding accuracy of the neural coding data of different brain regions according to the classification results after decoding, includes: Inputting the neural coding data of different brain regions in the plurality of neural data into corresponding classifiers or learners for decoding, and obtaining a classification label corresponding to each neural coding data; Based on the labels obtained by classifying multiple neural coding data of the same brain region and the actual labels, the decoding accuracy of the neural coding data of each brain region is determined.

[0007] Optionally, grouping the neural coding data of different brain regions based on the decoding accuracy of the neural coding data of different brain regions includes: The decoding accuracy of the neural coding data of different brain regions is sorted, and the neural coding data of different brain regions are grouped according to the sorting results.

[0008] Optionally, the adaptive topological decoding model includes a feature extraction layer, an adaptive dimensionality reduction layer, a topological feature extraction layer, a feature fusion layer and a ViT layer; The feature extraction layer is used to extract neural features of each set of neural coding data; The adaptive dimensionality reduction layer is used to perform dimensionality reduction processing on the neural features; The topological feature extraction layer is used to generate a topological network graph from the features after dimensionality reduction of the neural features as topological features to reflect the distribution and local clusters of each group of neural coding data; The feature fusion layer is used to fuse the corresponding neural features of each set of neural coding data and the topological network graph to obtain fused features; The ViT layer is used to learn and classify the fusion features to obtain classification results.

[0009] Optionally, the adaptive dimension reduction layer adopts an ada-PCA method to dynamically select an optimal dimension to reduce the neural features.

[0010] Optionally, the topological feature extraction layer uses a Mapper algorithm to process the reduced neural features to generate a topological network graph.

[0011] Optionally, the influence of different brain regions on decoding accuracy is analyzed based on the decoding accuracy corresponding to each group of data, including: The decoding accuracies corresponding to different groups of data are compared, and when the second group of data contains all the neural encoding data in the first group of data and the neural encoding data of the first brain region, the influence of the neural encoding data of the first brain region on the decoding accuracy is determined according to the difference between the decoding accuracies.

[0012] According to another aspect of the present application, an adaptive topological decoding test device is provided, comprising: An acquisition unit is configured to acquire neural data collected by a brain under multiple stimuli, and construct a data set by taking a group of neural data as a sample; the neural data includes neural encoding data of multiple brain regions; A first decoding unit is configured to decode the neural encoding data of different brain regions in the neural data respectively, and acquire decoding accuracies of the neural encoding data of different brain regions according to the classification results after decoding; A grouping unit is configured to group the neural encoding data of different brain regions based on the decoding accuracies of the neural encoding data of different brain regions; each group of data after grouping includes at least neural encoding data of two different brain regions; A training unit is configured to train and test an adaptive topological decoding model by using the grouped data, and obtain decoding accuracies of each group of data; An analysis unit is configured to analyze the influence of data in different brain regions on decoding accuracy based on the decoding accuracy corresponding to each group of data. According to another aspect of the present application, an electronic device is provided, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the adaptive topological decoding test method according to any one of the embodiments of the present application.

[0013] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for causing a processor to implement the adaptive topology decoding test method according to any of the embodiments of the present application when executed.

[0014] The technical solution of the embodiment of the present application groups the neural encoding data of different brain regions based on the decoding accuracy of the neural encoding data of different brain regions; each group of data after grouping includes at least the neural encoding data of two different brain regions; the data after grouping is used to train and test the adaptive topology decoding model to obtain the decoding accuracy of each group of data; and the influence of the data in different brain regions on the decoding accuracy is analyzed based on the decoding accuracy corresponding to each group of data. By decoding the neural data according to the hierarchical information gradient, and constructing an interaction module across brain regions to complete the data unification between different brain regions, the hierarchical features of the neural data are analyzed, and the influence of the data in different brain regions on the decoding accuracy is analyzed.

[0015] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is a flowchart of an adaptive topology decoding test method according to an embodiment of the present application; Figure 2 is a flowchart of an adaptive topology decoding test method according to an embodiment of the present application; Figure 3 is a flowchart of Fine-grained decoding and AT-ViT used in an embodiment of the present application; Figure 4 is a schematic diagram of the decoding accuracy of each group of data after testing the AT-ViT model using neural decoding data of different groups of brain regions in an embodiment of the present application; Figure 5 is a structural schematic diagram of an adaptive topology decoding test device according to an embodiment of the present application; Figure 6 is a structural schematic diagram of an electronic device implementing the adaptive topology decoding test method according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the protection scope of the present application.

[0019] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0020] Embodiment one Figure 1 A flowchart of an adaptive topology decoding test method is provided for the first embodiment of the present application. As shown in the figure, the method comprises: Figure 1 S101, acquiring neural data collected under multiple stimuli of the brain, and constructing a data set with a set of neural data as a sample; the neural data includes multiple brain region neural encoding data.

[0021] It should be noted that the brain includes multiple brain regions, for example, for the 15 brain regions of the mouse cerebral cortex: visual cortex VISp, VISam, VISal, VISrl, VISpm, VISI, midbrain / thalamic region LGv, LGd, APN and LP, and hippocampal region CA1, CA2, CA3, DG, SUB. When the mouse brain is subjected to visual stimulation, the neural signals of each brain region can be collected, i.e. the neural data of the mouse under the visual stimulation can be collected.

[0022] ​To obtain sufficient data samples, the embodiment can train multiple mice to receive visual stimuli of multiple scenes respectively, and collect neural signals of multiple brains of the mice at the same time, so as to construct a data set by taking the neural signals collected after one mouse receives one kind of visual stimuli as one sample.

[0023] In addition, the neural data is a pulse signal collected at one time point, and the embodiment can pre-process the neural data, that is, convert the pulse signal into frequency encoding data.

[0024] S102, respectively decode the neural encoding data of different brain regions in the neural data, and obtain the decoding accuracy of the neural encoding data of different brain regions according to the classification results after decoding.

[0025] In the embodiment, the data set of the neural encoding data of different brain regions can be obtained from the data set, for example, the VISp brain region data set, the VISam brain region data set, the VISal brain region data set, the VISrl brain region data set, the VISpm brain region data set, the VISI brain region data set, etc. The data in these data sets are respectively input into the corresponding brain region decoder for decoding, and the data classification label corresponding to each neural encoding data can be obtained, so that the decoding accuracy of the corresponding brain region can be calculated according to the label obtained by decoding and the actual label. It should be noted that by decoding the neural data collected from each brain region of the mouse, 128 image classifications can be decoded, that is, 128 data classification labels are obtained.

[0026] In the embodiment, the Fine-grained decoding test method can be used, which can focus on the decoding of a single brain region to realize accurate decoding of a single brain region.

[0027] S103, grouping the neural encoding data of different brain regions based on the decoding accuracy of the neural encoding data of different brain regions; each group of data after grouping includes at least neural encoding data of two different brain regions.

[0028] For the decoding accuracy of the neural encoding data of different brain regions obtained in step S102, different brain regions can be sorted according to the decoding accuracy. For example, when the decoding accuracy is sorted from large to small, the result is VISp, VISam, VISal, VISrl, VISpm, VISI, LGv, LGd, APN, LP, CA1, CA2, CA3, DG, and SUB, then VISp, VISam, VISal, VISrl, VISpm, and VISI can be divided into a first group, the brain regions in the first group and LGv, LGd, APN, and LP can be divided into a second group, and the brain regions in the first group and the second group and CA1, CA2, CA3, DG, and SUB can be divided into a third group.

[0029] In an embodiment, in order to obtain a more intuitive influence of each brain region on decoding performance after subsequent testing of the adaptive topology decoding model, the neural encoding data of different brain regions can be divided into N groups, where the Mth group (M is less than N) includes neural encoding data corresponding to A brain regions, and the M+1th group includes neural encoding data corresponding to A brain regions in the Mth group and neural encoding data corresponding to B brain regions. A brain region and B brain region each include neural encoding data corresponding to at least one brain region. Specifically, the visual cortex VISp, VISam, VISal, VISrl, VISpm, and VISI can be divided into a first group; the visual cortex VISp, VISam, VISal, VISrl, VISpm, VISI, and the midbrain / thalamic region LGv and LGd can be divided into a second group; the visual cortex VISp, VISam, VISal, VISrl, VISpm, VISI, the midbrain / thalamic region LGv, LGd, APN, and LP can be divided into a third group; and the visual cortex VISp, VISam, VISal, VISrl, VISpm, VISI, the midbrain / thalamic region LGv, LGd, APN, LP, and the hippocampal region CA1, CA2, CA3, DG, and SUB can be divided into a fourth group.

[0030] S104, training and testing the adaptive topology decoding model using the grouped data to obtain the decoding accuracy of each group of data.

[0031] Each group of neural encoding data after grouping can be used as a sample to construct a new data set, and the data set can be divided into a training set and a test set for training and testing the adaptive topology decoding model, and finally the decoding accuracy of each group of data can be obtained.

[0032] S105, based on the decoding accuracy corresponding to each group of data, analyzing the influence of data in different brain regions on decoding accuracy.

[0033] Since the neural encoding data of each group of data includes multiple brain regions, the influence of the neural firing data of each brain region on the decoding accuracy can be obtained by analyzing the differences between the brain regions of different groups of data and the decoding accuracy corresponding to each group of data.

[0034] For example, the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI are divided into a first group; the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI and the midbrain / thalamic region LGv and LGd are divided into a second group; the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI, the midbrain / thalamic region LGv and LGd, the APN and LP are divided into a third group, and the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI, the midbrain / thalamic region LGv and LGd, the APN, LP, and the hippocampal region CA1, CA2, CA3 and DG are divided into a fourth group. Since the second group of data has two more brain regions, the midbrain / thalamic region LGv and LGd, the decoding accuracy of the second group of data can be subtracted from the decoding accuracy of the first group of data, and the influence of the neural data of the two brain regions LGv and LGd on the decoding accuracy can be obtained.

[0035] The technical scheme of the embodiment of the present application groups the neural encoding data of different brain regions based on the decoding accuracy of the neural encoding data of different brain regions; each group of data after grouping includes at least neural encoding data of two different brain regions; the self-adaptive topological decoding model is trained and tested using the grouped data to obtain the decoding accuracy of each group of data; and the influence of the data in different brain regions on the decoding accuracy is analyzed based on the decoding accuracy corresponding to each group of data. By decoding the neural data according to the hierarchical information gradient and constructing an interactive module across brain regions to unify the data between different brain regions, the hierarchical features between the neural data are analyzed, and the influence of the data in different brain regions on the decoding accuracy is analyzed.

[0036] Embodiment two Figure 2 A flowchart of a self-adaptive topological decoding test method provided by the second embodiment of the present application is shown in FIG. 2. Figure 2 As shown in the figure, the method comprises the following steps. S201, neural data collected under multiple stimuli of the brain is obtained, and a group of neural data is taken as a sample to construct a data set; the neural data includes neural encoding data of multiple brain regions.

[0037] The embodiment can directly obtain the Allen Brain Science dataset containing rich visual tasks. The dataset is mainly formed by training multiple mice to receive visual stimuli of multiple scenes respectively, recording the corresponding brain activity neural data, and repeating multiple times. Since the collected is the pulse data discharged at a time point, the pulse data needs to be preprocessed. The preprocessing process is mainly to convert the pulse data into frequency encoded data that can be used for decoding.

[0038] Specifically, 15 brain regions of the mouse cerebral cortex can be obtained, including visual cortex VISp, VISam, VISal, VISrl, VISpm, VISI, midbrain / thalamic region LGv, LGd, APN and LP, and hippocampal region CA1, CA2, CA3, DG and SUB. When the mouse brain is visually stimulated, the neural signals of each brain region can be collected respectively, that is, the neural data of the mouse under the visual stimulation can be collected.

[0039] S202, input the neural encoding data of different brain regions in the plurality of neural data into the corresponding classifier or learner respectively for decoding to obtain the classification label corresponding to each neural encoding data.

[0040] The neural encoding data of different brain regions can be obtained from the dataset, for example, the VISp brain region dataset, the VISam brain region dataset, the VISal brain region dataset, the VISrl brain region dataset, the VISpm brain region dataset, the VISI brain region dataset, the LGv brain region dataset, the LGd brain region dataset, the APN brain region dataset, the LP brain region dataset, the CA1 brain region dataset, the CA2 brain region dataset, the CA3 brain region dataset, the DG brain region dataset, and the SUB brain region dataset.

[0041] The embodiment can use the Fine-grained decoding test method to decode the neural encoding data of a single brain region. As shown in Figure 3 Figure 3 The single brain region of VISp, VISam, VISal, VISrl, VISpm, VISI, LGv, LGd, APN, LP, CA1, CA2, CA3, DG and SUB is decoded by the learner through multiple classifiers, and the label corresponding to each single brain region can be decoded respectively.

[0042] S203, based on the labels obtained by classifying the plurality of neural encoding data of the same brain region and the actual labels, determining the decoding accuracy of the neural encoding data of each brain region.

[0043] ​The data in each brain region corresponding data set is respectively input into the decoder of the corresponding brain region for decoding, so that the data classification label corresponding to each neural encoding data can be obtained, and thus the decoding accuracy of the corresponding brain region can be calculated according to the decoded label and the actual label.

[0044] S204, the decoding accuracies of the neural encoding data of different brain regions are sorted, and the neural encoding data of different brain regions are grouped according to the sorting result.

[0045] In this embodiment, the decoding accuracies of the neural encoding data of the visual cortex VISp, VISam, VISal, VISrl, VISpm, VISI, the midbrain / thalamic region LGv, LGd, APN and LP, and the hippocampal region CA1, CA2, CA3, DG and SUB can be sorted, and each group of data after grouping includes at least neural encoding data of two different brain regions.

[0046] Specifically, the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI can be classified into a first group; the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI and the midbrain / thalamic region LGv and LGd can be classified into a second group; the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI, the midbrain / thalamic region LGv, LGd, APN and LP can be classified into a third group, and the visual cortex VISp, VISam, VISal, VISrl, VISpm and VISI, the midbrain / thalamic region LGv, LGd, APN, LP, and the hippocampal region CA1, CA2, CA3, DG and SUB can be classified into a fourth group.

[0047] S205, the data after grouping is used to train and test the adaptive topology decoding model, and the decoding accuracy of each group of data is obtained.

[0048] In an embodiment, the adaptive topology decoding model includes a feature extraction layer, an adaptive dimension reduction layer, a topology feature extraction layer, a feature fusion layer and a ViT layer; the feature extraction layer is used to extract neural features of each group of neural encoding data; the adaptive dimension reduction layer is used to perform dimension reduction processing on the neural features; the topology feature extraction layer is used to generate a topology network graph as a topology feature from the dimension-reduced features of the neural features, so as to reflect the distribution and local cluster of each group of neural encoding data; the feature fusion layer is used to fuse the corresponding neural features and the topology network graph of each group of neural encoding data to obtain fusion features; and the ViT layer is used to learn and classify the fusion features to obtain a classification result.

[0049] In this embodiment, the adaptive topology decoding model can be as followsFigure 3 The adaptive topology decoding (AT-ViT) model is shown. The adaptive dimension reduction layer can use the ada-PCA method to dynamically select the optimal dimension reduction dimension according to the characteristics of the input neural features. The optimal dimension reduction parameter is mainly selected by the Bayesian method. Compared with the traditional fixed dimension reduction method, ada-PCA can better preserve key information while reducing redundancy.

[0050] In an embodiment, the topology feature extraction layer uses the Mapper algorithm to process the neural features after dimension reduction, and generates a topology network graph. The Mapper algorithm mainly adopts (1) a filter function (Filter) to project and reduce the data, (2) a cover (Cover) to divide the projected data, (3) a clustering (Clustering) to cluster each divided data, and (4) a graph construction (Graph) to obtain a topology structure of the data represented by a graph composed of points and edges. Overall, the Mapper algorithm reveals the global structure and local details of the data by constructing a topology network graph, thereby providing a basis for subsequent feature extraction.

[0051] As shown in Figure 3 In this embodiment, the data of each group after grouping is respectively input to the feature extraction layer and the topology feature extraction layer. The feature extraction layer is used to extract the neural features of the neural encoding data. The extracted neural features are processed by the adaptive dimension reduction layer for dimension reduction, and then the global structure features of the neural data are extracted by the topology feature extraction layer. In addition, the topology feature extraction layer is also used to extract the local detail features of the neural encoding data. The combination of the global structure features and the local detail features can obtain the topology information between different brain regions. Then the neural features and the topology features are fused and input to the ViT layer. The fused features are learned and classified to obtain the classification result. In this embodiment, the ViT model can set the number of Transformer layers, the number of attention heads, and the size of the hidden layer according to the data type and size.

[0052] S206, compare the decoding accuracies corresponding to different groups of data. When the second group of data contains all the neural encoding data in the first group of data and the neural encoding data of the first brain region, the influence of the neural encoding data of the first brain region on the decoding accuracy is determined according to the difference between the decoding accuracies.

[0053] The first brain region is a brain region corresponding to the neural encoding data in the first group of data, and the first brain region includes at least one brain region. For example, when the first group of data includes VISp, VISam, VISal, VISrl, VISpm, and VISI brain regions in the visual cortex, the second group of data can include the brain regions VISp, VISam, VISal, VISrl, VISpm, and VISI of the visual cortex, and the brain regions LGv and LGd of the midbrain / thalamic region.

[0054] In this embodiment, the visual cortex VISp, VISam, VISal, VISrl, VISpm, and VISI are divided into a first group; the visual cortex VISp, VISam, VISal, VISrl, VISpm, and VISI and the midbrain / thalamic region LGv and LGd are divided into a second group; the visual cortex VISp, VISam, VISal, VISrl, VISpm, and VISI, the midbrain / thalamic region LGv and LGd, APN, and LP are divided into a third group; and the visual cortex VISp, VISam, VISal, VISrl, VISpm, and VISI, the midbrain / thalamic region LGv and LGd, APN, LP, and the hippocampal region CA1, CA2, CA3, and DG are divided into a fourth group. The decoding accuracy corresponding to the neural decoding data of each group of brain regions can be obtained by testing, and the details are shown in Table 1. Figure 4 Figure 4 As can be seen from Table 1, the visual cortex and the midbrain / thalamic region both contain more visual information, and the hippocampal region contains less visual information, which has a greater impact on decoding accuracy. In addition, the AT-ViT method used in the present application can better distinguish the differences in visual information contained in different brain regions compared to existing solutions. That is, the present application can layer the visual encoding data of different brain regions according to the decoding accuracy of each brain region through a hierarchical processing mechanism, and effectively extract the topological features of neural data between different brain regions through the constructed AT-ViT model, so as to more accurately identify the visual information of neural data.

[0055] ​The technical scheme of the embodiment of the present application groups the neural encoding data of different brain regions based on the decoding accuracy of the neural encoding data of different brain regions, so as to better extract the hierarchical features between the visual information contained in different brain regions. The data of each group after grouping is respectively input into a feature extraction layer and a topological feature extraction layer. The feature extraction layer is used to extract the neural features of the neural encoding data. After the extracted neural features are processed by the adaptive dimension reduction layer, the global structure features of the neural data are extracted by the topological feature extraction layer. The topological feature extraction layer is also used to extract the local detail features of the neural encoding data. The global structure features and the local detail features are combined to obtain the topological information of the data between different brain regions, so that the data of different brain regions can interact across brain regions, and the visual information of the neural data can be more accurately recognized.

[0056] Embodiment three Figure 5 A structural schematic diagram of an adaptive topological decoding test device provided by the third embodiment of the present application is shown in FIG. 3. Figure 5 As shown in FIG. 3, the device comprises: An acquisition unit 501 is configured to acquire neural data collected by a brain under multiple stimuli, and construct a data set by taking a group of neural data as a sample. The neural data comprises neural encoding data of multiple brain regions. A first decoding unit 502 is configured to decode the neural encoding data of different brain regions in the neural data respectively, and obtain the decoding accuracy of the neural encoding data of different brain regions according to the classification results after decoding. A grouping unit 503 is configured to group the neural encoding data of different brain regions based on the decoding accuracy of the neural encoding data of different brain regions. Each group of data after grouping comprises at least the neural encoding data of two different brain regions. A training unit 504 is configured to train and test an adaptive topological decoding model by using the data after grouping, and obtain the decoding accuracy of each group of data. An analysis unit 505 is configured to analyze the influence of the data in different brain regions on the decoding accuracy based on the decoding accuracy corresponding to each group of data.

[0057] The adaptive topological decoding test device provided by the embodiment of the present application can execute the adaptive topological decoding test method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0058] Embodiment four Figure 6A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0059] like Figure 6 As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0060] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0061] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as an adaptive topology decoding test method.

[0062] In some embodiments, an adaptive topology decoding test method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the adaptive topology decoding test method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform an adaptive topology decoding test method by other means, e.g., with the aid of firmware.

[0063] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0064] Computer programs used to implement the test methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0065] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0066] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0067] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0068] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0069] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0070] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An adaptive topology decoding test method, characterized in that: include: Obtaining neural data collected from the brain under various stimuli, and constructing a data set using a set of neural data as a sample; the neural data includes neural coding data of multiple brain regions; Decoding the neural coding data of different brain regions in the neural data respectively, and obtaining decoding accuracy of the neural coding data of different brain regions according to the classification results after decoding; Based on the decoding accuracy of neural coding data of different brain regions, the neural coding data of different brain regions are grouped; each group of data after grouping includes neural coding data of at least two different brain regions; The adaptive topology decoding model is trained and tested using the grouped data to obtain the decoding accuracy of each group of data; Based on the decoding accuracy corresponding to each set of data, the impact of data in different brain regions on the decoding accuracy is analyzed.

2. The adaptive topology decoding test method according to claim 1, wherein: The neural coding data of different brain regions in the neural data are respectively decoded, and decoding accuracy of the neural coding data of different brain regions is obtained according to the classification results after decoding, including: Inputting the neural coding data of different brain regions in the plurality of neural data into corresponding classifiers or learners for decoding, and obtaining a classification label corresponding to each neural coding data; Based on the labels obtained by classifying multiple neural coding data of the same brain region and the actual labels, the decoding accuracy of the neural coding data of each brain region is determined.

3. The adaptive topology decoding test method according to claim 1, wherein: The neural coding data of different brain regions are grouped based on the decoding accuracy of the neural coding data of different brain regions, including: The decoding accuracy of the neural coding data of different brain regions is sorted, and the neural coding data of different brain regions are grouped according to the sorting results.

4. The adaptive topology decoding test method according to claim 1, wherein: The adaptive topological decoding model includes a feature extraction layer, an adaptive dimensionality reduction layer, a topological feature extraction layer, a feature fusion layer and a ViT layer; The feature extraction layer is used to extract neural features of each set of neural coding data; The adaptive dimensionality reduction layer is used to perform dimensionality reduction processing on the neural features; The topological feature extraction layer is used to generate a topological network graph from the features after dimensionality reduction of the neural features as topological features to reflect the distribution and local clusters of each group of neural coding data; The feature fusion layer is used to fuse the corresponding neural features of each set of neural coding data and the topological network graph to obtain fused features; The ViT layer is used to learn and classify the fusion features to obtain classification results.

5. The adaptive topology decoding test method according to claim 4, characterized in that: The adaptive dimensionality reduction layer adopts the ada-PCA method to dynamically select the optimal dimension to reduce the dimensionality of the neural features.

6. The adaptive topology decoding test method according to claim 4, characterized in that: The topological feature extraction layer uses the Mapper algorithm to process the neural features after dimensionality reduction to generate a topological network diagram.

7. The adaptive topology decoding test method according to claim 1, characterized in that: The decoding accuracy corresponding to each set of data is analyzed to determine the impact of different brain regions on the decoding accuracy, including: The decoding accuracies corresponding to different groups of data are compared. When the second group of data contains all the neural coding data in the first group of data and the neural coding data of the first brain region, the influence of the neural coding data of the first brain region on the decoding accuracy is judged based on the difference between the decoding accuracies.

8. An adaptive topology decoding test device, characterized in that: include: An acquisition unit is used to acquire neural data collected by the brain under various stimuli, and to construct a data set using a set of neural data as a sample; the neural data includes neural coding data of multiple brain regions; a first decoding unit, configured to respectively decode the neural coding data of different brain regions in the neural data, and obtain decoding accuracy of the neural coding data of different brain regions according to the classification results after decoding; a grouping unit, configured to group the neural coding data of different brain regions based on the decoding accuracy of the neural coding data of different brain regions; each group of data after grouping includes neural coding data of at least two different brain regions; A training unit is used to train and test the adaptive topology decoding model using the grouped data to obtain the decoding accuracy of each group of data; The analysis unit is used to analyze the impact of data in different brain regions on the decoding accuracy based on the decoding accuracy corresponding to each set of data.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the adaptive topology decoding test method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the adaptive topology decoding test method according to any one of claims 1 to 7 when executed.

Images