Data processing system, method and equipment constructed based on micro-macro relationship and medium
Through a data processing system built based on micro-macro relationship, using fMRI micro data and scale data, combined with multi-scale graph convolution network and attention mechanism, the problem of insufficient cognitive testing accuracy in the existing technology is solved, and more efficient cognitive ability classification and characterization is achieved.
Patent Information
- Application Number
- CN202411653759.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-02-28
AI Technical Summary
When fMRI technology is introduced into the prior art for cognitive testing, the test accuracy is insufficient and the challenge of multi-feature fusion is faced, and the population graph lacks the ability to learn and represent network relationships.
The data processing system is adopted based on micro-macro relationships. The micro-relationship building block uses phase-locked value PLV technology to analyze fMRI micro data to capture synchronous relationships at the whole brain scale; the macro-relationship building block acquires scale data, and uses paired similar encoder PAE and fuzzy clustering algorithm to capture paired relationships and subtype relationships; the multi-scale graph convolution module performs convolution learning; the fusion module fuses the importance of different channels through the attention mechanism, and outputs target representation information for cognitive ability classification.
Improve the accuracy of cognitive testing and optimize the representation of complex brain signals, providing new tools for cognitive testing.
Smart Images
Figure CN119538003A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing system, method, device and medium based on micro-macro relationship. Background Art
[0002] Cognitive ability refers to the ability of the human brain to process, store and retrieve information, which includes psychological processes such as perception, memory, attention, thinking and imagination. In some specific fields, cognitive ability testing is crucial. For example, in the field of civil aviation, as air traffic management becomes increasingly complex, air traffic controllers generally bear a huge cognitive load. Cognitive ability testing is crucial for both pilots and air traffic controllers.
[0003] Among the related technologies, with the development of functional imaging technology including functional magnetic resonance imaging (fMRI), the research on cognitive testing has reached a new level. Functional imaging can be used to more objectively judge the brain function of the subjects, and then judge the cognitive ability of the subjects. fMRI uses magnetic resonance imaging to measure the changes in blood dynamics caused by neuronal activity. fMRI has the advantages of being non-invasive and without radiation exposure.
[0004] However, brain cognitive testing in related technologies usually relies on electroencephalograms and near-infrared spectroscopy, and the introduction of fMRI technology for rapid and accurate testing is still in urgent need of research. Currently, fMRI usually evaluates brain function by constructing a crowd map. When scanning the neuroimaging data of the subjects, there will be different degrees of deviation based only on the microscopic physiological data. The data accuracy needs to be improved, and there are challenges in multi-feature fusion. The crowd map is not capable of learning and representing network relationships. Summary of the invention
[0005] In view of the deficiencies of the prior art, the present application provides a data processing system, device and storage medium constructed based on a micro-macro relationship, which solves the problem of insufficient test accuracy when introducing fMRI technology into cognitive testing.
[0006] To achieve the above objectives, this application is implemented through the following technical solutions:
[0007] In a first aspect, an embodiment of the present application provides a data processing system based on micro-macro relationship construction, the data processing system comprising: a micro relationship construction module, a macro relationship construction module, a multi-scale graph convolution module and a fusion module.
[0008] Specifically, the micro-relationship construction module is used to analyze the acquired fMRI micro-data using the phase-locked value (PLV) technology to capture the synchronization relationship of different target objects at the whole-brain scale; the macro-relationship construction module is used to obtain the scale data of the target object, and uses the paired similarity encoder (PAE) to capture the paired relationships between different target objects, and captures the subtype relationship through the fuzzy clustering algorithm; among them, the paired relationship and the subtype relationship are used to characterize the cognitive level of the target object; the multi-scale graph convolution module is used to construct a multi-channel graph convolution network, and convolution learning is performed on the synchronization relationship, paired relationship and subtype relationship respectively to obtain preliminary representation information; the fusion module is used to obtain the attention coefficient between the corresponding brain areas of different target objects, and learns the importance of different channels in the multi-scale graph convolution module through the preset attention mechanism, and fuses different preliminary representation information into target representation information for cognitive ability classification.
[0009] According to the first aspect of the embodiment of the present application, the micro-relationship construction module includes: a filtering processing unit, a signal conversion unit and a computing unit; the filtering processing unit is used to apply an FIR filter to pre-process the BOLD signal corresponding to each target object, extract the frequency components of a specific frequency band to reduce its noise and splice it into a filtered signal at the whole brain scale to obtain an fMRI target sequence; the signal conversion unit is used to convert the filtered signal into an analytical signal through a Hilbert transform, and extract the instantaneous phase information; the computing unit is used to calculate the stability of the instantaneous phase difference based on the instantaneous phase information to obtain the PLV value to characterize the synchronization relationship.
[0010] According to the first aspect of the embodiment of the present application, the micro-relationship construction module also includes a matrix unit, which is used to set a threshold and establish a synchronization relationship matrix based on the PLV value and the threshold; wherein the synchronization relationship matrix includes multiple target elements, and when the PLV value is greater than the threshold, the corresponding target element is 1; when the PLV value is less than or equal to the threshold, the corresponding target element is 0.
[0011] According to a first aspect of the embodiment of the present application, the fMRI target sequence satisfies the expression:
[0012] S=[s1||s2||…‖s K ]∈R K×T
[0013] Where S represents the fMRI target sequence, s i represents the brain region corresponding to the i-th target object, 1≤i≤K, K represents the number of brain regions, and T represents the number of brain region time points;
[0014] Calculate the two target objects S i and S j The PLV values between satisfy the expression:
[0015]
[0016] In the formula, is the instantaneous phase difference of the signal, For the target object S i The corresponding instantaneous phase information, For the target object S j The corresponding instantaneous phase information, e represents the natural constant, and γ represents the imaginary unit.
[0017] According to the first aspect of the embodiment of the present application, the macro relationship construction module includes a capture construction unit and a cluster evaluation unit. The capture construction unit is used to capture the similarity of basic information between target objects using a pairwise similarity encoder PAE to construct a macro pairwise relationship; wherein the basic information includes age, gender and IQ; the cluster evaluation unit is used to cluster the subtype information using fuzzy clustering, and evaluate the subtype group relationship between target objects based on the fuzzy membership and the set membership threshold; wherein the subtype group relationship is used to characterize the segmentation pre-test level of the target object.
[0018] According to the first aspect of the embodiment of the present application, the macro data corresponding to the macro relationship construction module includes different segmented pre-test data and paired basic information; the cluster evaluation unit is also used to explore the subtype relationship between the target objects by using fuzzy clustering, and determine the target object S by setting the membership threshold ξ. i and the target object S j whether they belong to the same common subtype, and quantify the relationship between different target objects into a subtype network;
[0019] In μ ik >ξ and μ jk >ξ, determine the target object S i and the target object S j belong to the same subtype; otherwise, determine the target object S i and the target object S j do not belong to the same subtype; among them, μ ik Represents the target object S i Belongs to the corresponding subtype c k The degree, μ jk Represents the target object S j Belongs to the corresponding subtype c k degree.
[0020] According to a first aspect of the embodiment of the present application, the pairwise relationship satisfies the expression:
[0021]
[0022] in, represents the pairwise relationship between the i-th target object and the j-th target object, φ i and φ j They respectively represent the standardized input representations of the basic information corresponding to the i-th target object and the j-th target object obtained by preprocessing in the mapping space;
[0023] The objective function L corresponding to fuzzy clustering satisfies the expression:
[0024]
[0025] The constraint condition corresponding to the objective function L satisfies the expression:
[0026] In the formula, N represents the number of target objects, c k is the cluster center, |C| is the number of cluster centers, and m is the fuzzy parameter.
[0027] According to the first aspect of the embodiment of the present application, the multi-scale graph convolution module includes: a design unit, a construction unit and a learning unit; the design unit is used to learn the graph embedding of each scale through the graph convolutional neural network technology, and design an adaptive multi-channel graph convolutional network; the construction unit is used to construct a multi-scale crowd graph based on synchronization relationships, pair relationships and subtype relationships; the learning unit is used to learn the multi-scale crowd graph through the multi-channel graph convolutional network to obtain preliminary representation information.
[0028] In the second aspect, an embodiment of the present application provides a data processing method based on micro-macro relationship construction, which includes: using the phase-locked value PLV technology to analyze the acquired fMRI micro data through the micro relationship construction module to capture the synchronization relationship of different target objects at the whole brain scale; obtaining the scale data of the target object through the macro relationship construction module, using the paired similarity encoder PAE to capture the paired relationship between different target objects, and capturing the subtype relationship through the fuzzy clustering algorithm; wherein the paired relationship and the subtype relationship are used to characterize the cognitive level of the target object; constructing a multi-channel graph convolution network through a multi-scale graph convolution module, and performing convolution learning on the synchronization relationship, paired relationship and subtype relationship respectively to obtain preliminary representation information; obtaining the attention coefficient between the corresponding brain areas of different target objects through the fusion module, learning the importance of different channels in the multi-scale graph convolution module through a preset attention mechanism, and fusing different preliminary representation information into target representation information for cognitive ability classification.
[0029] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the data processing method based on micro-macro relationship in the aforementioned second aspect is implemented.
[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the data processing method based on the micro-macro relationship in the aforementioned second aspect is implemented.
[0031] The present application provides a data processing system, method, device and medium based on micro-macro relationship. Compared with the prior art, it has the following beneficial effects:
[0032] The present application is to evaluate and classify the cognitive abilities of multiple target objects, and comprehensively considers and analyzes both microscopic and macroscopic perspectives. With regard to the microscopic perspective, the present application introduces functional magnetic resonance imaging (fMRI) technology to obtain fMRI microscopic data of each target object, and uses phase-locked value (PLV) technology for data analysis to capture and determine the synchronization relationship at the whole-brain scale; with regard to the macroscopic perspective, based on the acquired scale data, the paired relationships and subtype relationships between different target objects are analyzed and determined; then, the obtained synchronization relationships, paired relationships, and subtype relationships are convolutionally learned through a convolutional network, and fused based on the attention mechanism to output target representation information, thereby classifying the target objects' cognitive abilities; the present application optimizes the representation of complex brain signals, improves test accuracy, and provides a new tool for cognitive testing. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0034] Figure 1 It is a structural diagram of a data processing system based on micro-macro relationship provided by an embodiment of the present application;
[0035] Figure 2 It is a flowchart of a data processing method based on micro-macro relationship construction provided in an embodiment of the present application;
[0036] Figure 3 yes Figure 2 An exemplary flow chart of S110;
[0037] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0040] The embodiments of the present application solve the problem of insufficient test accuracy when fMRI technology is introduced into cognitive testing by providing a data processing system, method, device and medium based on micro-macro relationship.
[0041] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:
[0042] Cognitive ability refers to the ability of the human brain to process, store and retrieve information, which includes psychological processes such as perception, memory, attention, thinking and imagination. In some specific fields, cognitive ability testing is crucial. For example, in the field of civil aviation, as air traffic management becomes increasingly complex, air traffic controllers generally bear a huge cognitive load. Cognitive ability testing is crucial for both pilots and air traffic controllers.
[0043] Among the related technologies, with the development of functional imaging techniques including functional magnetic resonance imaging (fMRI), the research on cognitive testing has reached a new level. Functional imaging can be used to more objectively judge the brain function of the subjects, and then judge the cognitive ability of the subjects. fMRI uses magnetic resonance imaging to measure the changes in hemodynamics caused by neuronal activity. fMRI has the advantages of being non-invasive and without radiation exposure.
[0044] However, brain cognitive testing in related technologies usually relies on electroencephalograms and near-infrared spectroscopy, and the introduction of fMRI technology for rapid and accurate testing is still under study. Currently, fMRI usually faces the following challenges in the process of evaluating brain function by constructing a population map:
[0045] (1) Most crowd graph construction methods rely only on the vector distance between the regions of interest of the subjects, which is limited by the vector distance; this ignores the actual test performance, such as the judgment results of professionals, macro question-and-answer tests, and written tests.
[0046] (2) When the subjects are scanned for neuroimaging data, there will be different degrees of deviation based only on the microscopic physiological data, which will lead to obvious deviations between the clinical measured data and the neuroimaging data. The data accuracy needs to be improved.
[0047] (3) In the process of developing multi-feature fusion technology, how to effectively fuse and optimize the features of different modalities is a technical challenge, which directly affects the ability of crowd graphs to learn and represent network relationships.
[0048] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0049] The following first introduces a data processing system based on micro-macro relationship provided by an embodiment of the present application.
[0050] The structural diagram of a data processing system 100 based on a micro-macro relationship provided in an embodiment of the present application is as follows: Figure 1 As shown, the data processing system 100 may include the following modules:
[0051] A microscopic relationship building module 110, which is used to analyze the acquired fMRI microscopic data using the phase-locked value PLV technology to capture the synchronization relationship of different target objects at the whole brain scale; it can be understood that the target object represents the person participating in the test;
[0052] A macro-relationship construction module 120, which is used to obtain the scale data of the target object, use the pairwise similarity encoder PAE to capture the pairwise relationships between different target objects, and use the fuzzy clustering algorithm to capture the subtype relationships; wherein the pairwise relationships and subtype relationships are used to characterize the cognitive level of the target object;
[0053] A multi-scale graph convolution module 130, which is used to construct a multi-channel graph convolution network, and perform convolution learning on synchronization relationships, pair relationships, and subtype relationships respectively to obtain preliminary representation information;
[0054] Fusion module 140, which is used to obtain the attention coefficient between the brain regions corresponding to different target objects, learn the importance of different channels in the multi-scale graph convolution module through a preset attention mechanism, and fuse different preliminary representation information into target representation information for cognitive ability classification. It can be understood that after obtaining the target representation information, the output target representation information is processed by the multi-layer perception MLP to obtain the predicted label, and the prediction loss is estimated using the cross entropy function.
[0055] According to an embodiment of the present application, any multiple modules among the micro-relationship construction module 110, the macro-relationship construction module 120, the multi-scale graph convolution module 130 and the fusion module 140 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.
[0056] It should be noted that functional magnetic resonance imaging (fMRI) is a non-invasive imaging technology that can be used to study human brain function and cognition in healthy individuals and groups with abnormal brain states. In addition, whether in pulse neural network simulation or in actual EEG data analysis, phase locking value (PLV) is a more commonly used feature. PLV reflects the overall "trend" of the phase difference between two real signals.
[0057] The above is a specific implementation of a data processing system based on a micro-macro relationship provided in an embodiment of the present application. It can be understood that the present application comprehensively considers and analyzes the micro and macro perspectives for the purpose of evaluating and classifying the cognitive abilities of multiple target objects. Regarding the micro perspective, the present application introduces functional magnetic resonance imaging (fMRI) technology to obtain fMRI micro data of each target object, and uses phase-locked value (PLV) technology for data analysis to capture and determine the synchronization relationship at the whole-brain scale; regarding the macro perspective, based on the acquired scale data, the paired relationships and subtype relationships between different target objects are analyzed and determined.
[0058] Furthermore, the present application performs convolutional learning on the obtained synchronization relationships, pair relationships and subtype relationships through a convolutional network, and fuses them based on an attention mechanism to output target representation information, thereby classifying the cognitive abilities of the target objects; the present application optimizes the representation of complex brain signals and improves test accuracy.
[0059] In some embodiments, the aforementioned micro-relationship construction module 110 includes:
[0060] The filter processing unit 111 is used to pre-process the BOLD signal corresponding to each target object by applying an FIR filter, extract the frequency components of a specific frequency band, reduce its noise, and splice it into a filtered signal at the whole brain scale to obtain an fMRI target sequence;
[0061] A signal conversion unit 112, configured to convert the filtered signal into an analytical signal by Hilbert transform and extract instantaneous phase information;
[0062] The calculation unit 113 is used to calculate the stability of the instantaneous phase difference according to the instantaneous phase information to obtain a PLV value to characterize the synchronization relationship.
[0063] In the embodiments of the present application, it can be understood that in the process of preprocessing the BOLD signal, the Configurable Pipeline for the Analysis of Connectomes (CPAC) processing flow in the official website of the ABIDE dataset is used for processing, and bandpass filtering is enabled to filter the data within a certain frequency range; global signal regression is disabled, and the regression of the global signal is not considered in the data processing.
[0064] In some embodiments, the aforementioned micro-relationship construction module 110 may also include a matrix unit 114, which is used to set a threshold and establish a synchronization relationship matrix based on the PLV value and the threshold; wherein the synchronization relationship matrix includes multiple target elements, and when the PLV value is greater than the threshold, the corresponding target element is 1; when the PLV value is less than or equal to the threshold, the corresponding target element is 0.
[0065] In one example, the fMRI target sequence satisfies the expression:
[0066] S=[s1||s2||…‖s K ]∈R K×T
[0067] Where S represents the fMRI target sequence, s i represents the brain region corresponding to the i-th target object, 1≤i≤K, K represents the number of brain regions, and T represents the number of brain region time points;
[0068] Calculate the two target objects S i and S j The PLV values between satisfy the expression:
[0069]
[0070] In the formula, is the instantaneous phase difference of the signal, For the target object S iThe corresponding instantaneous phase information, For the target object S j The corresponding instantaneous phase information, e represents the natural constant, and γ represents the imaginary unit.
[0071] In some embodiments, the aforementioned macro relationship building module 120 may include:
[0072] A capture construction unit 121 is used to capture the similarity of basic information between target objects using a pairwise similarity encoder PAE to construct a macroscopic pairwise relationship; wherein the basic information includes age, gender, and IQ;
[0073] The cluster evaluation unit 122 is used to cluster the subtype information using fuzzy clustering, and evaluate the subtype group relationship between the target objects according to the fuzzy membership and the set membership threshold; wherein the subtype group relationship is used to characterize the subdivision pre-test level of the target object.
[0074] In the embodiments of the present application, it can be understood that in order to further improve the accuracy of cognitive ability testing, in addition to collecting data at a microscopic level based on functional magnetic resonance imaging (fMRI) technology, the present application also performs segmented pre-tests on the target objects being tested from a macroscopic perspective to make specific judgments from another perspective, thereby combining with the microscopic perspective to perform feature merging to ensure the accuracy and reliability of data processing and classification.
[0075] It should be noted that the subtype groups correspond to multiple classification levels for pre-testing from a macro perspective, the subtype group relationships correspond to the basic information and are used to reflect the actual test performance of the target objects from a macro perspective, and the subtype group relationships can take into account at least one of the judgment results of professionals, question-and-answer tests, and written tests.
[0076] In some embodiments, the macro data corresponding to the macro relationship building module includes different segmented pre-test data and paired basic information; the cluster evaluation unit is also used to explore the subtype relationship between the target objects by using fuzzy clustering, and determine the target object S by setting the membership threshold ξ i and the target object S j Whether they belong to the same common subtype, and quantify the relationship between different target objects into a subtype network.
[0077] Specifically, in μ ik >ξ and μ jk >ξ, determine the target object S i and the target object S j belong to the same subtype; otherwise, determine the target object S i and the target object S j do not belong to the same subtype; among them, μ ik Represents the target object S iBelongs to the corresponding subtype c k The degree, μ jk Represents the target object S j Belongs to the corresponding subtype c k degree.
[0078] In one example, the aforementioned pairwise relationship satisfies the expression:
[0079]
[0080] in, represents the pairwise relationship between the i-th target object and the j-th target object, φ i and φ j They respectively represent the standardized input representations of the basic information corresponding to the i-th target object and the j-th target object obtained by preprocessing in the mapping space.
[0081] In another example, the objective function L corresponding to the aforementioned fuzzy clustering satisfies the expression:
[0082]
[0083] The constraint condition corresponding to the objective function L satisfies the expression:
[0084] In the formula, N represents the number of target objects, c k is the cluster center, |C| is the number of cluster centers, and m is the fuzzy parameter.
[0085] In some embodiments, the aforementioned multi-scale graph convolution module 130 includes:
[0086] A design unit 131 is used to learn graph embedding of each scale through graph convolutional neural network technology and design an adaptive multi-channel graph convolutional network;
[0087] A construction unit 132, for constructing a multi-scale population graph based on the synchronization relationship, the pairwise relationship and the subtype relationship;
[0088] The learning unit 133 is used to learn the multi-scale crowd graph through a multi-channel graph convolutional network to obtain preliminary representation information.
[0089] In the embodiment of the present application, it can be understood that after the multi-scale crowd graph is constructed, the preliminary representation information in the multi-scale crowd graph includes data feature information of multiple target objects. After the cognitive test of multiple target objects is completed, the test classification result of each target object can be added to the multi-scale crowd graph and bound to the corresponding data feature information. Based on this, when conducting the next round of cognitive testing, the analysis data of the new target object can be compared with the multi-scale crowd graph to determine the closest data feature information and its test classification results, thereby realizing a rapid judgment of cognitive ability.
[0090] In some embodiments, the present application also provides a data processing method based on micro-macro relationship construction, such as Figure 2 As shown, the data processing method may include the following steps:
[0091] S210, using the phase-locked value (PLV) technique to analyze the acquired fMRI micro data through the micro relationship building module to capture the synchronization relationship of different target objects at the whole brain scale;
[0092] S220, obtaining the scale data of the target object through the macro-relationship construction module, using the paired similarity encoder PAE to capture the paired relationships between different target objects, and capturing the subtype relationships through the fuzzy clustering algorithm; wherein the paired relationships and subtype relationships are used to characterize the cognitive level of the target object;
[0093] S230, constructing a multi-channel graph convolution network through a multi-scale graph convolution module, performing convolution learning on synchronization relationship, pair relationship and subtype relationship respectively, and obtaining preliminary representation information;
[0094] S240. Obtain the attention coefficients between the brain regions corresponding to different target objects through the fusion module, learn the importance of different channels in the multi-scale graph convolution module through the preset attention mechanism, and fuse different preliminary representation information into target representation information to perform cognitive ability classification.
[0095] Figure 2 Each step in the method shown has the function of realizing each module in the aforementioned data processing system constructed based on the micro-macro relationship, and can achieve its corresponding technical effect. For the sake of concise description, it will not be repeated here.
[0096] In some embodiments, Figure 3 As shown, the data processing method based on micro-macro relationship construction also includes:
[0097] S310, preprocessing the BOLD signal corresponding to each target object by applying an FIR filter through a filter processing unit, extracting frequency components of a specific frequency band, reducing its noise, and splicing them into a filtered signal at a whole-brain scale to obtain an fMRI target sequence;
[0098] S320, converting the filtered signal into an analytical signal through Hilbert transform by a signal conversion unit, and extracting instantaneous phase information;
[0099] S330 , calculating the stability of the instantaneous phase difference according to the instantaneous phase information by a calculation unit to obtain a PLV value to characterize the synchronization relationship.
[0100] In some embodiments, the data processing method based on micro-macro relationship construction also includes: setting a threshold through a matrix unit, and establishing a synchronization relationship matrix based on the PLV value and the threshold; wherein the synchronization relationship matrix includes multiple target elements, and when the PLV value is greater than the threshold, the corresponding target element is 1; when the PLV value is less than or equal to the threshold, the corresponding target element is 0.
[0101] In some embodiments, the data processing method based on micro-macro relationship construction further includes:
[0102] S410, using a pairwise similarity encoder PAE to capture the similarity of basic information between target objects through a capture construction unit, and constructing a macroscopic pairwise relationship; wherein the basic information includes age, gender, and IQ;
[0103] S420, clustering the subtype information by using fuzzy clustering through a cluster evaluation unit, and evaluating the subtype group relationship between the target objects according to the fuzzy membership and the set membership threshold; wherein the subtype group relationship is used to characterize the subdivision pre-test level of the target object.
[0104] In some embodiments, the data processing method based on micro-macro relationship construction further includes:
[0105] S510, design an adaptive multi-channel graph convolutional network by learning graph embedding of each scale through graph convolutional neural network technology through design units;
[0106] S520, constructing a multi-scale population graph based on synchronization relationships, pair relationships, and subtype relationships by constructing units;
[0107] S530, learning a multi-scale crowd graph through a multi-channel graph convolutional network through the learning unit to obtain preliminary representation information.
[0108] In some embodiments, the present application provides an electronic device, the structure diagram of the electronic device is as follows Figure 4 shown.
[0109] The electronic device may include a processor 610 and a memory 620 storing computer program instructions.
[0110] Specifically, the processor 610 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0111] The memory 620 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 620 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 620 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 620 is a non-volatile solid-state memory.
[0112] The memory 620 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory 620 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in the data processing method based on micro-macro relationship construction in the above-mentioned embodiment.
[0113] The processor 610 implements the data processing method based on micro-macro relationship construction in the above embodiment by reading and executing computer program instructions stored in the memory 620.
[0114] In one example, the electronic device may further include a communication interface 630 and a bus 600. Figure 4 As shown, the processor 610, the memory 620, and the communication interface 630 are connected via a bus 600 and communicate with each other.
[0115] The communication interface 630 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0116] Bus 600 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 600 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.
[0117] In addition, in combination with the data processing method based on micro-macro relationship construction in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, the data processing method based on micro-macro relationship construction in the above embodiment is implemented.
[0118] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0119] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0120] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0121] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0122] In summary, compared with the prior art, this application has the following beneficial effects:
[0123] 1. From a microscopic perspective, this application introduces functional magnetic resonance imaging (fMRI) technology to obtain fMRI microscopic data of each target object, and uses phase-locked value (PLV) technology for data analysis to capture and determine the synchronization relationship at the whole brain scale; from a macroscopic perspective, based on the acquired scale data, the pairwise relationship and subtype relationship between different target objects are analyzed and determined. This application comprehensively considers and analyzes the microscopic and macroscopic perspectives to ensure the accuracy of cognitive testing.
[0124] 2. This application uses a convolutional network to perform convolutional learning on the obtained synchronization relationships, pair relationships, and subtype relationships, and fuses them based on the attention mechanism to output target representation information, thereby classifying the cognitive abilities of the target objects, optimizing the representation of complex brain signals, improving test accuracy, and providing a new tool for cognitive testing.
[0125] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data processing system based on micro-macro relationship, characterized in that: include: A microscopic relationship building module, wherein the microscopic relationship building module is used to analyze the acquired fMRI microscopic data using the phase-locked value (PLV) technology to capture the synchronization relationship of different target objects at the whole brain scale; A macro-relationship construction module, the macro-relationship construction module is used to obtain the scale data of the target object, use a pairwise similarity encoder PAE to capture the pairwise relationships between different target objects, and use a fuzzy clustering algorithm to capture subtype relationships; wherein the pairwise relationships and the subtype relationships are used to characterize the cognitive level of the target object; A multi-scale graph convolution module, wherein the multi-scale graph convolution module is used to construct a multi-channel graph convolution network, and to perform convolution learning on the synchronization relationship, the pair relationship, and the subtype relationship, respectively, to obtain preliminary representation information; A fusion module is used to obtain the attention coefficients between the brain regions corresponding to the different target objects, learn the importance of different channels in the multi-scale graph convolution module through a preset attention mechanism, and fuse the different preliminary representation information into target representation information for cognitive ability classification.
2. The data processing system based on micro-macro relationship construction as claimed in claim 1, characterized in that: The micro-relationship building module includes: A filtering processing unit, wherein the filtering processing unit is used to pre-process the BOLD signal corresponding to each target object by applying an FIR filter, extract the frequency components of a specific frequency band, reduce the noise thereof, and splice them into a filtered signal at a whole-brain scale to obtain an fMRI target sequence; A signal conversion unit, the signal conversion unit is used to convert the filtered signal into an analytical signal through Hilbert transform and extract instantaneous phase information; A calculation unit is used to calculate the stability of the instantaneous phase difference according to the instantaneous phase information to obtain a PLV value to characterize the synchronization relationship.
3. The data processing system based on micro-macro relationship construction as claimed in claim 2, characterized in that: The microscopic relationship building module further includes a matrixing unit, the matrixing unit is used to set a threshold value and establish a synchronization relationship matrix based on the PLV value and the threshold value; The synchronization relationship matrix includes multiple target elements. When the PLV value is greater than the threshold, the corresponding target element is 1; when the PLV value is less than or equal to the threshold, the corresponding target element is 0.
4. The data processing system based on micro-macro relationship construction as claimed in claim 2, characterized in that: The fMRI target sequence satisfies the expression: S=[s1||s2||…||s K ]∈R K×T Where, S represents the fMRI target sequence, s i represents the brain region corresponding to the i-th target object, 1≤i≤K, K represents the number of brain regions, and T represents the number of brain region time points; Calculate the two target objects S i and S j The PLV values between satisfy the expression: In the formula, is the instantaneous phase difference of the signal, For the target object S i The corresponding instantaneous phase information, For the target object S j The corresponding instantaneous phase information, e represents the natural constant, and γ represents the imaginary unit.
5. The data processing system based on micro-macro relationship construction as claimed in claim 1, characterized in that: The macro relationship building module includes: A capture construction unit, wherein the capture construction unit is used to capture the similarity of basic information between target objects using a pairwise similarity encoder PAE to construct a macroscopic pairwise relationship; wherein the basic information includes age, gender, and IQ; A clustering evaluation unit, wherein the clustering evaluation unit is used to cluster the subtype information using fuzzy clustering, and to evaluate the subtype group relationship between the target objects according to the fuzzy membership and the set membership threshold; wherein the subtype group relationship is used to characterize the subdivision pre-test level of the target object.
6. The data processing system based on micro-macro relationship construction as claimed in claim 5, characterized in that: The macro data corresponding to the macro relationship building module includes different segmented pre-test data and the basic information in pairs; the cluster evaluation unit is also used to explore the subtype relationship between target objects by using fuzzy clustering, and determine the target object S by setting the membership threshold ξ i and the target object S j whether they belong to the same common subtype, and quantify the relationship between different target objects into a subtype network; In μ ik >ξ and μ jk >ξ, determine the target object S i and the target object S j belong to the same subtype; otherwise, determine the target object S i and the target object S j do not belong to the same subtype; among them, μ ik Represents the target object S i Belongs to the corresponding subtype c k The degree, μ jk Represents the target object S j Belongs to the corresponding subtype c k the extent of; The pairwise relationship satisfies the expression: In the formula, represents the pairwise relationship between the i-th target object and the j-th target object, φ i and φ j They respectively represent the standardized input representations of the basic information corresponding to the i-th target object and the j-th target object obtained by preprocessing in the mapping space; The objective function L corresponding to the fuzzy clustering satisfies the expression: The constraint condition corresponding to the objective function L satisfies the expression: In the formula, N represents the number of target objects, c k is the cluster center, |C| is the number of cluster centers, and m is the fuzzy parameter.
7. The data processing system based on micro-macro relationship construction as claimed in claim 1, characterized in that: The multi-scale graph convolution module includes: A design unit, wherein the design unit is used to learn graph embedding of each scale through a graph convolutional neural network technology and design an adaptive multi-channel graph convolutional network; A construction unit, the construction unit is used to construct a multi-scale population graph based on the synchronization relationship, the pair relationship and the subtype relationship; A learning unit is used to learn the multi-scale crowd graph through the multi-channel graph convolutional network to obtain preliminary representation information.
8. A data processing method based on micro-macro relationship construction, characterized in that: include: The fMRI micro data acquired by the micro-relationship building module are analyzed using the phase-locked value (PLV) technique to capture the synchronization relationship of different target objects at the whole-brain scale. The scale data of the target object is obtained through a macro-relationship construction module, the pairwise relationship between different target objects is captured using a pairwise similarity encoder PAE, and the subtype relationship is captured through a fuzzy clustering algorithm; wherein the pairwise relationship and the subtype relationship are used to characterize the cognitive level of the target object; A multi-channel graph convolution network is constructed through a multi-scale graph convolution module, and convolution learning is performed on the synchronization relationship, the pair relationship, and the subtype relationship to obtain preliminary representation information; The attention coefficients between the brain regions corresponding to the different target objects are obtained through the fusion module, the importance of different channels in the multi-scale graph convolution module is learned through the preset attention mechanism, and the different preliminary representation information is fused into the target representation information to perform cognitive ability classification.
9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the data processing method based on micro-macro relationship construction as claimed in claim 8 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the data processing method based on micro-macro relationship construction as claimed in claim 8 is implemented.