Functional magnetic resonance imaging classification method, system, device and medium
By extracting the time series characteristics and demographic information of the fMR image, combining the information relationship attention mechanism and graph adaptive convolution, the model is optimized for fMR image classification, which solves the problem of poor individual classification effect and achieves accurate Image classification and biomarker mining.
Patent Information
- Application Number
- CN202210489108.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-07
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-05-07
AI Technical Summary
The prior art has poor results in individual classification of fMR images, making it difficult to effectively learn and dig information from local community groups, resulting in low classification accuracy.
By extracting time series features that react to changes in blood oxygen signal, brain function link features are generated, and redundant features are eliminated through correlation coefficient method and feature recursion method, combined with demographic information assisted modeling, using information relationship attention mechanism and graph adaptive volume The local group information is accumulated and the cross entropy loss and adaptive learning rate optimization model is adopted.
It realizes accurate classification prediction of functional magnetic resonance images, dynamically defines group information relationships, fully explores the intrinsic connections of the group, and provides valuable biomarkers for medical diagnosis.
Smart Images

Figure CN114842258B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of image recognition technology, and in particular to a functional magnetic resonance imaging classification method, system, device and medium. Background Art
[0002] At present, the role of functional magnetic resonance imaging in clinical diagnosis is becoming increasingly prominent. Compared with other brain imaging techniques, functional magnetic resonance imaging is less invasive to the brain. The brain function link data extracted based on functional magnetic resonance imaging has a convincing ability in characterizing brain neural pathways, and provides a powerful way to observe brain region abnormalities between normal people and patients. Doctors usually need to analyze a large amount of brain function link data in clinical diagnosis, which is a difficult task. Therefore, it is urgent to establish an objective classification model for functional magnetic resonance imaging to assist doctors in analyzing patient data. In the patient population, there is often an implicit relationship between individuals, which is instructive for the analysis of specific patients. Since the graph structure has a strong expressive ability to model relationships, graph-based methods are also widely used in image classification. Tong et al. proposed to construct a graph based on manual verification of each data modality, and then combine it into a unified graph for classification. Gao et al. used a graph convolutional neural network to learn the embedding of patients on the constructed multi-graph. Graph convolutional neural networks have a good effect on how to learn the information of patients in local community groups. Therefore, how to establish the information relationship of the group through the information of patients is crucial for the application of graph convolutional neural networks in image classification.
[0003] Knowledge embedding was proposed by Google in 2012. It is used to link an entity with another entity into a knowledge triple through multiple knowledge or information links. Based on this, multiple image knowledge triplets form the knowledge embedding of image information, and image knowledge embedding is used to characterize the information relationship of the group. Image knowledge embedding provides a reference information relationship template for graph convolutional neural networks to learn individual information in local community groups. Rotmensch et al. extracted medical concepts from a large number of identified patient records and constructed knowledge embedding using maximum likelihood estimation of three probability models. Pham et al. conceptualized the data in the survey report into a heterogeneous knowledge embedding and used it to classify individuals. Agrawal et al. used a nonlinear function to establish causal knowledge embedding to better understand existing model assumptions.
[0004] Therefore, with regard to the problem of individual functional magnetic resonance imaging classification, how to effectively learn and mine the information of local community groups and find valuable information for functional magnetic resonance imaging classification is urgently needed to be studied in the present invention. Summary of the invention
[0005] In view of this, the embodiments of the present disclosure provide a functional magnetic resonance imaging classification method, system, device and medium, which at least partially solve the problem of poor classification effect for individuals in the prior art.
[0006] In a first aspect, the present disclosure provides a functional magnetic resonance imaging classification method, comprising:
[0007] Step 1: extract the time series features reflecting the changes in blood oxygen signals based on functional magnetic resonance imaging, and generate brain functional connection features through the correlation coefficient method;
[0008] Step 2, eliminating redundant features in the brain function link features by a feature recursive method to obtain target function link features;
[0009] Step 3, using the target functional link features and demographic information to assist in modeling group knowledge embedding;
[0010] Step 4: Use the information relationship attention mechanism to learn the knowledge expression between entities in group knowledge embedding, and refer to the information relationship in group knowledge embedding through graph adaptive convolution to learn local group information for functional magnetic resonance imaging classification;
[0011] Step 5: Use two cross entropy losses to train the model, and use the Adam optimizer based on adaptive learning rate and label smoothing formula to optimize the model to obtain the classification model.
[0012] According to a specific implementation of the embodiment of the present disclosure, step 1 specifically includes:
[0013] The time series data reflecting the blood oxygen signal were extracted from the functional magnetic resonance imaging. The brain function connection matrix was generated using the Pearson correlation coefficient based on the time series data of different brain regions. The upper triangular features of the brain function connection matrix were extracted as the brain function connection features.
[0014] According to a specific implementation manner of the embodiment of the present disclosure, the feature recursive method is a recursive feature elimination method based on ridge regression.
[0015] According to a specific implementation of the embodiment of the present disclosure, the structure of group knowledge embedding is: Among them, Dis is the distance of individual feature vectors, Sim is the similarity measure of demographic information composition, Dis(S i ,S j ) is individual S i With S j The distance between i ,S j ) is individual S i With S jThe similarity between discrete phenotypes is measured, W is the knowledge embedding, and W(i,j) is the information relationship between entities i and j.
[0016] According to a specific implementation of the embodiment of the present disclosure, step 4 specifically includes:
[0017] The knowledge expression of entities in group knowledge embedding is learned through the information relationship attention mechanism, and then the local group features are learned through graph adaptive convolution. The information relationship between individuals is expressed as follows:
[0018]
[0019] where α ij is the normalized information relation attention weight, h i ,h j is the individual feature vector, N i is the group associated with individual i, T represents transposition, || represents the connection operation, and after the group knowledge embedding is generated, the graph convolutional neural network is used to learn the local group information. The specific formula is as follows:
[0020]
[0021] in To add a self-loop knowledge graph, H (l) is the result of the l-th graph convolution layer, is the degree matrix of the knowledge graph, W (l) is the learnable weight parameter of the lth layer, and σ is the Relu nonlinear activation function.
[0022] According to a specific implementation of the embodiment of the present disclosure, step 5 specifically includes:
[0023] The multi-layer features are optimized through a cross entropy loss, and the optimized features are input into the multi-layer fusion module to generate individual shared features and specific features;
[0024] According to the shared features and the specific features, another cross entropy loss is used to perform a label smoothing operation on the individual true labels to obtain the classification model.
[0025] In a second aspect, the present disclosure provides a functional magnetic resonance imaging classification system, comprising:
[0026] A generation module is used to extract time series features reflecting changes in blood oxygen signals based on functional magnetic resonance imaging, and generate brain functional connection features through a correlation coefficient method;
[0027] An elimination module, used for eliminating redundant features in the brain function link features by a feature recursive method to obtain a target function link feature;
[0028] A modeling module, for assisting modeling of group knowledge embedding using the target functional link features and demographic information;
[0029] The learning module is used to learn the knowledge expression between entities in group knowledge embedding by using the information relationship attention mechanism, and learn local group information for functional magnetic resonance imaging classification by referring to the information relationship in group knowledge embedding through graph adaptive convolution;
[0030] In the training module, two cross entropy losses are used to train the model, and the Adam optimizer based on adaptive learning rate and label smoothing formula are used to optimize the model to obtain the classification model.
[0031] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:
[0032] at least one processor; and,
[0033] a memory communicatively connected to the at least one processor; wherein,
[0034] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the functional magnetic resonance imaging classification method in the aforementioned first aspect or any implementation of the first aspect.
[0035] In a fourth aspect, an embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the functional magnetic resonance imaging classification method in the aforementioned first aspect or any implementation method of the first aspect.
[0036] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the functional magnetic resonance imaging classification method in the aforementioned first aspect or any implementation of the first aspect.
[0037] The functional magnetic resonance imaging classification scheme in the disclosed embodiment includes: step 1, extracting time series features reflecting changes in blood oxygen signals according to functional magnetic resonance imaging, and generating brain function link features by correlation coefficient method; step 2, eliminating redundant features in the brain function link features by feature recursion method to obtain target function link features; step 3, using the target function link features and demographic information to assist in modeling group knowledge embedding; step 4, using the information relationship attention mechanism to learn the knowledge expression between entities in group knowledge embedding, and using graph adaptive convolution to refer to the information relationship in group knowledge embedding, learn local group information for functional magnetic resonance imaging classification; step 5, using two cross entropy loss training models, and using the Adam optimizer based on adaptive learning rate and label smoothing formula to optimize the model to obtain a classification model.
[0038] The beneficial effects of the embodiments of the present disclosure are:
[0039] (1) The present invention proposes a graph adaptive convolutional neural network based on knowledge embedding, which can dynamically define the information relationship of a group and learn local group information based on the above information relationship to achieve accurate classification prediction of functional magnetic resonance imaging.
[0040] (2) In the further optimization method of the present invention, a multi-layer feature fusion method is proposed, which targets the different levels of group information learned by individuals, fully explores the internal connections between different groups, provides assistance for the final prediction classification and provides biomarkers with reference significance. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0042] Figure 1 A schematic diagram of a flow chart of a functional magnetic resonance imaging classification method provided in an embodiment of the present disclosure;
[0043] Figure 2 A schematic diagram of another functional magnetic resonance imaging classification method provided by an embodiment of the present disclosure;
[0044] Figure 3 A comparison diagram of a multi-layer fusion module and self-attention fusion provided in an embodiment of the present disclosure;
[0045] Figure 4 A schematic diagram of the structure of a functional magnetic resonance imaging classification system provided by an embodiment of the present disclosure;
[0046] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0047] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0048] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.
[0049] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.
[0050] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0051] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.
[0052] The present disclosure provides a functional magnetic resonance imaging classification method, which can be applied to the functional magnetic resonance imaging analysis process in medical clinical diagnosis scenarios.
[0053] See also Figure 1 , is a flow chart of a functional magnetic resonance imaging classification method provided by an embodiment of the present disclosure. Figure 1 and Figure 2 As shown, the method mainly comprises the following steps:
[0054] Step 1: extract the time series features reflecting the changes in blood oxygen signals based on functional magnetic resonance imaging, and generate brain functional connection features through the correlation coefficient method;
[0055] Furthermore, the step 1 specifically includes:
[0056] The time series data reflecting the blood oxygen signal were extracted from the functional magnetic resonance imaging. The brain function connection matrix was generated using the Pearson correlation coefficient based on the time series data of different brain regions. The upper triangular features of the brain function connection matrix were extracted as the brain function connection features.
[0057] In specific implementation, the time series data of the blood oxygen signal can be extracted from the functional magnetic resonance imaging, and then the brain function connection matrix can be generated using the Pearson correlation coefficient based on the time series data of different brain regions. Due to the symmetry of the brain function connection matrix, the upper triangular features of the brain function connection matrix can be extracted as the individual feature vector.
[0058] Step 2, eliminating redundant features in the brain function link features by a feature recursive method to obtain target function link features;
[0059] Optionally, the feature recursion method is a recursive feature elimination method based on ridge regression.
[0060] In specific implementation, considering the high dimensionality and redundancy of brain functional connection data, a recursive feature elimination method based on ridge regression can be used. The specific formula is as follows:
[0061]
[0062] where x (i) is the characteristic vector of the sample, y (i) is the label value of the sample, h θ is a learnable weight parameter, and θ is a regression coefficient. Of course, other algorithms can be used for feature elimination according to actual needs, which are not listed here.
[0063] Step 3, using the target functional link features and demographic information to assist in modeling group knowledge embedding;
[0064] Optionally, the structure of group knowledge embedding is Among them, Dis is the distance of individual feature vectors, Sim is the similarity measure of demographic information composition, Dis(S i,S j ) is individual S i With S j The distance between i ,S j ) is individual S i With S j The similarity between discrete phenotypes is measured, W is the knowledge embedding, and W(i,j) is the information relationship between entities i and j.
[0065] In the specific implementation, considering that in graph convolution, local group information is learned, group knowledge embedding must be carefully constructed to accurately model the relationship between entities. In the present invention, we define each entity as a node and the information relationship between two entities as an edge. Knowledge embedding consists of two parts, one is the distance of individual feature vectors, denoted as Dis, and the other is a similarity measure composed of demographic information, denoted as Sim. For an individual Si, we use F(Si) to represent its feature vector. The distance of the subject's feature vector is defined as the following formula:
[0066]
[0067] Where Dis(S i ,S j ) is individual S i With S j The distance between them, ρ(·,·) is the distance between the subject feature vectors. Demographic information can provide key information to explain the association between the feature vectors of individuals. My goal is to use this information to define a more accurate knowledge embedding structure to mine more information for graph convolutional neural networks. Through the performance information in demographic information, such as gender, age, and education level. The similarity calculation based on discrete phenotypic measurement is defined as the following formula:
[0068]
[0069] Where P m (S i ) represents S i Based on the above formula, the final knowledge embedding structure can be expressed as follows:
[0070]
[0071] Where W is the knowledge embedding and W(i,j) is the information relationship between entities i and j.
[0072] Step 4: Use the information relationship attention mechanism to learn the knowledge expression between entities in group knowledge embedding, and refer to the information relationship in group knowledge embedding through graph adaptive convolution to learn local group information for functional magnetic resonance imaging classification;
[0073] Based on the above embodiment, step 4 specifically includes:
[0074] The knowledge expression of entities in group knowledge embedding is learned through the information relationship attention mechanism, and then the local group features are learned through graph adaptive convolution. The information relationship between individuals is expressed as follows:
[0075]
[0076] where α ij is the normalized information relation attention weight, h i ,h j is the individual feature vector, N i is the group associated with individual i, T represents transposition, || represents the connection operation, and after the group knowledge embedding is generated, the graph convolutional neural network is used to learn the local group information. The specific formula is as follows:
[0077]
[0078] in To add a self-loop knowledge graph, H (l) is the result of the l-th graph convolution layer, is the degree matrix of the knowledge graph, W (l) is the learnable weight parameter of the lth layer, and σ is the Relu nonlinear activation function.
[0079] When implementing it, Figure 3 As shown in the figure, considering the static nature of the constructed group knowledge embedding, the information relationship attention mechanism can be used to learn specific information between individuals in a targeted manner. Specifically, more valuable knowledge relationships can be screened out from the group knowledge embedding through a threshold, and then the information relationship attention mechanism can be used to learn knowledge from paired entities to generate more meaningful information relationships. The information relationship attention mechanism is expressed as the following formula:
[0080] E ij =a(Wh i ,Wh j )
[0081] Where a is the information relationship attention mechanism, h i ,h j is the individual feature vector, E ijis the learned information relationship attention score, which represents the information reference importance of individual j to individual i. In order to make the information relationship attention score easy to compare between different individuals, the softmax function can be used to normalize all the choices of individual j:
[0082]
[0083] where α ij is the normalized information relation attention weight, N i is the group associated with individual i. In my invention, the information relationship attention mechanism a is a single-layer feedforward neural network, and the Leakyrelu activation function can be used to ensure the nonlinearity of the neural network. After full expansion, the information relationship between individuals can be expressed as follows:
[0084]
[0085] Where T represents transposition and || represents the connection operation. After the group knowledge embedding is generated, the graph convolutional neural network can be used to learn the local group information. The specific formula can be expressed as follows:
[0086]
[0087] in To add self-loop knowledge embedding, H (l) is the result of the l-th graph convolution layer, is the degree matrix of knowledge embedding, W (l) is the learnable weight parameter of the lth layer, and σ is the Relu nonlinear activation function.
[0088] Step 5: Use two cross entropy losses to train the model, and use the Adam optimizer based on adaptive learning rate and label smoothing formula to optimize the model to obtain the classification model.
[0089] Furthermore, the step 5 specifically includes:
[0090] The multi-layer features are optimized through a cross entropy loss, and the optimized features are input into the multi-layer fusion module to generate individual shared features and specific features;
[0091] According to the shared features and the specific features, another cross entropy loss is used to perform a label smoothing operation on the individual true labels to obtain the classification model.
[0092] In specific implementation, two cross entropy losses can be used, one to supervise the effect of multi-layer data fusion and accelerate the convergence speed, where each layer can be regarded as different levels of local group information learned by individuals. The multi-layer fusion loss can be expressed as the following formula:
[0093]
[0094] Where C is the number of sample categories, y is the sample label, w is the category weight, which is a parameter set to alleviate the category imbalance problem, and X is the input sample feature, which can be specifically expressed as X=concat(x1,x2...x l ) where l is the number of graph convolution layers. After preliminarily optimizing the sample features through multi-layer fusion loss, I further fused the sample features to generate shared features and specific features. The specific formula is as follows:
[0095] Output sp =Softmax(Cosine(Norm((XW q ,X)W cos )))
[0096]
[0097] Output sp is a specific feature, Output sh is the shared feature, W q , W cos , W v are all learnable weight parameters, For the cross entropy loss function of the final result, I smooth the individual true label y to further alleviate the category imbalance problem. The specific formula can be expressed as follows:
[0098]
[0099] Where K is the number of individual categories, y is the true label, and y s is the smoothed label, a is a hyperparameter, which is set to 0.1 in the present invention.
[0100] The functional magnetic resonance imaging classification method provided in this embodiment proposes a graph adaptive convolutional neural network based on knowledge embedding, which can dynamically define the information relationship of the group, and learn local group information based on the above information relationship to achieve accurate classification prediction of functional magnetic resonance imaging. Then, in the further optimization method, a multi-layer feature fusion method is proposed, which fully explores the internal connection of different groups based on the different levels of group information learned by individuals, provides assistance for the final prediction classification and provides reference biomarkers.
[0101] Corresponding to the above method embodiment, see Figure 4 The present disclosure also provides a functional magnetic resonance imaging classification system 40, including:
[0102] A generation module 401 is used to extract time series features reflecting changes in blood oxygen signals based on functional magnetic resonance imaging, and generate brain function connection features through a correlation coefficient method;
[0103] Elimination module 402, used to eliminate redundant features in the brain function link features by a feature recursive method to obtain target function link features;
[0104] A modeling module 403, for assisting modeling of group knowledge embedding using the target functional link features and demographic information;
[0105] A learning module 404 is used to learn the knowledge expression between entities in group knowledge embedding by using the information relationship attention mechanism, and learn local group information for functional magnetic resonance imaging classification by referring to the information relationship in the group knowledge embedding through graph adaptive convolution;
[0106] The training module 405 uses two cross entropy loss training models, and uses an Adam optimizer based on an adaptive learning rate and a label smoothing formula to optimize the model to obtain a classification model.
[0107] Figure 4 The system shown can correspondingly execute the contents of the above method embodiment. For the parts not described in detail in this embodiment, refer to the contents recorded in the above method embodiment and will not be repeated here.
[0108] See also Figure 5 The embodiment of the present disclosure further provides an electronic device 50, which includes: at least one processor and a memory connected to the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the functional magnetic resonance imaging classification method in the aforementioned method embodiment.
[0109] The disclosed embodiment also provides a non-transitory computer-readable storage medium, which stores computer instructions for causing the computer to execute the functional magnetic resonance imaging classification method in the aforementioned method embodiment.
[0110] The present disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the functional magnetic resonance imaging classification method in the aforementioned method embodiment.
[0111] Reference below Figure 5, which shows a schematic diagram of the structure of an electronic device 50 suitable for implementing the embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0112] like Figure 5 As shown, the electronic device 50 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 50 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0113] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication device 509 may allow the electronic device 50 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 50 with various devices is shown in the figure, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
[0114] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0115] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0116] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0117] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can execute the relevant steps of the method embodiment.
[0118] Alternatively, the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device can perform the relevant steps of the method embodiment.
[0119] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0120] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0121] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware.
[0122] It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
[0123] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A functional magnetic resonance imaging classification method, characterized in that: include: Step 1: extract the time series features reflecting the changes in blood oxygen signals based on functional magnetic resonance imaging, and generate brain functional connection features through the correlation coefficient method; The step 1 specifically includes: Extract the time series data of blood oxygen signal according to the functional magnetic resonance imaging, generate the brain function connection matrix according to the time series data of different brain regions using the Pearson correlation coefficient, and extract the upper triangular features of the brain function connection matrix as the brain function connection features; Step 2, eliminating redundant features in the brain function link features by a feature recursive method to obtain target function link features; Step 3, using the target function link feature and demographic information to assist in modeling group knowledge embedding, wherein the structure of group knowledge embedding is Among them, Dis is the distance of individual feature vectors, Sim is the similarity measure of demographic information composition, Dis(S i ,S j ) is individual S i With S j The distance between i ,S j ) is individual S i With S j The similarity between discrete phenotypes, W is the knowledge embedding, and W(i,j) is the information relationship between entities i and j; Step 4: Use the information relationship attention mechanism to learn the knowledge expression between entities in group knowledge embedding, and refer to the information relationship in group knowledge embedding through graph adaptive convolution to learn local group information for functional magnetic resonance imaging classification; The step 4 specifically includes: The knowledge expression of entities in group knowledge embedding is learned through the information relationship attention mechanism, and then the local group features are learned through graph adaptive convolution. The information relationship between individuals is expressed as follows: where α ij is the normalized information relation attention weight, h i ,h j is the individual feature vector, N i is the group associated with individual i, T represents transposition, || represents the connection operation, and after the group knowledge embedding is generated, the graph convolutional neural network is used to learn the local group information. The specific formula is as follows: in To add a self-loop knowledge graph, H (l) is the result of the l-th graph convolution layer, is the degree matrix of the knowledge graph, W (l) is the learnable weight parameter of the lth layer, σ is the Relu nonlinear activation function; Step 5: Use two cross entropy losses to train the model, and use the Adam optimizer based on adaptive learning rate and label smoothing formula to optimize the model to obtain the classification model.
2. The method according to claim 1, characterized in that ,The feature recursion method is a recursive feature elimination method based on ridge ,regression.
3. The method according to claim 1, characterized in that , the step 5 specifically includes: The multi-layer features are optimized through a cross entropy loss, and the optimized features are input into the multi-layer fusion module to generate individual shared features and specific features; According to the shared features and the specific features, another cross entropy loss is used to perform a label smoothing operation on the individual true labels to obtain the classification model.
4. A functional magnetic resonance imaging classification system, characterized in that: include: A generation module is used to extract time series features reflecting changes in blood oxygen signals based on functional magnetic resonance imaging, and generate brain functional connection features through a correlation coefficient method; The specific process of generating the module is as follows: Extract the time series data of blood oxygen signal according to the functional magnetic resonance imaging, generate the brain function connection matrix according to the time series data of different brain regions using the Pearson correlation coefficient, and extract the upper triangular features of the brain function connection matrix as the brain function connection features; An elimination module, used for eliminating redundant features in the brain function link features by a feature recursive method to obtain a target function link feature; A modeling module is used to use the target function link feature and demographic information to assist in modeling group knowledge embedding, wherein the structure of the group knowledge embedding is Among them, Dis is the distance of individual feature vectors, Sim is the similarity measure of demographic information composition, Dis(S i ,S j ) is individual S i With S j The distance between i ,S j ) is individual S i With S j The similarity between discrete phenotypes, W is the knowledge embedding, and W(i,j) is the information relationship between entities i and j; The learning module is used to learn the knowledge expression between entities in group knowledge embedding by using the information relationship attention mechanism, and learn local group information for functional magnetic resonance imaging classification by referring to the information relationship in group knowledge embedding through graph adaptive convolution; The specific process of the learning module is as follows: The knowledge expression of entities in group knowledge embedding is learned through the information relationship attention mechanism, and then the local group features are learned through graph adaptive convolution. The information relationship between individuals is expressed as follows: where α ij is the normalized information relation attention weight, h i ,h j is the individual feature vector, N i is the group associated with individual i, T represents transposition, || represents the connection operation, and after the group knowledge embedding is generated, the graph convolutional neural network is used to learn the local group information. The specific formula is as follows: in To add a self-loop knowledge graph, H (l) is the result of the l-th graph convolution layer, is the degree matrix of the knowledge graph, W (l) is the learnable weight parameter of the lth layer, σ is the Relu nonlinear activation function In the training module, two cross entropy losses are used to train the model, and the Adam optimizer based on adaptive learning rate and label smoothing formula are used to optimize the model to obtain the classification model.
5. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to perform the functional magnetic resonance imaging classification method according to any one of claims 1 to 3. 6 . A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the functional magnetic resonance imaging classification method according to claim 1 .
