Method, device and equipment for processing brain image and storage medium
By combining radiomics features and functional brain networks, and using 3D CNN, GCN, and MLP network models, the problem of the inability to accurately analyze the degree of functional impairment in brain imaging in existing technologies has been solved. This has enabled precise differentiation of target brain regions and analysis of functional connectivity abnormalities, providing accurate clinical reference information.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2026-05-15
AI Technical Summary
Current brain imaging processing methods cannot accurately analyze the degree of functional impairment in target areas of a patient based on the symptoms they present, thus failing to provide doctors with precise clinical reference information.
By combining radiomics features of brain images with functional brain networks, the target classification features of each brain region are determined, thereby accurately distinguishing target sites and analyzing their functional connectivity. 3D CNN, GCN and MLP network models are used for feature analysis and classification.
It enables accurate differentiation of target brain regions and accurate analysis of functional connectivity abnormalities, providing doctors with precise clinical reference information and assisting them in assessing the degree of functional connectivity impairment.
Smart Images
Figure CN116597208B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, specifically to a method, apparatus, device, and storage medium for processing brain images. Background Technology
[0002] Various lesions in the brain (such as brain tumors, schizophrenia, and multiple sclerosis) often threaten people's lives, and the diagnosis of brain lesions is complex and difficult to treat. Therefore, accurately segmenting the various lesion sites in brain imaging is of great clinical significance.
[0003] Currently, deep learning methods are typically used to perform feature analysis on patients' magnetic resonance imaging (MRI) images to extract corresponding brain region features, thereby identifying target areas in the patient's brain that require specific clinical observation.
[0004] However, because the degree of damage to the target site varies among patients, it can lead to different degrees of functional decline. For example, stroke patients may experience numbness in their hands and feet, slurred speech, and other functional declines. Current brain imaging techniques cannot accurately analyze the degree of functional damage to the target site based on the patient's symptoms, thus failing to provide doctors with precise clinical reference information. Summary of the Invention
[0005] This application provides a method, apparatus, device, and storage medium for processing brain images, enabling accurate differentiation of target brain regions and accurate analysis of abnormal functional connectivity in these regions. This helps to effectively analyze the degree of functional connectivity damage in target regions, thereby providing doctors with precise clinical reference information for analyzing target regions.
[0006] In a first aspect, embodiments of this application provide a method for processing brain images, the method comprising:
[0007] Based on radiomics features and functional brain networks within brain images, target classification features for each brain region are determined.
[0008] Based on the target classification features of each brain region, the corresponding target location is determined;
[0009] The functional connectivity of the target site is determined based on the classification importance of each brain region toward the target site.
[0010] Secondly, embodiments of this application provide a brain image processing apparatus, the apparatus comprising:
[0011] The feature determination module is used to determine the target classification features for each brain region based on radiomics features and functional brain networks within brain images.
[0012] The target location determination module is used to determine the corresponding target location based on the target classification features of each brain region.
[0013] The functional connectivity analysis module is used to determine the functional connectivity of the target site based on the classification importance of each brain region toward the target site.
[0014] Thirdly, embodiments of this application provide an electronic device, which includes:
[0015] A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the brain image processing method provided in the first aspect of this application.
[0016] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the brain image processing method provided in the first aspect of this application.
[0017] Fifthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the brain image processing method provided in the first aspect of this application.
[0018] This application provides a method, apparatus, device, and storage medium for processing brain images. First, based on radiomics features and functional brain networks within the brain images, the target classification features of each brain region are comprehensively determined to identify the corresponding target location, achieving accurate differentiation of target locations within the brain images. Furthermore, based on the classification importance of each brain region to the target location, the functional connectivity of the target location is determined, enabling accurate analysis of functional connectivity abnormalities in the target location. This helps doctors effectively analyze the degree of functional connectivity damage in target brain locations, thereby providing precise clinical reference information for doctors' analysis of the target locations. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 A flowchart illustrating a brain image processing method according to an embodiment of this application;
[0021] Figure 2 This is a schematic diagram of a functional brain network shown in an embodiment of this application;
[0022] Figure 3 This is a flowchart illustrating the method for determining target classification features for each brain region in an embodiment of this application.
[0023] Figure 4 This is a schematic diagram illustrating the principle of brain image processing in an embodiment of this application;
[0024] Figure 5 This is a flowchart illustrating the method for determining the functional connectivity of a target location according to an embodiment of this application.
[0025] Figure 6 This is a schematic block diagram of a brain image processing device according to an embodiment of this application;
[0026] Figure 7 This is a schematic block diagram of an electronic device shown in an embodiment of this application. Detailed Implementation
[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0029] To address the problem that patient symptoms alone cannot accurately analyze the degree of functional impairment in target brain regions, thus failing to provide doctors with precise clinical reference information, this application presents a brain imaging processing scheme. Based on radiomics features and functional brain networks within brain images, the scheme comprehensively determines the target classification features of each brain region to identify the corresponding target location, thereby achieving accurate differentiation of target brain regions. Furthermore, by determining the functional connectivity of the target location based on the classification importance of each brain region to the target location, the scheme enables accurate analysis of functional connectivity abnormalities in the target brain region. This helps doctors effectively analyze the degree of functional connectivity impairment in target brain regions, thus providing doctors with precise clinical reference information for their analysis of the target location.
[0030] Figure 1 This is a flowchart illustrating a method for processing brain images, as shown in an embodiment of this application. (Refer to...) Figure 1 The method may include the following steps:
[0031] S110, based on radiomics features and functional brain networks within brain images, determines the target classification features for each brain region.
[0032] In order to comprehensively and accurately distinguish target areas in the patient's brain that require special clinical observation, this application first uses a specific scanning device to scan the patient's brain to obtain a three-dimensional brain image.
[0033] In some possible implementations, the brain images in this application may include, but are not limited to, MRI images obtained by scanning the brains of various patients using magnetic resonance imaging (MRI) technology and FMRI data obtained by scanning the brains of various patients using functional magnetic resonance imaging (FMRI) technology.
[0034] MRI images can describe the internal morphology of a patient's brain by scanning the morphology of gray matter, white matter, and cerebrospinal fluid, while fMRI data can describe the connectivity between different brain regions by detecting blood flow in the brain.
[0035] Considering that different brain functional areas exist within brain imaging, namely the brain regions described in this application, and that different brain regions also have different forms of functional connections, thus forming a complex and vast brain network. Moreover, different brain diseases involve different brain tissues, causing varying degrees of abnormal effects on the specific functions of each brain region and the functional connections between different brain regions due to different disease sites.
[0036] Therefore, to ensure accurate differentiation of target brain regions, this application, after acquiring brain images, first locates an Anatomical Automatic Labeling (AAL) template for brain region partitioning. This AAL template can be obtained by performing standard brain region partitioning using the AAL algorithm. Then, according to this AAL template, each brain region within the brain image is determined. For example, the AAL template in this application provides a total of 116 brain regions, of which 90 belong to cerebral structures and the remaining 26 belong to cerebellar structures.
[0037] Then, we can comprehensively analyze the target classification characteristics of each brain region for lesions in different locations from two aspects: the specific structure of each brain region and the functional connections between different brain regions.
[0038] Specifically, by performing feature analysis on the brain morphological structures described by brain images, corresponding radiomics features can be obtained. These radiomics features can comprehensively describe the overall structural features of each brain region.
[0039] Furthermore, by performing feature analysis on the functional connectivity states of different brain regions described in brain images, it is possible to identify the individual brain regions present in the brain images and the functional connections between them. Then, as... Figure 2 As shown, each brain region is treated as a corresponding brain region node, and the functional connection between any two brain regions is treated as an edge between these two brain region nodes, thereby generating the corresponding functional brain network.
[0040] Then, for each brain region, a comprehensive analysis can be performed on the overall brain structure described by the corresponding radiomics features and the functional connectivity features between that brain region and other brain regions in the functional brain network to obtain the target classification features of that brain region. Following the same steps described above, the target classification features for each brain region can be obtained.
[0041] Taking multiple lesions in the brain as an example, if the brain image processing scheme provided in this application is applied to the analysis of a certain brain lesion, then the target classification features of each brain region can be the lesion classification features of each brain region.
[0042] S120 determines the corresponding target location based on the target classification features of each brain region.
[0043] After obtaining the target classification features of each brain region, the target classification features of the entire brain can be obtained by merging the target classification features of each brain region. Moreover, since the target classification features of each brain region are formed by the fusion of the overall structure of the brain region and the functional connectivity between brain regions, the merged target classification features of the entire brain can comprehensively represent the overall structure of each brain region and the functional connectivity between brain regions.
[0044] Considering that, to some extent, the pathogenesis of target brain regions is often due to abnormalities in some form of connectivity between related brain areas, feature analysis of the target classification characteristics of the entire brain after merging the images can be used to comprehensively judge the structural and functional connectivity abnormalities of each brain region, thereby identifying the corresponding target regions and achieving accurate differentiation of target regions within brain imaging.
[0045] In some feasible approaches, by performing corresponding feature analysis on the target classification features after merging the entire brain, the specific disease category of the target site can be identified, and the lesion area corresponding to the target site can be accurately segmented from the brain image to ensure accurate segmentation of brain lesions.
[0046] Taking multiple lesions in the brain as an example, if the brain image processing solution provided in this application is applied to the analysis of a certain brain lesion, then the target site in this application can be that brain lesion. That is, by analyzing the target classification features of each brain region, the corresponding brain lesion can be determined.
[0047] S130, determine the functional connectivity of the target site based on the classification importance of each brain region towards the target site.
[0048] Since functional degeneration caused by lesions in the target site is related to damage to functional connectivity within that site, and damage to functional connectivity between different brain regions can be represented by the functional connectivity of related brain regions within the target site, this application, after identifying the target site, further analyzes the functional connectivity of related brain regions within the target site to visually assess the degree of functional connectivity damage.
[0049] Considering that the pathogenesis of target brain regions is often due to abnormalities in some form of connectivity between related brain areas, meaning that target regions can be identified by analyzing whether there are structural or functional connectivity abnormalities in the relevant brain regions, the classification of target regions is closely related to the target classification characteristics of each brain region. Furthermore, the target classification characteristics of each brain region, as reference factors for target region classification, have varying degrees of influence on the accurate classification of target regions.
[0050] In this application, after identifying the target site, the influence of each brain region's target classification features on the target site classification can be analyzed by solving the partial derivative of the target classification features for each brain region. This allows for the determination of the classification importance of each brain region towards the target site. By analyzing the classification importance of each brain region towards the target site, structural abnormalities in each brain region and functional connectivity abnormalities with other brain regions can be determined, thereby assessing the functional connectivity of related brain regions within the target site. The degree of functional connectivity impairment indicated by the functional connectivity of related brain regions within the target site can provide physicians with effective clinical reference information for the treatment of the target site, improving the accuracy of assessing functional connectivity impairment at the target site.
[0051] The technical solution provided in this application firstly determines the target classification features of each brain region based on radiomics features and functional brain networks within brain images, thereby identifying the corresponding target sites and achieving accurate differentiation of target sites within brain images. Furthermore, based on the classification importance of each brain region to the target site, the functional connectivity of the target site is determined, enabling accurate analysis of functional connectivity abnormalities in the target site. This helps doctors effectively analyze the degree of functional connectivity damage in target brain sites, thus providing precise clinical reference information for doctors' analysis of the target sites.
[0052] As an optional implementation in this application, considering that radiomics features in brain images are obtained by analyzing the morphological structure of the brain, while functional brain networks in brain images are obtained by analyzing the functional connections between various brain regions, this application classifies brain images into two types—Class I brain images and Class II brain images—to ensure accurate differentiation of target brain regions.
[0053] The first type of brain imaging can be structural images of the brain, such as MRI images. The second type of brain imaging can be functional images of the brain, such as fMRI images.
[0054] The following section provides a detailed explanation of the specific process for determining the target classification features for each brain region.
[0055] Figure 3 This is a flowchart illustrating the method for determining target classification features for each brain region in an embodiment of this application, as shown below. Figure 3 As shown, the method may include the following steps:
[0056] S310, determine the image blocks after segmentation of the first type of brain image and the radiomics features of each image block.
[0057] Given the irregularity of brain regions within brain images, it is impossible to accurately segment the entire radiomics features of a brain image into the radiomics features of each brain region for analysis of its morphological structure. Therefore, to ensure the accuracy of the target classification features for each brain region, this application first segments the first type of brain image into image blocks of appropriate sizes according to the standard format of brain regions. Moreover, the size of the image blocks is smaller than the size of the brain region, allowing each brain region to be composed of multiple image blocks.
[0058] Then, for each image block within the first type of brain imaging, this application can assign a corresponding three-dimensional convolutional neural network (3DCNN) to each image block. By inputting each image block into the corresponding 3D CNN network, each 3D CNN network performs feature analysis on the corresponding image block, thereby extracting the radiomics features of each image block. The radiomics features of each image block can represent its morphological and structural features, so that the radiomics features of each brain region can be determined according to the relationship between the brain region and the image block, thereby analyzing the morphological structure of each brain region.
[0059] It is understood that the first type of brain image in this application is a three-dimensional image, which can be converted into a three-dimensional tensor. Therefore, each image block after segmentation of the first type of brain image can also be represented as a three-dimensional tensor. Then, by inputting the three-dimensional tensor represented by each image block into the corresponding 3DCNN network, the radiomics features of each image block can be obtained.
[0060] In order to ensure the accuracy of the extraction of radiomics features for each image block, this application allows multiple 3D CNN networks to share weights.
[0061] S320 generates a corresponding functional brain network based on the distribution of brain regions in the second type of brain imaging.
[0062] To analyze the functional connectivity features of each brain region with other brain regions, this application can perform corresponding brain region distribution analysis on the second type of brain images to determine the functional connectivity relationships between each brain region. Then, each brain region is treated as a node, and the functional connectivity features between every two brain regions are used as edges between these two corresponding brain region nodes, thereby generating the corresponding functional brain network.
[0063] S330, based on the brain region node to which each image block belongs in the functional brain network and the adjacency matrix of the functional brain network, determine the target classification features of each brain region.
[0064] After identifying the radiomics features and functional brain network of each image patch, the specific location and size of each image patch in the brain can be analyzed. Furthermore, based on the brain region attribute information carried by each brain region node in the functional brain network, the specific location and size of each brain region can be analyzed. Then, by analyzing the relationship between the specific location and size of each image patch in the brain and the specific location and size of each brain region, it can be determined which brain region the image patch belongs to, thus identifying the brain region node to which each image patch belongs in the functional brain network.
[0065] For each brain region, based on the affiliation between each image block and each brain region node, this application can merge the radiomics features of each image block belonging to the brain region node to obtain the radiomics features of the brain region, so as to accurately describe the morphological structure of the brain region.
[0066] Considering that different target sites involve different brain tissues, and that functional connectivity between different brain regions can be abnormally affected to varying degrees, this application, in order to comprehensively analyze the lesion classification characteristics of each brain region, also analyzes the functional connectivity characteristics between every two brain regions when determining the radiomic characteristics of each brain region, thereby assessing the impact on the lesions at the target site.
[0067] In a functional brain network, the edge between any two brain region nodes represents the functional connection between those two brain regions. The adjacency matrix of a functional brain network can be stored as a two-dimensional array to represent the relationships between the nodes in each brain region. For example, if the number of brain region nodes in a functional brain network is N, then the adjacency matrix of the functional brain network can be an N*N two-dimensional matrix.
[0068] Therefore, by analyzing the adjacency matrix of functional brain networks, the functional connectivity features between every two brain regions can be determined. Then, for each brain region, the radiomics features of that region and the functional connectivity features between that region and other related brain regions can be fused and analyzed to determine the target classification features of that brain region. Following the same method, the target classification features of each brain region can be obtained.
[0069] As an exemplary solution in this application, to ensure accurate differentiation of target regions within brain images, this application may establish a target region classification model. For example, Figure 4 As shown, the target part classification model can include multiple 3D CNN networks, Graph Convolutional Network (GCN) and Multilayer Perceptron (MLP) networks.
[0070] In this process, by inputting each image block after the first type of brain image is segmented into the corresponding 3DCNN network, the radiomics features of each image block can be obtained.
[0071] Then, when analyzing the target classification features of each brain region, the node feature matrix of the functional brain network can be determined based on the brain region node to which each image block belongs in the functional brain network; and the target classification features of each brain region can be determined based on the node feature matrix and the adjacency matrix of the functional brain network.
[0072] In other words, based on the attribution relationships between each image patch and the nodes of each brain region in the functional brain network, the radiomics features of each image patch can be assigned to the corresponding brain region node. By merging the radiomics features of each image patch belonging to each brain region, the radiomics features of each brain region node can be obtained. By merging the radiomics features of each brain region node, the node feature matrix of the functional brain network can be obtained, which can then be used for feature fusion with the functional connectivity features between every two brain region nodes represented by the adjacency matrix of the functional brain network.
[0073] By inputting the node feature matrix and adjacency matrix of the functional brain network into the trained GCN network, the structural and functional connectivity features of each brain region node in the functional brain network can be comprehensively analyzed through the GCN network to determine the target classification features of each brain region.
[0074] Finally, the target classification features of each brain region are merged, and the merged target classification features are input into the MLP network to analyze the target classification features of the entire brain, thereby determining the corresponding target location.
[0075] Assume that the i-th image block after segmentation of the first type of brain image is I. i The functional brain network corresponding to the second type of brain imaging is G. f(·) Therefore, the radiomics features of the i-th image block in this application can be represented as F. i =3DCNNs(I i In this way, the radiomics features of each image block can be calculated.
[0076] For radiomics features of each brain region, a formula can be used. To determine. Among them, denoted as radiomics features of the j-th brain region, where N is the number of image blocks.
[0077] Through G f(j) ({F i} 1:nIt determines whether each image patch belongs to the j-th brain region. Furthermore, by merging and compressing the radiomics features of each image patch belonging to the j-th brain region using the flatten() function, the radiomics features of the j-th brain region can be obtained.
[0078] For the target classification features of each brain region, the formula can be used. To determine. Among them, Let θ represent the target classification feature of the j-th brain region, R be the number of brain region nodes in the functional brain network, and θ be the target classification feature of the j-th brain region. t Let A be a network parameter in the t-th GCN network. f This is the adjacency matrix of a functional brain network.
[0079] By using the GCN network to perform comprehensive feature analysis on the radiomics features of each brain region and the adjacency matrix between each brain region, the target classification features of each brain region are determined.
[0080] For target areas within brain imaging, a formula can be used. The target region is determined by merging the target classification features of various brain regions using the `concat()` function. Then, the merged target classification features are analyzed using an MLP network to determine the corresponding target location.
[0081] It is understandable that, when training the target part classification model, this application can use a large number of sample test sets to jointly train the 3D CNN network, GCN network and MLP network in the target part classification model, and use sample validation sets to verify the training results of the 3D CNN network, GCN network and MLP network in the target part classification model.
[0082] The number of samples in the test set and the validation set can be allocated to the corresponding training samples in a 4:1 ratio.
[0083] The technical solution provided in this application analyzes the specific structure of each brain region and the functional connectivity between different brain regions by using the radiomics features of each image block and the adjacency matrix of the functional brain network. Then, it comprehensively analyzes the target classification features of each brain region from both the specific structure of each brain region and the functional connectivity between different brain regions, ensuring the comprehensiveness of brain region features and improving the accuracy of distinguishing target brain regions.
[0084] According to one or more embodiments of this application, to some extent, the pathogenesis of a target brain region is usually due to an abnormality in some form of connectivity between related brain regions. In other words, the lesion in the target region can be determined by analyzing whether there are structural or functional connectivity abnormalities in the related brain regions. To ensure the comprehensiveness of the functional connectivity of the target region, this application can provide a detailed description of the specific steps for determining the functional connectivity of the target region.
[0085] Figure 5 This is a flowchart illustrating the method for determining the functional connectivity of a target location as shown in an embodiment of this application. Figure 5 As shown, the method may include the following steps:
[0086] S510, determine the importance of each brain region based on the proportion of the classification influence of the target region on the target area according to the target classification features of each brain region.
[0087] This application analyzes the functional connectivity of related brain regions within a target site by considering the influence of the target classification features of each brain region on the accurate classification of the target site. Furthermore, the target classification features of each brain region can be comprehensively determined by both the specific structural features of that brain region and the functional connectivity features between related brain regions. Therefore, this application can analyze the functional connectivity of the target site from two aspects: the individual influence of each brain region on the classification of the target site and the impact of the functional connectivity between different brain regions on the classification of the target site.
[0088] After identifying the target location, this application employs a partial derivative-based interpretation algorithm to calculate the classification importance of each brain region's target classification feature in the target location classification process, thereby determining which target classification feature plays a crucial role in the target location classification process.
[0089] For example, the explanatory algorithm formula for solving partial derivatives can be shown below:
[0090]
[0091] Where z represents the product of the length and width of the pixel size of the brain image, and k is the feature layer channel of the brain image. c To classify the target regions in brain images using a target region classification model. The feature importance of the i-th image block in the j-th brain region of each feature layer k in the target region classification model.
[0092] Therefore, using the above formula, the loss function in the target site classification model can be used to solve the partial derivative of the target classification features of each brain region during the target site classification process, so as to analyze the influence of the target classification features of each brain region on the target site classification and thus determine the importance of each brain region.
[0093] As an optional implementation in this application, if each brain region includes multiple image blocks, then the target site classification model for each brain region can be obtained by fusing the radiomics features of each image block belonging to each brain region with the adjacency matrix representing the functional connectivity between every two brain regions in the functional brain network. Therefore, the brain region importance of this application can be determined through the following steps: determining the first importance of each target image block in the brain region based on the proportion of the classification influence of the radiomics features of the target image blocks in each brain region towards the target site; determining the second importance of each brain region node in the functional brain network based on the proportion of the classification influence of the node feature matrix of the functional brain network towards the target site; and determining the brain region importance of each brain region based on the first importance of each target image block and the second importance of each brain region node.
[0094] In other words, by identifying the relationships between various image patches and different brain regions, the target image patch belonging to each brain region can be determined. When determining the target region using a target region classification model, for each brain region, the loss function in the target region classification model can be used to calculate the partial derivative of the radiomics features of each target image patch within that brain region during the target region classification process. This determines the proportion of influence of the radiomics features of each target image patch on the classification of the target region, and serves as the primary importance of each target image patch within that brain region.
[0095] Furthermore, by merging the radiomics features of each target image patch belonging to each brain region, the radiomics features of that brain region can be obtained. Moreover, the radiomics features of each brain region can form a node feature matrix of a functional brain network. Therefore, by using the above formula and employing the loss function in the target site classification model to solve the partial derivative of the node feature matrix of the functional brain network during the target site classification process, we can determine the proportion of influence of radiomics features of each brain region on the classification of the target site, thereby obtaining the second importance of each brain region node in the functional brain network.
[0096] Then, for each brain region, the importance of each brain region can be determined by weighting and summing the first importance of each target image block belonging to that brain region and the second importance of the nodes in that brain region.
[0097] S520: Based on the proportion of classification influence of the adjacency matrix of the functional brain network towards the target site, determine the connectivity importance of each brain region.
[0098] This application analyzes the functional connectivity of related brain regions within a target site by examining the functional connectivity between each brain region and other brain regions. The functional connectivity between each brain region and other brain regions can be represented by the adjacency matrix A of the functional brain network. f Therefore, this application can use the above formula and the loss function in the target part classification model to evaluate the adjacency matrix A of the functional brain network during the target part classification process. f By performing partial derivatives, we can determine the proportion of functional connectivity between each brain region that affects the classification of the target area, thereby determining the connectivity importance of each brain region.
[0099] S530 determines the functional connectivity of the target site based on the importance of brain regions and connectivity.
[0100] By comprehensively analyzing the importance of each brain region and the importance of its connectivity with other brain regions, we can determine the independent characteristics of each brain region and the important role of the functional connectivity between each pair of brain regions in the classification of the target site, thereby determining the functional connectivity of the target site.
[0101] As an optional implementation scheme in this application, in order to ensure the intuitiveness of the functional connectivity of the target region, this application can label the brain region nodes and node connections in the functional brain network according to the importance of the brain region and the importance of connectivity, and generate a visual functional connectivity diagram of the target region.
[0102] In other words, this application allows for the pre-setting of different annotation information, such as different annotation colors, according to the importance of each brain region. Therefore, based on the importance of each brain region, corresponding labels can be assigned to the brain region nodes in the functional brain network according to the pre-set annotation information. Furthermore, based on the connectivity importance between any two brain regions, corresponding labels can also be assigned to the connections between any two brain region nodes in the functional brain network according to the pre-set annotation information. This generates a visualized functional connectivity diagram of the target site. This visualized functional connectivity diagram allows doctors to visually and effectively assess the degree of functional connectivity impairment between relevant brain regions within the target site, providing doctors with accurate and intuitive clinical reference information for the analysis of the target site.
[0103] Taking multiple lesions in the brain as an example, if the brain image processing solution provided in this application is applied to the analysis of a certain brain lesion, then the visual functional connectivity diagram of the target area can be the visual functional connectivity diagram of the brain lesion, enabling doctors to intuitively and effectively judge the degree of functional connectivity damage in the relevant brain regions within the patient's brain lesion, thereby accurately formulating corresponding treatment plans for the brain lesion.
[0104] The technical solution provided in this application determines the classification importance of each brain region to the target site from two aspects: the importance of each brain region and the importance of connectivity between each brain region. This determines the functional connectivity of the target site and enables accurate analysis of functional connectivity abnormalities in the target site. It helps doctors effectively analyze the degree of functional connectivity damage in the target brain site, thereby providing doctors with accurate clinical reference information for the analysis of the target site.
[0105] Figure 6 This is a schematic block diagram illustrating a brain image processing device according to an embodiment of this application. Figure 6 As shown, the device 600 may include:
[0106] The feature determination module 610 is used to determine the target classification features of each brain region based on radiomics features and functional brain networks in brain images.
[0107] The target location determination module 620 is used to determine the corresponding target location based on the target classification features of each brain region.
[0108] The functional connectivity analysis module 630 is used to determine the functional connectivity of the target region based on the classification importance of each brain region toward the target region.
[0109] In some implementations, the feature determination module 610 may include:
[0110] The image block determination unit is used to determine the image blocks after the first type of brain image is segmented and the radiomics features of each image block;
[0111] The functional brain network determination unit is used to generate corresponding functional brain networks based on the distribution of brain regions in the second type of brain images.
[0112] The classification feature determination unit is used to determine the target classification features of each brain region based on the brain region node to which each image block belongs in the functional brain network and the adjacency matrix of the functional brain network.
[0113] The first type of brain imaging consists of structural brain images, while the second type consists of functional brain images.
[0114] In some implementations, the classification feature determination unit can be specifically used for:
[0115] Based on the brain region node to which each image block belongs in the functional brain network, the node feature matrix of the functional brain network is determined;
[0116] Based on the node feature matrix and the adjacency matrix of the functional brain network, the target classification features of each brain region are determined.
[0117] In some implementations, the functional connectivity analysis module 630 may include:
[0118] A brain region importance determination unit is used to determine the importance of each brain region based on the proportion of the classification influence of the target classification features of each brain region toward the target site.
[0119] The connectivity importance determination unit is used to determine the connectivity importance of each brain region based on the proportion of classification influence of the adjacency matrix of the functional brain network toward the target site.
[0120] The functional connectivity analysis unit is used to determine the functional connectivity of the target site based on the importance of the brain region and the importance of connectivity.
[0121] In some implementations, if the brain region comprises multiple image blocks, the brain region importance determination unit can be specifically used for:
[0122] The first importance of each target image block in the brain region is determined based on the proportion of the classification influence of the radiomics features of the target image block in each brain region toward the target site.
[0123] The second importance of each brain region node in the functional brain network is determined based on the classification influence ratio of the node feature matrix of the functional brain network toward the target site.
[0124] The importance of each brain region is determined based on the first importance of each target image block within each brain region and the second importance of each brain region node.
[0125] In some implementations, the functional connectivity analysis unit can be specifically used for:
[0126] Based on the importance of the brain regions and the importance of connectivity, the brain region nodes and node connections in the functional brain network are labeled accordingly to generate a visual functional connectivity diagram of the target region.
[0127] In this embodiment, the target classification features of each brain region are first determined by comprehensively analyzing radiomics features and functional brain networks within brain images to identify the corresponding target sites and achieve accurate differentiation of target sites within brain images. Furthermore, based on the classification importance of each brain region to the target site, the functional connectivity of the target site is determined, enabling accurate analysis of functional connectivity abnormalities in the target site. This helps physicians effectively analyze the degree of functional connectivity impairment in target brain sites, thereby providing precise clinical reference information for physicians' analysis of the target sites.
[0128] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 6 The apparatus 600 shown can execute any of the method embodiments in this application, and the foregoing and other operations and / or functions of each module in the apparatus 600 are respectively for implementing the corresponding processes in the various methods in the embodiments of this application. For the sake of brevity, they will not be described in detail here.
[0129] The apparatus 600 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0130] Figure 7 This is a schematic block diagram of an electronic device shown in an embodiment of this application.
[0131] like Figure 7 As shown, the electronic device 700 may include:
[0132] The system includes a memory 710 and a processor 720. The memory 710 stores computer programs and transfers the program code to the processor 720. In other words, the processor 720 can retrieve and run the computer program from the memory 710 to implement the methods described in the embodiments of this application.
[0133] For example, the processor 720 can be used to execute the above-described method embodiments according to instructions in the computer program.
[0134] In some embodiments of this application, the processor 720 may include, but is not limited to:
[0135] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0136] In some embodiments of this application, the memory 710 includes, but is not limited to:
[0137] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
[0138] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 710 and executed by the processor 720 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.
[0139] like Figure 7 As shown, the electronic device may also include:
[0140] Transceiver 730, which can be connected to processor 720 or memory 710.
[0141] The processor 720 can control the transceiver 730 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 730 may include a transmitter and a receiver. The transceiver 730 may further include antennas, and the number of antennas may be one or more.
[0142] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.
[0143] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.
[0144] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0145] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0146] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0147] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0148] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for processing brain images, characterized in that, include: Identify image blocks after segmentation of the first type of brain images and the radiomics features of each image block; the first type of brain images are structural brain images; Based on the distribution of brain regions within the second type of brain images, corresponding functional brain networks are generated; the second type of brain images are functional brain images. Based on the brain region node to which each image block belongs in the functional brain network and the adjacency matrix of the functional brain network, the target classification features of each brain region are determined. Based on the target classification features of each brain region, the corresponding target location is determined; The importance of each brain region is determined based on the proportion of the classification influence of the target classification features of each brain region toward the target site. The connectivity importance of each brain region is determined based on the proportion of classification influence of the adjacency matrix of the functional brain network toward the target site. The functional connectivity of the target site is determined based on the importance of the brain region and the importance of connectivity.
2. The method according to claim 1, characterized in that, The step of determining the target classification features of each brain region based on the brain region node to which each image block belongs in the functional brain network and the adjacency matrix of the functional brain network includes: Based on the brain region node to which each image block belongs in the functional brain network, the node feature matrix of the functional brain network is determined; Based on the node feature matrix and the adjacency matrix of the functional brain network, the target classification features of each brain region are determined.
3. The method according to claim 1, characterized in that, If the brain region comprises multiple image patches, then determining the importance of each brain region based on the classification influence percentage of the lesion classification features of each brain region towards the target site includes: The first importance of each target image block in the brain region is determined based on the proportion of the classification influence of the radiomics features of the target image block in each brain region toward the target site. The second importance of each brain region node in the functional brain network is determined based on the classification influence ratio of the node feature matrix of the functional brain network toward the target site. The importance of each brain region is determined based on the first importance of each target image block within each brain region and the second importance of each brain region node.
4. The method according to claim 1, characterized in that, Determining the functional connectivity of the target site based on the importance of the brain region and the importance of connectivity includes: Based on the importance of the brain regions and the importance of connectivity, the brain region nodes and node connections in the functional brain network are labeled accordingly to generate a visual functional connectivity diagram of the target region.
5. A brain image processing device, characterized in that, include: The feature determination module is used to determine the image blocks after segmentation of the first type of brain images and the radiomics features of each image block; generate a corresponding functional brain network based on the brain region distribution in the second type of brain images; and determine the target classification features of each brain region based on the brain region nodes to which each image block belongs in the functional brain network and the adjacency matrix of the functional brain network; the first type of brain images are structural brain images; and the second type of brain images are functional brain images. The target location determination module is used to determine the corresponding target location based on the target classification features of each brain region. The functional connectivity analysis module is used to determine the importance of each brain region based on the proportion of classification influence of the target classification features of each brain region toward the target site; to determine the connectivity importance between each brain region based on the proportion of classification influence of the adjacency matrix of the functional brain network toward the target site; and to determine the functional connectivity of the target site based on the importance of the brain region and the connectivity importance.
6. An electronic device, characterized in that, include: A processor and a memory, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the brain image processing method according to any one of claims 1-4.
7. A computer-readable storage medium, characterized in that, Used to store computer programs that cause a computer to perform a method for processing brain images as described in any one of claims 1-4.
8. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instructions implement the brain image processing method as described in any one of claims 1-4.