A multi-level data fusion method and system based on cerebrovascular knowledge

Abstract

The present invention relates to the technical field of data fusion, and discloses a multi-level data fusion method and system based on cerebrovascular knowledge. The method comprises: segmenting brain image data to obtain cerebrovascular segmentation data and performing centerline extraction on the data to obtain cerebrovascular tree centerline data; after layered processing of the cerebrovascular segmentation and cerebrovascular knowledge data, using a vascular preprocessing algorithm set to perform feature quantization to obtain corresponding layered quantized features and set their weights; combining the cerebrovascular tree centerline data, performing feature graph fusion according to the layered quantized features to obtain a data fusion feature graph, constructing a subgraph based on the position of each node, obtaining each node subgraph and performing node weight update on the subgraph, obtaining a corresponding weighted subgraph, and fusing the subgraph to obtain a node weight adjustment feature graph, forming multi-level vascular knowledge fusion data, and combining the performance of a preset model to obtain optimal multi-level vascular knowledge fusion data, thereby clarifying the optimal processing path for the multi-level vascular knowledge fusion data.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion, and in particular to a multi-level data fusion method and system based on cerebrovascular knowledge. Background Art

[0002] Existing research has focused on the utilization of three-dimensional morphological features of blood vessels and the spatial distribution of aneurysms. Related studies have confirmed the importance of surface contours for aneurysm detection, but surface three-dimensional point sets rely on accurate surface reconstruction and will lose other features except contours, making it difficult to further improve performance.

[0003] In the task of improving aneurysm detection performance, relevant scholars have proposed various methods. Although various methods have achieved improvements in detection performance, the theoretical basis for the optimal processing method for input data is still unclear. For example, it is assumed that multi-channel data increases the input data volume of the convolutional network and thus achieves performance improvement. However, when the contour and high-density area are used as parallel channels for three-channel input, no corresponding improvement is obtained, proving that this assumption is incorrect. It is also assumed that the second channel of the continuous structure enables the network to enhance the learning of the continuity characteristics of the blood vessels, similar to the effect of hard attention. However, replacing the gradient contour information with the contour of the binary result or the centerline of the blood vessel does not significantly improve the training of the model, proving that this assumption is also not valid.

[0004] In summary, during the input phase of aneurysm detection models, the processing of supervised data (consisting of input data and labeled data) is strongly correlated with the model's lesion detection performance. Researchers have explored several supervised data generation methods that can improve lesion detection sensitivity. However, existing studies often suffer from one-sided descriptions and rely heavily on manual observation and experience. The optimal processing path for supervised data remains unclear, and finding the optimal supervised data generation method remains a major challenge for researchers. Summary of the Invention

[0005] In view of this, the present invention provides a multi-level data fusion method and system based on cerebrovascular knowledge, which conducts structured and hierarchical research on cerebral arteries and cerebral aneurysms based on cerebrovascular knowledge, and forms a set of multi-level data fusion methods for cerebral artery related knowledge by studying the implementation path of the structured hierarchical method on brain imaging data, solving the problem of optimizing the knowledge representation method in knowledge-guided lesion detection, and improving the detection effect of lesions, so as to solve the technical problems raised in the above background.

[0006] In a first aspect, the present invention provides a multi-level data fusion method based on cerebrovascular knowledge, the method comprising:

[0007] Acquire brain imaging data and cerebrovascular knowledge data;

[0008] Segment the brain image data to obtain cerebral vascular segmentation data, and extract the centerline to obtain the cerebral vascular tree centerline data;

[0009] The cerebral vascular segmentation data and cerebral vascular knowledge data are processed in layers respectively;

[0010] According to the cerebrovascular knowledge data after hierarchical processing, the vascular preprocessing algorithm set is used to quantify the features of the cerebrovascular segmentation data after hierarchical processing, obtain the corresponding hierarchical quantification features and set their corresponding weights;

[0011] Combined with the cerebral vascular tree centerline data, feature map fusion is performed according to the hierarchical quantitative features to obtain the data fusion feature map;

[0012] Based on the position of each node in the data fusion feature graph, a subgraph is constructed to obtain a subgraph of each node;

[0013] The node weights of each node subgraph are updated to obtain the corresponding weighted subgraph, and the weighted subgraphs are fused to obtain the node weight adjustment feature graph to form multi-level vascular knowledge fusion data; the optimal multi-level vascular knowledge fusion data is obtained based on the performance of the multi-level vascular knowledge fusion data and the preset model.

[0014] The present invention can conduct structured and hierarchical research on cerebral arteries and cerebral aneurysms based on cerebrovascular knowledge. By studying the implementation path of the structured and hierarchical approach in brain imaging data, the optimal processing path for multi-level vascular knowledge fusion data is clarified, and the problem of optimizing the knowledge representation method in knowledge-guided lesion detection is solved, thereby improving the lesion detection effect to a certain extent.

[0015] In an optional embodiment, the cerebrovascular knowledge data is text data, including: clinical guidelines, medical-related books, clinical consultation records and medical records; the brain imaging data includes: CT images, magnetic resonance angiography images and time-of-flight magnetic resonance angiography images.

[0016] This invention fuses textual data on cerebrovascular knowledge with image data on brain images, and uses the fused data for research on lesion detection in medicine. By using this method to represent cerebrovascular knowledge alongside brain image data, the present invention ensures high-quality and diverse input data for lesion detection models, improving the model's detection accuracy to a certain extent.

[0017] In an optional embodiment, the process of segmenting brain image data to obtain cerebral vascular segmentation data, and performing centerline extraction on the data to obtain cerebral vascular tree centerline data includes:

[0018] The brain image data is processed in sequence through vascular enhancement, automatic bone removal, seed point extraction, seed point regional density distribution statistics and region growing to obtain cerebral vascular segmentation data;

[0019] The cerebral vascular segmentation data is binarized and eroded to obtain the cerebral vascular tree centerline data. Its directed graph contains the nodes, connection relationships and node types of the blood vessels. The node types include: blood vessel bifurcation points, blood vessel upper points and blood vessel end points.

[0020] The segmentation and centerline extraction processing of brain image data performed by the present invention can obtain high-precision segmentation data and cerebral vascular tree centerline data, which to a certain extent guarantees the quality of multi-level vascular knowledge fusion data.

[0021] In an optional embodiment, the process of performing hierarchical processing on the cerebral vascular segmentation data and the cerebral vascular knowledge data includes:

[0022] The cerebrovascular segmentation data and cerebrovascular knowledge data are partitioned from global to local according to the vascular structure, including: whole brain, vascular tree, arterial region, arterial segment, and lesion localization; among them, the arterial region is divided into anterior cerebral artery, middle cerebral artery, posterior cerebral artery, internal carotid artery, vertebral artery, basilar artery, anterior cerebral circulation, posterior cerebral circulation, and anterior choroidal artery;

[0023] Vessel segmentation is further performed within each region of the arterial region.

[0024] The present invention performs hierarchical processing on cerebrovascular segmentation data and cerebrovascular knowledge data, including vascular zoning and segmentation, respectively. This refines the hierarchy of knowledge representation, improving the granularity of the knowledge representation and the accuracy of the results. Furthermore, based on a knowledge hierarchical quantitative representation approach, a corresponding representation mapping between cerebrovascular knowledge points at each level and brain imaging data is obtained. Furthermore, a pre-set model is used to optimize the multi-level vascular knowledge fusion data to obtain the optimal multi-level vascular knowledge fusion data. This overcomes the existing problem of heavy reliance on manual observation and experience in the acquisition of multi-level vascular knowledge fusion data.

[0025] In an optional embodiment, based on the cerebral vascular knowledge data after layered processing, a process of performing feature quantization on the cerebral vascular segmentation data after layered processing using a vascular preprocessing algorithm set to obtain corresponding layered quantized features and set corresponding weights thereof includes:

[0026] For the image features of cerebrovascular segmentation data corresponding to all cerebrovascular knowledge data after hierarchical processing, the algorithm adapted to the vascular preprocessing algorithm set is selected to perform quantitative processing on the corresponding cerebrovascular segmentation data, thereby extracting the corresponding hierarchical quantitative features. The hierarchical quantitative features include: vascular partitioning, vascular segmentation, image feature data, and measurement feature data. The vascular preprocessing algorithm set is a collection of multiple algorithms for processing vascular images and extracting vascular features, including: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometric measurement, and filters.

[0027] The weights of the hierarchical quantitative features are set according to the hierarchical processing results of the vascular partitioning and vascular segmentation of the cerebral vascular knowledge data.

[0028] The vascular preprocessing algorithm set adopted in the present invention can obtain different hierarchical features and image features, improve the diversity of features, and improve the quality of multi-level vascular knowledge fusion data to a certain extent.

[0029] In an optional embodiment, a process of constructing a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph of each node includes:

[0030] The data fusion feature graph is separated into multiple nodes based on the location of the vascular bifurcation point to obtain a node subgraph and obtain the location information of the corresponding node. The location information includes: the vascular area, vascular segment and length from the vascular bifurcation point to which the coordinate belongs;

[0031] Position coding is performed on each separated node. The position coding is used to represent the mapping relationship between each node and the data fusion feature map. Among them, the two-dimensional position coordinates of the point on the centerline data of the cerebral vascular tree are its corresponding position coding.

[0032] Based on the prior knowledge of vascular zoning and segmentation, the present invention constructs graph data based on the features of each level, and uses multiple subgraphs containing precise location information as input data for the preset model, which can meet the input requirements of the preset model and speed up the model training process to a certain extent.

[0033] In an optional embodiment, the process of obtaining optimal multi-level vascular knowledge fusion data based on the multi-level vascular knowledge fusion data and the performance of a preset model includes:

[0034] Multi-level vascular knowledge fusion data is used as the input data of the preset model, and a weighted set of sensitivity and specificity is used as the loss function of the model. The model is trained to minimize the loss function, so that the model achieves optimal performance and obtains the optimal multi-level vascular knowledge fusion data.

[0035] The present invention determines the optimal multi-level vascular knowledge fusion data through the performance of a preset model, clarifies the optimal processing path for the multi-level vascular knowledge fusion data, solves the problem of optimizing the knowledge representation method in knowledge-guided lesion detection, and helps to improve the detection effect of lesions.

[0036] In a second aspect, the present invention provides a multi-level data fusion system based on cerebrovascular knowledge, the system comprising:

[0037] A data acquisition module is used to acquire brain imaging data and cerebrovascular knowledge data;

[0038] The data processing module is used to segment the brain image data to obtain cerebral vascular segmentation data, and extract the centerline of the data to obtain the centerline data of the cerebral vascular tree;

[0039] Data stratification module, used to perform stratification processing on cerebral vascular segmentation data and cerebral vascular knowledge data respectively;

[0040] A feature quantification module is used to perform feature quantification on the cerebral vascular segmentation data after layered processing using a vascular preprocessing algorithm set based on the cerebral vascular knowledge data after layered processing, obtain corresponding layered quantification features and set their corresponding weights;

[0041] The feature fusion module is used to combine the cerebral vascular tree centerline data and perform feature map fusion based on the hierarchical quantitative features to obtain a data fusion feature map;

[0042] A subgraph construction module is used to construct a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph of each node;

[0043] The weight optimization module is used to update the node weight of each node subgraph to obtain the corresponding weighted subgraph, fuse each weighted subgraph to obtain a node weight adjustment feature graph, and form multi-level vascular knowledge fusion data; based on the multi-level vascular knowledge fusion data and the performance of the preset model, the node weight information is obtained to obtain the optimal node weight adjustment feature graph to form the optimal multi-level vascular knowledge fusion data.

[0044] The multi-level data fusion system based on cerebrovascular knowledge in the present invention can conduct structured and hierarchical research on cerebral arteries and cerebral aneurysms based on cerebrovascular knowledge. By studying the implementation path of the structured hierarchical method on brain imaging data, the optimal processing path of multi-level vascular knowledge fusion data is clarified, the problem of optimizing the knowledge representation method in knowledge-guided lesion detection is solved, and the detection effect of lesions is improved.

[0045] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute a multi-level data fusion method based on cerebrovascular knowledge according to the first aspect or any corresponding embodiment thereof.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute a multi-level data fusion method based on cerebrovascular knowledge according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 is a flowchart of a multi-level data fusion method based on cerebrovascular knowledge according to an embodiment of the present invention;

[0049] Figure 2 2. It is a schematic structural diagram of a multi-level data fusion method based on cerebrovascular knowledge according to an embodiment of the present invention;

[0050] Figure 3 is a flowchart of another multi-level data fusion method based on cerebrovascular knowledge according to an embodiment of the present invention;

[0051] Figure 4 is a structural block diagram of a multi-level data fusion system based on cerebrovascular knowledge according to an embodiment of the present invention;

[0052] Figure 5 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0054] Intracranial aneurysms are saccular malformations caused by abnormal expansion of the lumen of the intracranial artery due to damage to the arterial wall or local congenital defects. They occur in approximately 3%-5% of the population. Rupture of an aneurysm can cause subarachnoid hemorrhage, which is associated with high mortality and permanent disability. Therefore, early detection of aneurysms and prediction of the risk of rupture are of great clinical significance.

[0055] Existing research on cerebral aneurysm detection often uses raw image data directly for model training to detect and identify cerebral aneurysms. However, as a typical small-target detection task, cerebral aneurysm detection is difficult to achieve high model training efficiency and detection accuracy using only raw brain image data due to the complex and variable structure of cerebral arteries. Related research improves data uniformity by integrating vascular knowledge into cerebral artery data, thereby enhancing the accuracy and efficiency of cerebral aneurysm detection.

[0056] In previous studies, the inventors found that prior knowledge such as aneurysms and cerebral blood vessels can improve the performance of the lesion detection model, and the forms of prior knowledge representation are very diverse. Through preliminary experiments, it was found that the detection sensitivity of the model can be improved by inputting the vascular gradient contour information and the original vascular data as parallel channels into the detection model of the fully convolutional network; if the contour information is replaced by the binarization result of the threshold segmentation of the high-density area in the center of the blood vessel, the detection sensitivity can be further improved. In addition, when the optimal processing method for the input data is still unclear, the inventors started with the labeled data and replaced the labeled input with the texture features of the area after the aneurysm was expanded, achieving a performance improvement of about 5% in sensitivity, indicating that the knowledge representation form of the label data also has an impact on the detection performance. Therefore, in the input data stage of the aneurysm detection model, processing the input data and label data based on knowledge representation can improve the model detection performance.

[0057] However, the optimal processing path for the supervised data input to the model is still unclear and the knowledge representation forms are diverse. The existing technology cannot obtain the optimal aneurysm knowledge representation method and data fusion method to construct supervised data. The present invention provides a multi-level data fusion method and system based on cerebrovascular knowledge, which conducts a knowledge structured layered study of cerebral arteries and cerebral aneurysms based on cerebrovascular knowledge, and forms a set of multi-level data fusion methods for cerebral artery related knowledge by studying the implementation path of the structured layered method on brain imaging data. It can clarify the optimal processing path of the multi-level vascular knowledge fusion data as the supervised data input to the model, solve the problem of optimizing the knowledge representation method in knowledge-guided lesion detection, and improve the detection effect of aneurysms to a certain extent.

[0058] The embodiment of the present invention provides a multi-level data fusion method based on cerebrovascular knowledge, such as Figure 1 As shown, the method includes:

[0059] Step S101: Acquire brain image data and cerebral vascular knowledge data.

[0060] In this embodiment, brain imaging data and cerebrovascular knowledge data are obtained based on clinical data and existing public databases. The brain imaging data is image data obtained by commonly used clinical imaging techniques. For example, commonly used clinical imaging techniques include: digital subtraction angiography (DSA), computed tomography angiography (CTA), magnetic resonance angiography (MRA), and time of flight magnetic resonance angiography (TOF MRA). It should be noted that the clinical data, existing public databases, and image data of the specific imaging technology used in this embodiment are not specifically limited and are determined based on actual application requirements.

[0061] Step S102 : Segment the brain image data to obtain cerebral vascular segmentation data, and perform centerline extraction on the data to obtain cerebral vascular tree centerline data.

[0062] It should be noted that cerebral vascular segmentation is a research difficulty in the field of medical image processing. Accurate cerebral vascular segmentation can provide an important basis for the analysis of cerebral aneurysm images and can be used for cerebral vascular matching and three-dimensional reconstruction.

[0063] Step S103 : performing layer-wise processing on the cerebral vessel segmentation data and the cerebral vessel knowledge data.

[0064] It should be noted that the layered processing in this embodiment includes blood vessel partitioning and blood vessel segmentation, both of which are divided according to the specific structure of the blood vessels.

[0065] Step S104 : Based on the cerebrovascular knowledge data after the hierarchical processing, a vascular preprocessing algorithm set is used to perform feature quantization on the cerebrovascular segmentation data after the hierarchical processing, to obtain corresponding hierarchical quantified features and set corresponding weights thereof.

[0066] Step S105 , combining the cerebral vascular tree centerline data, performing feature map fusion according to the hierarchical quantitative features, and obtaining a data fusion feature map.

[0067] Step S106 , constructing a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph for each node.

[0068] It should be noted that, in the process of constructing a subgraph to obtain multiple subgraphs, a node may contain one or more hierarchical quantitative features, and the specific number of hierarchical quantitative features is determined according to actual applications.

[0069] In step S107, the node weights of each node subgraph are updated to obtain the corresponding weighted subgraph, and the weighted subgraphs are fused to obtain a node weight adjustment feature graph to form multi-level vascular knowledge fusion data; the optimal multi-level vascular knowledge fusion data is obtained based on the performance of the multi-level vascular knowledge fusion data and the preset model.

[0070] It should be noted that during the performance optimization process based on the preset model, the weights of each node sub-node of one or more hierarchical quantitative features contained within each node are updated. By analyzing the weights of different sub-nodes and combining them with the optimal performance of the preset model, a corresponding node weight-adjusted feature map, i.e., the optimal multi-level vascular knowledge fusion data, is obtained. In this embodiment, the preset model is determined based on actual application requirements and is not specifically limited here.

[0071] This embodiment uses a hierarchical representation form of knowledge quantification to obtain the mapping relationship between each level of knowledge and the corresponding brain imaging data, and uses the performance of a preset model to obtain the optimal multi-level vascular knowledge fusion data. It can clarify the optimal processing path for the multi-level vascular knowledge fusion data, solve the problem of optimizing the knowledge representation method in knowledge-guided lesion detection, and improve the detection effect of cerebral aneurysms to a certain extent.

[0072] In this embodiment, the cerebrovascular knowledge data is text data, including: clinical guidelines, medical-related books, clinical consultation records and medical records; the brain imaging data includes: CT images, magnetic resonance angiography images and time-of-flight magnetic resonance angiography images.

[0073] In a specific embodiment, the aneurysm text description obtained from a medical book states that "aneurysm refers to a localized outward bulging of all three layers of the arterial wall, often occurring at arterial branches. Approximately 85% are located in the anterior circulation, 30% to 40% originate from the anterior cerebral artery or anterior communicating artery, 30% are located in the posterior communicating artery, 20% to 30% are located in the branches of the middle cerebral artery, and 5% to 10% are located in the internal carotid artery. Approximately 15% originate from the posterior circulation, including the basilar apex, superior cerebellar artery, and posterior inferior cerebellar artery. Aneurysms appear as clearly defined, circular, slightly higher-density lesions on CT, sometimes with surrounding calcifications", and time-of-flight magnetic resonance angiography (TOF MRA) images were obtained from a hospital.

[0074] It should be noted that the text data can be manually annotated and paragraph entities can be extracted. The bolded ones are the annotated entities. The specific text data is as follows:

[0075] An aneurysm is a localized outward bulge in all three layers of the arterial wall, often arising at arterial bifurcations. Approximately 85% occur in the anterior circulation, 30%–40% arise from the anterior cerebral artery or anterior communicating artery, 30% in the posterior communicating artery, 20%–30% in branches of the middle cerebral artery, and 5%–10% in the internal carotid artery. Approximately 15% arise from the posterior circulation, including the basilar apex, superior cerebellar artery, and posterior inferior cerebellar artery. On CT, an aneurysm appears as a well-defined, round, slightly hyperattenuated lesion, sometimes with surrounding calcification. This is for illustration only and is not intended to be limiting.

[0076] In this example, textual data representing cerebrovascular knowledge and image data from brain imaging are fused and used in medical lesion detection research. This approach, in which cerebrovascular knowledge representations complement brain imaging data, ensures high-quality and diverse input data for aneurysm detection models, significantly improving the model's detection accuracy.

[0077] Specifically, the above step S102 includes:

[0078] In step S1021 , the brain image data is sequentially processed with blood vessel enhancement, automatic bone removal, seed point extraction, seed point regional density distribution statistics, and region growing to obtain brain vessel segmentation data.

[0079] It should be noted that this embodiment does not limit the brain image data segmentation method, and can be adaptively adjusted according to actual application needs and data segmentation accuracy requirements.

[0080] Step S1022 , binarization and corrosion processing are performed on the cerebral vascular segmentation data to obtain cerebral vascular tree centerline data, whose directed graph contains vascular nodes, connection relationships, and node types. Node types include: vascular bifurcation points, vascular upper points, and vascular end points.

[0081] In this embodiment, by performing segmentation and centerline extraction on brain image data, high-precision segmentation data and cerebral vascular tree centerline data can be obtained, which to a certain extent ensures the quality of multi-level vascular knowledge fusion data.

[0082] Specifically, the above step S103 includes:

[0083] Step S1031, performing global to local vascular partitioning of the cerebral vascular segmentation data and cerebral vascular knowledge data according to the vascular structure, including: whole brain, vascular tree, arterial region, arterial segment, and local lesion; wherein the arterial region is divided into anterior cerebral artery, middle cerebral artery, posterior cerebral artery, internal carotid artery, vertebral artery, basilar artery, anterior cerebral circulation, posterior cerebral circulation, and anterior choroidal artery.

[0084] Step S1032: perform blood vessel segmentation again in each region of the arterial region.

[0085] In one specific embodiment, cerebrovascular segmentation image data and textual data on cerebrovascular knowledge are subjected to global-to-local vascular partitioning based on vascular structure, resulting in the whole brain, vascular tree, arterial regions, arterial segments, and local lesion regions. The arterial regions are divided into the anterior cerebral artery (ACA), middle cerebral artery (MCA), posterior cerebral artery (PCA), internal carotid artery (ICA), vertebral artery (VA), basilar artery (BA), anterior cerebral circulation (ACOM), posterior cerebral circulation (PCOM), and anterior choroidal artery (AchA). Arterial segmentation is then performed, i.e., refined vascular segmentation within each of these arterial regions. For example, the anterior cerebral artery (ACA) can be further divided into segments A1, A2, A3, and A4. This is for illustrative purposes only; vascular segmentation in other arterial regions is determined based on actual conditions.

[0086] It should be noted that in this embodiment, in addition to stratifying blood vessels according to anatomical structures, the global-level stratification processing method can also stratify blood vessels according to areas in the image, that is, non-vascular anatomical areas. For example, the stratification method of classifying according to the central block of the image and the four quadrant blocks around it is only for illustration and is not limited to this.

[0087] In this embodiment, by performing hierarchical processing on cerebrovascular segmentation data and cerebrovascular knowledge data, including vascular zoning and segmentation, respectively, the knowledge representation hierarchy is refined, improving the granularity and accuracy of the representation. Furthermore, based on a knowledge hierarchical quantitative representation approach, corresponding representation mappings between cerebrovascular knowledge points at each level and brain imaging data are obtained. Combined with a pre-set model, the multi-level vascular knowledge fusion data is optimized to obtain optimal multi-level vascular knowledge fusion data. This overcomes the existing problem of heavy reliance on manual observation and experience in obtaining multi-level vascular knowledge fusion data.

[0088] Specifically, the above step S104 includes:

[0089] Step S1041: For the image features of the cerebral vascular segmentation data corresponding to all the cerebral vascular knowledge data after the hierarchical processing, select an algorithm suitable for the vascular preprocessing algorithm set to perform quantitative processing on the corresponding cerebral vascular segmentation data to extract corresponding hierarchical quantitative features. The hierarchical quantitative features include: vascular partition, vascular segmentation, image feature data, and measurement feature data. The vascular preprocessing algorithm set is a collection of multiple algorithms for processing vascular images and extracting vascular features, including: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometric measurement, and filters.

[0090] In this embodiment, the specific composition of the vascular preprocessing algorithm set includes:

[0091] 1. Edge operators, including Canny operator, Sobel operator and Sift operator;

[0092] 2. Texture features extracted by texture feature extraction algorithm, including: Hessian matrix and Haar wavelet features;

[0093] 3. Density distribution is histogram statistics;

[0094] 4. Threshold segmentation, including: global threshold segmentation, Otsu threshold segmentation and iterative threshold segmentation;

[0095] 5. Elements of geometric measurement, including: surface curvature, projected curvature, length, and angle;

[0096] 6. Types of filters, including Gaussian filters and Laplace filters.

[0097] It should be noted that the specific composition of the vascular preprocessing algorithm set is provided for illustrative purposes only and is not intended to be limiting. It should be adjusted based on actual application needs. The vascular preprocessing algorithm set employed can capture features from different hierarchical levels and image features, increasing feature diversity and, to a certain extent, improving the quality of the multi-level vascular knowledge fusion data.

[0098] Step S1042 : setting weights of hierarchical quantitative features according to hierarchical processing results of the vascular partitions and vascular segments of the cerebral vascular knowledge data.

[0099] It should be noted that the weighting method for the layered quantitative features in this embodiment is not specifically limited and is determined based on actual application requirements. For example, in the detection of cerebral aneurysms, the primary requirement is to detect the specific location of the aneurysm lesion in the relevant brain imaging data. After the layered processing, the probability of aneurysms occurring in specific locations can be calculated, or weights can be set based on physician experience. First, based on this method, the input data for the cerebral aneurysm detection model is obtained. Specifically, the obtained relevant text data and image data are layered and processed into vascular zoning and segmentation. Based on prior knowledge of aneurysms, the layered processing results are assigned a higher weight to the aneurysm lesion location that has a greater impact on detection accuracy, while the layered processing results are assigned a lower weight to the location that has a smaller impact on detection accuracy. Specifically, the weights for each location are set based on the probability of aneurysm occurrence. For example, the probability of aneurysms located in the anterior cerebral artery (ACA) is 40%, the middle cerebral artery (MCA) is 34%, the posterior cerebral circulation (PCOM) is 20%, the posterior cerebral artery (PCA) is 4%, and other regions are 2%. After normalization based on these proportions, the weights are assigned to each region. This is for illustration only and is not intended to be limiting.

[0100] Specifically, the above step S106 includes:

[0101] In step S1061, the data fusion feature graph is separated into multiple nodes based on the location of the vascular bifurcation point to obtain a node subgraph, and the location information of the corresponding node is obtained. The location information includes: the vascular area and vascular segment to which the coordinates belong, and the length from the vascular bifurcation point.

[0102] In step S1062, position coding is performed on each separated node. The position coding is used to represent the mapping relationship between each node and the data fusion feature map. The two-dimensional position coordinates of the point on the centerline data of the cerebral vascular tree are its corresponding position coding.

[0103] This embodiment is based on the prior knowledge of vascular division and segmentation, constructs graph data using features at each level, and uses multiple subgraphs containing precise location information as input data for the preset model, which can meet the input requirements of the preset model and speed up the model training process to a certain extent.

[0104] Specifically, the process of obtaining optimal multi-level vascular knowledge fusion data based on the performance of the multi-level vascular knowledge fusion data and the preset model in the above-mentioned step S107 includes: using the multi-level vascular knowledge fusion data as input data of the preset model, using a weighted set of sensitivity and specificity as the loss function of the model, and minimizing the loss function by training the model so that the model achieves optimal performance and obtains optimal multi-level vascular knowledge fusion data.

[0105] It should be noted that the preset model of this embodiment can be a medical-related detection model or classification model. The preset model can adopt models including: a fully convolutional network model, a dual-branch network model, and a U-Net model, which are only used as examples and are not intended to be limiting.

[0106] In this embodiment, the optimal multi-level vascular knowledge fusion data is determined by the performance of the preset model, the optimal processing path of the multi-level vascular knowledge fusion data is clarified, and the problem of optimizing the knowledge representation method in knowledge-guided lesion detection is solved, which helps to improve the detection effect of lesions.

[0107] In one embodiment, see Figure 2 Based on the cerebral aneurysm detection model, data preprocessing uses the multi-level data fusion method based on cerebrovascular knowledge provided in this embodiment for data fusion processing. As can be seen from the figure, the brain image data and cerebrovascular knowledge data obtained in this embodiment are cerebral artery text knowledge data, cerebral aneurysm knowledge data, and TOFMRA image data, respectively.

[0108] The preset model of this embodiment adopts the U-Net model. The specific process of the multi-level data fusion method based on cerebrovascular knowledge is shown in Figure 3As shown. As can be seen from the figure, this embodiment utilizes cerebral vascular segmentation data and cerebral aneurysm knowledge data and their hierarchical data, combined with the cerebral artery tree centerline data and the vascular preprocessing algorithm set, to process the cerebral artery segmentation data in a hierarchical quantitative manner, and then combines the vascular skeleton feature map and related data compact data cross-fusion processing to generate sub-nodes to obtain the corresponding data fusion feature map. And separate each node in the data fusion feature map to construct each node subgraph. Update the node weight of each node subgraph to obtain the corresponding weighted subgraph, fuse each weighted subgraph to obtain the node weight adjustment feature map, and form multi-level vascular knowledge fusion data; according to the multi-level vascular knowledge fusion data and the performance of the U-Net model, the optimal multi-level vascular knowledge fusion data is obtained. Among them

[0109] In a specific embodiment, a multi-level data fusion method based on cerebrovascular knowledge includes:

[0110] 1. The input data is TOF-MRA brain image data.

[0111] 2. Input data is a text description of a cerebral aneurysm: An aneurysm is a localized outward bulge of all three layers of the arterial wall, often arising at arterial bifurcations. Approximately 85% occur in the anterior circulation, 30%–40% arise from the anterior cerebral artery or anterior communicating artery, 30% in the posterior communicating artery, 20%–30% in branches of the middle cerebral artery, and 5%–10% in the internal carotid artery. Approximately 15% arise from the posterior circulation, including the basilar apex, superior cerebellar artery, and posterior inferior cerebellar artery. On CT, an aneurysm appears as a well-defined, round, slightly hyperdense lesion, sometimes with surrounding calcification.

[0112] 3. Perform cerebral artery segmentation on TOF-MRA brain image data, that is, perform vascular enhancement, automatic bone removal, seed point extraction, seed point area density statistics, and region growing in sequence to finally obtain the cerebral vascular segmentation result.

[0113] 4. Based on the cerebral vascular segmentation results, centerline extraction is performed to obtain the centerline of the cerebral vascular tree, including the points on the centerline and the directed connection relationship between the points.

[0114] 5. Collect a set of preprocessing algorithms for cerebral arteries.

[0115] 6. Hierarchical quantitative features, including the following levels:

[0116] (1) Vascular divisions (such as anterior cerebral artery, middle cerebral artery, basilar artery, etc.);

[0117] (2) vascular segmentation (e.g., the A1 and A2 segments of the anterior artery, the M1 and M2 segments of the middle artery);

[0118] (3) Image feature data (such as contour and density);

[0119] (4) Measure characteristic data (such as length, angle, curvature, etc.).

[0120] The data is layered according to the above levels, specifically including: first classifying the knowledge corresponding to different levels in the text knowledge, and then quantifying each specific knowledge entity using a preprocessing algorithm, for example:

[0121] (1) All three layers of the arterial wall bulge outward in a limited manner. The corresponding image features are enhanced with Canny contour data; and the surface curvature data are enhanced with the measurement features.

[0122] (2) It often occurs at the branching of arteries, and the corresponding measurement features are added with surface curvature and projection curvature data.

[0123] (3) About 85% are located in the anterior circulation, 30% to 40% originate from the anterior cerebral artery or anterior communicating artery, 30% are located in the posterior communicating artery, 20% to 30% are located in the branches of the middle cerebral artery, 5% to 10% are located in the internal carotid artery, and about 15% originate from the posterior circulation, including the basilar apex, superior cerebellar artery, and posterior inferior cerebellar artery. Different weights are assigned to the different characteristics of the regions and segments according to the prior knowledge of the regions and segments.

[0124] (4) The density of circles with clear boundaries is slightly higher, which corresponds to the image features and increases the Sobel contour data.

[0125] 7. Based on the above quantitative relationship, the points on the center line are nodes, the center line is the edge between the nodes, and the quantitative features are the node information to construct the corresponding data fusion feature graph.

[0126] 8. Construct sub-graphs of the data fusion feature graph based on the vascular bifurcation points.

[0127] 9. During the separation of the subgraphs, position-related information is added, namely, the vascular area, vascular segment, and length from the bifurcation point to which the coordinates belong.

[0128] 10. Position coding: Position coding is performed based on the mapping relationship between node features and the original image; the node comes from the point on the centerline data of the cerebral vascular tree, and the two-dimensional position coordinates of the point are the position code of the node.

[0129] 11. After the sub-graph data is positionally encoded, the features + positional encoding can be concatenated into a feature map and sent to the U-net model for training. The final classification layer is removed, and the output result is the probability map generated by the Softmax layer.

[0130] 12. The probability weights of multiple positions in the probability graph corresponding to different nodes are accumulated to obtain the weighted scores of the corresponding nodes, forming a new node weighted graph; the optimal node weighted graph is determined by the sensitivity and specificity performance indicators of the U-net model.

[0131] 13. Perform dot multiplication of the optimal node weighted graph and the atomic graph to obtain the final fusion data. This embodiment solves the problem of optimizing the knowledge representation method in knowledge-guided lesion detection and improves the lesion detection effect to a certain extent.

[0132] This embodiment also provides a multi-level data fusion system based on cerebrovascular knowledge. This system is used to implement the above-mentioned embodiments and preferred implementations, and details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the systems described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0133] The present invention provides a multi-level data fusion system based on cerebrovascular knowledge, such as Figure 4 As shown, the system includes:

[0134] The data acquisition module 401 is used to acquire brain image data and cerebral vascular knowledge data.

[0135] The data processing module 402 is used to segment the brain image data to obtain cerebral vascular segmentation data, and perform centerline extraction on the data to obtain cerebral vascular tree centerline data.

[0136] The data stratification module 403 is used to perform stratification processing on the cerebral vessel segmentation data and the cerebral vessel knowledge data respectively.

[0137] The feature quantification module 404 is used to perform feature quantification on the cerebral vascular segmentation data after the layered processing using a vascular preprocessing algorithm set according to the cerebral vascular knowledge data after the layered processing, obtain corresponding layered quantification features and set their corresponding weights.

[0138] The feature fusion module 405 is used to combine the cerebral vascular tree centerline data and perform feature map fusion according to the hierarchical quantitative features to obtain a data fusion feature map.

[0139] The subgraph construction module 406 is used to construct a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph for each node.

[0140] The weight optimization module 407 is used to update the node weights of each node subgraph to obtain the corresponding weighted subgraph, fuse the weighted subgraphs to obtain a node weight adjustment feature graph, and form multi-level vascular knowledge fusion data; and obtain the node weight information based on the multi-level vascular knowledge fusion data and the performance of the preset model to obtain the optimal node weight adjustment feature graph to form the optimal multi-level vascular knowledge fusion data.

[0141] The further functional description of each of the above modules is the same as that of the corresponding embodiment above and will not be repeated here. The multi-level data fusion system based on cerebrovascular knowledge of the present invention can conduct knowledge structured layered research on cerebral arteries and cerebral aneurysms based on cerebrovascular knowledge. By studying the implementation path of the structured layered approach in brain imaging data, the optimal processing path for multi-level vascular knowledge fusion data is clarified, solving the problem of optimizing knowledge representation methods in knowledge-guided lesion detection, and improving the detection effect of lesions to a certain extent.

[0142] The present invention also provides a computer device. Figure 5 , Figure 5 is a schematic diagram of the structure of the controller provided by an optional embodiment of the present invention, such as Figure 5 As shown, the controller includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0143] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0144] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0145] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0146] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0147] The controller further includes a communication interface 30 for the main control chip to communicate with other devices or a communication network.

[0148] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor main control chip or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0149] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A multi-level data fusion method based on cerebrovascular knowledge, characterized by: The method comprises: Acquiring brain imaging data and cerebrovascular knowledge data, wherein the cerebrovascular knowledge data is text data, including clinical guidelines, medical-related books, clinical consultation records, and medical records; and the brain imaging data includes CT images, magnetic resonance angiography images, and time-of-flight magnetic resonance angiography images; Segmenting the brain image data to obtain cerebral vascular segmentation data, and extracting the centerline thereof to obtain cerebral vascular tree centerline data, including: The brain image data is processed in sequence through vascular enhancement, automatic bone removal, seed point extraction, seed point regional density distribution statistics and region growing to obtain cerebral vascular segmentation data; Binarization and erosion processing are performed on the cerebral vascular segmentation data to obtain the cerebral vascular tree centerline data. The directed graph contains the nodes, connection relationships and node types of the blood vessels. The node types include: blood vessel bifurcation points, blood vessel upper points and blood vessel end points. The cerebral vascular segmentation data and the cerebral vascular knowledge data are respectively subjected to layered processing, including: The cerebrovascular segmentation data and cerebrovascular knowledge data are partitioned from global to local according to the vascular structure, including: whole brain, vascular tree, arterial region, arterial segment, and lesion localization; among them, the arterial region is divided into anterior cerebral artery, middle cerebral artery, posterior cerebral artery, internal carotid artery, vertebral artery, basilar artery, anterior cerebral circulation, posterior cerebral circulation, and anterior choroidal artery; Further segmenting blood vessels in each region of the arterial region; According to the cerebrovascular knowledge data after hierarchical processing, the vascular preprocessing algorithm set is used to quantify the features of the cerebrovascular segmentation data after hierarchical processing, obtain the corresponding hierarchical quantification features and set their corresponding weights, including: For the image features of cerebral vascular segmentation data corresponding to all cerebral vascular knowledge data after hierarchical processing, an algorithm adapted to the vascular preprocessing algorithm set is selected to perform quantization processing on the corresponding cerebral vascular segmentation data to extract corresponding hierarchical quantitative features; the hierarchical quantitative features include: vascular partition, vascular segmentation, image feature data, and measurement feature data; the vascular preprocessing algorithm set is a collection of multiple algorithms for processing vascular images and extracting vascular features, including: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometric measurement, and filters; The weights of hierarchical quantitative features are set according to the hierarchical processing results of vascular partitioning and vascular segmentation of cerebrovascular knowledge data; Combining the cerebral vascular tree centerline data, performing feature map fusion according to the hierarchical quantitative features to obtain a data fusion feature map; Constructing a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph of each node; The node weights of each node subgraph are updated to obtain the corresponding weighted subgraph, and the weighted subgraphs are fused to obtain the node weight adjustment feature graph to form multi-level vascular knowledge fusion data; the optimal multi-level vascular knowledge fusion data is obtained based on the performance of the multi-level vascular knowledge fusion data and the preset model.

2. The multi-level data fusion method based on cerebrovascular knowledge according to claim 1 is characterized in that: The process of constructing a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph of each node includes: Separate multiple nodes of the data fusion feature graph based on the location of the vascular bifurcation point to obtain a node subgraph, and obtain the location information of the corresponding node, the location information including: the vascular area and vascular segment to which the coordinate belongs and the length from the vascular bifurcation point; Position coding is performed on each separated node, and the position coding is used to represent the mapping relationship between each node and the data fusion feature map, wherein the two-dimensional position coordinates of a point on the centerline data of the cerebral vascular tree are its corresponding position coding.

3. The multi-level data fusion method based on cerebrovascular knowledge according to claim 1 is characterized in that: The process of obtaining the optimal multi-level vascular knowledge fusion data based on the multi-level vascular knowledge fusion data and the performance of the preset model includes: Multi-level vascular knowledge fusion data is used as the input data of the preset model, and a weighted set of sensitivity and specificity is used as the loss function of the model. The model is trained to minimize the loss function, so that the model achieves optimal performance and obtains the optimal multi-level vascular knowledge fusion data.

4. A multi-level data fusion system based on cerebrovascular knowledge, characterized by: The system comprises: a data acquisition module for acquiring brain imaging data and cerebrovascular knowledge data, wherein the cerebrovascular knowledge data is text data, including clinical guidelines, medical-related books, clinical consultation records, and medical records; and the brain imaging data includes CT images, magnetic resonance angiography images, and time-of-flight magnetic resonance angiography images; The data processing module is used to segment the brain image data to obtain cerebral vascular segmentation data, and perform centerline extraction on the data to obtain cerebral vascular tree centerline data, including: The brain image data is processed in sequence through vascular enhancement, automatic bone removal, seed point extraction, seed point regional density distribution statistics and region growing to obtain cerebral vascular segmentation data; Binarization and erosion processing are performed on the cerebral vascular segmentation data to obtain the cerebral vascular tree centerline data. The directed graph contains the nodes, connection relationships and node types of the blood vessels. The node types include: blood vessel bifurcation points, blood vessel upper points and blood vessel end points. The data stratification module is used to perform stratification processing on the cerebral vascular segmentation data and the cerebral vascular knowledge data, including: The cerebrovascular segmentation data and cerebrovascular knowledge data are partitioned from global to local according to the vascular structure, including: whole brain, vascular tree, arterial region, arterial segment, and lesion localization; among them, the arterial region is divided into anterior cerebral artery, middle cerebral artery, posterior cerebral artery, internal carotid artery, vertebral artery, basilar artery, anterior cerebral circulation, posterior cerebral circulation, and anterior choroidal artery; Further segmenting blood vessels in each region of the arterial region; The feature quantification module is used to perform feature quantification on the cerebral vascular segmentation data after layered processing using a vascular preprocessing algorithm set based on the cerebral vascular knowledge data after layered processing, obtain corresponding layered quantification features and set their corresponding weights, including: For the image features of cerebral vascular segmentation data corresponding to all cerebral vascular knowledge data after hierarchical processing, an algorithm adapted to the vascular preprocessing algorithm set is selected to perform quantization processing on the corresponding cerebral vascular segmentation data to extract corresponding hierarchical quantitative features; the hierarchical quantitative features include: vascular partition, vascular segmentation, image feature data, and measurement feature data; the vascular preprocessing algorithm set is a collection of multiple algorithms for processing vascular images and extracting vascular features, including: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometric measurement, and filters; The weights of hierarchical quantitative features are set according to the hierarchical processing results of vascular partitioning and vascular segmentation of cerebrovascular knowledge data; a feature fusion module, configured to combine the cerebral vascular tree centerline data and perform feature map fusion according to the hierarchical quantitative features to obtain a data fusion feature map; A subgraph construction module, configured to construct a subgraph based on the position of each node in the data fusion feature graph to obtain a subgraph for each node; The weight optimization module is used to update the node weight of each node subgraph to obtain the corresponding weighted subgraph, fuse each weighted subgraph to obtain a node weight adjustment feature graph, and form multi-level vascular knowledge fusion data; based on the multi-level vascular knowledge fusion data and the performance of the preset model, the node weight information is obtained to obtain the optimal node weight adjustment feature graph to form the optimal multi-level vascular knowledge fusion data.

5. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the multi-level data fusion method based on cerebrovascular knowledge according to any one of claims 1 to 3 by executing the computer instructions.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the multi-level data fusion method based on cerebrovascular knowledge according to any one of claims 1 to 3.

Images