Brain nuclear magnetic image focus reasoning method based on multi-hop memory network

Abstract

The invention discloses a brain nuclear magnetic imaging focus reasoning method based on a multi-hop memory network. The method comprises the following steps: S1, acquiring brain nuclear magnetic resonance imaging data and corresponding clinical annotation data; s2, generating preprocessed brain nuclear magnetic resonance image data; s3, constructing a medical priori knowledge graph; s4, constructing a multi-hop memory network model; s5, generating an image feature vector; s6, generating an image feature vector fused with the prior knowledge; s7, storing the updated features in a memory unit; s8, performing hop-by-hop calibration on the judgment of the focus position, the focus range and the focus property, and outputting a focus reasoning result; and S9, performing fusion display with the original brain nuclear magnetic resonance image data. According to the method, remarkable innovation is achieved on the algorithm level, and meanwhile the accuracy and clinical applicability of the focus reasoning result are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of brain magnetic resonance imaging lesions, and in particular to a method for inferring brain magnetic resonance imaging lesions based on a multi-hop memory network. Background Technique

[0002] With the rapid development of medical imaging technology and artificial intelligence, brain magnetic resonance imaging, as an important tool for diagnosing brain diseases, has become a widely used imaging method in the medical field due to its non-invasive, high-resolution, and multi-modal characteristics. However, in the face of the increasing amount of imaging data and complex lesion characteristics, traditional imaging analysis and lesion inference techniques still face many challenges in practical applications.

[0003] Currently, the analysis of brain magnetic resonance imaging usually relies on experienced radiologists to identify the specific location, scope, and nature of lesions through manual film reading. Although it can combine the professional knowledge and experience of doctors, there are also obvious limitations: the efficiency of manual film reading is relatively low, and it is easy to cause missed diagnosis or misdiagnosis due to visual fatigue when dealing with a large number of sliced imaging data. At the same time, there may be significant subjective differences in the diagnostic results among different doctors, making it difficult to ensure the consistency of diagnostic results. In addition, for small lesions or atypical lesions, doctors' judgments often rely on subjective experience, which further limits the reliability of manual diagnosis.

[0004] To make up for the deficiencies of manual diagnosis, in recent years, image analysis methods based on artificial intelligence have gradually been applied to the processing and inference of brain magnetic resonance imaging. Traditional algorithms usually extract image features by constructing convolutional neural networks or other machine learning models, and classify or segment lesions. There are still the following problems in practical applications:

[0005] 1. Insufficient depth of feature extraction: Most existing deep learning models only rely on single extraction of image features, making it difficult to capture the complex spatial relationships and semantic information of lesions, resulting in insufficient accuracy of lesion inference.

[0006] 2. Lack of integration of medical knowledge: Traditional models usually process image data independently and fail to effectively combine medical prior knowledge (anatomical structure, disease association). Therefore, their inference ability for complex lesions is limited, especially when dealing with rare diseases or lesions with strong heterogeneity.

[0007] 3. Opaque inference process: Existing technologies usually output inference results through a single network, lacking iterative calibration of the inference process, which is likely to miss small lesions or atypical features and affect the credibility of clinical diagnosis.

[0008] In summary, the existing technologies have significant deficiencies in the depth of image feature extraction, the effective integration of medical knowledge, and the accuracy and transparency of lesion reasoning, making it difficult to meet the clinical requirements for efficient and accurate diagnosis. These technical deficiencies not only limit the breadth of application of artificial intelligence in medical image analysis but also directly affect the diagnostic efficiency of doctors and the treatment outcomes of patients. There is an urgent need for a new method that combines deep feature extraction, multi-hop reasoning, and medical knowledge fusion to solve the above problems. Summary of the Invention

[0009] An object of the present invention is to propose a method for lesion reasoning of brain magnetic resonance imaging based on a multi-hop memory network. The present invention not only achieves significant innovation at the algorithm level but also significantly improves the accuracy and clinical applicability of the lesion reasoning results.

[0010] A method for lesion reasoning of brain magnetic resonance imaging based on a multi-hop memory network according to an embodiment of the present invention includes the following steps:

[0011] S1. Obtain brain magnetic resonance imaging data and corresponding clinical annotation data;

[0012] S2. Perform image denoising, artifact correction, spatial registration, image enhancement, and data normalization on the brain magnetic resonance imaging data to generate preprocessed brain magnetic resonance imaging data;

[0013] S3. Construct a medical prior knowledge graph, which includes brain anatomical structures, lesion locations of common diseases, the relationship between lesion characteristics and disease types, and corresponding clinical annotation data;

[0014] S4. Based on the medical prior knowledge graph and the preprocessed brain magnetic resonance imaging data, construct a multi-hop memory network model, including an image feature extraction module, a memory update module, a knowledge graph embedding module, and a multi-hop reasoning module;

[0015] S5. Perform multi-layer feature encoding on the preprocessed brain magnetic resonance imaging data through the image feature extraction module to extract multi-scale spatial features and lesion-related features of the brain magnetic resonance imaging data, and generate an image feature vector;

[0016] S6. Input the image feature vector into the knowledge graph embedding module, and combine the anatomical structures and disease association features in the medical prior knowledge graph to perform semantic association embedding on the image feature vector, and generate an image feature vector fused with prior knowledge;

[0017] S7. Input the image feature vector fused with prior knowledge into the memory update module, use the attention mechanism to screen and update the key regions of the current feature vector, and store the updated feature in the memory unit;

[0018] S8. Based on the multi-hop reasoning module, perform multiple iterative inferences on the image features stored in the memory unit, and in combination with the structural relationships of the medical prior knowledge graph, calibrate the judgments of the lesion location, lesion range, and lesion nature hop by hop, and output the lesion inference result;

[0019] S9. Generate a lesion annotation map of the brain magnetic resonance image according to the lesion inference result. The lesion annotation map includes the spatial location, size range, and disease type information of the lesion, and is fused and displayed with the original brain magnetic resonance image data.

[0020] Optionally, the S1 specifically includes:

[0021] S11. Obtain brain magnetic resonance image data from the medical image database:

[0022] I = {I T1 , I T2 , I FLAIR};

[0023] Among them, I T1 is the T1-weighted imaging data, I T2 is the T2-weighted imaging data, I FLAIR is the FLAIR imaging data;

[0024] S12. Obtain the clinical annotation data corresponding to the brain magnetic resonance image data:

[0025] A = {A 1 , A 2 , …, A n};

[0026] Among them, A i represents the lesion location and lesion type annotation on the i-th image slice;

[0027] S13. Match the brain magnetic resonance image data I with the corresponding clinical annotation data A. The matching basis includes the serial number of the image slice, the scanning time, and the spatial coordinates of the annotation area, and generate the matched brain magnetic resonance image data pair:

[0028] D = {(I i , A i )}.

[0029] Optionally, the S2 specifically includes:

[0030] S21. Use the non-local means denoising algorithm to perform image denoising processing on the brain magnetic resonance image data I, remove the random noise in the image data by analyzing the local similarity of each pixel point, and generate the denoised brain magnetic resonance image data;

[0031] S22. Perform artifact correction on the denoised brain MRI data, identify the frequency-domain characteristics of artifacts using Fourier transform, and remove the artifact components through frequency-domain filtering to generate the artifact-corrected brain MRI data;

[0032] S23. Perform spatial registration processing on the artifact-corrected brain MRI data using a registration method based on affine transformation, adjust the rotation, translation, and scaling parameters of the image according to the spatial relationship between image slices, and generate the spatially registered brain MRI data;

[0033] S24. Perform image enhancement processing on the spatially registered brain MRI data, optimize the contrast distribution of the image using the histogram equalization algorithm, and generate the image-enhanced brain MRI data by enhancing the global uniformity of the gray level;

[0034] S25. Perform normalization processing on the image-enhanced brain MRI data, normalize the pixel values of the image data to the standardized range [0, 1], and generate the normalized brain MRI data:

[0035] I norm ={I norm,T1 ,I norm,T2 ,I norm,FLAIR}}.

[0036] Optionally, the specific steps of S3 include:

[0037] S31. Based on the brain MRI data, collect the brain anatomical structure dataset E norm , the lesion location dataset E anat of common diseases, the dataset E loc on the relationship between lesion features and disease types, and the corresponding clinical annotation dataset E feat-type ; label ;

[0038] S32. Represent the anatomical structure entity e anat,j as a vector v anat,j , represent the lesion location entity e loc,k as a vector v loc,k , represent the entity e feat-type,l of the relationship between lesion features and disease types as a vector v feat-type,l , and represent the clinical annotation entity e label,u as a vector v label,u . All embedding vectors are defined in , where h is the embedding dimension;

[0039] S33. Define the set of relationships R between entities in the medical prior knowledge graph, including the anatomical structure association relationship r anat-rel , the location association relationship r loc-rel , and the feature - type association relationship r feat-type-rel . Define a transformation matrix for each relationship such that for an associated entity pair (e x , e y ) there is a corresponding relationship embedding v r (e x , e y ) = W r v x , where v x is the vector embedding of entity e x ;

[0040] S34. Construct a mapping function f from image coordinates to anatomical structure and lesion entities for the spatial coordinates in the brain magnetic resonance imaging data I norm . The mapping function maps the image coordinates (x, y, z) to specific anatomical structure entities and lesion location entities, establishing a corresponding relationship between the mapped entity embedding v map and v anat,j and the normalized image features; loc,k ;

[0041] S35. Based on the corresponding lesion type and feature annotation information in the clinical annotation data, establish an association relationship between the corresponding lesion features and disease type relationship entities v feat-type,l and clinical annotation entities v label,u in the medical prior knowledge graph, and iteratively calibrate the association information by updating the relationship transformation matrix W r to make the association relationship R between entities in the medical prior knowledge graph dynamically adapt to the characteristics of clinical annotation data;

[0042] S36. Obtain the medical prior knowledge graph constructed through embedding and association relationship construction:

[0043] G opt = (E, R, V, W);

[0044] where, E = E anat ∪ E loc ∪ E feat-type ∪ E label is the set of entities, R is the set of relationships, V is the set of entity embedding vectors, and W is the set of relationship transformation matrices.

[0045] Optionally, the specific steps of S5 include:

[0046] S51. Normalize the brain magnetic resonance imaging data I normIn the first-layer feature encoding of the image feature extraction module of the input multi-hop memory network model, local feature extraction is performed in the multi-channel dimension to capture the initial spatial texture and potential lesion structure information, and the initial low-level structural features are extracted:

[0047]

[0048] Among them, α, β, γ are the spatial coordinates of the feature map, τ is the input channel index, corresponding to the multi-modal channel of the posterior brain magnetic resonance imaging data I norm The multi-modal channel of 1 k is the size of the first-layer convolution kernel, c is the number of input channels, μ, ν are the spatial offsets of the convolution kernel; is the weight parameter of the first-layer convolution kernel, b (1) is the first-layer bias term, φ(·) is the non-linear activation function;

[0049] S52. Input the initial low-level structural features F (1) into the second-layer feature encoding, perform multi-scale convolution feature extraction on it, and capture different-scale features of the lesion area at the multi-scale level to generate a multi-scale feature map set {F (2,1) , F (2 ,2) , …, F (2,M)}:

[0050]

[0051] Among them, m = 1, …, M is the multi-scale index, d is the number of channels after the first-layer feature map, k 2,m is the size of the m-scale convolution kernel in the second layer, is the weight parameter of the m-scale convolution kernel, b (2,m) is the m-scale bias term, ρ, σ are the spatial offsets of the m-scale convolution kernel;

[0052] S53. Concatenate the obtained multi-scale feature map set along the channel dimension to effectively fuse features of different scales in the same representation space, and obtain the comprehensive feature map F (c) :

[0053]

[0054] Among them, f cat (·) is the channel concatenation operation, d m is the number of channels of F (2,m) ;

[0055] S54. For the comprehensive feature map F (c)Perform global spatial pooling and combine with the spatial weighting function M(α,β,γ) to focus on aggregating the lesion regions of interest, and obtain the final image feature vector v img :

[0056]

[0057] wherein, is the channel index, X, Y, Z are the spatial dimensions of the comprehensive feature map F (c) , |Ω| is the normalization factor, and M(α,β,γ) is the spatial weighting function, which is used to highlight the key lesion regions so that they have higher weights in the global representation.

[0058] Optionally, the S6 specifically includes:

[0059] S61. Input the image feature vector into the knowledge graph embedding module. Taking the image feature vector as the leading factor, through constructing a dynamic semantic mapping mechanism, initially associate the image features with the anatomical structure embedding vector and the disease association feature embedding vector in the medical prior knowledge graph, and generate an initial semantic vector containing anatomical and lesion semantic associations;

[0060] S62. According to the uneven distribution of the image features in the semantic space in the optimized medical prior knowledge graph, through the multi-dimensional association analysis mechanism, perform deep semantic enhancement on the initial semantic vector, extract high-order association relationships from the medical prior knowledge graph, and perform dynamic mapping and update of the image features with multi-level anatomical structures, lesion features, and disease types, and generate an enhanced semantic association embedding;

[0061] S63. On the basis of the enhanced semantic association embedding, introduce a weight normalization method based on dynamic attention distribution, analyze the spatio-temporal matching of the image features and the relevant entity embeddings in the medical prior knowledge graph, assign adaptive importance weights to different entities, optimize the influence of important medical entities in the final features, and generate a weight-normalized embedding vector that fuses the image features and the knowledge graph features;

[0062] S64. Input the weight-normalized embedding vector into the feature integration and optimization layer, perform multi-dimensional optimization on the fused features through the semantic difference compensation and feature conflict resolution mechanism, and finally generate an image feature vector with multi-level semantic association capabilities.

[0063] Optionally, the S8 specifically includes:

[0064] S81. Use the image feature vector with multi-level semantic association capabilities as the multi-hop reasoning initial query vector q (0) , and obtain the updated feature memory set from the memory unit:

[0065] M(0) = {m 1 , m 2 , …, m Nm};

[0066] Among them, represents the feature memory vector related to the brain MRI lesion, including the knowledge embedding related to the anatomical structure, lesion location, lesion characteristics, and disease type;

[0067] S82. Calculate the correlation degree between the initial query vector q (t-1) and the feature memory set M (t-1) in the t-th inference iteration, obtain the attention weight through the attention mechanism, and perform weighted summation on the memory to form the updated context vector c (t) :

[0068]

[0069] Among them, is the memory vector stored after the (t - 1)-th iteration of the memory unit, and this memory vector implicitly contains the feature information of the lesion location, scope, and nature in the brain MRI. q (t-1) is the updated query vector after the previous iteration, which is used to guide the attention allocation of the current iteration. is the attention weight, which is used to emphasize the attention to a specific memory vector;

[0070] S83. Use the updated context vector c (t) to update the query vector q (t-1) , generate the query vector q (t) for the next iteration, and perform multi-dimensional reconstruction of the features during the update process to calibrate the judgment of the lesion characteristics hop by hop;

[0071] S84. After T multi-hop inference iterations, obtain the final query vector q (T) , input q (T) into the output mapping function f dec (·), and decode the lesion location, lesion scope, and lesion nature information from the final query vector through this function to generate the final lesion inference result:

[0072] L pos , L rng , L typ = f dec (q (T) );

[0073] Among them, L pos represents the inferred lesion spatial location, and L rng represents the lesion spatial scope information, and Ltyp Indicates the information on the nature category of the lesion.

[0074] The beneficial effects of the present invention are as follows:

[0075] (1) Through the multi-hop memory network model, the present invention deeply combines image features with the medical prior knowledge graph by means of multiple iterative inferences, enabling the output result of each inference to dynamically calibrate the lesion features, and gradually improving the judgment accuracy of the lesion location, scope, and nature. Traditional single-inference models often tend to miss important features or produce misjudgments when dealing with complex or tiny lesions. The multi-hop memory network of the present invention utilizes multiple memory update mechanisms to progressively excavate the deep semantic associations of lesion features, and can accurately identify tiny lesions and atypical lesions in complex image data.

[0076] (2) The present invention introduces a medical prior knowledge graph to deeply integrate image features with the anatomical structure, lesion location, and disease association features in medical knowledge, and realizes the semantic association embedding of image features and prior knowledge through the knowledge graph embedding module, enhancing the interpretability and robustness of the inference process. Traditional image analysis models usually only rely on image data for feature extraction and are difficult to effectively combine the structured knowledge in the medical field, so they perform inadequately when dealing with lesion features with strong heterogeneity. The present invention dynamically associates and enhances the high-order relationships of image features through the optimized knowledge graph, enabling the inference model to more accurately judge the characteristics of complex lesions under the guidance of medical knowledge, and improving the reliability and clinical applicability of the inference results.

[0077] (3) The present invention introduces a dynamic attention mechanism and a weight normalization strategy into the multi-hop memory network, dynamically adjusts the attention distribution according to the importance of different lesion regions, enabling the inference process to focus more on key lesion regions. Through the weight normalization method, according to the correlation degree between image features and entity embeddings in the knowledge graph, higher weights are assigned to important entities, thereby amplifying the influence of key features in the inference results. Compared with the inference errors caused by the uniform processing of all features in traditional methods, the present invention effectively reduces the feature confusion problem through the gradual calibration of the lesion location and scope, and improves the spatial resolution and semantic accuracy of the inference results. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0079] Figure 1 is a flowchart of a method for inferring brain MRI lesions based on a multi-hop memory network proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0080] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0081] Reference Figure 1 , a method for inferring brain MRI lesions based on a multi-hop memory network, comprising the following steps:

[0082] S1. Obtain brain magnetic resonance imaging data and corresponding clinical annotation data;

[0083] S2. Perform image denoising, artifact correction, spatial registration, image enhancement, and data normalization on the brain magnetic resonance imaging data to generate preprocessed brain magnetic resonance imaging data;

[0084] S3. Construct a medical prior knowledge graph, which includes brain anatomical structures, lesion locations of common diseases, the relationship between lesion characteristics and disease types, and corresponding clinical annotation data;

[0085] S4. Based on the medical prior knowledge graph and the preprocessed brain magnetic resonance imaging data, construct a multi-hop memory network model, including an image feature extraction module, a memory update module, a knowledge graph embedding module, and a multi-hop reasoning module;

[0086] S5. Perform multi-layer feature encoding on the preprocessed brain magnetic resonance imaging data through the image feature extraction module, extract multi-scale spatial features and lesion-related features of the brain magnetic resonance imaging data, and generate an image feature vector;

[0087] S6. Input the image feature vector into the knowledge graph embedding module, and combine the anatomical structure and disease association features in the medical prior knowledge graph to perform semantic association embedding on the image feature vector, generating an image feature vector fused with prior knowledge;

[0088] S7. Input the image feature vector fused with prior knowledge into the memory update module, use the attention mechanism to screen and update the key regions of the current feature vector, and store the updated feature in the memory unit;

[0089] S8. Based on the multi-hop reasoning module, perform multiple iterative inferences on the image features stored in the memory unit, and combine the structural relationship of the medical prior knowledge graph to calibrate the judgment of the lesion location, lesion range, and lesion nature hop by hop, and output the lesion inference result;

[0090] S9. Generate a lesion annotation map of the brain magnetic resonance imaging according to the lesion inference result. The lesion annotation map includes the spatial location, size range, and disease type information of the lesion, and is fused and displayed with the original brain magnetic resonance imaging data.

[0091] In this embodiment, S1 specifically includes:

[0092] S11. Obtain brain magnetic resonance imaging data from a medical image database:

[0093] I = {I T1 , I T2 , I FLAIR};

[0094] where I T1 is T1-weighted imaging data, I T2 is T2-weighted imaging data, I FLAIR is FLAIR imaging data;

[0095] S12. Obtain clinical annotation data corresponding to the brain magnetic resonance imaging data:

[0096] A = {A 1 , A 2 , …, A n};

[0097] where A i represents the lesion location and lesion type annotation on the i-th image slice;

[0098] S13. Match the brain magnetic resonance imaging data I with the corresponding clinical annotation data A. The matching basis includes the serial number of the image slice, the scanning time, and the spatial coordinates of the annotation area, and generate a pair of matched brain magnetic resonance imaging data:

[0099] D = {(I i , A i )}.

[0100] In this embodiment, S2 specifically includes:

[0101] S21. Perform image denoising processing on the brain magnetic resonance imaging data I using the non-local means denoising algorithm, and remove random noise in the image data by analyzing the local similarity of each pixel point to generate denoised brain magnetic resonance imaging data;

[0102] S22. Perform artifact correction on the denoised brain magnetic resonance imaging data. Use Fourier transform to identify the frequency domain characteristics of artifacts, and remove the artifact components through frequency domain filtering to generate artifact-corrected brain magnetic resonance imaging data;

[0103] S23. Perform spatial registration processing on the artifact-corrected brain magnetic resonance imaging data using an affine transformation-based registration method, and adjust the rotation, translation, and scaling parameters of the image according to the spatial relationship between the image slices to generate spatially registered brain magnetic resonance imaging data;

[0104] S24. Perform image enhancement processing on the spatially registered brain MRI image data, optimize the contrast distribution of the image using the histogram equalization algorithm, and generate the enhanced brain MRI image data by improving the global balance of the gray levels.

[0105] S25. Perform normalization processing on the enhanced brain MRI image data, normalize the pixel values of the image data to the standardized range [0,1], and generate the normalized brain MRI image data:

[0106] I norm ={I norm,T1 ,I norm,T2 ,I norm,FLAIR}}.

[0107] In this embodiment, S3 specifically includes:

[0108] S31. Based on the brain MRI image data, collect the brain anatomical structure dataset E norm , the lesion location dataset E anat of common diseases, the dataset E loc of the relationship between lesion features and disease types, and the corresponding clinical annotation dataset E feat-type ; label

[0109] S32. Represent the anatomical structure entity e anat,j as a vector v anat,j , represent the lesion location entity e loc,k as a vector v loc,k , represent the relationship entity e feat-type,l between lesion features and disease types as a vector v feat-type,l , represent the clinical annotation entity e label,u as a vector v label,u , and all embedding vectors are defined in , where h is the embedding dimension;

[0110] S33. Define the set of relationships R between entities in the medical prior knowledge graph, including the anatomical structure association relationship r anat-rel , the location association relationship r loc-rel , and the feature - type association relationship r feat-type-rel . Define a transformation matrix for each relationship such that for the associated entity pair (e x , e y ) there is a corresponding relationship embedding v r (e x , e y ) = W r vx , where v x is the vector embedding of entity e x ;

[0111] S34. Construct a mapping function f from image coordinates to anatomical structures and lesion entities for the spatial coordinates in the brain MRI image data I norm , and the mapping function maps the image coordinates (x, y, z) to specific anatomical structure entities and lesion location entities, so that the mapped entity embedding v map establishes a correspondence with v anat,j and between v loc,k and the normalized image features;

[0112] S35. According to the corresponding lesion type and feature annotation information in the clinical annotation data, establish an association relationship between the corresponding lesion features and disease type entities v feat-type,l and the clinical annotation entity v label,u in the medical prior knowledge graph, and iteratively calibrate the association information by updating the relationship transformation matrix W r to make the association relationship R between entities in the medical prior knowledge graph dynamically adapt to the characteristics of clinical annotation data;

[0113] S36. Obtain the medical prior knowledge graph constructed through embedding and association relationship:

[0114] G opt =(E, R, V, W);

[0115] where E = E anat ∪E loc ∪E feat-type ∪E label is the entity set, R is the relationship set, V is the entity embedding vector set, and W is the relationship transformation matrix set.

[0116] In this embodiment, S5 specifically includes:

[0117] S51. Input the normalized brain MRI image data I norm into the first-layer feature encoding of the image feature extraction module of the multi-hop memory network model, perform local feature extraction in the multi-channel dimension, capture the initial spatial texture and potential lesion structure information, and extract the initial low-level structure features:

[0118]

[0119] where α, β, γ are the spatial coordinates of the feature mapping, τ is the input channel index, corresponding to the multi-modal channels of the posterior brain MRI image data I norm , and k 1is the size of the first-layer convolutional kernel, c is the number of input channels, and μ, ν are the spatial offsets of the convolutional kernel; is the weight parameter of the first-layer convolutional kernel, b (1) is the first-layer bias term, and φ(·) is the non-linear activation function;

[0120] S52. Input the initial low-level structural feature F (1) into the second-layer feature encoding, perform multi-scale convolutional feature extraction on it, and capture different-scale features of the lesion area at the multi-scale level to generate a multi-scale feature map set {F (2,1) , F (2 ,2) , …, F (2,M) }:

[0121]

[0122] where m = 1, …, M is the multi-scale index, d is the number of channels after the first-layer feature map, and k 2,m is the size of the m-th scale convolutional kernel in the second layer, is the weight parameter of the m-th scale convolutional kernel, b (2,m) is the m-th scale bias term, and ρ, σ are the spatial offsets of the m-th scale convolutional kernel;

[0123] S53. Concatenate the obtained multi-scale feature map set along the channel dimension to effectively fuse features of different scales in the same representation space, and obtain the comprehensive feature map F (c) :

[0124]

[0125] where f cat (·) is the channel concatenation operation, and d m is the number of channels of F (2,m) ;

[0126] S54. Perform global spatial pooling on the comprehensive feature map F (c) and combine the spatial weighting function M(α, β, γ) to focus on aggregating the lesion areas of interest, and obtain the final image feature vector v img :

[0127]

[0128] where, is the channel index, X, Y, Z are the spatial dimensions of the comprehensive feature map F (c) , |Ω| is the normalization factor, and M(α, β, γ) is the spatial weighting function, which is used to highlight the key lesion areas so that they have higher weights in the global representation.

[0129] In this embodiment, S6 specifically includes:

[0130] S61. Input the image feature vector into the knowledge graph embedding module. Taking the image feature vector as the leading factor, through constructing a dynamic semantic mapping mechanism, initially associate the image features with the anatomical structure embedding vector and the disease association feature embedding vector in the medical prior knowledge graph, and generate an initial semantic vector containing the semantic association between anatomy and lesions.

[0131] S62. According to the uneven distribution of image features in the semantic space in the optimized medical prior knowledge graph, perform deep semantic enhancement on the initial semantic vector through a multi-dimensional association analysis mechanism, extract high-order association relationships from the medical prior knowledge graph, and perform dynamic mapping and update on the image features with multi-level anatomical structures, lesion features, and disease types to generate an enhanced semantic association embedding.

[0132] S63. On the basis of the enhanced semantic association embedding, introduce a weight normalization method based on dynamic attention distribution, analyze the spatio-temporal matching of image features with relevant entity embeddings in the medical prior knowledge graph, assign adaptive importance weights to different entities, optimize the influence of important medical entities in the final features, and generate a weight-normalized embedding vector that fuses image features and knowledge graph features.

[0133] S64. Input the weight-normalized embedding vector into the feature integration and optimization layer, perform multi-dimensional optimization on the fused features through a semantic difference compensation and feature conflict resolution mechanism, and finally generate an image feature vector with multi-level semantic association capabilities.

[0134] In this embodiment, S8 specifically includes:

[0135] S81. Take the image feature vector with multi-level semantic association capabilities as the multi-hop reasoning initial query vector q (0) , and obtain the updated feature memory set from the memory unit:

[0136] M (0) ={m 1 ,m 2 ,…,m Nm};

[0137] Among them, represents the feature memory vector related to the lesions of brain MRI images, including the knowledge embeddings related to anatomical structures, lesion locations, lesion features, and disease types;

[0138] S82. Calculate the degree of association between the initial query vector q (t-1) and the feature memory set M (t-1) in the t-th inference iteration, and obtain the attention weight through the attention mechanism And perform weighted summation on the memories to form an updated context vector c (t) :

[0139]

[0140] Wherein, is the memory vector stored after the (t - 1)-th iteration of the memory unit, and this memory vector implicitly contains the feature information of the lesion location, scope, and nature in the brain MRI, q (t-1) is the query vector updated after the previous iteration, which is used to guide the attention allocation of the current iteration, is the attention weight, which is used to emphasize the attention degree to a specific memory vector;

[0141] S83. Use the updated context vector c (t) to update the query vector q (t-1) to generate the query vector q (t) for the next iteration, and perform multi-dimensional reconstruction of the features during the update process to calibrate the judgment of the lesion characteristics hop by hop;

[0142] S84. After T multi-hop inference iterations, obtain the final query vector q (T) , and input q (T) into the output mapping function f dec (·), and decode the lesion location, lesion scope, and lesion nature information from the final query vector through this function to generate the final lesion inference result:

[0143] L pos , L rng , L typ = f dec (q (T) );

[0144] Wherein, L pos represents the inferred lesion spatial location, L rng represents the spatial scope information of the lesion, and L typ represents the lesion nature category information.

[0145] Example 1:

[0146] In the radiology department of a tertiary hospital in a provincial capital city, a 45-year-old male patient came to see a doctor due to persistent headache, nausea, and mild memory impairment. The doctor arranged a brain magnetic resonance imaging examination for him, generating multi-modal image data of the patient, including T1-weighted images, T2-weighted images, and FLAIR images. The images showed a suspected abnormal area in the left frontal lobe. However, due to the blurred boundary and uneven signal, it was difficult for the doctor to accurately determine the lesion scope and nature with the naked eye.

[0147] To improve diagnostic accuracy, the hospital adopted the brain MRI lesion reasoning method based on multi-hop memory network of the present invention. After the imaging data was input into the system, the system first processed the patient's imaging data through denoising and artifact correction steps. The system prompted that the imaging process was completed and displayed a data quality assessment report: the data integrity score was 96%, and the data noise level score was 90%. After the doctor confirmed the preprocessing quality, the system automatically entered the imaging feature extraction stage.

[0148] The feature extraction module performed multi-scale encoding on the T1-weighted image and found multiple high-signal areas in the left frontal lobe in the preliminary results; the T2-weighted image showed that the boundaries of these areas presented irregular shapes, while the FLAIR image captured a suspected edema phenomenon. The system fused the extracted imaging features and performed embedding association on them through a medical knowledge graph. During the knowledge graph query process, the system prompted: "The correlation degree with the features of grade II and grade III gliomas is 87%, and the correlation degree with metastatic tumors is 45%."

[0149] Subsequently, the system entered the multi-hop reasoning stage. In the first reasoning iteration, the attention mechanism focused on the central position of the lesion in the left frontal lobe. The system recorded the preliminary reasoning result: the estimated lesion range was 15.6 cubic centimeters, and the boundary did not completely cover the signal abnormal area. The system displayed: "The reasoning confidence level is 73%, and the next iteration is started."

[0150] In the second iteration, the system dynamically adjusted the attention weights and expanded and calibrated the lesion boundary by combining the edema features of the FLAIR image. The reasoning result showed: "The lesion range is updated to 18.4 cubic centimeters, and the morphology conforms to the features of grade III glioma, and the confidence level is increased to 91%."

[0151] Finally, in the third iteration, the system fused the anatomical structure and feature distribution information, further confirmed the lesion location, and marked the areas where adjacent important functional areas might be affected. After the reasoning ended, the system automatically generated a diagnostic report, the content of which included:

[0152] Lesion location: near the precentral gyrus in the left frontal lobe;

[0153] Lesion range: 18.4 cubic centimeters;

[0154] Lesion nature: suspected grade III glioma;

[0155] Risk warning: The lesion may affect the motor cortex area.

[0156] The system also generated a 3D visualization image, with the lesion area highlighted in red and the surrounding edema area marked in yellow. The doctor further analyzed the patient in combination with the system report and confirmed the accuracy of the system reasoning result.

[0157] To evaluate the efficiency of the method of the present invention, the hospital compared the performance of the method of the present invention with traditional methods (manual film reading by doctors and traditional neural network analysis). During a one-month trial, the system processed 100 suspected cases of glioma, and the results are as follows:

[0158] Table 1 Comparative analysis of lesion inference methods for brain MRI images

[0159]

[0160] During the actual treatment process of this patient, the doctor quickly formulated a treatment plan based on the report generated by the method of the present invention. The postoperative pathological results showed that the patient's lesion was grade III glioma, and the scope was highly consistent with the system inference result. No lesions were missed during the postoperative review.

[0161] In summary, this embodiment not only verifies the superiority of the present invention in complex brain lesion inference, but also demonstrates its significant application value in actual clinical scenarios, providing reliable technical support for doctors' diagnosis and treatment.

[0162] The present invention uses a multi-hop memory network model to deeply combine image features with a medical prior knowledge graph through multiple iterative inferences, enabling the output result of each inference to dynamically calibrate the lesion features and gradually improving the judgment accuracy of the lesion location, scope, and nature. Traditional single-inference models often easily miss important features or produce misjudgments when dealing with complex or tiny lesions. The multi-hop memory network of the present invention uses multiple memory update mechanisms to progressively excavate the deep semantic associations of lesion features and can accurately identify tiny lesions and atypical lesions in complex image data.

[0163] The present invention introduces a medical prior knowledge graph to deeply integrate image features with anatomical structures, lesion locations, and disease-associated features in medical knowledge, and realizes the semantic association embedding of image features and prior knowledge through a knowledge graph embedding module, enhancing the interpretability and robustness of the inference process. Traditional image analysis models usually only rely on image data for feature extraction and are difficult to effectively combine structured knowledge in the medical field, so they perform poorly when dealing with lesion features with strong heterogeneity. The present invention dynamically associates and enhances the high-order relationships of image features through an optimized knowledge graph, enabling the inference model to more accurately judge the characteristics of complex lesions under the guidance of medical knowledge and improving the reliability and clinical applicability of the inference results.

[0164] The present invention introduces a dynamic attention mechanism and a weight normalization strategy into the multi-hop memory network, dynamically adjusts the attention distribution according to the importance of different lesion regions, enabling the inference process to focus more on key lesion regions. Through the weight normalization method, higher weights are assigned to important entities according to the correlation between image features and entity embeddings in the knowledge graph, thereby amplifying the influence of key features in the inference result. Compared with the inference error caused by the uniform processing of all features in the traditional method, the present invention effectively reduces the problem of feature confusion through the gradual calibration of the lesion position and range, improving the spatial resolution and semantic accuracy of the inference result.

[0165] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered within the protection scope of the present invention.

Claims

1. A brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network, characterized in that: The steps include: S1. Obtain brain magnetic resonance imaging data and corresponding clinical annotation data; S2. performing image denoising, artifact correction, spatial registration, image enhancement and data normalization processing on the brain magnetic resonance imaging data to generate pre-processed brain magnetic resonance imaging data; S3. Construct a priori medical knowledge graph, which includes the anatomical structure of the brain, the location of lesions of common diseases, the relationship between lesion characteristics and disease types, and the corresponding clinical annotation data; S4. Based on the medical prior knowledge graph and the preprocessed brain MRI image data, a multi-hop memory network model is constructed, including an image feature extraction module, a memory update module, a knowledge graph embedding module, and a multi-hop reasoning module; S5. Performing multi-layer feature encoding on the preprocessed brain magnetic resonance imaging data through an image feature extraction module, extracting multi-scale spatial features and lesion-related features of the brain magnetic resonance imaging data, and generating an image feature vector; S6. Input the image feature vector into the knowledge graph embedding module, combine the anatomical structure and disease association features in the medical prior knowledge graph, perform semantic association embedding on the image feature vector, and generate an image feature vector fused with the prior knowledge; S7. Input the image feature vector fused with prior knowledge into the memory update module, use the attention mechanism to screen and update the key areas of the current feature vector, and store the updated features into the memory unit; S8. Perform multiple iterative reasoning on the image features stored in the memory unit based on the multi-hop reasoning module, combine the structural relationship of the medical prior knowledge graph, calibrate the judgment of the lesion location, lesion range and lesion nature hop by hop, and output the lesion reasoning result; S9. Generate a lesion annotation map of the brain MRI image based on the lesion inference result, the lesion annotation map includes the spatial location, size range and disease type information of the lesion, and fuse it with the original brain MRI image data for display.

2. According to claim 1, a brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network is characterized in that: The S1 specifically includes: S11. Obtain brain MRI data from the medical imaging database: I={I T1 ,I T2 ,I FLAIR }; Among them, I T1 is T1-weighted imaging data, I T2 is T2-weighted imaging data, I FLAIR is the FLAIR imaging data; S12. Obtain clinical annotation data corresponding to brain magnetic resonance imaging data: <h2 style=";text-align:left;direction:ltr">A = {A1,A2,…,A<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr">}; Among them, A i Indicates the lesion location and lesion type annotation on the i-th image slice; S13. Match the brain MRI image data I with the corresponding clinical annotation data A, and the matching basis includes the sequence number of the image slice, the scanning time and the spatial coordinates of the annotated area to generate a matched brain MRI image data pair: D={(I i ,A i )}。 3. The brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network according to claim 1, characterized in that: The S2 specifically includes: S21. Performing image denoising on the brain magnetic resonance imaging data I using a non-local mean denoising algorithm, removing random noise in the image data by analyzing the local similarity of each pixel point, and generating denoised brain magnetic resonance imaging data; S22. performing artifact correction on the denoised brain MRI image data, identifying artifact frequency domain features using Fourier transform, and removing artifact components by frequency domain filtering to generate artifact-corrected brain MRI image data; S23. Using an affine transformation-based registration method to perform spatial registration processing on the artifact-corrected brain MRI image data, adjusting the image rotation, translation and scaling parameters according to the spatial relationship between the image slices, and generating the spatially registered brain MRI image data; S24. performing image enhancement processing on the brain magnetic resonance imaging data after spatial registration, optimizing the contrast distribution of the image by using a histogram equalization algorithm, and generating brain magnetic resonance imaging data after image enhancement by improving the global balance of gray levels; S25. Perform normalization processing on the brain magnetic resonance imaging data after image enhancement, normalize the pixel values ​​of the imaging data to a standardized range of [0,1], and generate normalized brain magnetic resonance imaging data: I norm ={I norm,T1 ,I norm,T2 ,I norm,FLAIR }。 4. The brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network according to claim 1, characterized in that: The S3 specifically includes: S31. Based on the brain magnetic resonance imaging data D and the normalized brain magnetic resonance imaging data I norm Collecting brain anatomical structure dataset E anat , Common disease lesion location dataset E loc , Lesion characteristics and disease type relationship dataset E feat-type And the corresponding clinical annotation dataset E label ; S32. Anatomical structure entity e anat,j Represented as a vector v anat,j , the lesion location entity e loc,k Represented as a vector v loc,k , the relationship entity e between the lesion feature and the disease type feat-type,l Represented as a vector v feat-type,l , the clinical annotation entity e label,u Represented as a vector v label,u , all embedding vectors are in where h is the embedding dimension; S33. Define the relationship set R between entities in the medical prior knowledge graph, including the anatomical structure association relationship r anat-rel , position association relationship loc-rel And the feature-type association relationship r feat-type-rel , define the transformation matrix for each relationship So that for the associated entity pair (e x ,e y ) has a corresponding relational embedding v r (e x ,e y )=W r v x , where v x For entity e x Vector embedding of ; S34. Brain MRI Data I norm The spatial coordinates in construct the mapping function f from the image coordinates to the anatomical structure and lesion entity map The mapping function corresponds to the specific anatomical structure entity and lesion location entity according to the image coordinates (x, y, z) so that the mapped entity is embedded in v anat,j With v loc,k Establishing a corresponding relationship with the normalized image features; S35. According to the corresponding lesion type and feature annotation information in the clinical annotation data, the corresponding lesion feature and disease type relationship entity v feat-type,l With clinical annotation entity v label,u Establish association relationships in the medical prior knowledge graph and update the relationship transformation matrix W r Iteratively calibrate the association information so that the association relationship R between entities in the medical prior knowledge graph dynamically adapts to the characteristics of clinical annotation data; S36. Obtain the medical prior knowledge graph constructed through embedding and association relationships: G opt =(E,R,V,W); Where E = E anat ∪E loc ∪E feat-type ∪E label is the entity set, R is the relationship set, V is the entity embedding vector set, and W is the relationship transformation matrix set.

5. The brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network according to claim 1, characterized in that: The S5 specifically includes: S51. Normalize the brain magnetic resonance imaging data I norm In the first-layer feature encoding of the image feature extraction module input to the multi-hop memory network model, local feature extraction is performed in multi-channel dimensions to capture the initial spatial texture and potential lesion structure information and extract the initial low-level structural features: Among them, α, β, γ are the spatial coordinates of the feature map, τ is the input channel index, corresponding to the posterior brain magnetic resonance imaging data I norm The multimodal channel of , k1 is the size of the convolution kernel of the first layer, c is the number of input channels, μ, ν is the spatial offset of the convolution kernel; is the weight parameter of the first layer convolution kernel, b (1) is the bias term of the first layer, φ(·) is the nonlinear activation function; S52. Initial low-level structural features F (1) Input into the second layer feature encoding, perform multi-scale convolution feature extraction on it, capture the different scale features of the lesion area at the multi-scale level to generate a multi-scale feature mapping set {F (2,1) ,F (2 ,2) ,…,F (2,M) }: Among them, m=1,…,M is the multi-scale index, d is the number of channels after the first layer feature mapping, k 2,m is the size of the m-th convolution kernel in the second layer, is the weight parameter of the m-th scale convolution kernel, b (2,m) is the m-th scale bias term, ρ,σ are the spatial offsets of the m-th scale convolution kernel; S53. The obtained multi-scale feature map set is cascaded along the channel dimension to effectively fuse the features of different scales in the same representation space to obtain a comprehensive feature map F (c) : Among them, f cat (·) is the channel cascade operation, d m F (2,m) The number of channels; S54. Comprehensive feature map F (c) Perform global spatial pooling and combine the spatial weighting function M(α, β, γ) to focus on the lesion area of ​​interest and obtain the final image feature vector v img : Among them, ζ is the channel index, X, Y, Z are the comprehensive feature maps F (c) is the spatial dimension of , |Ω| is the normalization factor, and M(α, β, γ) is the spatial weighting function, which is used to highlight the key lesion area so that it has a higher weight in the global representation.

6. The brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network according to claim 1, characterized in that: The S6 specifically includes: S61. Input the image feature vector into the knowledge graph embedding module, take the image feature vector as the main factor, and construct a dynamic semantic mapping mechanism to preliminarily associate the image features with the anatomical structure embedding vector and the disease-related feature embedding vector in the medical prior knowledge graph to generate an initial semantic vector containing the semantic association between anatomy and lesions; S62. According to the optimization of the medical prior knowledge graph, the image features are unevenly distributed in the semantic space. The initial semantic vector is deeply semantically enhanced through a multi-dimensional association analysis mechanism. High-order association relationships are extracted from the medical prior knowledge graph. Image features are dynamically mapped and updated with multi-level anatomical structures, lesion features, and disease types to generate enhanced semantic association embedding. S63. Based on the enhanced semantic association embedding, a weight normalization method based on dynamic attention distribution is introduced to analyze the spatiotemporal matching between image features and related entity embeddings in the medical prior knowledge graph, assign adaptive importance weights to different entities, optimize the influence of important medical entities in the final features, and generate a weight normalized embedding vector that integrates image features and knowledge graph features; S64. The weighted normalized embedding vector is input into the feature integration optimization layer, and the fusion features are multi-dimensionally optimized through the semantic difference compensation and feature conflict resolution mechanism, and finally an image feature vector with multi-level semantic association capability is generated.

7. The brain magnetic resonance imaging lesion reasoning method based on a multi-hop memory network according to claim 1, characterized in that: The S8 specifically includes: S81. Image feature vectors with multi-level semantic association capabilities As the initial query vector q for multi-hop reasoning (0) , and obtain the updated feature memory set from the memory unit: in, Represents the feature memory vector related to brain MRI lesions, including knowledge embedding related to anatomical structure, lesion location, lesion characteristics and disease type; S82. In the tth inference iteration, the initial query vector q (t-1) and feature memory set M (t-1) Perform correlation calculation and obtain attention weight through attention mechanism And perform weighted summation on the memory to form the updated context vector c (t) : in, is the memory vector stored by the memory unit after the t-1th iteration. This memory vector contains the characteristic information of the location, range and nature of the lesion in the brain MRI. (t-1) is the query vector updated after the previous iteration, which is used to guide the attention allocation of the current iteration. is the attention weight, which is used to emphasize the attention to a specific memory vector; S83. Using the updated context vector c (t) For the query vector q (t-1) Update and generate the query vector q for the next iteration (t) ,During the updating process, the features are reconstructed in multiple dimensions, and the judgment of the lesion characteristics is calibrated jump by jump; S84. After T multi-hop reasoning iterations, the final query vector q is obtained (T) , q (T) Input to output mapping function f dec (·), through which the lesion location, lesion range and lesion property information are decoded from the final query vector to generate the final lesion inference result: L pos ,L rng ,L typ =f dec (q (T) ); Among them, L pos represents the inferred spatial location of the lesion, L rng Indicates the spatial range information of the lesion, L typ Indicates the category information of the lesion nature.

Images