Intracranial aneurysm imaging detection system and method based on deep learning

Abstract

The present application provides an intracranial aneurysm imaging detection system and method based on deep learning, which relates to the field of intelligent imaging detection. It first obtains DSA imaging data and medical record information corresponding to the DSA imaging data, and then uses data processing and image analysis algorithms based on artificial intelligence and deep learning to analyze the DSA imaging data and medical record information at the back end, as well as feature aggregation processing based on feature interaction and screening ideas, thereby realizing intelligent analysis and detection of the state of intracranial aneurysms to determine whether the aneurysm is occluded. This can not only eliminate or reduce the differences caused by human factors, but also comprehensively utilize the key semantics and complex interactive information in DSA images and medical record information to assist doctors in diagnosing intracranial aneurysms, providing strong support for clinicians.

Description

Technical Field

[0001] The present application relates to the field of intelligent image detection, and more specifically, to a deep learning-based intracranial aneurysm image detection system and method. Background Art

[0002] Intracranial aneurysm is a serious cerebrovascular disease, usually manifested as local dilation or protrusion of the cerebral blood vessel wall, which may lead to serious consequences such as cerebral hemorrhage and ischemic stroke. The use of interventional embolization technology has gradually become an important treatment for intracranial aneurysms. After interventional embolization treatment of intracranial aneurysms, it is necessary to assess the risk of recurrence of intracranial aneurysms after embolization. Traditional methods mainly rely on the doctor's experience and visual assessment, or assist in judgment based on follow-up examination results to determine whether intracranial aneurysm occlusion will occur. This may vary greatly due to differences in the operator's human eye resolution ability, fatigue level, cognitive experience, etc. At the same time, because the operator is prone to misjudgment when making judgments, the operability is poor.

[0003] In response to the above technical problems, Chinese patent CN113130078B proposes a method, device and equipment for predicting intracranial aneurysm occlusion, which analyzes DSA image data and case information corresponding to these image data to predict whether there is a jet of contrast agent in the aneurysm, the filling degree in the aneurysm, whether the contrast agent eventually exists in the capillary phase and the venous sinus phase, and the probability of aneurysm occlusion, and then uses the weighted sum of these prediction data to compare with a preset threshold to obtain the final prediction result of aneurysm occlusion.

[0004] In the above-mentioned method for predicting intracranial aneurysm occlusion, the features of DSA image data and case information corresponding to the DSA image data are extracted respectively, and then the two features are supervised trained to obtain a training model, and the training model is used to predict aneurysm occlusion. Although this method can reduce the differences caused by human factors, its utilization rate of information in DSA image data is low, and it is difficult to comprehensively and carefully analyze the DSA image semantics. At the same time, some subtle but important feature information will be lost, thereby affecting the prediction accuracy. In addition, when using DSA image data features and case information features to predict aneurysm occlusion, the key semantics and complex interactive information between the two are not paid attention to. This may make it difficult to discover the intrinsic connection between DSA image semantics and case information semantics, resulting in errors in the judgment of the status of intracranial aneurysms, thereby affecting doctors' treatment decisions.

[0005] Therefore, an optimized imaging system for detecting intracranial aneurysms is desired. Summary of the invention

[0006] In order to solve the above technical problems, this application is proposed.

[0007] According to one aspect of the present application, a deep learning-based intracranial aneurysm imaging detection system is provided, which includes:

[0008] A data acquisition module, used to obtain DSA image data and medical record information corresponding to the DSA image data;

[0009] An image data multi-scale feature extraction module, used for performing multi-scale feature extraction on the DSA image data to obtain DSA image shallow-deep joint interactive features;

[0010] A medical record information text semantic coding module, used to semantically code the medical record information corresponding to the DSA image data to obtain a medical record information text semantic coding feature;

[0011] A medical record information-image semantic interaction screening module, used for performing feature interaction screening and encoding on the DSA image shallow-deep joint interaction features and the medical record information text semantic coding features to obtain medical record information-DSA image interaction screening features; wherein the medical record information-image semantic interaction screening module includes: a feature mapping processing unit, used for performing linear transformation and masking processing on the medical record information text semantic coding features to obtain a masked first medical record information text semantic coding auxiliary feature mapping pattern representation and a masked second medical record information text semantic coding auxiliary feature mapping pattern representation; a feature interaction screening unit, used for performing feature screening on the DSA image shallow-deep joint interaction features based on the masked first medical record information text semantic coding auxiliary feature mapping pattern representation and the masked second medical record information text semantic coding auxiliary feature mapping pattern representation to obtain the medical record information-DSA image interaction screening features;

[0012] The aneurysm occlusion detection module is used to determine a detection result indicating whether the aneurysm is occluded based on the medical record information-DSA image interactive screening feature.

[0013] Furthermore, the image data multi-scale feature extraction module includes:

[0014] An image data feature capture unit, used for inputting the DSA image data into a multi-scale image feature extractor based on a dilated pyramid network to obtain a DSA image shallow feature map and a DSA image deep feature map;

[0015] The image shallow-deep feature joint interaction unit is used to perform feature joint interaction encoding based on kernel association on the DSA image shallow feature map and the DSA image deep feature map to obtain the DSA image shallow-deep joint interaction feature.

[0016] Furthermore, the image shallow-deep feature joint interaction unit includes:

[0017] An image feature distillation subunit, used for inputting the DSA image shallow feature map and the DSA image deep feature map into a kernel feature distillation network to obtain a DSA image shallow kernel feature vector and a DSA image deep kernel feature vector;

[0018] An image kernel feature global correlation coding subunit is used to perform global correlation coding on the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector to obtain a global coding matrix between the DSA image shallow and deep kernel features;

[0019] The image shallow-deep kernel feature sparse mask subunit is used to input the global encoding matrix between the DSA image shallow-deep kernel features into the sparse mask unit to obtain the DSA image shallow-deep kernel feature correlation sparse weight matrix;

[0020] An image feature position-by-position association interaction subunit is used to input the DSA image shallow feature map and the DSA image deep feature map into a position-by-position association interaction network to obtain a DSA image shallow-deep position-by-position interaction fusion feature map;

[0021] The image shallow-deep layer position-by-position interactive fusion feature significant sparse modulation subunit is used to perform feature significant sparse modulation on the DSA image shallow-deep layer position-by-position interactive fusion feature map based on the associated sparse weight matrix between the shallow-deep kernel features of the DSA image to obtain the DSA image shallow-deep joint interactive feature map as the DSA image shallow-deep joint interactive feature.

[0022] Furthermore, the image feature distillation subunit includes:

[0023] The image shallow feature decomposition secondary subunit is used to perform feature decomposition on the DSA image shallow feature map to obtain a sequence of DSA image shallow decomposition feature vectors;

[0024] The image shallow decomposition feature difference calculation secondary subunit is used to calculate the difference characteristics of each DSA image shallow decomposition feature vector in the sequence of DSA image shallow decomposition feature vectors relative to other DSA image shallow decomposition feature vectors to obtain a DSA image shallow decomposition feature difference representation vector;

[0025] The secondary subunit for normalization processing of the image shallow decomposition feature difference representation is used to normalize each eigenvalue in the DSA image shallow decomposition feature difference representation vector to obtain a normalized DSA image shallow decomposition feature difference representation vector;

[0026] The secondary subunit for calculating the weighted sum of the shallow decomposition features of the image is used to calculate the weighted sum of the sequence of the shallow decomposition feature vectors of the DSA image using each eigenvalue in the normalized DSA image shallow decomposition feature difference representation vector as a weight to obtain the shallow kernel feature vector of the DSA image.

[0027] Furthermore, the secondary subunit for calculating the difference in image shallow layer decomposition features is used to:

[0028] Calculating the norm of the positional difference between each DSA image shallow decomposition feature vector and other DSA image shallow decomposition feature vectors in the sequence of DSA image shallow decomposition feature vectors to obtain a sequence of DSA image shallow decomposition feature difference norm representation values;

[0029] Calculating the average value of the sequence of the DSA image shallow decomposition feature difference-norm representation values ​​to obtain the DSA image shallow decomposition feature difference-norm average representation value;

[0030] A plurality of the DSA image shallow decomposition feature difference one-norm average representation values ​​are vectorized and arranged to obtain the DSA image shallow decomposition feature difference representation vector.

[0031] Furthermore, the medical record information text semantic encoding module is used to: perform semantic encoding on the medical record information corresponding to the DSA image data to obtain a medical record information text semantic encoding feature vector as the medical record information text semantic encoding feature.

[0032] Furthermore, the feature mapping processing unit comprises:

[0033] A medical record information text feature linear change subunit, used to perform a linear transformation on the medical record information text semantic coding feature vector based on a first weight modifier matrix and a second weight modifier matrix to obtain a first medical record information text semantic coding auxiliary feature mapping mode representation vector and a second medical record information text semantic coding auxiliary feature mapping mode representation vector;

[0034] A first medical record information text gradient amplitude mask processing subunit is used to perform gradient amplitude-based mask processing on the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain a masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation;

[0035] The second medical record information text gradient amplitude mask processing subunit is used to perform gradient amplitude-based mask processing on the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain a masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation.

[0036] Furthermore, the feature interactive screening unit includes:

[0037] A preliminary feature screening subunit, used for performing preliminary feature screening on the DSA image shallow-deep joint interactive feature map based on the masked first medical record information text semantic encoding auxiliary feature mapping mode representation vector and the hyperbolic tangent function to obtain the DSA image shallow-deep joint interactive feature map after preliminary screening;

[0038] The secondary feature screening subunit is used to perform secondary feature screening on the shallow-deep joint interactive feature map of the DSA image after the preliminary screening based on the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector and the Sigmoid function to obtain the medical record information-DSA image interactive screening feature map as the medical record information-DSA image interactive screening feature.

[0039] Furthermore, the aneurysm occlusion detection module is used to: input the medical record information-DSA image interactive screening feature map into a classifier-based detection result generator to obtain a detection result, and the detection result is used to indicate whether the aneurysm is occluded.

[0040] According to another aspect of the present application, a method for detecting intracranial aneurysms based on deep learning is provided, comprising:

[0041] Step 1, obtaining DSA image data and medical record information corresponding to the DSA image data;

[0042] Step 2, performing multi-scale feature extraction on the DSA image data to obtain DSA image shallow-deep joint interactive features;

[0043] Step 3, semantically encoding the medical record information corresponding to the DSA image data to obtain semantic encoding features of the medical record information text;

[0044] Step 4, performing feature interactive screening and encoding on the DSA image shallow-deep joint interactive features and the medical record information text semantic coding features to obtain medical record information-DSA image interactive screening features; wherein, step 4 includes: performing linear transformation and masking processing on the medical record information text semantic coding features to obtain a masked first medical record information text semantic coding auxiliary feature mapping pattern representation and a masked second medical record information text semantic coding auxiliary feature mapping pattern representation; based on the masked first medical record information text semantic coding auxiliary feature mapping pattern representation and the masked second medical record information text semantic coding auxiliary feature mapping pattern representation, performing feature screening on the DSA image shallow-deep joint interactive features to obtain the medical record information-DSA image interactive screening features;

[0045] Step 5: Determine a test result indicating whether the aneurysm is occluded based on the medical record information-DSA image interactive screening feature.

[0046] Compared with the prior art, the present application provides a deep learning-based intracranial aneurysm imaging detection system and method, which first obtains DSA image data and medical record information corresponding to the DSA image data, and then uses data processing and image analysis algorithms based on artificial intelligence and deep learning to analyze the DSA image data and medical record information at the back end, as well as feature aggregation processing based on feature interaction and screening ideas, thereby realizing intelligent analysis and detection of the state of intracranial aneurysms to determine whether the aneurysm is occluded. This can not only eliminate or reduce the differences caused by human factors, but also comprehensively utilize the key semantics and complex interactive information in DSA images and medical record information to assist doctors in diagnosing intracranial aneurysms, providing strong support for clinicians. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0048] Figure 1 4 is a system block diagram of an intracranial aneurysm imaging detection system based on deep learning according to an embodiment of the present application.

[0049] Figure 2 Schematic diagram of data flow of an intracranial aneurysm imaging detection system based on deep learning according to an embodiment of the present application.

[0050] Figure 3It is a block diagram of a multi-scale feature extraction module for image data in a deep learning-based intracranial aneurysm imaging detection system according to an embodiment of the present application.

[0051] Figure 4 It is a block diagram of the image shallow-deep feature joint interaction unit in the intracranial aneurysm image detection system based on deep learning according to an embodiment of the present application.

[0052] Figure 5 It is a block diagram of the medical record information-image semantic interactive screening module in the deep learning-based intracranial aneurysm imaging detection system according to an embodiment of the present application.

[0053] Figure 6 Flow chart of a method for intracranial aneurysm imaging detection based on deep learning according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0055] Intracranial aneurysms are abnormal dilations or bulges of the walls of arteries within the brain, usually forming at the site of arterial branching. They may occur due to weakness in the arterial wall, genetic factors, hypertension, or arteriosclerosis. Most intracranial aneurysms do not cause symptoms when they are small, but when they grow or rupture, they may cause serious complications such as subarachnoid hemorrhage, severe headaches, vomiting, blurred vision, and even life-threatening. Treatments include monitoring, drug therapy, and surgical interventions such as clipping or interventional embolization. Traditional methods for determining intracranial aneurysm occlusion have relatively low information utilization of DSA image data, making it difficult to comprehensively and meticulously analyze image semantics. In addition, the key semantics and complex interactions between DSA image features and case information features may be ignored, which may affect the accurate judgment of the status of intracranial aneurysms and ultimately affect doctors' treatment decisions.

[0056] In view of the above technical problems, in the technical solution of the present application, a deep learning-based intracranial aneurysm imaging detection system is proposed, which can analyze the DSA image data and the medical record information corresponding to the DSA image data by using data processing and image analysis algorithms based on artificial intelligence and deep learning at the back end, so as to capture the multi-scale feature joint interaction information of the DSA image and the medical record text semantic information. Then, the multi-scale semantics of the DSA image and the medical record text semantics are subjected to feature aggregation processing based on feature interaction and screening ideas, so as to obtain the key feature complex interaction information between the medical record information text semantics and the DSA image semantics, thereby enhancing the robustness and performance of the detection system. In this way, the intelligent analysis and detection of the state of the intracranial aneurysm can be realized to determine whether the aneurysm is occluded, which can not only eliminate or reduce the differences caused by human factors, but also comprehensively utilize the key semantics and complex interaction information in the DSA image and medical record information to assist doctors in diagnosing intracranial aneurysms, provide strong support for clinicians, and help them make more scientific and reasonable judgments when making treatment decisions.

[0057] Specifically, Figure 1 This is a system block diagram of an intracranial aneurysm imaging detection system based on deep learning according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of an intracranial aneurysm imaging detection system based on deep learning according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, an intracranial aneurysm imaging detection system 100 based on deep learning includes: a data acquisition module 110, used to obtain DSA image data and medical record information corresponding to the DSA image data; an image data multi-scale feature extraction module 120, used to perform multi-scale feature extraction on the DSA image data to obtain DSA image shallow-deep joint interaction features; a medical record information text semantic encoding module 130, used to perform semantic encoding on the medical record information corresponding to the DSA image data to obtain medical record information text semantic encoding features; a medical record information-image semantic interactive screening module 140, used to perform feature interactive screening and encoding on the DSA image shallow-deep joint interaction features and the medical record information text semantic encoding features to obtain medical record information-DSA image interactive screening features; an aneurysm occlusion detection module 150, used to determine a detection result indicating whether an aneurysm is occluded based on the medical record information-DSA image interactive screening features.

[0058] In the above-mentioned intracranial aneurysm imaging detection system 100 based on deep learning, the data acquisition module 110 is used to obtain DSA image data and medical record information corresponding to the DSA image data. It should be understood that DSA (digital subtraction angiography) is a high-precision medical imaging technology, which is mainly used for vascular imaging to help doctors diagnose and treat vascular diseases. This technology provides clear vascular images through contrast enhancement and subtraction technology, making the morphology and lesions of the blood vessels more prominent. DSA is usually used for interventional treatment, such as stent implantation, vascular embolization, etc., and its process involves the injection of contrast agents and X-ray imaging, which can effectively display the dynamic changes of blood flow and has important clinical application value in the fields of neurology, cardiology and oncology. Specifically, first, the doctor will decide whether a DSA examination is needed based on the patient's clinical manifestations and related examinations (such as CT, MRI, etc.). During the examination, the patient usually receives local anesthesia or general anesthesia, and then, under the guidance of the imaging equipment, the catheter is inserted into the target blood vessel (such as the femoral artery or radial artery). Through the catheter, the contrast agent is injected, and the image changes of the blood vessel are monitored in real time on the imaging device. Ultimately, the DSA images generated by the device can help doctors analyze the morphology and blood flow of blood vessels, helping to make a more accurate diagnosis. DSA images include blood vessel diameter, aneurysm morphology (such as size, shape, location) and blood flow velocity. Medical record information is the patient's personal health record, including past medical history, family medical history, current symptoms, and test results, which can be obtained through the database. In the technical solution of this application, the combination of imaging data and medical record information can provide a comprehensive patient assessment, helping doctors make more accurate decisions when performing interventional treatments or formulating follow-up observation plans.

[0059] In the above-mentioned intracranial aneurysm imaging detection system 100 based on deep learning, the image data multi-scale feature extraction module 120 is used to perform multi-scale feature extraction on the DSA image data to obtain the shallow-deep joint interactive features of the DSA image. It should be understood that DSA image data contains rich structural and functional information, in which features at different levels can reveal different pathological conditions in the image. Multi-scale feature extraction can capture this information more comprehensively. For example, small lesions may not be obvious at certain scales, but can be clearly seen at other scales. By combining shallow and deep features, it helps to form a more accurate and comprehensive image analysis model, allowing doctors to obtain more valuable information when diagnosing.

[0060] Figure 3 FIG. 4 is a block diagram of a multi-scale feature extraction module for image data in an intracranial aneurysm image detection system based on deep learning according to an embodiment of the present application. Figure 3As shown, the image data multi-scale feature extraction module 120 includes: an image data feature capture unit 121, which is used to input the DSA image data into a multi-scale image feature extractor based on a diaphanous pyramid network to obtain a DSA image shallow feature map and a DSA image deep feature map; an image shallow-deep feature joint interaction unit 122, which is used to perform kernel association-based feature joint interaction encoding on the DSA image shallow feature map and the DSA image deep feature map to obtain the DSA image shallow-deep joint interaction feature.

[0061] Specifically, in the image data feature capture unit 121 of the technical solution of the present application, the DSA image data is input into a multi-scale image feature extractor based on a dilated pyramid network for feature mining to extract shallow features and deep semantic features in the DSA image data, thereby obtaining a DSA image shallow feature map and a DSA image deep feature map. It should be understood that in medical images, different lesions and structures often exhibit different characteristics at different spatial scales. Traditional feature extraction methods may not be able to effectively capture these diversities, resulting in information loss. The dilated pyramid network can expand the receptive field while maintaining resolution by introducing dilated convolution, so that the model can effectively extract features at different scales. Specifically, shallow features mainly include edges, textures, and local structures of images, which are usually very helpful for the identification of smaller blood vessels and their lesions; while deep features involve more complex patterns and semantic information, such as the overall morphology of blood vessels and a wide range of lesions. Combining shallow features and deep features helps to improve the accuracy of lesion identification.

[0062] Furthermore, in the image shallow-deep feature joint interaction unit 122, since the DSA image shallow feature map and the DSA image deep feature map respectively contain shallow feature and deep semantic feature information about the DSA image, both features play an important role in the subsequent aneurysm state detection task. In order to comprehensively consider the interaction between the DSA image shallow features and the DSA image deep semantic features, and extract the most representative and discriminative features therefrom, to provide a basis for subsequent DSA image type recognition and detection, in the technical solution of the present application, the DSA image shallow feature map and the DSA image deep feature map are further subjected to kernel-related feature joint interaction encoding to obtain the DSA image shallow-deep joint interaction feature map. Through the kernel-association-based feature joint interactive coding process, the most representative and discriminative feature information can be extracted from the original DSA image shallow feature map and the DSA image deep feature map through multi-level processing steps, and on this basis, these two multi-scale features at different levels of the DSA image are effectively fused to obtain a new feature representation, which includes both the shallow features of the DSA image and the deep semantics, such as the morphology, size, position of the aneurysm and its relationship with surrounding tissues, thereby providing a more comprehensive DSA image description.

[0063] Figure 4 FIG. 1 is a block diagram of an image shallow-deep feature joint interaction unit in an intracranial aneurysm image detection system based on deep learning according to an embodiment of the present application. Figure 4As shown, the image shallow-deep feature joint interaction unit 122 includes: an image feature distillation subunit 1221, which is used to input the DSA image shallow feature map and the DSA image deep feature map into the kernel feature distillation network to obtain the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector; an image kernel feature global correlation encoding subunit 1222, which is used to perform global correlation encoding on the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector to obtain the DSA image shallow-deep kernel feature global encoding matrix; an image shallow-deep kernel feature sparse mask subunit 1223, which is used to input the DSA image shallow-deep kernel feature global encoding matrix into the kernel feature distillation network; A sparse mask unit is used to obtain a sparse weight matrix associated with the shallow-deep kernel features of the DSA image; an image feature position-by-position associated interaction subunit 1224 is used to input the DSA image shallow feature map and the DSA image deep feature map into a position-by-position associated interaction network to obtain a DSA image shallow-deep position-by-position interactive fusion feature map; an image shallow-deep position-by-position interactive fusion feature significant sparse modulation subunit 1225 is used to perform feature significant sparse modulation on the DSA image shallow-deep position-by-position interactive fusion feature map based on the sparse weight matrix associated with the shallow-deep kernel features of the DSA image to obtain a DSA image shallow-deep joint interactive feature map as the DSA image shallow-deep joint interactive feature.

[0064] Specifically, in the image feature distillation subunit 1221, the DSA image shallow feature map and the DSA image deep feature map are input into the kernel feature distillation network to obtain the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector. It should be understood that the kernel feature distillation network utilizes the powerful representation ability of the neural network, which not only reduces redundant information, but also improves the expressiveness and robustness of the DSA image shallow features and the DSA image deep features. By identifying and enhancing the expressiveness of these features, it can best distinguish or describe the characteristic attributes of the essence of the data, that is, the core attribute information in the shallow and deep structural features of the DSA image.

[0065] More specifically, the image feature distillation subunit 1221 includes: a secondary subunit for image shallow feature decomposition, which is used to perform feature decomposition on the DSA image shallow feature map to obtain a sequence of DSA image shallow decomposition feature vectors; a secondary subunit for calculating shallow decomposition feature differences of an image, which is used to calculate the difference features of each DSA image shallow decomposition feature vector in the sequence of DSA image shallow decomposition feature vectors relative to other DSA image shallow decomposition feature vectors to obtain a DSA image shallow decomposition feature difference representation vector; a secondary subunit for normalizing the image shallow decomposition feature difference representation, which is used to normalize each eigenvalue in the DSA image shallow decomposition feature difference representation vector to obtain a normalized DSA image shallow decomposition feature difference representation vector; and a secondary subunit for calculating shallow decomposition feature weighted sum, which is used to calculate the weighted sum of the sequence of DSA image shallow decomposition feature vectors using each eigenvalue in the normalized DSA image shallow decomposition feature difference representation vector as a weight to obtain the DSA image shallow kernel feature vector.

[0066] More specifically, the secondary subunit for calculating the shallow decomposition feature difference of the image is used to: calculate the norm of the positional difference between each DSA image shallow decomposition feature vector and other DSA image shallow decomposition feature vectors in the sequence of the DSA image shallow decomposition feature vectors to obtain a sequence of DSA image shallow decomposition feature difference norm representation values; calculate the average value of the sequence of DSA image shallow decomposition feature difference norm representation values ​​to obtain a DSA image shallow decomposition feature difference norm average representation value; vectorize and arrange a plurality of the DSA image shallow decomposition feature difference norm average representation values ​​to obtain the DSA image shallow decomposition feature difference representation vector.

[0067] More specifically, the secondary subunit of normalization processing of shallow decomposition feature difference representation of the image is used to normalize each eigenvalue in the shallow decomposition feature difference representation vector of the DSA image to obtain a normalized DSA image shallow decomposition feature difference representation vector, including: calculating the positional exponential function value with a natural constant as the base of each eigenvalue in the shallow decomposition feature difference representation vector of the DSA image to obtain a DSA image shallow decomposition feature difference class support representation vector; calculating the sum of each eigenvalue in the shallow decomposition feature difference class support representation vector of the DSA image to obtain a sum of DSA image shallow decomposition feature difference class support; calculating the positional division between each eigenvalue in the DSA image shallow decomposition feature difference class support representation vector and the sum of DSA image shallow decomposition feature difference class support to obtain the normalized DSA image shallow decomposition feature difference representation vector.

[0068] More specifically, in the image kernel feature global correlation coding subunit 1222, the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector are globally correlated to obtain a DSA image shallow-deep kernel feature global coding matrix. That is, the shallow shape and deep structure kernel feature vectors of the DSA image are processed using the global correlation coding technology to construct a kernel feature global coding matrix that can reflect the correlation and interaction between kernel features at different levels of the DSA image, which helps to better analyze and understand the status of different DSA images and aneurysms, so as to more accurately perform the aneurysm occlusion detection task.

[0069] More specifically, the image kernel feature global association coding subunit 1222 includes: calculating the multiplication between the transposed vectors of the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector to obtain the global coding matrix between the DSA image shallow and deep kernel features.

[0070] More specifically, in the image shallow-deep kernel feature sparse mask subunit 1223, the global encoding matrix between the DSA image shallow-deep kernel features is input into the sparse mask unit to obtain the DSA image shallow-deep kernel feature correlation sparse weight matrix. That is, the sparsity principle is introduced to filter out the less important DSA image shallow-deep kernel correlation feature information, while highlighting those parts that are most critical to the overall information transmission, which can effectively condense the key information flow paths in the original data. In this way, the system can significantly reduce the computing cost while maintaining or even improving performance, which is particularly important for actual deployment, especially for applications in resource-constrained environments.

[0071] More specifically, the image shallow-deep kernel feature sparse mask subunit 1223 includes: performing a sparse mask operation on the eigenvalues ​​at each position in the global coding matrix between the shallow-deep kernel features of the DSA image to obtain a sparse weight matrix associated with the shallow-deep kernel features of the DSA image; wherein, in response to the eigenvalues ​​in the global coding matrix between the shallow-deep kernel features of the DSA image being greater than twice a preset threshold, the eigenvalues ​​greater than twice the preset threshold are multiplied by a constant 3; in response to the eigenvalues ​​in the global coding matrix between the shallow-deep kernel features of the DSA image being less than or equal to the preset threshold, the eigenvalues ​​less than or equal to the preset threshold are divided by a constant 3; in response to the eigenvalues ​​in the global coding matrix between the shallow-deep kernel features of the DSA image being greater than the preset threshold and less than or equal to twice the preset threshold, the eigenvalues ​​are taken as original values ​​to obtain a sparse weight matrix associated with the shallow-deep kernel features of the DSA image.

[0072] More specifically, on another data processing link, the image feature position-by-position association interaction subunit 1224 inputs the DSA image shallow feature map and the DSA image deep feature map into a position-by-position association interaction network to obtain a DSA image shallow-deep position-by-position interaction fusion feature map. In this way, it is possible to focus on the feature interaction between the shallow shape features and the deep structural features of the DSA image at the local detail level, and to achieve information exchange and integration at each corresponding position across the feature maps through a carefully designed network structure, ultimately forming a DSA image shallow-deep position-by-position interaction fusion feature map.

[0073] More specifically, the image feature position-by-position association interaction subunit 1224 includes: performing position point multiplication on the DSA image shallow feature map and the DSA image deep feature map to obtain the DSA image shallow-deep position-by-position interactive fusion feature map.

[0074] More specifically, in the image shallow-deep layer position-by-position interactive fusion feature significant sparse modulation subunit 1225, based on the correlation sparse weight matrix between the DSA image shallow-deep layer kernel features, the DSA image shallow-deep layer position-by-position interactive fusion feature map is subjected to feature significant sparse modulation. This modulation process adjusts the contribution of each part according to the importance level indicated by the correlation sparse weight matrix, so that the adjusted result (DSA image shallow-deep layer joint interactive feature map) can better reflect the most representative feature combination in the original input and the most relevant to the subsequent aneurysm state detection and recognition tasks. By assigning higher weights to these features, the system can better adapt to the complex and changeable actual application scenario requirements and provide a basis for subsequent aneurysm occlusion detection.

[0075] More specifically, the image shallow-deep position-by-position interactive fusion feature significant sparse modulation subunit 1225 includes: multiplying each feature matrix along the channel dimension in the DSA image shallow-deep position-by-position interactive fusion feature map by the associated sparse weight matrix between the DSA image shallow-deep kernel features to obtain the DSA image shallow-deep joint interactive feature map.

[0076] Specifically, the DSA image shallow feature map and the DSA image deep feature map are subjected to kernel association-based feature joint interaction encoding using the following feature joint interaction formula to obtain a DSA image shallow-deep joint interaction feature map as the DSA image shallow-deep joint interaction feature;

[0077] Among them, the feature joint interaction formula is: ; ; ; ; ; ; ;in, and are respectively the DSA image shallow feature map and the DSA image deep feature map, For kernel feature distillation, For kernel feature distillation, is the feature map that needs to be processed by kernel feature distillation. and are the shallow kernel feature vector of DSA image and the deep kernel feature vector of DSA image, respectively. is the transposed vector, is a decomposition operation, are the first decomposition eigenvector, the second decomposition eigenvector, The decomposed eigenvectors, The decomposition eigenvectors and decomposition eigenvectors, is the one-norm of the vector, is the sequence of decomposed eigenvectors, is the number of vectors in the sequence of decomposed eigenvectors minus one, To decompose the eigenvalues ​​of each position in the feature difference representation vector, is the length of the decomposed feature difference representation vector, is the exponential operation, is the kernel eigenvector, is the global encoding matrix between the shallow and deep kernel features of the DSA image, is the feature value of each position in the global encoding matrix between the shallow and deep kernel features of the DSA image, is the preset threshold, is a sparse mask operation, is the sparse weight matrix of the correlation between the shallow and deep kernel features of the DSA image, It is the DSA image shallow-deep interactive fusion feature map, It is the point multiplication by position. is matrix multiplication, It is the shallow-deep joint interaction feature map of the DSA image.

[0078] In the above-mentioned intracranial aneurysm imaging detection system 100 based on deep learning, the medical record information text semantic encoding module 130 is used to semantically encode the medical record information corresponding to the DSA image data to obtain the medical record information text semantic encoding features. It should be understood that medical record information is an important part of medical data, covering the patient's basic condition, medical history, and examination results. Unlike structured data (such as laboratory test values), medical record information exists in the form of natural language, which is difficult for computers to understand directly. Semantic coding can convert this text information into a numerical form that can be processed by a machine learning model, and capture potential patterns, symptom associations, and disease characteristics in the text data, thereby improving the accuracy of disease prediction, providing doctors with auxiliary diagnosis and treatment recommendations, and improving clinical decision-making efficiency.

[0079] Specifically, the medical record information text semantic encoding module 130 is used to: semantically encode the medical record information corresponding to the DSA image data to obtain the medical record information text semantic encoding feature vector as the medical record information text semantic encoding feature. More specifically, first, the medical record information corresponding to the DSA image data is preprocessed to reduce redundant information and improve the efficiency of subsequent processing. This includes removing noise (such as punctuation, stop words), word segmentation (decomposing sentences into words or phrases), and stem extraction (restoring words to their basic form). Then, by counting the frequency of occurrence of words in the text, the text is converted into a word frequency vector. Then, different weights are assigned according to the importance of words in a specific text. Next, through word embedding encoding (Word2Vec), word vectors are trained to represent the semantics of words to capture the similarity and contextual relationship between words. Finally, the contextual information of the text is processed using a long short-term memory network to generate a richer semantic representation. In this way, it can help doctors make more accurate and efficient decisions in diagnosis and treatment plan formulation.

[0080] In the above-mentioned intracranial aneurysm imaging detection system 100 based on deep learning, the medical record information-image semantic interaction screening module 140 is used to perform feature interaction screening and encoding on the shallow-deep joint interaction features of the DSA image and the medical record information text semantic coding features to obtain the medical record information-DSA image interaction screening features. It should be understood that since the DSA image shallow-deep joint interaction feature map and the medical record information text semantic coding feature vector respectively contain the deep-shallow joint interaction features and medical record information text semantic features related to the DSA image. Therefore, in order to be able to combine and comprehensively represent these two features to capture the complex association and interactive relationship between the DSA image semantics and the medical record information text semantics, so as to perform feature interaction to enrich the characterization of the state of the aneurysm and provide a basis for subsequent aneurysm occlusion detection, in the technical solution of the present application, the DSA image shallow-deep joint interaction features and the medical record information text semantic coding features are further feature-interactively screened and encoded to obtain the medical record information-DSA image interaction screening features.

[0081] Specifically, Figure 5 FIG. 1 is a block diagram of a medical record information-image semantic interaction screening module in a deep learning-based intracranial aneurysm image detection system according to an embodiment of the present application. Figure 5 As shown, the medical record information-image semantic interaction screening module 140 is used to perform feature interaction screening and encoding on the shallow-deep joint interaction features of the DSA image and the semantic coding features of the medical record information text to obtain the medical record information-DSA image interaction screening features, including: a feature mapping processing unit 141, used to perform linear transformation and masking processing on the medical record information text semantic coding features to obtain a masked first medical record information text semantic coding auxiliary feature mapping pattern representation and a masked second medical record information text semantic coding auxiliary feature mapping pattern representation; a feature interaction screening unit 142, used to perform feature screening on the shallow-deep joint interaction features of the DSA image based on the masked first medical record information text semantic coding auxiliary feature mapping pattern representation and the masked second medical record information text semantic coding auxiliary feature mapping pattern representation to obtain the medical record information-DSA image interaction screening features.

[0082] It should be understood that in the feature mapping processing unit 141, a linear transformation based on the first weight modifier matrix and the second weight modifier matrix is ​​applied to the semantic encoding feature vector of the medical record information text to expand the expression range of the semantic features of the medical record information text through linear transformation, so that the model can capture the relationship between the semantic features of the medical record information text and the shallow-deep joint interaction feature map of the DSA image from different angles.

[0083] More specifically, the feature mapping processing unit 141 includes: a medical record information text feature linear change subunit, which is used to perform a linear transformation on the semantic coding feature vector of the medical record information text based on the first weight modifier matrix and the second weight modifier matrix to obtain the first medical record information text semantic coding auxiliary feature mapping pattern representation vector and the second medical record information text semantic coding auxiliary feature mapping pattern representation vector; a first medical record information text gradient amplitude mask processing subunit, which is used to perform a gradient amplitude mask on the first medical record information text semantic coding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the first medical record information text semantic coding auxiliary feature mapping pattern representation vector. The second medical record information text gradient amplitude mask processing subunit is used to perform gradient amplitude-based mask processing on the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain a masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation.

[0084] More specifically, the medical record information text feature linear change subunit performs linear transformation on the medical record information text semantic coding feature vector based on the first weight modifier matrix and the second weight modifier matrix to obtain the first medical record information text semantic coding auxiliary feature mapping mode representation vector and the second medical record information text semantic coding auxiliary feature mapping mode representation vector, including: calculating the medical record information text semantic coding feature vector and the transpose of the first weight modifier matrix, multiplying it, and then adding it positionally with the first bias modifier vector to obtain the first medical record information text semantic coding auxiliary feature mapping mode representation vector; calculating the medical record information text semantic coding feature vector and the transpose of the second weight modifier matrix, multiplying it, and then adding it positionally with the second bias modifier vector to obtain the second medical record information text semantic coding auxiliary feature mapping mode representation vector.

[0085] More specifically, the first medical record information text gradient amplitude mask processing subunit is used to perform gradient amplitude-based mask processing on the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain a masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation, including: calculating the square of the difference between the previous position feature value and the next position feature value of each position feature value in the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain the first medical record information text semantic encoding auxiliary feature mapping pattern representation. information text semantic gradient amplitude representation vector; calculate the absolute value of the square root of each first medical record information text semantic gradient amplitude in the first medical record information text semantic gradient amplitude representation vector, and input the obtained feature vector into the mask function for processing to obtain the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector; wherein, in the mask function, in response to the first medical record information text semantic gradient amplitude representation vector having a semantic gradient amplitude greater than a predetermined threshold, the first medical record information text semantic gradient amplitude greater than the predetermined threshold is set to 1, otherwise it is set to 0 to obtain the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector.

[0086] More specifically, in the feature interaction screening unit 142, based on the gradient amplitude of each position in the auxiliary feature mapping pattern representation vector of the semantic encoding of the first medical record information text, a mask processing based on the gradient amplitude is performed. Similarly, the same processing is performed on the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector. It should be understood that the gradient amplitude reflects the degree of change of each element in the medical record information text semantic encoding auxiliary feature mapping pattern representation vector. Through mask processing, features with large changes can be highlighted, thereby enhancing attention to important medical record information text semantic features in the subsequent feature selection expression process.

[0087] More specifically, the feature interaction screening unit 142 includes: a preliminary feature screening subunit, which is used to perform preliminary feature screening on the DSA image shallow-deep joint interaction feature map based on the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector and hyperbolic tangent function to obtain the DSA image shallow-deep joint interaction feature map after preliminary screening; a secondary feature screening subunit, which is used to perform secondary feature screening on the DSA image shallow-deep joint interaction feature map after preliminary screening based on the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector and Sigmoid function to obtain the medical record information-DSA image interaction screening feature map as the medical record information-DSA image interaction screening feature.

[0088] More specifically, in the preliminary feature screening subunit, the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector is combined with the hyperbolic tangent function to perform preliminary screening on the DSA image shallow-deep joint interactive feature map. It should be understood that the hyperbolic tangent function is a commonly used activation function with an output range between -1 and +1, which can effectively distinguish different areas in the feature map, thereby highlighting those feature representations related to the semantics of the DSA image in the preliminary screening stage.

[0089] More specifically, the preliminary feature screening subunit is used to perform preliminary feature screening on the DSA image shallow-deep joint interaction feature map based on the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector and the hyperbolic tangent function to obtain the DSA image shallow-deep joint interaction feature map after preliminary screening, including: using the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the channel weight vector, performing weighted enhancement on the DSA image shallow-deep joint interaction feature map along the channel dimension, and then inputting the weighted enhanced feature map into the hyperbolic tangent function for activation to obtain the DSA image shallow-deep joint interaction feature map after preliminary screening.

[0090] More specifically, in the secondary feature screening subunit, based on the preliminary screening, the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector is used in combination with the Sigmoid function to further screen the DSA image shallow-deep joint interaction feature map after the preliminary screening to determine which DSA image shallow-deep joint interaction features are truly important, so that the medical record information text semantic features can be used to guide the enhancement of the expression ability of the DSA image shallow-deep joint interaction features, and generate the medical record information-DSA image interaction screening feature map accordingly. It should be understood that through feature interaction and screening, the relevant features of the DSA image shallow-deep fine-grained joint perception feature map can be enhanced under the guidance of the medical record information text semantics, so that it can better reflect the true state of the aneurysm. In this way, the enhanced feature map not only contains the original shape information, but also incorporates a deep level of semantic understanding, providing a basis for subsequent aneurysm occlusion detection tasks.

[0091] More specifically, the secondary feature screening subunit includes: using the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the channel weight vector, weightedly enhancing the shallow-deep joint interaction feature map of the DSA image after the preliminary screening along the channel dimension, and inputting the weighted enhanced feature map into the Sigmoid function for activation to obtain the medical record information-DSA image interaction screening feature map.

[0092] Specifically, the DSA image shallow-deep joint interaction feature and the medical record information text semantic coding feature are subjected to feature interactive screening encoding using the following feature interactive screening formula to obtain the medical record information-DSA image interactive screening feature;

[0093] The feature interaction screening formula includes: ; ; ; ; ; ; ;in, is the semantic encoding feature vector of the medical record information text, and are the first weight modifier matrix and the first bias modifier vector, respectively, and are the second weight modifier matrix and the second bias modifier vector, respectively, and are the first linear transformation and the second linear transformation respectively, and are respectively the first medical record information text semantic encoding auxiliary feature mapping mode representation vector and the second medical record information text semantic encoding auxiliary feature mapping mode representation vector, and are respectively the first medical record information text semantic encoding auxiliary feature mapping mode representation vector Position and The eigenvalues ​​at the positions, and are respectively the first and second medical record information text semantic encoding auxiliary feature mapping mode representation vectors Position and The eigenvalues ​​at the positions, is a mask operation, is a mask operation, and The first medical record information text semantic encoding auxiliary feature mapping mode representation vector and the second medical record information text semantic encoding auxiliary feature mapping mode representation vector are respectively The eigenvalues ​​at the positions, is the preset threshold, is the shallow-deep joint interactive feature map of the DSA image, A representation vector of the semantic encoding auxiliary feature mapping pattern for the masked first medical record information text, A semantic encoding auxiliary feature mapping pattern representation vector for the masked second medical record information text, is the position-wise weighted multiplication process along the channel dimension, is the hyperbolic tangent function, It is the shallow-deep joint interactive feature map of DSA images after preliminary screening. for function, A feature map for the medical record information-DSA image interaction screening.

[0094] In the above-mentioned intracranial aneurysm imaging detection system 100 based on deep learning, the aneurysm occlusion detection module 150 is used to determine the detection result indicating whether the aneurysm is occluded based on the medical record information-DSA image interactive screening feature. It should be understood that DSA image data may be affected by technical factors, patient position, etc., resulting in differences in interpretation. By combining medical record information with DSA image data, the deficiencies of image analysis can be supplemented to effectively integrate image and text data, so as to more accurately assess the state of the aneurysm. Specifically, the medical record information-DSA image interactive screening feature is passed through a trained classifier model to output a label of "occluded" or "non-occluded", thereby obtaining a detection result of whether the aneurysm is occluded. Among them, the classifier model should include occluded and non-occluded labeled samples during the training process, and a cross-validation method should be used to evaluate the performance of the model to avoid overfitting. In this way, intelligent analysis and detection of the status of intracranial aneurysms can be achieved to determine whether the aneurysm is occluded. This can not only eliminate or reduce the differences caused by human factors, but also comprehensively utilize the key semantics and complex interactive information in DSA images and medical records to assist doctors in diagnosing intracranial aneurysms, providing strong support for clinicians and helping them make more scientific and reasonable judgments when making treatment decisions.

[0095] More specifically, the aneurysm occlusion detection module 150 is used to: input the medical record information-DSA image interactive screening feature map into a classifier-based detection result generator to obtain a detection result, and the detection result is used to indicate whether the aneurysm is occluded.

[0096] Preferably, in an example of the present application, the medical record information-DSA image interaction screening feature map is input into a classifier-based detection result generator to obtain a detection result, including: calculating the sum of the absolute values ​​of each eigenvalue of the medical record information-DSA image interaction screening feature map to obtain a first medical record information-DSA image interaction screening spatial structure value, and calculating the square root of the sum of the squares of each eigenvalue of the medical record information-DSA image interaction screening feature map to obtain a second medical record information-DSA image interaction screening spatial structure value; multiplying each eigenvalue of the medical record information-DSA image interaction screening feature map with the first medical record information-DSA image interaction screening spatial structure value and the second medical record information-DSA image interaction screening spatial structure value to obtain a first medical record information-DSA image interaction screening structure reference value and a second medical record information-DSA image interaction screening structure reference value corresponding to each eigenvalue; multiplying each eigenvalue of the medical record information-DSA image interaction screening feature map with the scale of the medical record information-DSA image interaction screening feature map and the square root of the scale to obtain a value corresponding to each eigenvalue. The first medical record information-DSA image interactive screening scale transformation value and the second medical record information-DSA image interactive screening scale transformation value corresponding to the eigenvalue; dividing the first medical record information-DSA image interactive screening structure reference value by the difference between the first medical record information-DSA image interactive screening spatial structure value and the first medical record information-DSA image interactive screening scale transformation value to obtain the first medical record information-DSA image interactive screening transformation adjustment value; dividing the second medical record information-DSA image interactive screening structure reference value by the difference between the second medical record information-DSA image interactive screening spatial structure value and the second medical record information-DSA image interactive screening scale transformation value to obtain the second medical record information-DSA image interactive screening transformation adjustment value; calculating the weighted sum of the first medical record information-DSA image interactive screening transformation adjustment value and the second medical record information-DSA image interactive screening transformation adjustment value to obtain each eigenvalue of the optimized medical record information-DSA image interactive screening feature map; and inputting the optimized medical record information-DSA image interactive screening feature map into a classifier-based detection result generator to obtain a detection result.

[0097] Here, the medical record information-DSA image interactive screening feature map is recorded as The optimization expression is: ; ; ; ; ; ; ; ; ;in, Interactive screening feature map for medical record information-DSA image, Represents the eigenvalues ​​of the medical record information-DSA image interactive screening feature map, represents the first medical record information-DSA image interactive screening transformation adjustment value, represents the second medical record information-DSA image interactive screening transformation adjustment value, Indicates the spatial structure value of the first medical record information-DSA image interactive screening, Indicates the spatial structure value of the second medical record information-DSA image interaction screening, represents the scale of the medical record information-DSA image interactive screening feature map, represents the summation function, represents the absolute value function, represents the set of real numbers, represents the width of the feature map, represents the height of the feature map, Indicates the number of channels of the feature map, A first medical record information-DSA image interactive screening transformation adjustment feature graph obtained by vectorizing a plurality of the first medical record information-DSA image interactive screening transformation adjustment values, A second medical record information-DSA image interactive screening transformation adjustment feature graph obtained by vectorizing a plurality of the second medical record information-DSA image interactive screening transformation adjustment values, It means adding by position. represents the interactive screening transformation adjustment weighted value, represents positional multiplication, Represents the optimized medical record information-DSA image interactive screening feature map.

[0098] That is, the applicant of this application took into account that the semantic coding feature vector of the medical record information text and the shallow-deep joint interaction feature map of the DSA image respectively represent the semantic coding features of the medical record information and the multi-scale bidirectional global attention enhanced image semantic features of the DSA image data. When performing feature interaction screening encoding based on metadata blocks, the feature representation differences of different dimensions of heterogeneous source data will also make the spatial composite semantic representation of the medical record information-DSA image interaction screening feature map have a complex spatial structure based on different local spatial distribution structures, thereby expecting to improve its classification regression convergence and generalization effects under complex spatial structures.

[0099] Therefore, in the preferred example, for the spatial structural information of the feature set of the medical record information-DSA image interactive screening feature map in the high-dimensional space, the scale-based frame transformation of each feature value of the medical record information-DSA image interactive screening feature map is performed by using the class norm space structured representation of the medical record information-DSA image interactive screening feature map as a reference window, and the spatial structure-based frame attention weight adjustment of each feature value of the medical record information-DSA image interactive screening feature map is realized to ensure the invariance of the spatial transformation (translation, scaling and rotation) of the medical record information-DSA image interactive screening feature map under the feature space interaction, thereby improving the convergence and generalization effect of the classification regression of the feature set of the medical record information-DSA image interactive screening feature map under the complex spatial structure representation, and improving the accuracy of the analysis result obtained by inputting it into the structural analysis module based on the classifier. In this way, intelligent analysis and detection of the state of intracranial aneurysms can be realized to determine whether the aneurysms are occluded, which can not only eliminate or reduce the differences caused by human factors, but also provide strong support for clinicians.

[0100] In summary, the intracranial aneurysm imaging detection system 100 based on deep learning according to the embodiment of the present application is explained, which first obtains DSA imaging data and medical record information corresponding to the DSA imaging data, and then uses data processing and image analysis algorithms based on artificial intelligence and deep learning to analyze the DSA imaging data and medical record information at the back end, as well as feature aggregation processing based on feature interaction and screening ideas, thereby realizing intelligent analysis and detection of the state of intracranial aneurysms to determine whether the aneurysm is occluded. This can not only eliminate or reduce the differences caused by human factors, but also comprehensively utilize the key semantics and complex interactive information in DSA images and medical record information to assist doctors in diagnosing intracranial aneurysms, providing strong support for clinicians.

[0101] As described above, the intracranial aneurysm imaging detection system 100 based on deep learning according to the embodiment of the present application can be implemented in various terminal devices. In one example, the intracranial aneurysm imaging detection system 100 based on deep learning can be integrated into the terminal device as a software module and / or a hardware module. For example, the intracranial aneurysm imaging detection system 100 based on deep learning can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the intracranial aneurysm imaging detection system 100 based on deep learning can also be one of the many hardware modules of the terminal device.

[0102] Alternatively, in another example, the deep learning-based intracranial aneurysm imaging detection system 100 and the terminal device may also be separate devices, and the deep learning-based intracranial aneurysm imaging detection system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0103] Figure 6 FIG. 1 is a flow chart of a method for detecting intracranial aneurysms based on deep learning according to an embodiment of the present application. Figure 6 As shown, according to the deep learning-based intracranial aneurysm imaging detection method of the embodiment of the present application, it includes: step 1, obtaining DSA image data and medical record information corresponding to the DSA image data; step 2, performing multi-scale feature extraction on the DSA image data to obtain the DSA image shallow-deep joint interaction feature; step 3, semantically encoding the medical record information corresponding to the DSA image data to obtain the medical record information text semantic coding feature; step 4, performing feature interactive screening encoding on the DSA image shallow-deep joint interaction feature and the medical record information text semantic coding feature to obtain the medical record information-DSA image interactive screening feature; wherein, step 4, It includes: performing linear transformation and masking processing on the semantic coding features of the medical record information text to obtain a masked first medical record information text semantic coding auxiliary feature mapping pattern representation and a masked second medical record information text semantic coding auxiliary feature mapping pattern representation; based on the masked first medical record information text semantic coding auxiliary feature mapping pattern representation and the masked second medical record information text semantic coding auxiliary feature mapping pattern representation, performing feature screening on the DSA image shallow-deep joint interaction feature to obtain the medical record information-DSA image interaction screening feature; step 5, based on the medical record information-DSA image interaction screening feature, determining the detection result used to indicate whether the aneurysm is occluded.

[0104] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned intracranial aneurysm imaging detection method based on deep learning have been referred to above. Figures 1 to 5 It has been introduced in detail in the description of the deep learning-based intracranial aneurysm imaging detection system, and therefore, its repeated description will be omitted.

[0105] In summary, the intracranial aneurysm imaging detection method based on deep learning according to the embodiment of the present application is explained, which first obtains DSA imaging data and medical record information corresponding to the DSA imaging data, and then uses data processing and image analysis algorithms based on artificial intelligence and deep learning to analyze the DSA imaging data and medical record information at the back end, as well as feature aggregation processing based on feature interaction and screening ideas, thereby realizing intelligent analysis and detection of the status of intracranial aneurysms to determine whether the aneurysm is occluded. This can not only eliminate or reduce the differences caused by human factors, but also comprehensively utilize the key semantics and complex interactive information in DSA images and medical record information to assist doctors in diagnosing intracranial aneurysms, providing strong support for clinicians.

Claims

1. A deep learning-based intracranial aneurysm imaging detection system, characterized in that: include: A data acquisition module, used to obtain DSA image data and medical record information corresponding to the DSA image data; An image data multi-scale feature extraction module, used for performing multi-scale feature extraction on the DSA image data to obtain DSA image shallow-deep joint interactive features; A medical record information text semantic coding module, used to semantically code the medical record information corresponding to the DSA image data to obtain a medical record information text semantic coding feature; A medical record information-image semantic interaction screening module, used for performing feature interaction screening and encoding on the DSA image shallow-deep joint interaction features and the medical record information text semantic coding features to obtain medical record information-DSA image interaction screening features; wherein the medical record information-image semantic interaction screening module includes: a feature mapping processing unit, used for performing linear transformation and masking processing on the medical record information text semantic coding features to obtain a masked first medical record information text semantic coding auxiliary feature mapping pattern representation and a masked second medical record information text semantic coding auxiliary feature mapping pattern representation; a feature interaction screening unit, used for performing feature screening on the DSA image shallow-deep joint interaction features based on the masked first medical record information text semantic coding auxiliary feature mapping pattern representation and the masked second medical record information text semantic coding auxiliary feature mapping pattern representation to obtain the medical record information-DSA image interaction screening features; The aneurysm occlusion detection module is used to determine a detection result indicating whether the aneurysm is occluded based on the medical record information-DSA image interactive screening feature.

2. The intracranial aneurysm imaging detection system based on deep learning according to claim 1, characterized in that: The image data multi-scale feature extraction module includes: An image data feature capture unit, used for inputting the DSA image data into a multi-scale image feature extractor based on a dilated pyramid network to obtain a DSA image shallow feature map and a DSA image deep feature map; The image shallow-deep feature joint interaction unit is used to perform feature joint interaction encoding based on kernel association on the DSA image shallow feature map and the DSA image deep feature map to obtain the DSA image shallow-deep joint interaction feature.

3. The intracranial aneurysm imaging detection system based on deep learning according to claim 2, characterized in that: The image shallow-deep feature joint interaction unit includes: An image feature distillation subunit, used for inputting the DSA image shallow feature map and the DSA image deep feature map into a kernel feature distillation network to obtain a DSA image shallow kernel feature vector and a DSA image deep kernel feature vector; An image kernel feature global correlation coding subunit is used to perform global correlation coding on the DSA image shallow kernel feature vector and the DSA image deep kernel feature vector to obtain a global coding matrix between the DSA image shallow and deep kernel features; The image shallow-deep kernel feature sparse mask subunit is used to input the global encoding matrix between the DSA image shallow-deep kernel features into the sparse mask unit to obtain the DSA image shallow-deep kernel feature correlation sparse weight matrix; An image feature position-by-position association interaction subunit is used to input the DSA image shallow feature map and the DSA image deep feature map into a position-by-position association interaction network to obtain a DSA image shallow-deep position-by-position interaction fusion feature map; The image shallow-deep layer position-by-position interactive fusion feature significant sparse modulation subunit is used to perform feature significant sparse modulation on the DSA image shallow-deep layer position-by-position interactive fusion feature map based on the associated sparse weight matrix between the shallow-deep kernel features of the DSA image to obtain the DSA image shallow-deep joint interactive feature map as the DSA image shallow-deep joint interactive feature.

4. The intracranial aneurysm imaging detection system based on deep learning according to claim 3, characterized in that: The image feature distillation subunit comprises: The image shallow feature decomposition secondary subunit is used to perform feature decomposition on the DSA image shallow feature map to obtain a sequence of DSA image shallow decomposition feature vectors; The image shallow decomposition feature difference calculation secondary subunit is used to calculate the difference characteristics of each DSA image shallow decomposition feature vector in the sequence of DSA image shallow decomposition feature vectors relative to other DSA image shallow decomposition feature vectors to obtain a DSA image shallow decomposition feature difference representation vector; The secondary subunit for normalization processing of the image shallow decomposition feature difference representation is used to normalize each eigenvalue in the DSA image shallow decomposition feature difference representation vector to obtain a normalized DSA image shallow decomposition feature difference representation vector; The secondary subunit for calculating the weighted sum of the shallow decomposition features of the image is used to calculate the weighted sum of the sequence of the shallow decomposition feature vectors of the DSA image using each eigenvalue in the normalized DSA image shallow decomposition feature difference representation vector as a weight to obtain the shallow kernel feature vector of the DSA image.

5. The intracranial aneurysm imaging detection system based on deep learning according to claim 4, characterized in that: The secondary subunit for calculating the difference in image shallow layer decomposition features is used to: Calculating the norm of the positional difference between each DSA image shallow decomposition feature vector and other DSA image shallow decomposition feature vectors in the sequence of DSA image shallow decomposition feature vectors to obtain a sequence of DSA image shallow decomposition feature difference norm representation values; Calculating the average value of the sequence of the DSA image shallow decomposition feature difference-norm representation values ​​to obtain the DSA image shallow decomposition feature difference-norm average representation value; A plurality of the DSA image shallow decomposition feature difference one-norm average representation values ​​are vectorized and arranged to obtain the DSA image shallow decomposition feature difference representation vector.

6. The intracranial aneurysm imaging detection system based on deep learning according to claim 5, characterized in that: The medical record information text semantic encoding module is used to: semantically encode the medical record information corresponding to the DSA image data to obtain a medical record information text semantic encoding feature vector as the medical record information text semantic encoding feature.

7. The intracranial aneurysm imaging detection system based on deep learning according to claim 6, characterized in that: The feature mapping processing unit comprises: A medical record information text feature linear change subunit, used to perform a linear transformation on the medical record information text semantic coding feature vector based on a first weight modifier matrix and a second weight modifier matrix to obtain a first medical record information text semantic coding auxiliary feature mapping mode representation vector and a second medical record information text semantic coding auxiliary feature mapping mode representation vector; A first medical record information text gradient amplitude mask processing subunit is used to perform gradient amplitude-based mask processing on the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the first medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain a masked first medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the masked first medical record information text semantic encoding auxiliary feature mapping pattern representation; The second medical record information text gradient amplitude mask processing subunit is used to perform gradient amplitude-based mask processing on the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector based on the gradient amplitude of each position in the second medical record information text semantic encoding auxiliary feature mapping pattern representation vector to obtain a masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector as the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation.

8. The intracranial aneurysm imaging detection system based on deep learning according to claim 7, characterized in that: The feature interactive screening unit comprises: A preliminary feature screening subunit, used for performing preliminary feature screening on the DSA image shallow-deep joint interactive feature map based on the masked first medical record information text semantic encoding auxiliary feature mapping mode representation vector and the hyperbolic tangent function to obtain the DSA image shallow-deep joint interactive feature map after preliminary screening; The secondary feature screening subunit is used to perform secondary feature screening on the shallow-deep joint interactive feature map of the DSA image after the preliminary screening based on the masked second medical record information text semantic encoding auxiliary feature mapping pattern representation vector and the Sigmoid function to obtain the medical record information-DSA image interactive screening feature map as the medical record information-DSA image interactive screening feature.

9. The intracranial aneurysm imaging detection system based on deep learning according to claim 8, characterized in that: The aneurysm occlusion detection module is used to: input the medical record information-DSA image interactive screening feature map into a classifier-based detection result generator to obtain a detection result, and the detection result is used to indicate whether the aneurysm is occluded.

10. A method for detecting intracranial aneurysms based on deep learning, characterized in that: include: Step 1, obtaining DSA image data and medical record information corresponding to the DSA image data; Step 2, performing multi-scale feature extraction on the DSA image data to obtain DSA image shallow-deep joint interactive features; Step 3, semantically encoding the medical record information corresponding to the DSA image data to obtain semantic encoding features of the medical record information text; Step 4, performing feature interactive screening and encoding on the DSA image shallow-deep joint interactive features and the medical record information text semantic coding features to obtain medical record information-DSA image interactive screening features; wherein, step 4 includes: performing linear transformation and masking processing on the medical record information text semantic coding features to obtain a masked first medical record information text semantic coding auxiliary feature mapping pattern representation and a masked second medical record information text semantic coding auxiliary feature mapping pattern representation; based on the masked first medical record information text semantic coding auxiliary feature mapping pattern representation and the masked second medical record information text semantic coding auxiliary feature mapping pattern representation, performing feature screening on the DSA image shallow-deep joint interactive features to obtain the medical record information-DSA image interactive screening features; Step 5: Determine a test result indicating whether the aneurysm is occluded based on the medical record information-DSA image interactive screening feature.

Images