A method, device and medium for constructing a tubular feature prediction model

Abstract

The present invention provides a method, system, device, medium and program product for constructing a tubular feature prediction model, and relates to the field of intelligent medical care. The method includes: obtaining paired NCE-CT and CE-CT images of aortic dissection in a training set sample; high-scale features and low-scale features of tubular structures in the NCE-CT images; obtaining an offset based on the high-scale features; guiding the position offset of the low-scale features based on the offsets, inputting the values ​​of the low-scale features after position offset and the offsets into a multi-head self-attention network for fusion to obtain tubular feature data; obtaining a student model classification result based on the tubular feature data; adjusting the pre-trained large model based on the CE-CT images to obtain a teacher model, and obtaining a corresponding teacher model classification result; optimizing the student model based on the teacher model classification result and the student model classification result, using knowledge distillation to obtain a constructed student model, which is the prediction model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical care, and more specifically, to a method, device, medium and program product for constructing a tubular feature prediction model. Background Art

[0002] Clinically, contrast-enhanced computed tomography (CE-CT) is the gold standard for diagnosing aortic dissection. However, CE-CT is expensive, complex, and exposes patients to contrast agents, which can cause allergic reactions or renal impairment, and in severe cases, lead to death. Non-contrast-enhanced computed tomography (NCE-CT) can diagnose AD without the use of contrast agents. However, due to the lack of contrast agents, the contrast of NCE-CT images is low, making it difficult to accurately distinguish tubular parts of the human body, such as the aorta, from surrounding tissues, especially at the site of lesions, resulting in an increased risk of misdiagnosis and missed diagnosis. Summary of the Invention

[0003] The present invention aims to address at least one of the technical problems existing in the prior art. To this end, the present invention provides a method, device, medium, and program product for constructing a tubular feature prediction model. By developing a method capable of highly accurate tubular structure identification on NCE-CT, the present method is beneficial in reducing missed or misdiagnosed conditions in early clinical screening for diseases requiring tubular structure identification, such as aortic dissection.

[0004] In a first aspect, the present application discloses a method for constructing a tubular feature prediction model, the method comprising:

[0005] S1: Obtain paired NCE-CT and CE-CT images of aortic dissection in the training set samples;

[0006] S2: High-scale and low-scale features of tubular structures in the NCE-CT images;

[0007] S3: Obtaining an offset based on the high-scale feature; guiding the position shift of the low-scale feature based on the offset, inputting the low-scale feature after position shift and the offset into a multi-head self-attention network for fusion to obtain tubular feature data; obtaining a student model classification result based on the tubular feature data;

[0008] S4: Based on the CE-CT image, the pre-trained large model is adjusted to obtain a teacher model, and the corresponding teacher model classification result is obtained; based on the teacher model classification result and the student model classification result, the student model is optimized using knowledge distillation, and the constructed student model is the prediction model.

[0009] In some embodiments, the method for obtaining the offset includes: S301: sequentially inputting high-scale features into a convolution layer and a normalization layer to obtain a query vector to capture key features inside the aorta; S302: mapping the query vector to obtain the offset;

[0010] Optionally, the query vector is obtained in S301 as follows:

[0011] Q = LayerNorm(Conv(Fhigh)); where Q is the query vector, Fhigh is the high-scale feature, conv is the convolution layer, and LayerNorm is the normalization layer;

[0012] Optionally, the formula for obtaining the offset in S302 is:

[0013] offset=Conv(gelu(LayerNorm(Conv(Q))));where offset is the offset, Conv is the convolution layer, LayerNorm is the normalization layer, and gelu is the activation function;

[0014] Optionally, the value of the low-scale feature after position shift in S3 includes the deformed key K and value V. The deformed key K and value V and the query vector Q are input into the multi-head self-attention network for fusion to obtain the tubular feature data FDefor.

[0015] Optionally, the formulas for obtaining K and V include:

[0016] K,V=Conv(grid sample (F low ,offset))

[0017] Among them, Flow is a low-scale feature, grid_sample is a Pytorch function;

[0018] Optionally, the formula for obtaining the tubular characteristic data FDefor includes:

[0019] F Defor =MSA(Q, K, V); MSA is a multi-head self-attention module.

[0020] In some embodiments, between S3 and S4, the method further includes S31: performing enhancement processing on the tubular feature data, and obtaining a classification result based on the enhanced tubular feature data;

[0021] Optionally, the enhancement processing includes: adding the tubular feature data to the query vector to obtain an original intermediate feature Mid; sequentially inputting the original intermediate feature Mid into a normalization layer (LayerNorm), a multi-layer perceptron (MLP), and a dropout layer (Dropath) for processing to obtain a processed intermediate feature, and taking the sum of the processed intermediate feature and the original intermediate feature to obtain the enhanced tubular feature data;

[0022] Optionally, the calculation formula of the original intermediate feature Mid is:

[0023] Mid=Q+Dropath(F Defor ); where FDefor is the tubular feature data;

[0024] Optionally, the formula for obtaining the enhanced tubular feature data is:

[0025] Out=Mid+Dropath(MLP(LayerNorm(Mid))), where out is the tubular feature data after the enhancement process;

[0026] Optionally, the method further includes: applying path discarding to process the tubular feature data, and adding the tubular feature data after the path discarding process to the query vector Q to obtain an original intermediate feature.

[0027] In some embodiments, between S3 and S4, the method further includes: S32, and the order of S32 and S31 is not limited;

[0028] S321: Segmenting the NCE-CT image to obtain tubular segmentation features;

[0029] S322: reshape and encode the tubular feature data and the tubular segmentation feature respectively to obtain processed tubular features and processed segmentation features;

[0030] S323: Obtain tubular hints, input the processed tubular features, tubular hints, and processed segmentation features into a cross-attention network with a multi-head attention network to obtain fused tubular features;

[0031] S324: adding the fused tubular feature to the tubular feature data element by element, and outputting a student model classification result based on the result obtained by the element-by-element addition;

[0032] Optionally, the tubular feature data in S324 is pooled data.

[0033] In some embodiments, after S1 and before S2, the method further comprises: performing standardized preprocessing on the paired NCE-CT and CE-CT images to obtain standardized NCE-CT data and standardized CE-CT data;

[0034] Optionally, the standardization preprocessing includes: extracting an ROI region of the aorta in the image to obtain an aortic mask; increasing the proportion of the aorta in the image based on the aortic mask to obtain an image with an increased aortic proportion; downsampling, adjusting the CT threshold, and performing data enhancement on the image with the increased aortic proportion to obtain the standardized NCE-CT data and the standardized CE-CT data;

[0035] Optionally, the method of increasing the proportion of the aorta in the image includes: cropping the circumscribed matrix of the image based on the aorta mask to increase the proportion of the aorta mask in the image;

[0036] Optionally, the CT threshold of the NCE-CT data is set to [0, 200];

[0037] Optionally, the CT threshold of the CE-CT data is set to [0, 800];

[0038] Optionally, the teacher model and the student model use any of the following algorithms: Total Segmentator, nn-Unet, ViT, CNN-BiLSTM, MobileViT; preferably, Total Segmentator.

[0039] A second aspect of the present application discloses a method for segmenting aortic tubular features, the method comprising:

[0040] Obtaining the NCE-CT image to be tested;

[0041] Inputting the NCE-CT image into the prediction model described in the first aspect of the present application to obtain a classification result;

[0042] Optionally, when the test is to detect whether the aorta contains dissection, the classification result is a result of containing dissection and not containing dissection.

[0043] A third aspect of the present application discloses a method for intraoperative blood vessel navigation, the method comprising:

[0044] Obtain intraoperative medical images (intraoperative CT or intraoperative ultrasound images);

[0045] extracting high-scale features and low-scale features of blood vessels in the intraoperative medical image;

[0046] An offset is obtained based on the high-scale feature, and the spatial displacement of the low-scale feature is guided based on the offset to obtain a real-time blood vessel feature.

[0047] In a fourth aspect, the present application discloses a computer device, comprising: a memory and a processor; the memory is used to store a computer program; and the processor executes the computer program to implement the steps of the above method.

[0048] In a fifth aspect, the present application discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned method when the computer program is executed by a processor.

[0049] In a sixth aspect, the present application discloses a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0050] This application has the following beneficial effects:

[0051] This paper designs a multi-task learning framework that can simultaneously learn and optimize multiple related tasks. Through shared representation and collaborative learning between tasks, the generalization ability and overall performance of the model are improved.

[0052] 1. The present application innovatively discloses a method for constructing a tubular feature prediction model. The method first uses a residual network to extract high-scale and low-scale features of tubular structures in an image, learns a feature set from the high-scale features, and adds it to the low-scale features, which easily causes feature displacement. This displacement shifts the network's focus to the interior of the tubular structure and introduces cross-scale information into the network to output tubular feature data; the accuracy of the tubular feature data is also ensured by enhancing the tubular feature data. The above is generally summarized as an aortic tubular feature guidance module in this application. This module can dynamically select and optimize features according to the specificity of the data and task requirements. By introducing this module, deformable attention technology is introduced, enabling the network to focus on the interior of the tubular structure in the image (for example, the inside of the aorta). At the same time, by introducing the tubular segmentation features based on NCE-CT images, the tubular segmentation features and the tubular feature data output by the aortic tubular feature guidance module are respectively input into the cross-attention network with a multi-head attention network to obtain the fused tubular features. The fused tubular features and the tubular feature data are added element by element to highlight the proportion of tubular features in the image and output tubular features with higher recognition accuracy.

[0053] 2. The model disclosed in this application has a wide range of applications, not only for the detection of aortic dissections. When segmentation masks for the vascular lumen, true lumen, and false lumen are present, they can be used as input to the module. This module then extracts these features and outputs them as higher-dimensional fused features. Furthermore, it can be used in intraoperative navigation scenarios.

[0054] The automated system developed in this application is capable of highly accurate identification of aortic structure in NCE-CT and thus accurate detection of aortic dissection, which is beneficial to reducing missed or misdiagnoses in early clinical screening of aortic dissection. The NCE-CT technology that combines the deformable self-attention module and the adaptive geometric information extraction module, the two modules are combined to achieve highly reliable AD diagnosis, significantly improving the recognition accuracy of aortic structure in NCE-CT images, overcoming the limitations of existing technologies in complex AD geometric morphology and low-contrast tissue recognition, and ensuring high-accuracy diagnosis of AD in the absence of contrast agents. Among them, the deformable self-attention module adaptively focuses on important features in the image at different scales, and enhances the ability to capture details and extract features of the aortic structure by adaptively adjusting the attention weights. The adaptive geometric information extraction module can effectively identify, extract and utilize the geometric information of the aorta, and effectively separate the geometric structure of the aorta from the NCE-CT image by using geometric constraints and morphological features, thereby improving the resolution of complex geometric morphology.

[0055] The technical solution of the present application can well solve the following technical problems: 1. The contrast between the aorta and the surrounding tissues is low, and it is difficult for the network to focus on the tubular structure of the aorta; 2. The correlation between the aortic geometric information and AD is not certain, and it is difficult to establish a relationship model between the aortic geometric information and AD. The existing modeling of aortic geometric information and AD simply extracts low-dimensional information and ignores the complexity of the relationship between the aortic geometric structure and AD. Existing blood vessel segmentation technology can extract the technical features of blood vessels, but because low-scale features contain more morphological features and high-scale features contain more semantic information, the existing blood vessel segmentation does not take into account the lack of interaction between feature levels; 3. The intensity value of the aortic intimal valve is similar to the intensity value of blood, which brings challenges to the network's recognition of the false lumen. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1This is a schematic diagram of the method flow provided by the first aspect of the embodiment of the present invention;

[0058] Figure 2 2 is a schematic diagram of a segmentation system for aortic tubular features provided by a second aspect of an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of the architecture of an exemplary computing device provided by an embodiment of the present invention;

[0061] Figure 5 is a schematic diagram of a storage medium provided by an embodiment of the present invention;

[0062] Figure 6 This is a diagram of the overall network structure provided by an embodiment of the present invention;

[0063] Figure 7 This is a network structure diagram of an aorta tubular feature guidance module provided by an embodiment of the present invention;

[0064] Figure 8 is a network structure diagram of an adaptive feature fusion module provided by an embodiment of the present invention;

[0065] Figure 9 This is the quantitative evaluation result of the model provided by the embodiment of the present invention compared with the commonly used segmentation model;

[0066] Figure 10 A comparison diagram of the aorta segmentation structure between the model provided by the embodiment of the present invention and a commonly used segmentation model;

[0067] Figure 11 It is a quantitative evaluation result obtained by internal verification of a commonly used deep learning classification model provided by an embodiment of the present invention;

[0068] Figure 12 It is the quantitative evaluation result of external validation of the commonly used deep learning classification model provided by the embodiment of the present invention using the data set of an external center;

[0069] Figure 13 This is a rendering of the aorta tubular feature guidance module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0071] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0072] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0073] Figure 1 1 is a flow chart of a method for constructing a tubular feature prediction model provided by an embodiment of the present invention. Specifically, the method includes the following steps:

[0074] S1: Obtain paired NCE-CT and CE-CT images of aortic dissection in the training set samples;

[0075] In some embodiments, NCE-CT non-enhanced computed tomography lacks contrast agents, and the contrast of NCE-CT images is low, making it difficult to accurately distinguish the aorta from surrounding tissues, especially lesions, resulting in an increased risk of misdiagnosis and missed diagnosis;

[0076] In some embodiments, after S1 and before S2, the method further comprises: performing standardized preprocessing on the paired NCE-CT and CE-CT images to obtain standardized NCE-CT data and standardized CE-CT data;

[0077] Optionally, the standardization preprocessing includes: extracting an ROI region of the aorta in the image to obtain an aortic mask; increasing the proportion of the aorta in the image based on the aortic mask to obtain an image with an increased aortic proportion; downsampling, adjusting the CT threshold, and performing data enhancement on the image with the increased aortic proportion to obtain the standardized NCE-CT data and the standardized CE-CT data;

[0078] Optionally, the method of increasing the proportion of the aorta in the image includes: cropping the circumscribed matrix of the image based on the aorta mask to increase the proportion of the aorta mask in the image;

[0079] Optionally, the CT threshold of the NCE-CT data is set to [0, 200]; the CT threshold of the CE-CT data is set to [0, 800];

[0080] S2: High-scale features and low-scale features of the tubular structure in the NCE-CT image; using a residual network to extract features in the NCE-CT image.

[0081] In some embodiments, the tubular structure may be a tubular structure such as a blood vessel in the human body, preferably the aorta;

[0082] S3: Obtaining an offset based on the high-scale feature; guiding the position offset of the low-scale feature based on the offset, inputting the low-scale feature after position offset and the offset into a multi-head self-attention network for fusion to obtain tubular feature data; obtaining a student model classification result based on the tubular feature data;

[0083] In some embodiments, the method for obtaining the offset includes: S301: sequentially inputting high-scale features into a convolution layer and a normalization layer to obtain a query vector Q to capture key features inside the aorta; S302: mapping the query vector Q to obtain the offset, and processing Q through additional convolution and activation function gelu to generate an offset;

[0084] Optionally, the query vector is obtained in S301 as follows:

[0085] Q = LayerNorm(Conv(Fhigh)); where Q is the query vector, Fhigh is the high-scale feature, conv is the convolution layer, and LayerNorm is the normalization layer;

[0086] Optionally, the formula for obtaining the offset in S302 is:

[0087] offset=Conv(gelu(LayerNorm(Conv(Q))));where offset is the offset, Conv is the convolution layer, LayerNorm is the normalization layer, and gelu is the activation function;

[0088] Optionally, the value of the low-scale feature after position shift in S3 includes the deformed key K and value V. The deformed key K and value V and the query vector Q are input into the multi-head self-attention network for fusion to obtain the tubular feature data FDefor.

[0089] Optionally, the formulas for obtaining K and V include:

[0090] K,V=Conv(grid sample (F low ,offset))

[0091] Among them, Flow is a low-scale feature, grid_sample is a Pytorch function;

[0092] Optionally, the formula for obtaining the tubular characteristic data FDefor includes:

[0093] F Defor =MSA(Q, K, V); MSA is a multi-head self-attention module.

[0094] In some embodiments, between S3 and S4, the method further includes S31: performing enhancement processing on the tubular feature data, and obtaining a classification result based on the enhanced tubular feature data;

[0095] Optionally, the enhancement processing includes: adding the tubular feature data FDefor to the query vector Q to obtain an original intermediate feature Mid; sequentially inputting the original intermediate feature Mid into a normalization layer (LayerNorm), a multi-layer perceptron (MLP), and a dropout layer (Dropath) for processing to obtain a processed intermediate feature, and taking the sum of the processed intermediate feature and the original intermediate feature to obtain enhanced tubular feature data. This process enhances the nonlinear representation capability of the feature, enabling the network to better capture complex internal structural information of the aorta;

[0096] Optionally, the calculation formula of the original intermediate feature Mid is:

[0097] Mid=Q+Dropath(F Defor ); where FDefor is the tubular feature data;

[0098] Optionally, the formula for obtaining the enhanced tubular feature data is:

[0099] Out=Mid+Dropath(MLP(LayerNorm(Mid))), where out is the tubular feature data after the enhancement process;

[0100] Optionally, the method further includes: applying path discarding to process the tubular feature data, and adding the tubular feature data after the path discarding process to the query vector Q to obtain an original intermediate feature.

[0101] In some embodiments, between S3 and S4, the method further includes S32, and the order of S32 and S31 is not limited;

[0102] S321: Segmenting the NCE-CT image to obtain tubular segmentation features;

[0103] S322: reshape and encode the tubular feature data and the tubular segmentation feature respectively to obtain processed tubular features and processed segmentation features; in some embodiments, reshape the tubular feature data and the tubular segmentation feature, and use a transform encoder to encode the reshaped aortic tubular feature and the aortic segmentation feature to obtain encoded tubular features and encoded segmentation features; specifically, reshape the aortic tubular feature Fcls, and use a transform encoder to encode the reshaped aortic tubular feature Fcls (deformed self-attention network) to obtain the encoded tubular feature Xcls (Xcls). cls =En cls (Reshape(F cls ), Reshape(F cls ))); Reshape the segmentation feature Fseg, use the transformation encoder to encode the reshaped segmentation feature Fseg (deformation self-attention network), and obtain the encoded segmentation feature Xseg (X seg =En seg (Reshape(F seg ), Reshape(F seg )));

[0104] S323: Obtain tubular prompt promptcls, input the processed tubular features, tubular prompts, and processed segmentation features into a cross-attention network with a multi-head attention network to obtain a fused tubular feature; in some embodiments, obtain tubular prompt promptcls, input the encoded tubular feature Xcls, tubular prompt promptcls(Query), and encoded segmentation feature Xseg(K and V) into a cross-attention network with a multi-head attention network to obtain a fused tubular feature Ycls(Y cls =Pool(En share (Concat(X cls , prompt cls ), X seg )));

[0105] S324: adding the fused tubular feature to the tubular feature data element by element, and outputting a student model classification result based on the result obtained by the element-by-element addition;

[0106] Optionally, the tubular feature data in S324 is pooled data.

[0107] S4: Based on the CE-CT images, the pre-trained large model is adjusted to obtain a teacher model, and the corresponding teacher model aortic dissection classification result is obtained; based on the classification results of the teacher model and the classification results of the student model, the student model is optimized using knowledge distillation, and the constructed student model is the prediction model.

[0108] In some embodiments, a teacher model is used to train a student model based on knowledge distillation technology. Optionally, the teacher model and the student model use any of the following algorithms: Total Segmentator, ViT, CNN-BiLSTM, MobileViT, preferably Total Segmentator, which covers aorta learning.

[0109] In some embodiments, between S3 and S4, the method further includes S33 (the order of S33, S32 and S31 is not limited):

[0110] Get the segmentation prompt promptseg, input the encoded segmentation feature and the segmentation prompt and the encoded tubular feature into the cross-attention network with the multi-head attention network to obtain the fused segmentation feature Yseg; Specifically, get the segmentation prompt promptseg, input the encoded segmentation feature Xseg and the segmentation prompt promptseg(Query), and the encoded tubular feature Xcls(K and V) into the cross-attention network with the multi-head attention network to obtain the fused segmentation feature Yseg(Y seg =Reshape(En share (Concat(X seg , prompt seg ), X cls ))); Through this adaptive geometric feature fusion module, by learning the importance weights of features, effective feature screening and combination are achieved. In addition to being used in the aorta segmentation field disclosed in this embodiment, the method of this module can also be used in specific application scenarios such as medical image registration and diagnosis of different diseases: a first image and a second image to be tested are obtained, features in the first image and the second image are extracted to obtain first image features and second image features; the first image features and the second image features are fused. The first image and the second image are images of different modalities or images at different time points, or images of different individuals;

[0111] The fused segmentation feature Yseg is input into a multi-head self-attention network. The result obtained after input is element-wise added to the aortic segmentation feature and upsampled (enlarged image) through a U-Net to obtain the aortic segmentation result. The result obtained after input is the learned fused segmentation feature Yseg; the learned fused segmentation feature Yseg is used as the high-level fused segmentation feature for the aortic dissection segmentation task. Optionally, the standardized NCE-CT data is roughly segmented using the totalsegmentator to obtain a coarse segmentation feature, which cannot accurately segment the patient's aortic structure.

[0112] A second aspect of the present application discloses a method for segmenting aortic tubular features, the method comprising:

[0113] Obtaining the NCE-CT image to be tested;

[0114] The NCE-CT image is input into the prediction model described in the first aspect of the present application to obtain a classification result; when the test is to detect whether the aorta contains a dissection, the classification result is a result of containing a dissection or not containing a dissection.

[0115] In some embodiments, the terms "subject" or "person to be tested" or "test sample" as used herein refer to any animal (e.g., mammal), including but not limited to humans, non-human primates, rodents, etc., which will be the recipient of a specific treatment. Generally, the terms "subject" and "patient" are used interchangeably herein when referring to human subjects. Preferably, the subject is a human. In some embodiments, the test sample is a patient clinically used to undergo a prognostic assessment.

[0116] A third aspect of the present application discloses a method for intraoperative blood vessel navigation, the method comprising:

[0117] Obtain intraoperative medical images (intraoperative CT or intraoperative ultrasound images);

[0118] extracting high-scale features and low-scale features of blood vessels in the intraoperative medical image;

[0119] An offset is obtained based on the high-scale feature, and the spatial displacement of the low-scale feature is guided based on the offset to obtain a real-time blood vessel feature.

[0120] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention, such as Figure 3 As shown, the device may include: one or more processors, and one or more memories; wherein the memories store computer-readable codes, and when the computer-readable codes are run by the one or more processors, the method described above may be executed.

[0121] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It can implement or execute the various methods, operations, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor can be a microprocessor or any conventional processor, etc., and can be an X86 architecture or an ARM architecture.

[0122] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the disclosed embodiments are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0123] For example, the method or apparatus according to the embodiment of the present disclosure may also be implemented by Figure 4 The architecture of the computing device 3000 shown in FIG. Figure 4 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the method provided in the present disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of a computing device are shown.

[0124] The embodiment of the present invention further provides a computer-readable storage medium, such as Figure 5As shown, it is a schematic diagram of a storage medium provided in an embodiment of the present invention, and computer-readable instructions 4010 are stored on the computer storage medium 4020. When the computer-readable instructions 4010 are executed by the processor, the method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0125] The embodiments of the present disclosure further provide a computer program product or system, including a computer program, which implements the steps of the above method when executed by a processor.

[0126] In some embodiments, this embodiment further discloses a system for constructing a tubular feature prediction model, the system comprising:

[0127] The first acquisition module 201 is used to obtain paired NCE-CT and CE-CT images of aortic dissection in the training set samples;

[0128] A feature extraction module 202 for extracting high-scale features and low-scale features of tubular structures in the NCE-CT image;

[0129] The tubular feature data calculation module 203 is configured to obtain an offset based on the high-scale feature; guide the position shift of the low-scale feature based on the offset, input the low-scale feature position shift value and the offset into a multi-head self-attention network for fusion to obtain tubular feature data; and obtain a student model classification result based on the tubular feature data.

[0130] The model optimization module 204 is used to adjust the pre-trained large model based on the CE-CT image to obtain a teacher model and obtain the corresponding teacher model classification result; based on the teacher model classification result and the student model classification result, the student model is optimized by using knowledge distillation to obtain the constructed student model, which is the prediction model.

[0131] In some embodiments, this embodiment further discloses a system for segmenting aortic tubular features, the system comprising:

[0132] The second acquisition module is used to acquire the NCE-CT image to be tested;

[0133] The segmentation result output module is used to input the NCE-CT image into the prediction model described in the first aspect of the present application to obtain a classification result.

[0134] In some embodiments, this embodiment further discloses a system for intraoperative blood vessel navigation, the system comprising:

[0135] A third acquisition module is used to acquire intraoperative medical images (intraoperative CT or intraoperative ultrasound images);

[0136] a multidimensional feature extraction module, configured to extract high-scale features and low-scale features of blood vessels in the intraoperative medical image;

[0137] The real-time blood vessel feature extraction module is used to obtain an offset based on the high-scale feature, and to guide the spatial displacement of the low-scale feature based on the offset to obtain the real-time blood vessel feature.

[0138] Example

[0139] The overall network structure diagram designed in this embodiment is as follows Figure 6 As shown, specifically, the flow chart of the end-to-end detection system for aortic dissection, wherein the high-dimensional feature is the high-scale feature Fhigh, the low-dimensional feature is the low-scale feature Flow, and the tubular feature is Fdefor.

[0140] The aortic dissection detection system proposed in the present invention mainly includes: first, paired NCE-CT and CE-CT images of aortic dissection patients from multiple centers are collected as the input of the system, and the data are subjected to standardized preprocessing such as image cropping and resampling through a multi-center data preprocessing module. The standardized data is then subjected to an aortic tubular feature guidance module and an adaptive geometric feature fusion module. Finally, the fused features are sent to a deep learning classification network for automatic diagnosis to obtain a diagnosis result.

[0141] Specifically, 1) Multi-center data preprocessing module: First, the NCE-CT and CE-CT images are processed using a deep learning medical segmentation model to extract the region of interest (ROI) of the aorta, thereby obtaining an aortic mask. Based on this mask, the image's bounding box matrix is ​​cropped to increase the proportion of the aorta in the image. Next, the image is downsampled and the CT threshold is adjusted, and data augmentation is performed to ensure that the multi-center data is input in a uniform size of 128×128×256. The CT thresholds for NCE-CT and CE-CT are set to [0,200] and [0,800], respectively. Finally, these preprocessed data are learned end-to-end through a neural network.

[0142] 2) Aortic tubular feature guidance module: After preprocessing, the image is input into the aortic tubular feature guidance module. This module uses a multi-scale convolutional neural network (CNN) and a self-attention structure to extract features at different scales from the image. The self-attention mechanism is an attention mechanism used to capture long-range dependencies in the input sequence. It plays an important role in the tubular feature guidance module of the end-to-end aortic dissection detection system, helping the model to better learn the tubular features in the aortic image. Specifically, the input of this module includes high-scale features Fhigh and low-scale features Flow. We first map Fhigh to Q and then map Q to an offset.

[0143] Q=LayerNorm(Conv(Fhigh))#(1)

[0144] offset=Conv(gelu(LayerNorm(Conv(Q))))#(2)

[0145] The offset is then applied to F low , and mapped to K and V, and then the deformation feature map is obtained through the multi-head self-attention module (MSA), and F Defor express.

[0146] K,V=Conv(grid sample (F low ,offset))#(3)

[0147] F Defor =MSA(Q, K, V)#(4)

[0148] Finally, perform the following coding to obtain the output result of the tubular feature.

[0149] Mid=Q+Dropath(F Defor )#(5)

[0150] Out=Mid+Dropath(MLP(LayerNorm(Mid)))#(6)

[0151] Among them, LayerNorm is the normalization layer, Conv is the convolution, gelu is the activation function, grid_sample is the Pytorch function: torch.nn.functional.grid sample, MSA is multi-head self-attention, Dropath is path elimination, and MLP is multi-layer perceptron. The specific implementation network structure of this module is as follows Figure 7 As shown, the high-dimensional feature is the high-scale feature Fhigh, the low-dimensional feature is the low-scale feature Flow, and the tubular feature is Fdefor.

[0152] 3) Adaptive feature fusion module

[0153] Given F seg The high-level segmentation feature map and F cls After representing the aorta tubular high-level features, they are first reshaped and then sent to the transformer encoder for preliminary encoding.

[0154] X cls =En cls (Reshape(F cls ), Reshape(F cls ))#(7)

[0155] X seg =En seg (Reshape(F seg ), Reshape(F seg ))#(8)

[0156] Then a cross-attention module is designed. To enable this module to distinguish the sources of query features, segmentation and classification cues Xcls and Xseg are added for differentiation.

[0157] Y cls =Pool(En share (Concat(X cls , prompt cls ), X seg ))#(9)

[0158] Y seg =Reshape(En share (Concat(X seg , prompt seg ), X cls ))#(10)

[0159] Among them, Encls, Enseg and Enshare are transform encoders. The two results Ycls and Yseg will return the classification network and segmentation network respectively to enhance their functions. The specific implementation network structure of this module is shown in the figure below: Figure 8 As shown; the segmentation feature at the input end is the tubular segmentation feature in S3021, and the tubular feature is Figure 7 The output tubular feature data, the fused segmentation feature at the output end is Yseg, and the fused tubular feature is Ycls.

[0160] 4) Deep learning classification module

[0161] During network training, network parameters are updated through loss calculation, gradient descent, and backpropagation. The network loss consists of three parts: the classification loss Losscls, the classification loss Lossseg, and the similarity loss Losssim. Specifically, Losscls is calculated using the cross-entropy loss, as shown in Equation 11; Lossseg is calculated using the weighted sum of the Dice loss LossDice and the cross-entropy loss LossCrEn, as shown in Equation 14. The total model loss is shown in Equation 15.

[0162] Loss cls =-γ1ylog p-γ2(1-y)log(1-p)#(11)

[0163] Loss CrEn =-γ3ylog p-γ4(1-y)log(1-p)#(12)

[0164]

[0165] Loss seg =Loss CrEn +Loss Dice #(14)

[0166] Loss total =Loss cls +Loss seg #(15)

[0167] The AdamW optimizer is used in combination with the total loss function for network learning, and finally the diagnostic output results are obtained.

[0168] 5) Diagnostic output module

[0169] In the embodiment of the present invention, quantitative evaluation is used to analyze the performance of the model, and the advantages of each module are demonstrated by combining ablation comparative experiments. For the segmentation task, we use Dic eThe DSC, Jaccard index (JI), mean intersection over union (MIoU), and frequency weighted intersection over union (FWIoU) are used for quantitative evaluation. e It is a method used to measure the overlap between the segmentation result and the true segmentation result. JI is an indicator similar to the Dice similarity coefficient, which is used to measure the overlap between the segmentation result and the true segmentation result. MIoU is the average of the DSC or JI indicators of multiple segmentation results. FWIoU is an indicator based on area, which is used to measure the overlap between the segmentation result and the true segmentation result. The expression is as follows:

[0170]

[0171] For classification tasks, the following are used: Accuracy, Sensitivity, Specificity, F1 score (F1) and Area Under Curve (AUC). Accuracy refers to the proportion of the number of samples correctly predicted by the model to the total number of samples. Sensitivity, also known as recall rate, refers to the proportion of all true positive cases correctly identified by the model. Specificity, also known as precision rate, refers to the proportion of all negative cases correctly identified by the model. The F1 score is the harmonic mean of sensitivity and specificity, which can comprehensively measure the recall and precision of the model. Accuracy refers to the proportion of the number of samples correctly predicted by the model to the total number of samples. AUC is the value of the area under the ROC curve, which can measure the model's ability to classify positive and negative samples. The ROC curve is a curve that plots the relationship between sensitivity and specificity. The evaluation index expression is as follows:

[0172]

[0173] Where TP is the number of samples predicted to be positive and actually positive, FP is the number of samples predicted to be positive and actually negative, and FN is the number of samples predicted to be negative and actually positive. A_i is the area of ​​the i-th true segmentation region, and A_tot is the sum of the areas of all true segmentation regions.

[0174] The present invention is trained and internally validated on aortic dissection datasets obtained from three medical centers. For the segmentation task, the quantitative indicators of the segmentation results are as follows: Figure 9As shown. Compared with common medical image segmentation models, our framework achieved the best results. Specifically, the segmentation results of U-net achieved DSC of 0.864, JI of 0.762, MioU of 0.878, and FWIoU of 0.991. It is lower than nn-U-net, with DSC of 0.880, JI of 0.786, MIoU of 0.891, and FWIoU of 0.992. In comparison, compared with nn-U-net, the DSC of our proposed framework is improved by 4.8%, JI is improved by 8.1%, MIoU is improved by 4.1%, and FWIoU is improved by 0.3%267. These segmentation results are also reflected in the visualization, as shown in Figure 10 As shown in Figure 2, the cavity segmentation result of U-net is not smooth enough, as shown by the arrow, and the bifurcation segmentation result of nn-U-net shows obvious under-segmentation, as shown by the arrow. Compared with them, our result is smoother and achieves better results on the aortic bifurcation.

[0175] In the classification task, we compared several of the most commonly used deep learning classification models. The quantitative classification results of internal verification are as follows: Figure 11 As shown in Figure 2, compared with common classification networks and previous studies on AD classification, our framework achieved the highest accuracy of 91.1%, the highest sensitivity of 92.5%, and the highest F1 score of 92.2%. In addition, to test the generalization performance of the model, we used an external center dataset for external validation, and the quantitative results are shown in Figure 2. Figure 12 As shown in Figure 3, our framework still achieves the highest accuracy of 84.0%, the highest sensitivity of 88.8%, and the highest F1 score of 83.6%.

[0176] Finally, we show grad-CAM to visualize whether the feature-guided module focuses the network’s attention from the surrounding tissue to the tubular structure of the aorta. Figure 13 As shown in the figure, we display grad-CAM in the coronal and sagittal planes before and after feature guidance to visually demonstrate the effect of the module. In the sagittal plane, before adding feature guidance, the model's attention was focused on the aortic arch, but the attention strength was insufficient. After adding feature guidance, the model's attention to the aortic tubular structure region was significantly enhanced. In the coronal plane, before using feature guidance, the model's attention area deviated slightly from the lumen region. After adding feature guidance, the model's attention was correctly focused on the aortic tubular structure region. It is clear that after adding feature guidance, the network's attention is more focused on the aortic tubular structure.

[0177] 1) This paper proposes a deep learning-based framework that can effectively process and fuse multimodal medical image data, enabling efficient data utilization and comprehensive analysis. This innovation ensures greater accuracy and robustness when processing multimodal data.

[0178] 2) Establish a standard process for preprocessing multi-center aortic NCE-CT and CE-CT imaging data

[0179] 3) The present invention introduces a feature guidance module that can dynamically select and optimize features according to the characteristics of the data and task requirements, thereby improving the performance and adaptability of the model.

[0180] 4) The present invention introduces adaptive feature fusion, which achieves effective feature screening and combination by learning the importance weights of features.

[0181] In summary, this invention enables end-to-end, accurate detection of aortic dissection using NCE-CT images. Current image processing algorithms and feature extraction methods can only extract low-dimensional geometric information of the aorta. Our model, guided by the aortic tubular features and combined with adaptive feature fusion, allows the model to focus on the aortic interior and address the geometric complexity of the aortic structure. Furthermore, within our multi-task model framework, standardized preprocessing of data collected from multiple centers automatically segments the aortic mask, reducing diagnostic errors associated with novel or rare lesions.

[0182] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0183] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the disclosed embodiments are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0184] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0186] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0188] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will appreciate that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A method for constructing a tubular feature prediction model, characterized in that: The method comprises: S1: Obtain paired NCE-CT and CE-CT images of aortic dissection in the training set samples; S2: Extract high-scale and low-scale features of tubular structures in NCE-CT images; S3: obtaining an offset based on the high-scale feature; guiding the position offset of the low-scale feature based on the offset, inputting the value of the low-scale feature after position offset and the offset into the multi-head self-attention network for fusion to obtain tubular feature data; obtaining the student model classification result based on the tubular feature data; the method for obtaining the offset includes: S301: sequentially inputting the high-scale features into the convolution layer and the normalization layer to obtain a query vector to capture the key features inside the aorta; S302: mapping the query vector to obtain the offset; the query vector acquisition formula is: ; Where Q is the query vector, Fhigh is the high-scale feature, conv is the convolution layer, and LayerNorm is the normalization layer; the formula for obtaining the offset is: ; Among them, offset is the offset, Conv is the convolution layer, LayerNorm is the normalization layer, and gelu is the activation function; S4: Based on the CE-CT image, the pre-trained large model is adjusted to obtain a teacher model, and the corresponding teacher model classification result is obtained; based on the teacher model classification result and the student model classification result, the student model is optimized using knowledge distillation, and the constructed student model is the prediction model.

2. The method for constructing a tubular feature prediction model according to claim 1, characterized in that: The value of the low-scale feature after position shift in S3 includes the deformed key K and value V. The deformed key K and value V and the query vector Q are input into the multi-head self-attention network for fusion to obtain the tubular feature data F. Defor .

3. The method for constructing a tubular feature prediction model according to claim 2, characterized in that: The formulas for obtaining K and V include: ; Among them, Flow is a low-scale feature, grid sample is a Pytorch function.

4. The method for constructing a tubular feature prediction model according to claim 3, characterized in that: The tubular characteristic data F Defor The formula for obtaining includes: ; MSA is the multi-head self-attention module.

5. The method for constructing a tubular feature prediction model according to claim 4, characterized in that: Between S3 and S4, the method further includes S31: performing enhancement processing on the tubular feature data, and obtaining a classification result based on the enhanced tubular feature data.

6. The method for constructing a tubular feature prediction model according to claim 5, characterized in that: The enhancement processing includes: adding the tubular feature data to the query vector to obtain the original intermediate feature Mid; sequentially inputting the original intermediate feature Mid into a normalization layer, a multi-layer perceptron and a discard layer for processing to obtain a processed intermediate feature, taking the sum of the processed intermediate feature and the original intermediate feature to obtain the tubular feature data after enhancement processing.

7. The method for constructing a tubular feature prediction model according to claim 6, characterized in that: The calculation formula of the original intermediate feature Mid is: Among them, F Defor It is tubular feature data, and Dropath is the discarded layer.

8. The method for constructing a tubular feature prediction model according to claim 7, characterized in that: The formula for obtaining the tubular feature data after the enhancement processing is: , where out is the tubular feature data after the enhancement processing, and MLP is a multi-layer perceptron.

9. The method for constructing a tubular feature prediction model according to claim 8, characterized in that: Between S3 and S4, the method further includes S32, and the order of S32 and S31 is not limited; S32 includes: S321: performing segmentation based on the NCE-CT image to obtain tubular segmentation features; S322: reshape and encode the tubular feature data and the tubular segmentation feature respectively to obtain processed tubular features and processed segmentation features; S323: Obtain tubular hints, input the processed tubular features, tubular hints, and processed segmentation features into a cross-attention network with a multi-head attention network to obtain fused tubular features; S324: Add the fused tubular feature and the tubular feature data element by element, and output the student model classification result based on the result obtained by the element-by-element addition.

10. The method for constructing a tubular feature prediction model according to claim 9, characterized in that: The tubular feature data in S324 is pooled data.

11. The method for constructing a tubular feature prediction model according to any one of claims 1 to 10, characterized in that: After S1 and before S2, the method further includes: performing standardized preprocessing on the paired NCE-CT and CE-CT images to obtain standardized NCE-CT data and standardized CE-CT data.

12. The method for constructing a tubular feature prediction model according to claim 11, characterized in that: The standardized preprocessing includes: extracting the ROI area of ​​the aorta in the image to obtain an aortic mask; increasing the proportion of the aorta in the image based on the aortic mask to obtain an image with an increased aortic proportion; downsampling, CT threshold adjustment, and data enhancement of the image with an increased aortic proportion to obtain the standardized NCE-CT data and standardized CE-CT data.

13. The method for constructing a tubular feature prediction model according to claim 12, characterized in that: The method for increasing the proportion of the aorta in the image includes: cropping the circumscribed matrix of the image based on the aorta mask, and increasing the proportion of the aorta mask in the image.

14. The method for constructing a tubular feature prediction model according to claim 13, characterized in that: The CT threshold of the NCE-CT data is set to [0, 200].

15. The method for constructing a tubular feature prediction model according to claim 13, characterized in that: The CT threshold of the CE-CT data is set to [0, 800].

16. The method for constructing a tubular feature prediction model according to claim 9, characterized in that: The teacher model and student model use any of the following algorithms: Total Segmentator, nn-Unet, ViT, CNN-BiLSTM, MobileViT.

17. A method for segmenting aortic tubular features, characterized in that: The method comprises: Acquire the NCE-CT image to be tested; The NCE-CT image is input into the prediction model constructed by the method described in any one of claims 1 to 16 to obtain a classification result.

18. The method for segmenting the aorta tubular feature according to claim 17, characterized in that: When the test is to detect whether the aorta contains dissection, the classification result is the result of containing dissection and not containing dissection.

19. A computer device, characterized in that: The device comprises: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 18.

20. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 18 are implemented.

21. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 18 are implemented.

Images