Two-stage full 3D aneurysm segmentation method and system

Abstract

The application belongs to the technical field of image processing, and specifically discloses a two-stage full 3D aneurysm segmentation method and system. The method acquires a 3D aneurysm detection image and performs pretreatment. A random data enhancement method is used to perform data enhancement on the pretreated 3D aneurysm detection image. A plurality of samples are generated on the data-enhanced 3D aneurysm detection image. A coarse detection network is optimized through neural network architecture search. Based on the plurality of samples, the coarse detection network is trained, and a region of interest in the sample is extracted. The region of interest is taken as input to train a fine segmentation network. The fine segmentation network segments an aneurysm in the region of interest from the background to obtain a final fine segmentation image. With the technical solution, the trained coarse detection network is used to infer the input image, quickly locate the position of the aneurysm target, and then the fine segmentation network is used to finely segment the region of interest, thereby improving the accuracy and detail expression ability of aneurysm segmentation.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a two-stage full 3D aneurysm segmentation method and system. Background Technology

[0002] An aneurysm is a common vascular disease, typically referring to a localized expansion and aneurysmal dilation of the blood vessel wall. The formation of an aneurysm can lead to weakening of the vessel wall, rupture, and bleeding, which can be life-threatening in severe cases. Therefore, early diagnosis and treatment of aneurysms are crucial for improving patient survival rates.

[0003] Currently, medical imaging technologies such as computed tomography (CT) and magnetic resonance imaging (MRI) are widely used in the diagnosis and evaluation of aneurysms. However, due to the diverse morphology and location of aneurysms in images, accurate segmentation remains a challenging problem. Traditional segmentation methods rely on manual operation, which is time-consuming, labor-intensive, and prone to subjective errors. Automated segmentation methods typically extract aneurysm regions based on specific image features or thresholds, but their accuracy and robustness are limited due to the variability in aneurysm morphology and location.

[0004] 2D segmentation and 3D segmentation are two commonly used methods in medical image processing, each with different characteristics and application scenarios when processing and analyzing three-dimensional medical images. 2D segmentation treats each two-dimensional slice in a medical image as a separate image, typically using pixel-level image processing and machine learning algorithms. 2D segmentation methods are relatively simple and fast, suitable for processing large numbers of two-dimensional images, such as slices from CT scans or MRI sequences. It can be used to detect and segment smaller structures or lesions, such as tumors, blood vessels, and lesions. However, 2D segmentation ignores information in the depth direction of the image and may not accurately capture the morphology and positional relationships of three-dimensional structures.

[0005] 3D segmentation directly segments complete three-dimensional medical images. This method considers the spatial relationships of the image in three dimensions, enabling more accurate capture of the morphology and location of three-dimensional structures. 3D segmentation methods are typically based on voxel-level image processing and machine learning algorithms, labeling voxels in the 3D image as structures or lesions of interest. It is suitable for tasks requiring consideration of 3D contextual information, such as organ segmentation, aneurysm segmentation, and brain tumor segmentation. Compared to 2D segmentation, 3D segmentation offers higher accuracy and robustness, but also requires more computational resources and time. In summary, 2D segmentation is suitable for processing large numbers of 2D images and has certain advantages for segmenting smaller structures or lesions; while 3D segmentation considers the three-dimensional spatial relationships of the image, making it suitable for tasks requiring consideration of the morphological and positional relationships of three-dimensional structures, and providing more accurate segmentation results.

[0006] Currently, there are various deep learning solutions for medical image processing, primarily focusing on optimizing the U-Net model. Inspired by image translation tasks, U-Net's encoder is analogous to the image encoding process, while the decoder is analogous to the image decoding and reconstruction process. U-Net has been widely applied in medical image segmentation, such as lung segmentation, cell segmentation, and liver segmentation. Its simple yet effective structure makes it an important benchmark model in image segmentation tasks and provides a foundation for subsequent research and improvement. However, most existing methods typically require significant computational resources and time because they rely on complex neural networks, which may not be suitable for situations requiring real-time results in clinical practice. They also require large-scale labeled medical image data for training, which is difficult to obtain, especially for rare diseases or novel aneurysms. Furthermore, they may be prone to overfitting on small sample data, leading to poor performance on new data. Due to the wide scale variation of aneurysms, some methods struggle to adapt to aneurysms of different scales, thus requiring further customization and tuning. Summary of the Invention

[0007] The purpose of this invention is to provide a two-stage full 3D aneurysm segmentation method and system, which improves the accuracy and detail representation of aneurysm segmentation.

[0008] To achieve the above objectives, the basic solution of the present invention is: a two-stage full 3D aneurysm segmentation method, comprising the following steps:

[0009] Acquire 3D aneurysm detection images and perform preprocessing;

[0010] A random data augmentation method was used to augment the preprocessed 3D aneurysm detection images.

[0011] Multiple samples are generated on the data-enhanced 3D aneurysm detection image;

[0012] Optimize the coarse detection network through neural network architecture search;

[0013] A coarse detection network is trained based on multiple samples, and regions of interest are extracted from the samples.

[0014] The region of interest is used as input to train a fine segmentation network, which then segments the aneurysm within the region of interest from the background to obtain the final finely segmented image.

[0015] The working principle and beneficial effects of this basic scheme are as follows: In the first stage, this technical scheme uses a pre-trained coarse detection network to infer the location of the aneurysm target from the input image, quickly locating the position of the aneurysm target. Through the coarse detection network, candidate aneurysm target regions can be obtained, and these regions are considered to have a high probability of containing an aneurysm.

[0016] In the second stage, the candidate aneurysm target regions obtained in the first stage are extracted, and these regions are further precisely segmented using a trained fine segmentation network. The fine segmentation network makes pixel-level predictions for each candidate region, distinguishing the aneurysm target from the background. This two-stage inference method fully utilizes the fast localization capability of the coarse detection network and the fine segmentation capability of the fine segmentation network.

[0017] The coarse detection stage can quickly filter out candidate regions, reducing the computational burden of segmenting the entire image. The fine segmentation stage then performs more precise segmentation of the candidate regions, improving the accuracy and detail representation of aneurysm segmentation. This method effectively improves aneurysm detection and segmentation in practical applications, and has high computational efficiency, making it suitable for processing large-scale and complex medical image data.

[0018] Furthermore, the method for preprocessing 3D aneurysm detection images is as follows:

[0019] Remove image data with labeled voxels below the threshold, and then resample, standardize, and normalize the images;

[0020] Assuming the pixel spacing of the original image is `original_spacing`, the size of the original image is `original_size`, the new pixel spacing is `new_spacing`, and the new image size is `new_size`; the bilinear interpolation operation is represented by the function `interpolate`, and resampling is:

[0021]

[0022] new_size=original_size×scale_factor

[0023] resampled_image=interpolate(original_image,new_size)

[0024] Where scale_factor is the scaling factor, resampled_image is the resampled image, and original_image is the original image;

[0025] Based on the segmentation labels of the original aneurysm image, corresponding target detection labels are generated. The target detection labels are obtained by calculating the minimum bounding matrix of the segmentation labels.

[0026] bbox=(min_row,min_col,max_row,max_col,min_slice,max_slice)

[0027] Where bbox is the minimum bounding matrix, representing the coordinates of the top-left and bottom-right corners of the rectangle. Specifically, min_row and min_col are the minimum row and column values; max_row and max_col are the maximum row and column values; and min_slice and max_slice are the minimum and maximum slice values.

[0028] The center coordinates (x, y, z) and radius information are obtained through the circumscribed matrix:

[0029]

[0030]

[0031]

[0032] r x = (bbox[3]-bbox[0]+expand number )×ele arr[0]

[0033] r y = (bbox[4]-bbox[1]+expand) number )×ele arr[1]

[0034] r z = (bbox[5]-bbox[2]+expand) number )×ele arr[2]

[0035] r = max(r x ,r y ,r z )

[0036] Where bbox[0], bbox[1], bbox[2], bbox[3], bbox[4], and bbox[5] correspond to the min_row, min_col, max_row, max_col, min_slice, and max_slice contained in the minimum bounding matrix, respectively; r x r y r zThese are the radii of the x, y, and z axes, respectively; expand number To expand the pixel value, set it to 2; ele arr It's the pixel pitch, off. arr This is the offset; the final target detection label format is: img_info={'box':boxes,'image':image_name,'label':labels}, which are the detection box information, image path, and aneurysm identifier, respectively.

[0037] Preprocessing steps help improve data quality and the stability of network training.

[0038] Furthermore, the method for optimizing the coarse detection network through neural network architecture search is as follows:

[0039] Define the search space of the coarse detection network. The search space includes the network's layer structure, connection method, number of channels, depth and width of convolutional layers, activation function used, pooling method, and the range of other hyperparameters.

[0040] In neural architecture search, the scale permutation of intermediate and output blocks is searched first. Since there are N intermediate and output blocks, and the order of the 5 output blocks is fixed, a search space of size (N-5)!5! is obtained. The size of the search space is... The search is based on the connection pattern, where the parent block is any block with a lower ranking or a block from the stem network, where m is the number of candidate blocks in the stem network. This indicates the probability that each block has two input connections, where N is the total number of intermediate blocks;

[0041] For each block in the search space, define two input connections. The parent block is any block with a lower ranking or a block from the stem network. When connecting blocks from different feature levels, resample the space and feature dimensions.

[0042] Blocks are allowed to adjust their scale level and type; intermediate blocks are scaled using {-1,0,1,2} to obtain a block of size 4. N-5 The search space is determined by selecting all blocks from the bottleneck block and the residual block, resulting in a size of 2. N The search space;

[0043] By using a search algorithm, new candidate structures are selectively generated and evaluated by evaluating different network structures and optimizing them according to a predefined objective function.

[0044] Guided by the search algorithm, a series of candidate network structures are generated based on the search space, and they are trained and evaluated. The 3D aneurysm detection images are divided into training and test sets. Each candidate structure is trained on the training set for multiple rounds of iterations and then evaluated on the validation set. They are ranked according to performance metrics.

[0045] Based on the evaluation results, the search space is updated to find a better network structure;

[0046] The search process terminates according to predefined termination conditions. After the search is completed, the network structure with the best performance is selected as the final optimization result based on the evaluation metrics.

[0047] The model's performance was further improved by introducing block adjustments into the search space.

[0048] Furthermore, when performing cross-scale feature fusion, there may be differences in resolution and feature dimensions between the parent block and the target block. Therefore, a resampling operation is designed to match the resolution and feature dimensions of the target block. For the bottleneck block: C in =C out = 4C, for residual blocks: C in =C out =C; C represents the feature dimension of the 3×3 convolution in the residual block or bottleneck block; C in C out These represent the input and output dimensions of the block, respectively.

[0049] Nearest neighbor interpolation is used for upsampling, or 3×3 convolution with a stride of 2 is used for downsampling to match the feature map. Finally, 1×1 convolution is used to match the target feature dimension.

[0050] It is simple to operate and easy to use.

[0051] Furthermore, the coarse detection network includes a feature extractor, a feature pyramid network, a classification head, and a regression head;

[0052] The feature extractor is used to extract features from the input 3D image. The feature extractor uses a residual network-based backbone network, which consists of multiple layers. Each layer contains multiple residual blocks. Deep feature representations are constructed through multiple residual blocks. The residual blocks include convolutional layers, batch normalization layers, activation functions, and downsampling operations.

[0053] Feature pyramid networks are used for feature fusion across different scales to detect targets at different scales. Feature pyramid networks consist of internal blocks and hierarchical blocks. Internal blocks use convolutional layers to reduce or increase the dimensionality of features in the backbone network to match features at different scales. Hierarchical blocks use convolutional layers to fuse features at different scales.

[0054] The classification head is used to predict the category of the target. The classification head includes a series of convolutional layers, grouping normalization layers, activation functions, and a final convolutional layer for predicting the category probability.

[0055] The regression head is used to predict the bounding box of the target. The regression head includes convolutional layers, grouping normalization layers, activation functions, and a last convolutional layer, which is used to predict the coordinates and size of the bounding box.

[0056] The network structure exhibits good performance in target detection tasks and can quickly locate aneurysm targets.

[0057] Furthermore, the method for training the fine segmentation network is as follows:

[0058] Crop and scale the region of interest;

[0059] The region of interest, after being resized, is input into the encoder for feature extraction. The encoder includes multiple convolutional and pooling layers to progressively extract high-level semantic features of the image.

[0060] A self-attention mechanism is introduced, and an adaptive learning scaling attention module and a cross-scaling attention module are designed. A learnable parameter is introduced into the attention mechanism to adjust the scaling factor in attention calculation.

[0061] After processing by the encoder and self-attention mechanism, the obtained feature map is fed into the decoder for feature fusion and segmentation result generation. The decoder includes a series of convolutional transpose layers and upsampling operations to gradually restore the spatial resolution of the feature map and generate pixel-level segmentation prediction results.

[0062] Design a U-Net segmentation network with adaptive learning scaling attention and cross-scaling attention mechanisms. This network structure can segment images more accurately.

[0063] Furthermore, the adaptive learning scaling attention module connects each element in the highest feature map to obtain a receptive field that includes all input images;

[0064] To accommodate the need for multi-size aneurysm segmentation, a scaling factor (scale_factor) is introduced as a learnable parameter.

[0065] During training, the scaling factor (scale_factor) is used for gradient updates based on the model's loss function, enabling the model to adaptively adjust the degree of attention scaling. The loss function is defined as follows: sf :

[0066] loss sf=MSE(predicted,target)+λ·Regularization(scale_factor)

[0067] Where MSE(predicted,target) is the mean squared error, used to measure the difference between the model output and the target value; Regularization(scale_factor) is the regularization term for the scaling factor, choosing to use L1 regularization to help control the size and sparsity of the scaling factor; λ is the weight of the regularization term, used to balance the two parts in the loss function.

[0068] The adaptive learning scaling attention module is as follows:

[0069]

[0070] Each row in the attention matrix Attention(Q,K,V) corresponds to the similarity of a given element in the query matrix Q with respect to all elements in the key matrix K. The attention function performs a weighted average of the elements in the value matrix V to account for all interactions between the query and the keys. In the segmentation task, matrices Q, K, and V have the same size and correspond to different learned embeddings of the highest-level feature map X, denoted as W. q W k and W v A represents attention weights, WqQK T In the expression WqQ, it represents a linear transformation of the query Q, mapping the query features to a new space, and then combining it with K. T Perform a dot product operation to obtain the similarity matrix between the query and the key. The overall representation is a linear transformation of the query and the key through the learned weight matrix, and the similarity between them is calculated to determine the degree of association between different positions.

[0071] Learnable scaling factors can make models more flexible in adapting to different tasks, different input distributions, and differences between different attention heads.

[0072] Furthermore, the cross-scaling attention module reduces the influence of irrelevant regions and highlights regions that have a significant impact on the target region by focusing on high-level feature maps in the skip connection features.

[0073] Assume F l F represents the feature map from a lower level. h Let A represent the feature map from a higher level, and A represent the attention weights. Then, the output Fcross_scale of the cross-scaling attention module is expressed as:

[0074] Fcross_scale =A⊙F h +(1-A)⊙F l

[0075] Where ⊙ represents element-wise multiplication, the attention weight A is scaled by adaptive learning; the structure of the cross-scaled attention module is achieved by gating the skip connections S, which is based on attention to the high-level feature map Y; the calculated weight values ​​are rescaled to between 0 and 1 by the sigmoid activation function.

[0076] The idea behind the cross-scaling attention module is to reduce the influence of irrelevant regions and highlight regions that have a significant impact on the target region by focusing on high-level feature maps in skip connection features.

[0077] The present invention also provides a two-stage full 3D aneurysm segmentation system, including an image acquisition module and a processing module. The image acquisition module is used to acquire 3D aneurysm detection images. The output end of the image acquisition module is connected to the input end of the processing module. The processing module executes the method described in the present invention to perform full 3D aneurysm segmentation.

[0078] This system employs a two-stage segmentation method to achieve more accurate and detailed segmentation. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the scale scaling process of the two-stage full 3D aneurysm segmentation method of the present invention;

[0080] Figure 2 This is a flowchart illustrating the optimized coarse detection network of the two-stage full 3D aneurysm segmentation method of the present invention.

[0081] Figure 3 This is a schematic diagram of the adaptive learning scaling attention module of the two-stage full 3D aneurysm segmentation method of the present invention;

[0082] Figure 4 This is a schematic diagram of the cross-scaling attention module in the two-stage full 3D aneurysm segmentation method of the present invention;

[0083] Figure 5 This is a flowchart illustrating the two-stage full 3D aneurysm segmentation method of the present invention;

[0084] Figure 6 This is a schematic diagram of the coarse detection network results of the two-stage full 3D aneurysm segmentation method of the present invention;

[0085] Figure 7 This is a schematic diagram of the fine segmentation network result of the two-stage full 3D aneurysm segmentation method of the present invention. Detailed Implementation

[0086] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0087] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0088] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0089] Aneurysms vary greatly in size and shape, making them difficult to locate and segment. Traditional detection and segmentation methods struggle with complex three-dimensional vascular structures and find it challenging to accurately extract the location and shape of intracranial aneurysms from large amounts of medical images. Furthermore, the number of aneurysms in medical images is often far less than the number of normal blood vessels, leading to class imbalance in the dataset. This results in trained models performing well in classifying normal blood vessels but poorly in classifying aneurysms, thus affecting model accuracy. Moreover, existing techniques cannot be specifically optimized for aneurysm segmentation, potentially leading to inaccurate segmentation results that fail to capture the details and shape features of the aneurysm.

[0090] This technical solution utilizes deep learning techniques and data augmentation methods to achieve rapid and accurate detection and segmentation of intracranial aneurysms, thereby improving the accuracy and reliability of rupture risk assessment. This invention employs methods such as undersampling, oversampling, and weighted loss functions to balance the dataset. Neural architecture search techniques can be used to optimize the network architecture for aneurysm segmentation tasks. By searching for the optimal network structure and hyperparameter settings suitable for the task, the performance and robustness of the segmentation model on aneurysm segmentation can be improved. Such optimization allows the model to better adapt to the shape, texture, and boundary features of the aneurysm, improving the accuracy of the segmentation results.

[0091] This invention discloses a two-stage, fully 3D aneurysm segmentation method, such as... Figure 5 As shown, it includes the following steps:

[0092] Acquire 3D aneurysm detection images and perform preprocessing;

[0093] Random data augmentation methods are employed to enhance the preprocessed 3D aneurysm detection images. For example, the probability of applying a flip transformation can be set to 0.5, randomly selecting whether to apply the flip transformation or not. Augmentation methods include geometric transformations and pixel transformations. Geometric transformations include image rotation, random cropping, and mirroring. These transformations geometrically alter the dataset in image space, increasing the diversity of training samples and thus improving the model's generalization ability. Pixel transformations include contrast enhancement, Gaussian blurring, and brightness enhancement. These transformations generate pixel masking points of different sizes and values ​​with different fill values, which are then mixed with the original image to disrupt some features of the original image, increasing the diversity of data samples. This effectively increases the diversity and quantity of data samples, improving the model's adaptability to different situations. It helps improve the model's robustness and generalization ability, enabling it to better handle aneurysm segmentation tasks in various practical application scenarios.

[0094] Multiple samples are generated on the data-augmented 3D aneurysm detection image; preferably, a sliding window method is used to generate samples on the 3D aneurysm image. The size of each sample is s = [96, 96, 64], which should encompass the target region and its surrounding context information as much as possible.

[0095] Optimize the coarse detection network through neural network architecture search;

[0096] Based on multiple samples, a coarse detection network is trained, and regions of interest (ROIs) are extracted from the samples, such as... Figure 6 As shown;

[0097] Cropping and resizing operations are performed to adjust the size of the ROI image to [192, 192, 192], resulting in a larger ROI image for more accurate subsequent fine segmentation. The ROI is then used as input to train a fine segmentation network, which separates the aneurysm from the background within the ROI, yielding the final finely segmented image, as shown below. Figure 7 As shown.

[0098] In a preferred embodiment of the present invention, the method for preprocessing 3D aneurysm detection images is as follows:

[0099] Removing image data with labeled voxels below a threshold, followed by resampling, standardization, and normalization of the images, helps improve data quality and network training stability. The image data is divided into training and test sets in an 8:2 ratio.

[0100] Assuming the pixel spacing of the original image is `original_spacing`, the size of the original image is `original_size`, the new pixel spacing is `new_spacing`, and the new image size is `new_size`; the bilinear interpolation operation is represented by the function `interpolate`, and resampling is:

[0101]

[0102] new_size=original_size×scale_factor

[0103] resampled_image=interpolate(original_image,new_size)

[0104] Where scale_factor is the scaling factor, resampled_image is the resampled image, and original_image is the original image;

[0105] The original data was resampled to standardize the pixel spacing. The sampling rate was [0.703125, 0.703125, 1.25].

[0106] Based on the segmentation labels of the original aneurysm image, corresponding target detection labels are generated. The target detection labels are obtained by calculating the minimum bounding matrix of the segmentation labels.

[0107] bbox=(min_row,min_col,max_row,max_col,min_slice,max_slice)

[0108] Where bbox is the minimum bounding matrix, representing the coordinates of the top-left and bottom-right corners of the rectangle. Specifically, min_row and min_col are the minimum row and column values; max_row and max_col are the maximum row and column values; and min_slice and max_slice are the minimum and maximum slice values.

[0109] The center coordinates (x, y, z) and radius information are obtained through the circumscribed matrix:

[0110]

[0111]

[0112]

[0113] r x = (bbox[3]-bbox[0]+expand number )×ele arr[0]

[0114] r y = (bbox[4]-bbox[1]+expand) number )×ele arr[1]

[0115] r z = (bbox[5]-bbox[2]+expand) number )×ele arr[2]

[0116] r = max(r x ,r y ,r z )

[0117] Where bbox[0], bbox[1], bbox[2], bbox[3], bbox[4], and bbox[5] correspond to the min_row, min_col, max_row, max_col, min_slice, and max_slice contained in the minimum bounding matrix, respectively; r x r y r z These are the radii of the x, y, and z axes, respectively; expand number To expand the pixel value, set it to 2; ele arr It's the pixel pitch, off. arr This is the offset; the final target detection label format is: img_info={'box':boxes,'image':image_name,'label':labels}, which are the detection box information, image path, and aneurysm identifier, respectively.

[0118] In a preferred embodiment of the present invention, such as Figure 2 As shown, the structure and hyperparameters of the coarse detection network are optimized through Neural Architecture Search (NAS) to improve network performance. The method is as follows:

[0119] Define the search space for coarse detection networks (such as 3D RetinaNet, which combines the single-stage detection framework of RetinaNet with 3D convolutional operations to achieve accurate detection and localization of aneurysms. 3D RetinaNet can handle volumetric data, capture dynamic features, and has high detection performance and robustness). The search space includes the network's layer structure, connection method, number of channels, depth and width of convolutional layers, activation function used, pooling method, and the range of other hyperparameters.

[0120] In neural architecture search, the scale permutation of intermediate and output blocks is searched first. Since there are N intermediate and output blocks, and the order of the 5 output blocks is fixed, a search space of size (N-5)!5! is obtained. The size of the search space is... The search is based on the connection pattern, where the parent block is any block with a lower ranking or a block from the stem network, and m is the number of candidate blocks in the stem network. This indicates the probability that each block has two input connections, where N is the total number of intermediate blocks;

[0121] For each block in the search space, define two input connections. The parent block is any block with a lower ranking or a block from the stem network. When connecting blocks from different feature levels, resample the space and feature dimensions.

[0122] Blocks are allowed to adjust their scale level and type; intermediate blocks are scaled using {-1,0,1,2} to obtain a block of size 4. N-5 The search space is determined by selecting all blocks from the bottleneck block and the residual block, resulting in a size of 2. N The search space;

[0123] By using a search algorithm, new candidate structures are selectively generated and evaluated by evaluating different network structures and optimizing them according to a predefined objective function.

[0124] Guided by search algorithms (such as genetic algorithms, reinforcement learning, evolutionary algorithms, etc., which selectively generate and evaluate new candidate structures by evaluating different network structures and optimizing them according to a predefined objective function), a series of candidate network structures are generated based on the search space, and they are trained and evaluated. The 3D aneurysm detection images are divided into training sets and test sets. Each candidate structure is trained iteratively on the training set for multiple rounds, and then evaluated on the validation set. They are ranked according to performance metrics (such as accuracy, loss function, etc.).

[0125] Based on the evaluation results, the search space is updated to find better network structures; for example, structures that performed well in previous rounds of searching are selected as the basis for the next round of searching, and then fine-tuned and modified. This allows the search to be deepened step by step to find better network structures.

[0126] The search process terminates based on predefined termination conditions (such as the number of search rounds, time limits, etc.). After the search is completed, the network structure with the best performance is selected as the final optimization result based on the evaluation metrics.

[0127] In a preferred embodiment of the present invention, such as Figure 1 As shown, when performing cross-scale feature fusion, there may be differences in resolution and feature dimensions between the parent block and the target block. Therefore, a resampling operation is designed to match the resolution and feature dimensions of the target block. For the bottleneck block: C in =C out = 4C, for residual blocks: C in =C out =C; C represents the feature dimension of the 3×3 convolution in the residual block or bottleneck block; C in C out These represent the input and output dimensions of the block, respectively.

[0128] Nearest neighbor interpolation is used for upsampling, or 3×3 convolution with a stride of 2 is used for downsampling to match the feature map. Finally, 1×1 convolution is used to match the target feature dimension.

[0129] In a preferred embodiment of the present invention, the coarse detection network includes a feature extractor, a feature pyramid network (FPN), a classification head, and a regression head;

[0130] The feature extractor is used to extract features from the input 3D image. The feature extractor uses a residual network-based backbone network, which consists of multiple layers. Each layer contains multiple residual blocks (ResNetBottleneck). Deep feature representations are constructed through multiple residual blocks. The residual blocks include convolutional layers (Conv3d), batch normalization layers (BatchNorm3d), activation functions (ReLU), and downsampling operations.

[0131] The Feature Pyramid Network is used for feature fusion across different scales to detect targets at different scales. The Feature Pyramid Network consists of inner blocks and layer blocks. The inner blocks use convolutional layers (Conv3d) to reduce or increase the dimensionality of features in the backbone network to match features at different scales. The layer blocks use convolutional layers to fuse features at different scales.

[0132] The classification head is used to predict the category of the target. The classification head includes a series of layers (Conv3d), a group normalization layer (GroupNorm), an activation function (ReLU), and a final convolutional layer (Conv3d) used to predict the category probability.

[0133] The regression head is used to predict the bounding box of the target. The regression head includes convolutional layers, grouping normalization layers, activation functions, and a last convolutional layer, which is used to predict the coordinates and size of the bounding box.

[0134] In a preferred embodiment of the present invention, the method for training the fine segmentation network is as follows:

[0135] Crop and scale the region of interest;

[0136] The region of interest, after being resized, is input into the encoder for feature extraction. The encoder includes multiple convolutional and pooling layers to progressively extract high-level semantic features of the image.

[0137] This paper introduces a self-attention mechanism and designs an Adaptive Learning Scales Attention (ALSA) module and a Cross-zoom Attention (CA) module. A learnable parameter is introduced into the attention mechanism to adjust the scaling factor in attention calculation. The main advantage of ALSA and CA modules is that they allow the model to learn a scaling factor that better suits the task requirements during training, rather than using a fixed default value. This learnable scaling factor allows the model to more flexibly adapt to different tasks, different input distributions, and differences between different attention heads.

[0138] After processing by the encoder and self-attention mechanism, the obtained feature map is fed into the decoder for feature fusion and segmentation result generation. The decoder includes a series of convolutional transpose layers and upsampling operations to gradually restore the spatial resolution of the feature map and generate pixel-level segmentation prediction results.

[0139] The introduction of a self-attention mechanism enhances the network's receptive field and attention mechanism, improving segmentation accuracy and robustness. This fine-segmentation network enables more refined segmentation results for aneurysms. In practical applications, this technical solution effectively improves aneurysm detection and segmentation, and possesses high computational efficiency, making it suitable for processing large-scale and complex medical image data.

[0140] Preferred, such as Figure 3 As shown, the adaptive learning scaling attention module connects each element in the highest feature map to obtain a receptive field that includes all input images;

[0141] To accommodate the need for multi-size aneurysm segmentation, a scaling factor (scale_factor) is introduced as a learnable parameter.

[0142] During training, the scaling factor (scale_factor) is used for gradient updates based on the model's loss function, enabling the model to adaptively adjust the degree of attention scaling. The loss function is defined as follows: sf :

[0143] loss sf =MSE(predicted,target)+λ·Regularization(scale_factor)

[0144] Where MSE(predicted,target) is the mean squared error, used to measure the difference between the model output and the target value; Regularization(scale_factor) is the regularization term for the scaling factor, choosing to use L1 regularization to help control the size and sparsity of the scaling factor; λ is the weight of the regularization term, used to balance the two parts in the loss function.

[0145] The adaptive learning scaling attention module is as follows:

[0146]

[0147] Each row in the attention matrix Attention(Q,K,V) corresponds to the similarity of a given element in the query matrix Q with respect to all elements in the key matrix K. The attention function performs a weighted average of the elements in the value matrix V to account for all interactions between the query and the keys. In the segmentation task, matrices Q, K, and V have the same size and correspond to different learned embeddings of the highest-level feature map X, denoted as W. q W k and W v A represents attention weights, WqQK TIn the expression WqQ, it represents a linear transformation of the query Q, mapping the query features to a new space, and then combining it with K. T Perform a dot product operation to obtain the similarity matrix between the query and the key. The overall representation is a linear transformation of the query and the key through the learned weight matrix, and the similarity between them is calculated to determine the degree of association between different positions.

[0148] More preferably, such as Figure 4 As shown, the cross-scaling attention module reduces the influence of irrelevant regions and highlights regions that have a significant impact on the target region by focusing on high-level feature maps in skip connection features.

[0149] Assume F l F represents the feature map from a lower level. h Let A represent the feature map from a higher level, and A represent the attention weights. Then, the output Fcross_scale of the cross-scaling attention module is expressed as:

[0150] F cross_scale =A⊙F h +(1-A)⊙F l

[0151] Here, ⊙ represents element-wise multiplication, and the attention weights A are scaled through adaptive learning; the structure of the cross-scaling attention module involves gating the skip connections S, which is based on attention to the high-level feature map Y; the calculated weight values ​​are rescaled to between 0 and 1 using the sigmoid activation function. Figure 4 In the diagram, the resulting tensor is denoted by Z, where low-amplitude elements indicate irrelevant regions to be reduced. The cleaned S is given by the Hadamard product Z and S. Finally, the result of this filtering operation is concatenated with the high-level feature tensor Y.

[0152] Adaptive learning scaling attention allows the model to focus on each element in the input while processing sequences or images, taking into account global contextual information. This helps capture long-range dependencies, enabling the model to better understand the relationships between elements in a sequence. Cross-scaling attention aims to highlight regions of significant task interest by focusing on low-level feature maps, thus closing out irrelevant or noisy regions. This is particularly useful for tasks such as image segmentation, improving the focus on targets of interest.

[0153] The present invention also provides a two-stage full 3D aneurysm segmentation system, including an image acquisition module and a processing module. The image acquisition module is used to acquire 3D aneurysm detection images. The output end of the image acquisition module is electrically connected to the input end of the processing module. The processing module executes the method described in the present invention to perform full 3D aneurysm segmentation.

[0154] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0155] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A two-stage, fully 3D aneurysm segmentation method, characterized in that, Includes the following steps: Acquire 3D aneurysm detection images and perform preprocessing; A random data augmentation method was used to augment the preprocessed 3D aneurysm detection images. Multiple samples are generated on the data-enhanced 3D aneurysm detection image; Optimize the coarse detection network through neural network architecture search; A coarse detection network is trained based on multiple samples, and regions of interest are extracted from the samples. The region of interest is used as input to train a fine segmentation network, which then segments the aneurysm within the region of interest from the background to obtain the final fine segmentation image. The method for training a fine segmentation network is as follows: Crop and scale the region of interest; The region of interest, after being resized, is input into the encoder for feature extraction. The encoder includes multiple convolutional and pooling layers to progressively extract high-level semantic features of the image. A self-attention mechanism is introduced, and an adaptive learning scaling attention module and a cross-scaling attention module are designed. A learnable parameter is introduced into the attention mechanism to adjust the scaling factor in attention calculation. After processing by the encoder and self-attention mechanism, the obtained feature map is fed into the decoder for feature fusion and segmentation result generation; the decoder includes a series of convolutional transpose layers and upsampling operations to gradually restore the spatial resolution of the feature map and generate pixel-level segmentation prediction results. The adaptive learning scaling attention module connects each element in the highest feature map to obtain a receptive field that includes all input images; To accommodate the need for multi-size aneurysm segmentation, a scaling factor (scale_factor) is introduced as a learnable parameter. During training, the scaling factor (scale_factor) is used for gradient updates based on the model's loss function, enabling the model to adaptively adjust the degree of attention scaling. The loss function is defined as follows: : ， in, It is the mean squared error, used to measure the difference between the model output and the target value; This is the regularization term for the scaling factor. L1 regularization is chosen to help control the size and sparsity of the scaling factor. It represents the weight of the regularization term, used to balance the two parts of the loss function.

2. The two-stage full 3D aneurysm segmentation method as described in claim 1, characterized in that, The method for preprocessing 3D aneurysm detection images is as follows: Remove image data with labeled voxels below the threshold, and then resample, standardize, and normalize the images; Assuming the pixel spacing of the original image is `original_spacing`, the size of the original image is `original_size`, the new pixel spacing is `new_spacing`, and the new image size is `new_size`; the bilinear interpolation operation is represented by the function `interpolate`, and resampling is: ， ， ， in, It is a scaling factor. This is the resampled image. It is the original image; Based on the segmentation labels of the original aneurysm image, corresponding target detection labels are generated. The target detection labels are obtained by calculating the minimum bounding matrix of the segmentation labels. ， Where bbox is the minimum bounding matrix, representing the coordinates of the top-left and bottom-right corners of the rectangle. Specifically, min_row and min_col are the minimum row and column values; max_row and max_col are the maximum row and column values; and min_slice and max_slice are the minimum and maximum slice values. The center coordinates are obtained through the outer matrix. , , ) and radius information: ， ， ， ， ， ， ， in, , , , , These correspond to min_row, min_col, max_row, max_col, min_slice, and max_slice, respectively, contained in the minimum bounding matrix. , , These are the radii of the x, y, and z axes, respectively; To expand the pixel value, set it to 2; It's the pixel pitch. It is the offset; the final target detection label format is: These are the detection box information, image path, and aneurysm identifier, respectively.

3. The two-stage full 3D aneurysm segmentation method as described in claim 1, characterized in that, The following method optimizes the coarse detection network through neural network architecture search: Define the search space of the coarse detection network. The search space includes the network's layer structure, connection method, number of channels, depth and width of convolutional layers, activation function used, pooling method, and the range of other hyperparameters. In neural architecture search, the scale permutation of intermediate and output blocks is searched first. Since there are N intermediate and output blocks, and the order of the 5 output blocks is fixed, a scale of [missing information] is obtained. The search space, the size of the search space is The search is based on the connection pattern, where the parent block is any block with a lower ranking or a block from the stem network, where m is the number of candidate blocks in the stem network. This indicates the probability that each block has two input connections, where N is the total number of intermediate blocks; For each block in the search space, define two input connections. The parent block is any block with a lower ranking or a block from the stem network. When connecting blocks from different feature levels, resample the space and feature dimensions. Blocks are allowed to adjust their scale level and type. Intermediate blocks are scaled using {-1, 0, 1, 2} to obtain a block of size. The search space is defined by selecting all blocks from the bottleneck block and the residual block, resulting in a size of [missing information]. The search space; By using a search algorithm, new candidate structures are selectively generated and evaluated by evaluating different network structures and optimizing them according to a predefined objective function. Guided by the search algorithm, a series of candidate network structures are generated based on the search space, and they are trained and evaluated. The 3D aneurysm detection images are divided into training and test sets. Each candidate structure is trained on the training set for multiple rounds of iterations and then evaluated on the validation set. They are ranked according to performance metrics. Based on the evaluation results, the search space is updated to find a better network structure; The search process terminates according to predefined termination conditions. After the search is completed, the network structure with the best performance is selected as the final optimization result based on the evaluation metrics.

4. The two-stage full 3D aneurysm segmentation method as described in claim 3, characterized in that, When performing cross-scale feature fusion, there may be differences in resolution and feature dimensions between the parent block and the target block. Therefore, a resampling operation is designed to match the resolution and feature dimensions of the target block. For bottleneck blocks: For residual blocks: C represents the feature dimension of the 3×3 convolution in the residual block or bottleneck block; These represent the input and output dimensions of the block, respectively. Nearest neighbor interpolation is used for upsampling, or 3×3 convolution with a stride of 2 is used for downsampling to match the feature map. Finally, 1×1 convolution is used to match the target feature dimension.

5. The two-stage full 3D aneurysm segmentation method as described in claim 1, characterized in that, The coarse detection network includes a feature extractor, a feature pyramid network, a classification head, and a regression head; The feature extractor is used to extract features from the input 3D image. The feature extractor uses a residual network-based backbone network, which consists of multiple layers. Each layer contains multiple residual blocks. Deep feature representations are constructed through multiple residual blocks. The residual blocks include convolutional layers, batch normalization layers, activation functions, and downsampling operations. Feature pyramid networks are used for feature fusion across different scales to detect targets at different scales. Feature pyramid networks consist of internal blocks and hierarchical blocks. Internal blocks use convolutional layers to reduce or increase the dimensionality of features in the backbone network to match features at different scales. Hierarchical blocks use convolutional layers to fuse features at different scales. The classification head is used to predict the category of the target. The classification head includes a series of convolutional layers, grouping normalization layers, activation functions, and a final convolutional layer for predicting the category probability. The regression head is used to predict the bounding box of the target. The regression head includes convolutional layers, grouping normalization layers, activation functions, and a last convolutional layer, which is used to predict the coordinates and size of the bounding box.

6. The two-stage full 3D aneurysm segmentation method as described in claim 1, characterized in that, The adaptive learning scaling attention module is as follows: ， Attention Matrix Each row in the matrix corresponds to the similarity of a given element in the query matrix Q with respect to all elements in the key matrix K. The attention function performs a weighted average of the elements of the value matrix V to account for all interactions between the query and the key. In the segmentation task, matrices Q, K, and V have the same size and correspond to different learned embeddings of the highest-level feature map X, with the embedding matrices denoted as follows: , and ; Indicates attention weights. middle This indicates that a linear transformation is performed on the query Q, mapping the query's features to a new space, and then... Perform a dot product operation to obtain the similarity matrix between the query and the key. The overall representation is a linear transformation of the query and the key through the learned weight matrix, and the similarity between them is calculated to determine the degree of association between different positions.

7. The two-stage full 3D aneurysm segmentation method as described in claim 1, characterized in that, The cross-scaling attention module reduces the influence of irrelevant regions and highlights regions that have a significant impact on the target region by focusing on high-level feature maps in the skip connection features. Assumption This represents a feature map from a lower level. Let A represent the feature map from a higher level, and A represent the attention weights. Then, the output of the cross-scaling attention module... Represented as: ， Where ⊙ represents element-wise multiplication, the attention weight A is scaled by adaptive learning; the structure of the cross-scaled attention module is achieved by gating the skip connections S, which is based on attention to the high-level feature map Y; the calculated weight values ​​are rescaled to between 0 and 1 by the sigmoid activation function.

8. A two-stage fully 3D aneurysm segmentation system, characterized in that, The device includes an image acquisition module and a processing module. The image acquisition module is used to acquire 3D aneurysm detection images. The output end of the image acquisition module is connected to the input end of the processing module. The processing module performs the method described in any one of claims 1-7 to perform full 3D aneurysm segmentation.

Images