Three-dimensional image classifier and classification method based on hierarchical feature extraction and structure perception

Abstract

The application discloses a three-dimensional image classifier and a classification method based on layered feature extraction and structure perception, the classifier comprising: a multi-scale intra-layer feature extraction module, a multi-scale cross-layer feature fusion module, a lesion region self-adaptive extraction module and a lesion structure feature extraction module, a first full connection layer and a second full connection layer connected in turn. The application provides a general three-dimensional image classifier and a classification method based on layered feature extraction and structure perception, which can accurately and efficiently extract anisotropic resolution three-dimensional features and obtain accurate classification results. Before classifying anisotropic three-dimensional data, the application does not need to perform an interpolation operation or the like to unify the resolution, thereby saving the calculation amount consumed in the interpolation process, and while saving the calculation amount, the application can also improve the classification accuracy. The application can adaptively extract lesion structure information for classification, and can improve the classification accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a three-dimensional image classifier and classification method based on hierarchical feature extraction and structure awareness. Background Technology

[0002] Digital image processing technology has developed rapidly and is used in many fields, such as facial recognition. Furthermore, digitizing medical images to achieve automatic classification also has broad application prospects.

[0003] Digital image processing algorithms based on convolutional neural networks (CNNs) have achieved good performance in many classification tasks. However, 3D medical images generally exhibit anisotropic resolution, meaning that the resolution is high within a slice layer but low across slices. This characteristic limits the feature extraction efficiency of 3D CNN-based algorithms. Furthermore, the structural information of lesions can also reflect their benign or malignant nature; however, the local feature extraction characteristic of convolution cannot effectively extract this structural information. Therefore, it is worthwhile to research and design a classification scheme for 3D anisotropic resolution medical images. Summary of the Invention

[0004] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a three-dimensional image classifier and classification method based on hierarchical feature extraction and structure awareness. This method can efficiently extract lesion structure information from three-dimensional images with anisotropic resolution for classification. This invention adopts a multi-stage design approach. First, the input data is used to independently extract intra-slice features through two-dimensional group convolution. Then, standard three-dimensional convolution is used to perform cross-layer feature fusion. Finally, a Transformer-based structural feature extraction module is used to obtain the structural information of the lesions for classification.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a three-dimensional image classifier based on hierarchical feature extraction and structure awareness, comprising: a multi-scale intra-layer feature extraction module, a multi-scale cross-layer feature fusion module, a lesion region adaptive extraction module and a lesion structure feature extraction module, a first fully connected layer and a second fully connected layer connected in series.

[0006] The multi-scale intra-layer feature extraction module is constructed based on two-dimensional grouped convolution and is used to achieve independent feature extraction within slices;

[0007] The multi-scale cross-layer feature fusion module is constructed based on standard three-dimensional convolution and is used to fuse two-dimensional features between slices;

[0008] The adaptive extraction module for lesion regions is used to remove irrelevant background regions from the feature map;

[0009] The lesion structure feature extraction module is built based on the deformable Transformer module and is used to obtain lesion structure information.

[0010] The first fully connected layer and the second fully connected layer are used to integrate and classify features;

[0011] The three-dimensional medical image to be classified is input into the three-dimensional image classifier by the multi-scale layer feature extraction module, and the classification result is output through the second fully connected layer.

[0012] Preferably, the multi-scale in-layer feature extraction module consists of a two-dimensional grouped convolutional layer with a kernel size of 7 and three residual modules connected in series. Each residual module contains 6 two-dimensional grouped convolutional layers, 6 batch normalization layers, 6 non-linear activation functions ReLU, and two skip connection operations.

[0013] In this design, the number of groups in all two-dimensional grouped convolutional layers is consistent with the number of slices in the input three-dimensional image.

[0014] Preferably, the processing method of the multi-scale cross-layer feature fusion module is as follows:

[0015] The output of each scale of the feature extraction module within the multi-scale layer is first subjected to a feature transformation operation to convert the two-dimensional feature map into a three-dimensional feature map. Then, the residual module composed of standard three-dimensional convolution is used to extract and fuse the three-dimensional features between slices to output a three-dimensional feature map.

[0016] Preferably, the feature transformation operation includes feature channel recombination and feature splitting, and the feature channel recombination operation is represented by the following formula:

[0017]

[0018] Where I represents the input two-dimensional feature map with dimension C. (i,j) ×H×W,C (i,j) This represents the j-th feature map in the i-th group; O represents the two-dimensional feature map after feature recombination; c represents the feature map concatenation operation; G represents the total number of groups;

[0019] The feature splitting operation is as follows: O is uniformly divided into G parts along the channel dimension to obtain the final three-dimensional feature map.

[0020] Preferably, the processing method of the adaptive lesion area extraction module is as follows:

[0021] The 3D feature maps of each scale output by the multi-scale cross-layer feature fusion module are first fed in parallel into a global max pooling layer and a global average pooling layer. Then, two fully connected layers with shared weights are used to obtain channel attention weights and assign weights to different channels. The channel-weighted feature maps are then fed in parallel into a channel average pooling layer and a channel max pooling layer to obtain two spatial attention weight maps. Finally, a set of 3D convolutions with kernel sizes of 7, 5, and 3 are used to fuse the two spatial attention weight maps, and the spatial attention weights are redistributed. After thresholding, the final lesion region mask is obtained.

[0022] Preferably, the lesion region adaptive extraction module uses a mask consistency loss function L. cls To ensure the accuracy of the three lesion region masks obtained by extracting lesion region masks from three different scales of 3D feature maps, the mask consistency loss function L... cls The formula is:

[0023]

[0024] Among them, M i and M j These are two mask images of different scales.

[0025] Preferably, the processing method of the lesion structure feature extraction module is as follows:

[0026] First, the image output by the adaptive extraction module of the lesion region is flattened into a one-dimensional vector. Then, a three-dimensional position embedding function based on the sine and cosine functions is used to maintain the original position correlation of the one-dimensional vector. Finally, it is input into the variable Transformer module to extract the lesion structure features and obtain the lesion structure information.

[0027] The expression for the 3D position embedding function is as follows:

[0028]

[0029] Among them, PE i∈{D,H,W} This represents the positional encoding of the three different dimensions of the original 3D feature map, where C represents the number of channels in the flattened one-dimensional vector, and PE... D PE H PE W The positional encoding of the feature is completed by adding the corresponding one-dimensional feature vector element by element. pos represents the position of the corresponding pixel in the original three-dimensional feature map, and k represents the dimension of the positional encoding.

[0030] Preferably, the variable Transformer module consists of eight deformable multi-head self-attention layers connected in series, using... The multi-scale feature map represents the input, where I l ∈R C×D×H×W The feature map represents the l-th scale, and there are a total of L feature maps of different scales; The normalized coordinates of the reference point representing each query vector q are then expressed by the following formula:

[0031]

[0032] Where m represents the attention head number, M represents the total number of attention heads; l represents the scale number, L represents the total number of different scales; k represents the reference point number, K represents the total number of reference points; W m and W′ m It is a learnable embedding matrix; A mlqk For attention weights, p mlqk ∈R 3 For sampling offset, σ l (·)yes Sampling at the l-th scale; M l It is a mask for the lesion area.

[0033] The present invention also provides a three-dimensional image classification method, comprising the following steps:

[0034] S1. Construct a 3D image classifier based on hierarchical feature extraction and structure awareness as described above;

[0035] S2. Constructing the training dataset: After cropping, normalizing, and data augmentation of the 3D image dataset, the training dataset is obtained.

[0036] S3. Train the 3D image classifier constructed in S1 using the training dataset;

[0037] S4. Input the three-dimensional medical image to be classified into the trained three-dimensional image classifier obtained in step S4 to obtain the classification result.

[0038] Preferably, in step S3, the 3D medical images and their corresponding category labels from the training dataset are input into the 3D image classifier, using the following loss function L. total Conduct training:

[0039]

[0040] Among them, L cls and L con These represent the mask consistency loss and classification loss for lesion locations, respectively, where N represents the total number of 3D medical image samples, and y i and p i These are the labels and the model's prediction results, respectively, where λ is the consistency loss weight, and M...i and M j These represent two mask images at different scales.

[0041] The beneficial effects of this invention are:

[0042] This invention provides a general 3D image classifier and classification method based on hierarchical feature extraction and structure awareness, which can accurately and efficiently extract 3D features with anisotropic resolution and obtain accurate classification results.

[0043] Before classifying anisotropic 3D data, this invention eliminates the need for interpolation or other resolution-uniforming operations, saving computational resources associated with the interpolation process. Furthermore, this invention improves classification accuracy while saving computational resources.

[0044] This invention can adaptively extract lesion structure information for classification, thereby improving classification accuracy. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of a three-dimensional image classifier based on hierarchical feature extraction and structure awareness in Embodiment 1 of the present invention;

[0046] Figure 2 This is a schematic diagram of the residual module in Embodiment 1 of the present invention;

[0047] Figure 3 This is a schematic diagram of the grouped convolution and feature recombination segmentation operation in Embodiment 1 of the present invention;

[0048] Figure 4 This is a schematic diagram of the adaptive lesion region extraction module in Embodiment 1 of the present invention;

[0049] Figure 5 This is a flowchart of the three-dimensional image classification method in Embodiment 2 of the present invention. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to embodiments, so that those skilled in the art can implement it based on the description.

[0051] It should be understood that terms such as “having,” “comprising,” and “including” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof.

[0052] Example 1

[0053] Reference Figure 1-3This embodiment of a three-dimensional image classifier based on hierarchical feature extraction and structure awareness includes: a multi-scale intra-layer feature extraction module, a multi-scale cross-layer feature fusion module, a lesion region adaptive extraction module, a lesion structure feature extraction module, a first fully connected layer, and a second fully connected layer connected in series.

[0054] The multi-scale intra-layer feature extraction module is constructed based on two-dimensional grouped convolution to achieve independent feature extraction within slices;

[0055] The multi-scale cross-layer feature fusion module is constructed based on standard 3D convolution and is used to fuse 2D features between slices;

[0056] The adaptive lesion region extraction module is used to remove irrelevant background regions in the feature map, ensuring that the subsequent lesion structure feature extraction module can better focus on the lesion location and accurately extract its structural information, while also reducing the computational load of the lesion structure feature extraction module.

[0057] The lesion structure feature extraction module is built based on the deformable Transformer module and is used to obtain lesion structure information;

[0058] The first and second fully connected layers are used to integrate and classify features;

[0059] The three-dimensional medical image to be classified is input into the three-dimensional image classifier by the multi-scale layer feature extraction module, and the classification result is output through the second fully connected layer.

[0060] In this embodiment, independent feature extraction within slices is achieved by constructing a multi-scale intra-layer feature extraction module based on two-dimensional grouped convolution. Specifically, the multi-scale intra-layer feature extraction module consists of a two-dimensional grouped convolutional layer with a kernel size of 7 and three residual modules connected in series. Each residual module contains 6 two-dimensional grouped convolutional layers, 6 batch normalization layers, 6 non-linear activation functions (ReLU), and two skip connection operations, as detailed below. Figure 2 As shown.

[0061] In this design, the number of groups in all two-dimensional grouped convolutional layers is consistent with the number of slices in the input three-dimensional image, which ensures that the intra-layer feature extraction between slices is independent of each other.

[0062] Reference Figure 3 In this embodiment, a multi-scale cross-layer feature fusion module based on standard 3D convolution is constructed to fuse 2D features between slices. Specifically, the processing method of the multi-scale cross-layer feature fusion module is as follows:

[0063] The output of each scale of the feature extraction module within the multi-scale layer is first subjected to a feature transformation operation to convert the two-dimensional feature map into a three-dimensional feature map. Then, the residual module, which consists of standard three-dimensional convolutions, is used to extract and fuse the three-dimensional features between slices to output a three-dimensional feature map.

[0064] The feature transformation operations include feature channel recombination and feature splitting. The feature channel recombination operation is represented by the following formula:

[0065]

[0066] Where I represents the input two-dimensional feature map with dimension C. (i,j) ×H×W,C (i,j) This represents the j-th feature map in the i-th group; O represents the two-dimensional feature map after feature recombination; c represents the feature map concatenation operation; G represents the total number of groups;

[0067] The feature splitting operation involves uniformly dividing O into G parts along the channel dimension to obtain the final 3D feature map. Channel recombination and splitting ensure that each 3D feature map contains the 2D features of all slices while preserving the correlation between adjacent slices.

[0068] In this embodiment, an adaptive lesion region extraction module is constructed to remove irrelevant background regions from the feature map, ensuring that the subsequent lesion structure feature extraction module can better focus on the lesion location and accurately extract its structural information. This also reduces the computational load of the lesion structure feature extraction module. Specifically, the processing method of the adaptive lesion region extraction module is as follows:

[0069] Reference Figure 4 First, the 3D feature maps of each scale output by the multi-scale cross-layer feature fusion module are fed in parallel into a global max pooling layer (GMP) and a global average pooling layer (GAP). Then, two fully connected layers with shared weights are used to obtain channel attention weights and assign weights to different channels. The channel-weighted feature maps are then fed in parallel into a channel average pooling layer (CAP) and a channel max pooling layer (CMP) to obtain two spatial attention weight maps. Finally, a set of 3D convolutions with kernel sizes of 7, 5 and 3 are used to fuse the two spatial attention weight maps, and the spatial attention weights are redistributed. After thresholding, the final lesion region mask is obtained.

[0070] Since lesion region masks need to be extracted from the three 3D feature maps at three different scales, three lesion region masks at different scales can be obtained in the end. To ensure the accuracy of the lesion masks at different scales, this embodiment uses the mask consistency loss function L. clsBy constraining the similarity of masks at different scales, the learning difficulty of the adaptive feature extraction module for lesion regions is reduced, and the mask consistency loss function L... cls The formula is:

[0071]

[0072] Among them, M i and M j It consists of two mask images at different scales (both masks are upsampled to the same resolution). The mean absolute error constraint is used to ensure the similarity of the generated masks at different scales, and it is also a way to introduce prior information.

[0073] The adaptive extraction module for lesion regions obtains a mask and then performs mask filtering (multiplying the mask and the input image pixel by pixel) to obtain a three-dimensional feature image.

[0074] In this embodiment, by constructing a lesion structure feature extraction module based on a deformable Transformer, the model can quickly and accurately acquire lesion structure information, thereby improving the accuracy of classification. Specifically, the processing method of the lesion structure feature extraction module is as follows:

[0075] First, the three-dimensional feature images of three different scales output by the adaptive extraction module of the lesion region and filtered by the lesion mask are flattened into one-dimensional vectors. Then, a three-dimensional position embedding function based on the sine and cosine functions is used to maintain the original position correlation of the one-dimensional vector (the same position embedding operation is required for the feature map of each scale). Then, it is input into the variable Transformer module to extract the lesion structure features and obtain the lesion structure information.

[0076] The expression for the 3D position embedding function is as follows:

[0077]

[0078] Among them, PE i∈{D,H,W} This represents the positional encoding of the three different dimensions of the original 3D feature map, where C represents the number of channels in the flattened one-dimensional vector, and PE... D PE H PE W The positional encoding of the feature is completed by adding the corresponding one-dimensional feature vector element by element. pos represents the position of the corresponding pixel in the original three-dimensional feature map, and k represents the dimension of the positional encoding.

[0079] The variable Transformer module consists of eight deformable multi-head self-attention layers connected in series. Each self-attention structure only needs to focus on the correlation between five reference points in the lesion region, which greatly reduces the computational cost of the Transformer structure. The multi-scale feature map represents the input, where I l ∈R C×D×H×W The feature map represents the l-th scale, and there are a total of L feature maps of different scales; The normalized coordinates of the reference point representing each query vector q are then expressed by the following formula:

[0080]

[0081] Where m represents the attention head number, M represents the total number of attention heads (M is set to 8 in this embodiment); l represents the scale number, L represents the total number of different scales (L is set to 3 in this embodiment); k represents the reference point number, K represents the total number of reference points (K is set to 5 in this embodiment); W m and W′ m It is a learnable embedding matrix; A mlqk For attention weights, p mlqk ∈R 3 For sampling offset, σ l (·)yes Sampling at the l-th scale; M l It is a mask for the lesion area.

[0082] In this embodiment, the first fully connected layer and the second fully connected layer are 512-dimensional and 256-dimensional fully connected layers, respectively. The multi-scale intra-layer feature extraction module, the multi-scale cross-layer feature fusion module, the lesion region adaptive extraction module, and the lesion structure feature extraction module are sequentially connected in series. Then, two more fully connected layers of 512-dimensional and 256-dimensional dimensions are connected in series for feature integration and classification, thus obtaining a complete hierarchical feature extraction and structure-aware classification network, i.e., the three-dimensional image classifier.

[0083] Example 2

[0084] Reference Figure 5 This embodiment provides a three-dimensional image classification method, including the following steps:

[0085] S1. Construct a 3D image classifier based on hierarchical feature extraction and structure awareness as described in Example 1;

[0086] S2. Constructing the training dataset: After cropping, normalizing, and data augmentation of the 3D image dataset, the training dataset is obtained.

[0087] Specifically, in this embodiment, a digital breast tomosynthesis (DBT) dataset is used, which is preprocessed to construct a training dataset. The preprocessing operation first uses the center of the lesion as the origin to crop an 8×512×512 image block containing the lesion. Then, maximum value normalization is used to normalize the pixel values ​​to between 0 and 1. Finally, offline data augmentation methods such as random horizontal flipping, vertical flipping, and random rotation from -90 degrees to 90 degrees are used to expand the data, thereby constructing the training dataset.

[0088] S3. Train the 3D image classifier constructed in S1 using the training dataset;

[0089] In this embodiment, the 3D medical images and their corresponding category labels from the training dataset are input into the 3D image classifier, and the loss function L is used. total Training is performed, and the loss function L is used. total It consists of two parts: lesion location mask consistency loss L cls And classification loss L con The total loss obtained by weighted summation of the two is used for backpropagation to update parameters and guide the optimization of the entire network. The loss function L total The definition is as follows:

[0090]

[0091] Where N represents the total number of 3D medical image samples, y i and p i These are the label and the model's prediction results, respectively. λ is the consistency loss weight; in this embodiment, λ is set to 0.5. i and M j These represent two mask images at different scales.

[0092] S4. Input the three-dimensional medical image to be classified into the trained three-dimensional image classifier obtained in step S4, and automatically obtain the classification result.

[0093] In one embodiment, experiments were conducted on digital breasttomosynthesis (DBT) images using the method of the present invention to verify the effectiveness of the method. DBT exhibits severe anisotropy, with an intra-layer resolution of 0.1 mm and an inter-layer resolution of approximately 1 mm. The dataset we used included 462 DBT cases, of which 174 were benign and 322 were malignant. Five-fold cross-validation was used in the experiments to compare algorithms based on 2D convolution, 3D convolution, and 3D anisotropic convolution. The experimental results are shown in Table 1 below:

[0094] Table 1. Comparative experimental results (mean ± standard deviation) of the method of this invention with other related classification algorithms.

[0095]

[0096] The experimental results show that the method of this invention has a high accuracy rate. The comparison algorithm is cited from the following references:

[0097] [1]RKSamala et al., "Deep-learning convolution neural network for computer-aided detection of microcalcifications in digital breasttomosynthesis," in Progr.Biomed.Opt.Imaging Proc.SPIE, vol.9785, Mar.2016, Art.No.97850Y.

[0098] [2] J.Wang, and Y.Yang. "A context-sensitive deep learning approach for microcalcification detection in mammograms," Pattern Recognit., vol.78, pp.12–22, Jun.2018.

[0099] [3]X.Shu, L.Zhang, Z,Wang, Q.Lv, and Y.Zhang, "Deep neural networks with region-based pooling structures for mammographic image classification," IEEETrans.Med.Imag., vol.39, no.6, pp.2246-2255, Jun.2020.

[0100] [4] K.He,

[0101] [5] I.Wichakam, J.Chayakulkheeree, and P.Vateekul, "Deep multi-label3dconvnet for breast cancer diagnosis in dbt with inversion augmentation," inProc.SPIE Int.Soc.Opt.Eng., vol.10806, Aug.2018, Art.No.Article number: 108065P.

[0102] [6]S.Liu et al., "3D anisotropic hybrid network:transferringconvolutional features from 2D images to 3D anisotropic volumes," inProc.Int.Conf.Med.Image Comput.Comput.-Assisted Intervent.(MICCAI), Sep.2018, pp.851-858.[7]J.Zheng,et al.,"3D context-aware convolutional neural network for false positive reduction in clustered microcalcifications detection,” IEEEJ.Biomed.Health Informat., vol.25, no.3, pp.764-773, Mar.2021.

[0103] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details.

Claims

1. A classification method for three-dimensional images based on hierarchical feature extraction and structure awareness, characterized in that, include: The multi-scale intra-layer feature extraction module, the multi-scale cross-layer feature fusion module, the lesion region adaptive extraction module, the lesion structure feature extraction module, the first fully connected layer, and the second fully connected layer are connected in series. The multi-scale intra-layer feature extraction module is constructed based on two-dimensional grouped convolution and is used to achieve independent feature extraction within slices; The multi-scale cross-layer feature fusion module is constructed based on standard three-dimensional convolution and is used to fuse two-dimensional features between slices; The adaptive extraction module for lesion regions is used to remove irrelevant background regions from the feature map; The lesion structure feature extraction module is built based on the deformable Transformer module and is used to obtain lesion structure information. The first fully connected layer and the second fully connected layer are used to integrate and classify features; The three-dimensional medical image to be classified is input into the three-dimensional image classifier by the multi-scale layer feature extraction module, and the classification result is output through the second fully connected layer; The processing method of the adaptive extraction module for the lesion area is as follows: The 3D feature maps of each scale output by the multi-scale cross-layer feature fusion module are first fed in parallel into a global max pooling layer and a global average pooling layer. Then, two fully connected layers with shared weights are used to obtain channel attention weights and assign weights to different channels. The channel-weighted feature maps are then fed in parallel into a channel average pooling layer and a channel max pooling layer to obtain two spatial attention weight maps. Finally, a set of 3D convolutions with kernel sizes of 7, 5, and 3 are used to fuse the two spatial attention weight maps, and the spatial attention weights are redistributed. After thresholding, the final lesion region mask is obtained.

2. The classification method for a 3D image classifier based on hierarchical feature extraction and structure awareness according to claim 1, characterized in that, The multi-scale in-layer feature extraction module consists of a two-dimensional grouped convolutional layer with a kernel size of 7 and three residual modules connected in series. Each residual module contains 6 two-dimensional grouped convolutional layers, 6 batch normalization layers, 6 non-linear activation functions ReLU, and two skip connection operations. In this design, the number of groups in all two-dimensional grouped convolutional layers is consistent with the number of slices in the input three-dimensional image.

3. The classification method for a 3D image classifier based on hierarchical feature extraction and structure awareness according to claim 2, characterized in that, The processing method of the multi-scale cross-layer feature fusion module is as follows: The output of each scale of the feature extraction module within the multi-scale layer is first subjected to a feature transformation operation to convert the two-dimensional feature map into a three-dimensional feature map. Then, the residual module composed of standard three-dimensional convolution is used to extract and fuse the three-dimensional features between slices to output a three-dimensional feature map.

4. The classification method for a 3D image classifier based on hierarchical feature extraction and structure awareness according to claim 3, characterized in that, The feature transformation operation includes feature channel recombination and feature splitting. The feature channel recombination operation is represented by the following formula: ； in, I The two-dimensional feature map representing the input has a dimension of . , Indicates the first i The first group j Each feature map; The two-dimensional feature map after feature recombination; This represents the feature map concatenation operation; G represents the total number of groups; The feature splitting operation is as follows: split along the channel dimension The image is uniformly divided into G parts to obtain the final three-dimensional feature map.

5. The classification method for a 3D image classifier based on hierarchical feature extraction and structure awareness according to claim 4, characterized in that, The adaptive lesion region extraction module employs a mask consistency loss function. To ensure the accuracy of the three lesion region masks obtained by extracting lesion region masks from three different scales of 3D feature maps, a mask consistency loss function is used. The formula is: ； in, and These are two mask images of different scales.

6. The classification method for a 3D image classifier based on hierarchical feature extraction and structure awareness according to claim 5, characterized in that, The processing method of the lesion structure feature extraction module is as follows: First, the image output by the adaptive extraction module of the lesion region is flattened into a one-dimensional vector. Then, a three-dimensional position embedding function based on sine and cosine functions is used to maintain the original position correlation of the one-dimensional vector. Finally, it is input into the deformable Transformer module to extract the lesion structure features and obtain the lesion structure information. The expression for the 3D position embedding function is as follows: ； in, The positional encodings represent the three different dimensions of the original 3D feature map. C This represents the number of channels in a flattened one-dimensional vector. The positional encoding of the feature is achieved by adding each element of the corresponding one-dimensional feature vector. pos This represents the position of the corresponding pixel in the original 3D feature map. k The dimension representing the location encoding.

7. The classification method for a 3D image classifier based on hierarchical feature extraction and structure awareness according to claim 6, characterized in that, in, The deformable Transformer module consists of eight deformable multi-head self-attention layers connected in series, using... The multi-scale feature map represents the input, where Representing the l The feature map of the scale has a total of L Feature maps at different scales; Represents each query vector q The normalized coordinates of the reference point then allow for the deformation of the multi-head self-attention layer. Expressed by the following formula: ； in, m M represents the number of attention heads; l The scale number represents the total number of different scales; k K represents the reference point number, and K represents the total number of reference points. and It is a learnable embedding matrix; For attention weights, For sampling offset, yes In the l Sampling at various scales; It is a mask for the lesion area.

8. A three-dimensional image classification method, characterized in that, Includes the following steps: S1. Construct a three-dimensional image classifier based on hierarchical feature extraction and structure awareness as described in any one of claims 1-7; S2. Constructing the training dataset: After cropping, normalizing, and data augmentation of the 3D image dataset, the training dataset is obtained. S3. Train the 3D image classifier constructed in S1 using the training dataset; S4. Input the three-dimensional medical image to be classified into the trained three-dimensional image classifier obtained in step S4 to obtain the classification result.

9. The three-dimensional image classification method according to claim 8, characterized in that, In step S3, the 3D medical images and their corresponding category labels from the training dataset are input into the 3D image classifier, using the following loss function. Conduct training: ； in, and These represent the mask consistency loss and classification loss for lesion locations, respectively, where N represents the total number of 3D medical image samples. and These are the labels and the model's prediction results, respectively, with λ representing the consistency loss weight. and These represent two mask images at different scales.

Images