Medical image classification method and system, electronic equipment and storage medium
By using multi -scale convolution self -attention modules and multiple convolutional layers in the medical image classification to extract and classify the image characteristics, the problem of difficulty capturing the complexity of the image in existing technology has been solved, and the higher image classification is accurate. Spend.
Patent Information
- Application Number
- CN202411386113.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2025-01-28
AI Technical Summary
Existing medical image classification techniques are difficult to effectively capture the complex relationship between the local details and global context of fundopauses of diabetic retinopathy, resulting in limited performance and accuracy in complex situations.
The image features are extracted by multi -scale convolution self -attention modules, combined with multiple convolutional layers for feature extraction, and classified the extracted feature input full connection layer.
Through multi -scale convolution self -attention modules, it can better capture the global context characteristics and local detail information in the image, enhance the sensitivity of the model, and improve the accuracy of medical image classification.
Smart Images
Figure CN119380075A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image classification, and in particular to a medical image classification method, system, electronic device and storage medium. Background Art
[0002] The medical image classification technologies currently used include traditional feature engineering algorithms, such as SVM, KNN, and random forest. Among them, medical image classification can be used to classify diabetic retinopathy fundus images. These traditional feature engineering algorithms first have doctors manually extract features from diabetic retinopathy fundus images, and then input the extracted features into algorithms such as the above for calculation and classification.
[0003] Traditional manual feature engineering relies on the knowledge and experience of domain experts to design and select appropriate features. Manual feature extraction may not work well in more complex tasks, and it is time-consuming and laborious to extract features manually. The models of traditional feature engineering algorithms require a large amount of data as training support to achieve higher accuracy. In addition, current deep convolutional neural networks cannot fully capture the complex relationship between local details and global context of diabetic retinopathy fundus images, which limits their performance and accuracy in complex situations. Summary of the invention
[0004] The present application aims to propose a medical image classification method, system, electronic device and storage medium, which can improve the accuracy of image classification.
[0005] In a first aspect, an embodiment of the present application provides a medical image classification method, the method comprising:
[0006] Acquire a medical image to be segmented, and preprocess the medical image to be segmented to obtain a preprocessed image;
[0007] Performing feature compression on the preprocessed image to obtain a first feature map;
[0008] Performing a convolution operation on the first feature map to obtain a second feature map;
[0009] Using a multi-scale convolutional self-attention module to extract features from the second feature map to obtain a first extracted feature map;
[0010] Performing a convolution operation on the first extracted feature map to obtain a third feature map;
[0011] Using multiple convolutional layers to extract features from the third feature map to obtain a second extracted feature map;
[0012] The second extracted feature map is input into the fully connected layer for classification to obtain a medical image classification result.
[0013] Compared with the prior art, the first aspect of the present application has the following beneficial effects:
[0014] This method uses a multi-scale convolutional self-attention module to extract features from the second feature map to obtain a first extracted feature map, and then uses multiple convolutional layers to extract features to obtain a second extracted feature map, and then inputs the second extracted feature map into a fully connected layer for classification to obtain a medical image classification result. In this way, the multi-scale convolutional self-attention module can better capture the global context features and local detail information in the image, enhance sensitivity, and thus improve the accuracy of image classification.
[0015] In some embodiments, the multi-scale convolutional self-attention module includes a self-attention submodule and a multi-scale convolution submodule, and the multi-scale convolutional self-attention module is used to extract features from the second feature map to obtain a first extracted feature map, including:
[0016] Using the self-attention submodule to perform feature extraction on the second feature map to obtain a self-attention feature map;
[0017] Using the multi-scale convolution submodule to perform feature extraction on the second feature map to obtain a multi-scale convolution feature map;
[0018] The self-attention feature map and the multi-scale convolution feature map are added pixel by pixel to obtain a first extracted feature map.
[0019] In some implementations, the using the self-attention submodule to extract features from the second feature map to obtain a self-attention feature map includes:
[0020] Projecting the second feature map into multiple groups of feature maps using convolution, wherein each group of feature maps includes a feature map as a query, a feature map as a key, and a feature map as a value;
[0021] Input each set of feature maps into a multi-head self-attention mechanism to obtain a position tensor of output features corresponding to each set of feature maps;
[0022] The position tensors of the output features corresponding to each group of feature maps are concatenated to obtain a self-attention feature map.
[0023] In some embodiments, each set of feature maps is input into a multi-head self-attention mechanism in the following manner to obtain a position tensor of output features corresponding to each set of feature maps:
[0024]
[0025] Among them, g ijRepresents the position tensor of the output features corresponding to each set of feature maps, N represents N sets of feature maps, represents a local area with a pixel space range of k centered at (i, j), f ab represents the position tensor in the local area centered at (i,j), represents the query projection matrix, f ij Represents the position tensor of the input features corresponding to each set of feature maps, represents the projection matrix of the key, The projection matrix representing the value, express Attention weights for internal features.
[0026] In some embodiments, the attention weight is calculated as follows:
[0027]
[0028] Among them, softmax represents the activation function, and d represents The characteristic dimension of .
[0029] In some implementations, the using the multi-scale convolution submodule to perform feature extraction on the second feature map to obtain a multi-scale convolution feature map includes:
[0030] Using multiple convolutional layers of different scales to extract features from the second feature map to obtain multiple convolutional feature maps;
[0031] Splicing the multiple convolution feature maps to obtain a spliced convolution feature map;
[0032] A one-dimensional convolution is used to perform a convolution operation on the concatenated convolution feature map to obtain a multi-scale convolution feature map.
[0033] In some implementations, the step of performing feature compression on the preprocessed image to obtain a first feature map includes:
[0034] Using a convolutional layer to compress the preprocessed image to obtain a compressed feature map;
[0035] The compressed feature map is compressed using a maximum pooling layer to obtain a first feature map.
[0036] In a second aspect, an embodiment of the present application further provides a medical image classification system, the system comprising:
[0037] An image processing unit, used for acquiring a medical image to be segmented, and preprocessing the medical image to be segmented to obtain a preprocessed image;
[0038] A feature compression unit, used for performing feature compression on the preprocessed image to obtain a first feature map;
[0039] A first convolution unit, configured to perform a convolution operation on the first feature map to obtain a second feature map;
[0040] A first feature extraction unit, configured to extract features from the second feature map by using a multi-scale convolutional self-attention module to obtain a first extracted feature map;
[0041] A second convolution unit, used for performing a convolution operation on the first extracted feature map to obtain a third feature map;
[0042] A second feature extraction unit, configured to extract features from the third feature map using a plurality of convolutional layers to obtain a second extracted feature map;
[0043] The medical image classification unit is used to input the second extracted feature map into the fully connected layer for classification to obtain a medical image classification result.
[0044] In a third aspect, an embodiment of the present application also provides an electronic device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a medical image classification method as described above.
[0045] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a medical image classification method as described above.
[0046] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0048] Figure 1 It is a flowchart of an embodiment of a medical image classification method provided by the present application;
[0049] Figure 2 It is a schematic diagram of the structure of convolution and MCAmix in Stage 1 in an embodiment of a medical image classification method provided by the present application;
[0050] Figure 3 is a schematic diagram of the structure of MCAmix in an embodiment of a medical image classification method provided by the present application;
[0051] Figure 4 is a schematic diagram of an attention calculation process in an embodiment of a medical image classification method provided by the present application;
[0052] Figure 5 is a schematic structural diagram of a multi-scale convolution submodule in an embodiment of a medical image classification method provided by the present application;
[0053] Figure 6 is a schematic diagram of a confusion matrix in an embodiment of a medical image classification method provided by the present application;
[0054] Figure 7 is a schematic diagram of a ROC curve in an embodiment of a medical image classification method provided by the present application;
[0055] Figure 8 is a feature heat map in an embodiment of a medical image classification method provided by the present application;
[0056] Fig. 9 is a structural schematic diagram of an embodiment of a medical image classification system provided by the present application;
[0057] Fig.10 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application. DETAILED DESCRIPTION
[0058] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.
[0059] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0060] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0061] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.
[0062] Traditional manual feature engineering relies on the knowledge and experience of domain experts to design and select appropriate features. Manual feature extraction may not work well in more complex tasks, and it is time-consuming and laborious to extract features manually. The models of traditional feature engineering algorithms require a large amount of data as training support to achieve higher accuracy. In addition, current deep convolutional neural networks cannot fully capture the complex relationship between local details and global context of diabetic retinopathy fundus images, which limits their performance and accuracy in complex situations.
[0063] In order to solve the above-mentioned problem of being unable to fully capture the complex relationship between local details and global context of diabetic retinopathy fundus images, the present application proposes a medical image classification method, system, electronic device and storage medium.
[0064] Reference Figure 1 The present invention provides a method for classifying medical images, which comprises the following steps:
[0065] Step S100, obtaining a medical image to be segmented, and preprocessing the medical image to be segmented to obtain a preprocessed image;
[0066] Step S200, performing feature compression on the preprocessed image to obtain a first feature map;
[0067] Step S300: performing a convolution operation on the first feature map to obtain a second feature map;
[0068] Step S400: extract features from the second feature map using a multi-scale convolutional self-attention module to obtain a first extracted feature map;
[0069] Step S500, performing a convolution operation on the first extracted feature map to obtain a third feature map;
[0070] Step S600, using multiple convolutional layers to perform feature extraction on the third feature map to obtain a second extracted feature map;
[0071] Step S700: input the second extracted feature map into the fully connected layer for classification to obtain a medical image classification result.
[0072] In this embodiment, a multi-scale convolutional self-attention module is used to extract features from the second feature map to obtain a first extracted feature map, and then multiple convolutional layers are used to extract features to obtain a second extracted feature map, and then the second extracted feature map is input into a fully connected layer for classification to obtain a medical image classification result. In this way, the multi-scale convolutional self-attention module can better capture the global context features and local detail information in the image, enhance sensitivity, and thus improve the accuracy of image classification.
[0073] The above-mentioned preprocessing of the medical image to be segmented may be to reduce the medical image to be segmented, or to enlarge the medical image to be segmented. Other existing technologies may also be used to preprocess the medical image to be segmented to increase the amount of image data. This embodiment does not specifically limit the preprocessing method.
[0074] The feature compression of the preprocessed image may be performed by using a 3×3 convolution operation to reduce the feature map by half (i.e., performing feature compression on the first feature map), or by using a 3×3 maximum pooling operation to reduce the size of the feature map.
[0075] The above-mentioned multi-scale convolutional self-attention module may include a self-attention sub-module and a multi-scale convolution sub-module.
[0076] The above-mentioned use of multiple convolutional layers to extract features from the third feature map may be to use convolutional layers of different scales to extract features from the third feature map, or to use convolutional layers of some same scales to extract features from the third feature map, and this embodiment does not make any specific limitation.
[0077] In some embodiments, the multi-scale convolutional self-attention module includes a self-attention submodule and a multi-scale convolutional submodule, and the multi-scale convolutional self-attention module is used to extract features from the second feature map to obtain a first extracted feature map, including:
[0078] The self-attention submodule is used to extract features from the second feature map to obtain a self-attention feature map;
[0079] A multi-scale convolution submodule is used to extract features from the second feature map to obtain a multi-scale convolution feature map;
[0080] The self-attention feature map and the multi-scale convolution feature map are added pixel by pixel to obtain the first extracted feature map.
[0081] In this embodiment, feature extraction is performed on the second feature map through the self-attention submodule and the multi-scale convolution submodule respectively, and then the self-attention feature map and the multi-scale convolution feature map are added pixel by pixel. The coarse-grained information and fine-grained information in the feature map can be obtained, and more image feature information can be obtained, that is, the global context features and local detail information in the image can be captured, thereby improving the accuracy of image classification.
[0082] In some embodiments, a self-attention submodule is used to extract features from the second feature map to obtain a self-attention feature map, including:
[0083] Using convolution to project the second feature map into multiple groups of feature maps, wherein each group of feature maps includes a feature map as a query, a feature map as a key, and a feature map as a value;
[0084] Input each set of feature maps into the multi-head self-attention mechanism to obtain the position tensor of the output features corresponding to each set of feature maps;
[0085] The position tensors of the output features corresponding to each set of feature maps are concatenated to obtain the self-attention feature map.
[0086] In this embodiment, the second feature map is projected into multiple groups of feature maps through the convolution layer, and then each group of feature maps is input into the multi-head self-attention mechanism, which can better capture the global context features in the image and lay a good data foundation for improving the accuracy of image classification. The multi-head self-attention mechanism can perform efficient and accurate classification without adding position encoding, and the use of convolution can share weights to speed up network training and reduce the memory required for training, and there is no need to break an image into several small blocks.
[0087] In some embodiments, each set of feature maps is input into a multi-head self-attention mechanism in the following manner to obtain a position tensor of output features corresponding to each set of feature maps:
[0088]
[0089] Among them, g ij Represents the position tensor of the output features corresponding to each set of feature maps, N represents N sets of feature maps, represents a local area with a pixel space range of k centered at (i, j), f ab represents the position tensor in the local area centered at (i,j), represents the query projection matrix, f ij Represents the position tensor of the input features corresponding to each set of feature maps, represents the projection matrix of the key, The projection matrix representing the value, express Attention weights for internal features.
[0090] In some embodiments, the attention weight is calculated as follows:
[0091]
[0092] Among them, softmax represents the activation function, and d represents The characteristic dimension of .
[0093] In some implementations, a multi-scale convolution submodule is used to extract features from the second feature map to obtain a multi-scale convolution feature map, including:
[0094] Using multiple convolutional layers of different scales to extract features from the second feature map to obtain multiple convolutional feature maps;
[0095] Splicing multiple convolutional feature maps to obtain a spliced convolutional feature map;
[0096] One-dimensional convolution is used to perform a convolution operation on the spliced convolution feature map to obtain a multi-scale convolution feature map.
[0097] In this embodiment, multiple convolutional layers of different scales are used to extract features from the second feature map. After multi-scale convolution, it is possible to focus on the local area of the input feature map, so that the multi-scale convolution can capture local spatial features, such as edges, textures, etc., laying a good data foundation for improving the accuracy of image classification.
[0098] In some implementations, performing feature compression on the preprocessed image to obtain a first feature map includes:
[0099] The convolutional layer is used to compress the preprocessed image to obtain a compressed feature map;
[0100] The maximum pooling layer is used to compress the compressed feature map to obtain the first feature map.
[0101] In this embodiment, by using a convolutional layer to compress the preprocessed image and using a maximum pooling layer to compress the compressed feature map, the size of the image can be reduced, thereby reducing the amount of calculation of the entire neural network model and speeding up the training speed of the entire neural network model.
[0102] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below:
[0103] After introducing the multi-scale convolution self-attention hybrid module in the neural network model proposed in this embodiment, it can focus on the key areas of lesions in the diabetic fundus lesion image, enhance the model's use of contextual information of diabetic retinopathy (DR) image data, enhance the model's understanding of lesion characteristics in DR images, and also enhance the model's feature expression capabilities. In the case of insufficient data, the use of the self-attention mechanism can obtain higher sensitivity on minority classes, alleviating the impact of data imbalance. By integrating the multi-scale convolution self-attention hybrid module into the model, the neural network model of this embodiment can better dig into the multi-scale lesion features in DR images and improve the model accuracy. The method of this embodiment specifically includes the following contents:
[0104] Reference Figure 2 In this embodiment, a new neural network structure is created. By adding a new neural network structure MCAmix (multi-scale convolution and self-attention) to the shallow neural network, the global features of the diabetic fundus lesion image are obtained to solve the problem of different lesion areas in the diabetic fundus lesion image. The network structure of the entire neural network model in this embodiment is shown in Table 1.
[0105] Table 1 Network structure of the entire neural network model in this embodiment
[0106]
[0107]
[0108] In Table 1, MSAmix-Net represents the neural network model of this embodiment, 3×3,64,stride 2 represents a convolutional layer with a convolution kernel size of 3×3, the number of output channels is 64, the stride is 2, Stage represents the stage, stride represents the stride, 3×3maxpool represents the maximum pooling layer of 3×3 size, conv represents the convolutional layer, and MCAmix represents the multi-scale convolutional self-attention module.
[0109] The specific structure of the multi-scale convolutional self-attention module is as follows Figure 3 As shown, it specifically includes the following contents:
[0110] The multi-scale convolution self-attention module has two sub-modules. In the self-attention sub-module, the feature maps formed by the projection of 1×1 convolution are clustered into N groups, each group contains 3 feature maps, and the 3 feature maps corresponding to each group are used as query, key and value respectively. Then, the feature maps generated by the self-attention mechanism are output, and then the feature maps of each group are spliced to obtain the final feature map (i.e., the self-attention feature map). In the multi-scale convolution sub-module, the input feature map is convolved with convolution kernels of different sizes and then spliced to generate the feature maps generated by each convolution kernel (i.e., the multi-scale convolution feature map). Finally, the feature maps of the two modules are added as the output feature map of the multi-scale convolution self-attention module (i.e., the first extracted feature map).
[0111] For the self-attention mechanism, in a self-attention module with N heads (i.e., multi-head self-attention mechanism), the input tensor is F∈R Cin×H×W , Cin represents the number of channels of the input feature map, H represents the height of the feature map, represents the width of the feature map, and the output feature is G∈R Cout×H×W , Cout represents the number of channels of the output feature map, f ij , g ij They correspond to the position tensors of the input tensor F(i,j) and the position tensors of the output features G. The output feature of the self-attention module is the output of the concatenation of the Value tensors calculated by multiple attention heads. The calculation formula is as follows:
[0112]
[0113] Among them, g ij Represents the position tensor of the output features corresponding to each set of feature maps, N represents N sets of feature maps, represents a local area with a pixel space range of k centered at (i, j), f ab represents the position tensor in the local area centered at (i,j), represents the query projection matrix, f ij Represents the position tensor of the input features corresponding to each set of feature maps, represents the projection matrix of the key, The projection matrix representing the value, express The attention weight assigned to the value depends on the matching similarity between the query and the key. The calculation formula is as follows:
[0114]
[0115] Among them, softmax represents the activation function, and d represents The characteristic dimension of .
[0116] Therefore, the self-attention module can first project the input into query, key and value through 1×1 convolution. Then perform self-attention calculation. Figure 4 The self-attention mechanism is decomposed into a 1×1 convolution process.
[0117] In this embodiment, the multi-scale convolution submodule uses three convolution kernels of 3×3, 5×5, and 7×7 to extract the features of the feature map, and finally reduces the number of channels of the spliced feature map through 1×1 convolution. Figure 5 shown.
[0118] In this embodiment, self-attention combined with multi-scale convolution can also focus on local areas of the input, which enables it to capture local spatial features such as edges, textures, etc.
[0119] For better explanation, the following experiments were performed in this embodiment:
[0120] In order to verify the segmentation performance of the proposed method, this example conducts a comprehensive and complete evaluation. The proposed method is compared with the existing model using the indicators of accuracy, specificity, sensitivity, and precision. The evaluation is performed on a combined dataset of the public dataset APTOS-2019 and Messidor-2.
[0121] Table 2 is a performance comparison of the method proposed in this embodiment and the prior art in the validation set of the merged dataset for image classification.
[0122] Accuracy Specificity Sensitivity Precision ResNet50 0.7572 0.5808 0.8532 0.6233 Inception v3 0.7724 0.6092 0.9023 0.6535 ConvNeXt-Tiny 0.7818 0.6255 0.9064 0.6960 EfficientNetV2-S 0.7984 0.6602 0.9150 0.6910 MCNN 0.8011 0.6618 0.9154 0.7177 The method of this embodiment 0.8233 0.7098 0.9274 0.7297
[0123] Among them, ResNet50, Inception v3, ConvNeXt-Tiny, EfficientNetV2-S and MCNN are all existing technologies and are not described in detail in this embodiment. Figure 6 ,According to the confusion matrix, the method proposed in this embodiment can identify a healthy retina with an accuracy rate of 96%, which is relatively ideal in practical applications.
[0124] Figure 7 The ROC curve in shows the trade-off between the sensitivity and specificity of the model. The macro-average AUC of the method proposed in this embodiment is 0.95, and the micro-average AUC is 0.97, which is ideal in the actual five-level classification of diabetic retinopathy.
[0125] At the same time, the feature heat map in the method is drawn, such as Figure 8 As shown, Figure 8(a), (b), (c) and (d) are all feature heat maps of DR images. Among them, the whiter areas indicate that the features contribute more to the model classification, while the gray areas indicate that the contribution is smaller. MSAmix-Net focuses more on and is closer to the lesion area. This fully demonstrates that the method of this embodiment (MSAmix-Net) can focus on the key areas of lesions in the image and deeply explore the multi-level lesion features in the image.
[0126] In this embodiment, by introducing multi-scale convolution and self-attention mechanism modules, the multi-scale feature learning ability of the model is significantly enhanced, so that it can better capture the global context features in the image. MSAmix-Net not only leads in accuracy and precision, but also performs particularly well in the attention and recognition of lesion areas. MSAmix-Net has higher sensitivity than other models and can solve the problem of S data imbalance. After adding a multi-scale convolution self-attention fusion module to the neural network model proposed in this embodiment, the overall depth of the model is reduced, the number of parameters is reduced, and the accuracy is still higher than that of other compared models. Parameter reduction can reduce overfitting, and model parameter reduction can speed up model training.
[0127] Reference Fig. 9 The embodiment of the present application further provides a medical image classification system, which includes an image processing unit 100, a feature compression unit 200, a first convolution unit 300, a first feature extraction unit 400, a second convolution unit 500, a second feature extraction unit 600 and a medical image classification unit 700, wherein:
[0128] The image processing unit 100 is used to obtain the medical image to be segmented, pre-process the medical image to be segmented, and obtain the pre-processed image;
[0129] A feature compression unit 200 is used to perform feature compression on the preprocessed image to obtain a first feature map;
[0130] A first convolution unit 300, configured to perform a convolution operation on the first feature map to obtain a second feature map;
[0131] A first feature extraction unit 400, configured to extract features from the second feature map using a multi-scale convolutional self-attention module to obtain a first extracted feature map;
[0132] A second convolution unit 500 is used to perform a convolution operation on the first extracted feature map to obtain a third feature map;
[0133] A second feature extraction unit 600, configured to extract features from the third feature map using multiple convolutional layers to obtain a second extracted feature map;
[0134] The medical image classification unit 700 is used to input the second extracted feature map into the fully connected layer for classification to obtain a medical image classification result.
[0135] It should be noted that, since a medical image classification system in this embodiment and a medical image classification method described above are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to this system embodiment and will not be described in detail here.
[0136] Reference Fig.10 The embodiment of the present application further provides an electronic device, the electronic device comprising:
[0137] at least one memory;
[0138] at least one processor;
[0139] at least one program;
[0140] The programs are stored in the memory, and the processor executes at least one program to implement the medical image classification method described above in the present disclosure.
[0141] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.
[0142] The electronic device according to the embodiment of the present application is described in detail below.
[0143] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present disclosure;
[0144] The memory 1700 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 may store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes the medical image classification method of the embodiment of the present disclosure.
[0145] Input / output interface 1800, used to implement information input and output;
[0146] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);
[0147] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );
[0148] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .
[0149] The embodiment of the present disclosure further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned medical image classification method.
[0150] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0151] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.
[0152] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present disclosure and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0153] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0154] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0155] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0156] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0157] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs. The above is a detailed description of the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Within the scope of knowledge possessed by ordinary technicians in the relevant technical field, various changes can be made without departing from the purpose of the present application.
Claims
1. A medical image classification method, characterized in that: The method comprises: Acquire a medical image to be segmented, and preprocess the medical image to be segmented to obtain a preprocessed image; Performing feature compression on the preprocessed image to obtain a first feature map; Performing a convolution operation on the first feature map to obtain a second feature map; Using a multi-scale convolutional self-attention module to extract features from the second feature map to obtain a first extracted feature map; Performing a convolution operation on the first extracted feature map to obtain a third feature map; Using multiple convolutional layers to extract features from the third feature map to obtain a second extracted feature map; The second extracted feature map is input into the fully connected layer for classification to obtain a medical image classification result.
2. The medical image classification method according to claim 1, characterized in that: The multi-scale convolution self-attention module includes a self-attention submodule and a multi-scale convolution submodule, and the multi-scale convolution self-attention module is used to extract features from the second feature map to obtain a first extracted feature map, including: Using the self-attention submodule to perform feature extraction on the second feature map to obtain a self-attention feature map; Using the multi-scale convolution submodule to perform feature extraction on the second feature map to obtain a multi-scale convolution feature map; The self-attention feature map and the multi-scale convolution feature map are added pixel by pixel to obtain a first extracted feature map.
3. The medical image classification method according to claim 2, characterized in that: The using the self-attention submodule to extract features from the second feature map to obtain a self-attention feature map includes: Projecting the second feature map into multiple groups of feature maps using convolution, wherein each group of feature maps includes a feature map as a query, a feature map as a key, and a feature map as a value; Input each set of feature maps into a multi-head self-attention mechanism to obtain a position tensor of output features corresponding to each set of feature maps; The position tensors of the output features corresponding to each group of feature maps are concatenated to obtain a self-attention feature map.
4. The medical image classification method according to claim 3, characterized in that: Each set of feature maps is input into the multi-head self-attention mechanism in the following way to obtain the position tensor of the output features corresponding to each set of feature maps: Among them, g ij Represents the position tensor of the output features corresponding to each set of feature maps, N represents N sets of feature maps, represents a local area with a pixel space range of k centered at (i, j), f ab represents the position tensor in the local area centered at (i,j), represents the query projection matrix, f ij Represents the position tensor of the input features corresponding to each set of feature maps, represents the projection matrix of the key, W v (l) The projection matrix representing the value, express Attention weights for internal features.
5. The medical image classification method according to claim 4, characterized in that: The attention weights are calculated as follows: Among them, softmax represents the activation function, and d represents W q (l) f ij The characteristic dimension of .
6. The medical image classification method according to claim 2, characterized in that: The adopting the multi-scale convolution submodule to perform feature extraction on the second feature map to obtain a multi-scale convolution feature map includes: Using multiple convolutional layers of different scales to extract features from the second feature map to obtain multiple convolutional feature maps; Splicing the multiple convolution feature maps to obtain a spliced convolution feature map; A one-dimensional convolution is used to perform a convolution operation on the concatenated convolution feature map to obtain a multi-scale convolution feature map.
7. The medical image classification method according to claim 1, characterized in that: The step of performing feature compression on the preprocessed image to obtain a first feature map includes: Using a convolutional layer to compress the preprocessed image to obtain a compressed feature map; The compressed feature map is compressed using a maximum pooling layer to obtain a first feature map.
8. A medical image classification system, characterized in that: The system comprises: An image processing unit, used for acquiring a medical image to be segmented, and preprocessing the medical image to be segmented to obtain a preprocessed image; A feature compression unit, used for performing feature compression on the preprocessed image to obtain a first feature map; A first convolution unit, configured to perform a convolution operation on the first feature map to obtain a second feature map; A first feature extraction unit, configured to extract features from the second feature map by using a multi-scale convolutional self-attention module to obtain a first extracted feature map; A second convolution unit, used for performing a convolution operation on the first extracted feature map to obtain a third feature map; A second feature extraction unit, configured to extract features from the third feature map using a plurality of convolutional layers to obtain a second extracted feature map; The medical image classification unit is used to input the second extracted feature map into the fully connected layer for classification to obtain a medical image classification result.
9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the medical image classification method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the medical image classification method according to any one of claims 1 to 7.