Brain glioma image classification method and device, electronic equipment and storage medium

Abstract

The invention relates to a brain glioma image classification method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a brain glioma image; the brain glioma image comprises a plurality of layers of magnetic resonance slice images; segmenting each layer of magnetic resonance slice image into a plurality of image blocks through an image block embedding layer, and mapping each image block into a corresponding embedding vector; setting a first position coding vector for each image block; adding a preset first classification mark vector, each embedding vector and each first position coding vector to obtain a first vector; performing feature extraction on the first vector through a feature extraction network layer to obtain a first feature vector; obtaining a second feature vector according to the first feature vector corresponding to each layer of magnetic resonance slice image and a deep learning neural network model; and obtaining a classification result of the brain glioma image according to the second feature vector and the classification network model, thereby improving the accuracy of the classification result of the brain glioma image.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular, to a method, device, electronic device, and storage medium for classifying glioma brain images. Background Art

[0002] Glioma is a malignant tumor derived from glial cells and is one of the most common types of brain tumors. Compared with other types of tumors, glioma has a more profound impact on the quality of life and survival prognosis of patients.

[0003] By obtaining the glioma brain image of a patient, classifying the glioma brain image, and obtaining a classification result. According to the classification result, an indirect basis can be provided for doctors' diagnosis. Combining with clinical features, doctors can determine the diagnosis result of glioma.

[0004] However, in the existing methods for classifying glioma brain images, the accuracy of the classification result is low. Summary of the Invention

[0005] Based on this, the purpose of the present application is to provide a method, device, electronic device, and storage medium for classifying glioma brain images, which can improve the accuracy of the classification result of glioma brain images.

[0006] According to the first aspect of the embodiments of the present application, a method for classifying glioma brain images is provided, including the following steps:

[0007] Obtain a glioma brain image; the glioma brain image includes a plurality of layers of magnetic resonance slice images;

[0008] Through an image patch embedding layer, each layer of magnetic resonance slice image is segmented into a plurality of image patches, and each image patch is mapped to a corresponding embedding vector;

[0009] Set a first position encoding vector for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image;

[0010] Add a preset first classification marker vector, each embedding vector, and each first position encoding vector to obtain a first vector;

[0011] Through a feature extraction network layer, perform feature extraction on the first vector to obtain a first feature vector;

[0012] According to the first feature vector corresponding to each layer of magnetic resonance slice image and a deep learning neural network model, obtain a second feature vector;

[0013] According to the second feature vector and a classification network model, obtain the classification result of the glioma brain image.

[0014] According to a second aspect of the embodiments of the present application, there is provided a classification device for brain glioma images, including:

[0015] An image acquisition module, configured to acquire brain glioma images; the brain glioma images include a plurality of layers of magnetic resonance slice images;

[0016] An embedded vector obtaining module, configured to divide each layer of magnetic resonance slice image into a plurality of image patches through an image patch embedding layer, and map each image patch to a corresponding embedded vector;

[0017] A position encoding vector obtaining module, configured to set a first position encoding vector for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image;

[0018] A first vector obtaining module, configured to add a preset first classification label vector, each embedded vector, and each first position encoding vector to obtain a first vector;

[0019] A first feature vector obtaining module, configured to perform feature extraction on the first vector through a feature extraction network layer to obtain a first feature vector;

[0020] A second feature vector obtaining module, configured to obtain a second feature vector according to the first feature vector corresponding to each layer of magnetic resonance slice image and a deep learning neural network model;

[0021] A classification result obtaining module, configured to obtain a classification result of the brain glioma image according to the second feature vector and a classification network model.

[0022] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the steps of the method in the first aspect.

[0023] According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the steps of the method in the first aspect.

[0024] In an embodiment of the present application, a brain glioma image is obtained; the brain glioma image includes several layers of magnetic resonance slice images; through an image patch embedding layer, each layer of magnetic resonance slice image is segmented into several image patches, and each image patch is mapped to a corresponding embedding vector; a first position encoding vector is set for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image; the preset first classification marker vector, each embedding vector, and each first position encoding vector are added together to obtain a first vector; through a feature extraction network layer, the first vector is subjected to feature extraction to obtain a first feature vector; according to the first feature vector corresponding to each layer of magnetic resonance slice image and a deep learning neural network model, a second feature vector is obtained; according to the second feature vector and a classification network model, a classification result of the brain glioma image is obtained. In the embodiment of the present application, based on the image patch embedding layer and the feature extraction network layer, local detail features and global semantic structure information of the brain glioma image can be effectively captured; based on the deep learning neural network model, the anatomical structure continuity between adjacent magnetic resonance slice images can be captured, thereby improving the accuracy of the classification result of the brain glioma image.

[0025] It should be understood that the above general description and subsequent detailed description are only exemplary and explanatory, and cannot limit the present application.

[0026] For better understanding and implementation, the present invention will be described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic flowchart of a method for classifying a brain glioma image provided by an embodiment of the present application;

[0028] Figure 2 It is a structural block diagram of a device for classifying a brain glioma image provided by an embodiment of the present application;

[0029] Figure 3 It is a schematic structural block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] To make the objectives, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0031] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0032] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present application. The singular forms "a", "the" and "said" used in the embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0033] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects and do not have to be used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0034] In addition, in the description of the present application, unless otherwise specified, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0035] Please refer to Figure 1 , which is a schematic flowchart of a method for classifying brain glioma images provided by an embodiment of the present application. The method for classifying brain glioma images provided by the embodiments of the present application includes the following steps:

[0036] S10: Obtain brain glioma images; the brain glioma images include several layers of magnetic resonance slice images.

[0037] Among them, the magnetic resonance slice image refers to an image of a cross-section, coronal plane or sagittal plane of the internal tissues of the human body obtained by magnetic resonance imaging technology, and is composed of images of multiple sequences, including but not limited to T1-weighted images, T2-weighted images, FLAIR (fluid-attenuated inversion recovery) images, and contrast-enhanced images, etc.

[0038] In the embodiments of the present application, magnetic resonance imaging technology can be used to scan the patient's brain to obtain brain glioma images. Brain glioma images can also be obtained from an existing database.

[0039] The glioma image of the brain includes several layers of magnetic resonance slice images, and each layer of magnetic resonance slice image is a cross-sectional image of the brain. Specifically, when scanning the brain using magnetic resonance technology, the magnetic resonance slice images of each layer from top to bottom of the brain can be obtained.

[0040] S20: Through the image patch embedding layer, each layer of magnetic resonance slice image is segmented into several image patches, and each image patch is mapped to a corresponding embedding vector.

[0041] In the embodiment of the present application, the image patch embedding layer is the patch embedding layer of the self-supervised learning model DinoV2 (DIstillation with NO labels, Version 2). The image patch embedding layer can segment each layer of magnetic resonance slice image into several non-overlapping image patches of a fixed size, and map each image patch to an embedding vector through linear projection. Among them, the DinoV2 model is a self-supervised pre-training model based on the Vision Transformer (abbreviation: ViT) architecture, and its training process is the prior art and will not be elaborated here.

[0042] Specifically, the representation of the embedding vector is as follows:

[0043] z p =Linear(x p ), where x p ∈R^(16×16×3), z p ∈R D

[0044] Among them, z p represents the embedding vector, and x p represents the image patch.

[0045] S30: Set a first position encoding vector for each image patch; among them, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image.

[0046] In the embodiment of the present application, by obtaining the horizontal and vertical position information of each image patch in the magnetic resonance slice image, a corresponding first position encoding vector is generated using the 1D position encoding method.

[0047] S40: Add the preset first classification token vector, each embedding vector, and each first position encoding vector to obtain a first vector.

[0048] Among them, the preset first classification token vector is a learnable embedding vector used to aggregate global features.

[0049] In the embodiment of the present application, the representation of the first vector is as follows:

[0050] Z = [z [CLS] ; z1; z2; …; z N + E pos

[0051] Wherein, Z represents the first vector, z [CLS] represents a preset first classification marker vector, z1, z2, …, z N respectively represent the 1st, 2nd, …, Nth embedding vectors, and E pos represents the first positional encoding vector.

[0052] S50: Through the feature extraction network layer, extract features from the first vector to obtain the first feature vector.

[0053] In the embodiments of the present application, the feature extraction network layer is the Transformer encoder in the DinoV2 model, which includes multiple feature extraction blocks (Transformer Block), and each feature extraction block is composed of a multi-head self-attention mechanism (Multi-Head Self-Attention, abbreviated as MHSA) and a feed-forward neural network (Feed-Forward Network, FFN).

[0054] Input the first vector into the feature extraction network layer to extract the semantic feature information of the magnetic resonance slice image and obtain the first feature vector.

[0055] S60: According to the first feature vector corresponding to each layer of magnetic resonance slice image and the deep learning neural network model, obtain the second feature vector.

[0056] In the embodiments of the present application, the deep learning neural network model is a masked autoencoder (Masked Autoencoder, abbreviated as MAE) model. The MAE model is a self-supervised learning model. By partially occluding the input data, the model is forced to learn deeper feature representations during the reconstruction process. The MAE model can not only use the previously extracted features for reconstruction, but also enhance the expressive ability of features through the learned context information.

[0057] Through the MAE model, perform deeper feature extraction on the first feature vector corresponding to each layer of magnetic resonance slice image to obtain the second feature vector.

[0058] S70: According to the second feature vector and the classification network model, obtain the classification result of the glioma image of the brain.

[0059] In the embodiment of the present application, the classification network model is a fully connected layer. The second feature vector is input into the fully connected layer to obtain the classification result of the brain glioma image. Specifically, the classification result of the brain glioma image is used to indicate the severity of glioma, including mild, moderate, and severe.

[0060] The classification result of the brain glioma image is used to provide an indirect basis for doctors to diagnose glioma diseases. That is, it can be used to assist doctors in diagnosing glioma diseases. Further, doctors need to combine the relevant characteristics of the patient, such as clinical characteristics, to determine the diagnosis result of glioma diseases.

[0061] Applying the embodiment of the present application, by obtaining a brain glioma image; the brain glioma image includes several layers of magnetic resonance slice images; through the image patch embedding layer, each layer of magnetic resonance slice image is segmented into several image patches, and each image patch is mapped to a corresponding embedding vector; a first position encoding vector is set for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image; adding the preset first classification marker vector, each embedding vector, and each first position encoding vector to obtain a first vector; through the feature extraction network layer, performing feature extraction on the first vector to obtain a first feature vector; obtaining a second feature vector according to the first feature vector corresponding to each layer of magnetic resonance slice image and the deep learning neural network model; and obtaining the classification result of the brain glioma image according to the second feature vector and the classification network model. The embodiment of the present application can effectively capture the local detail features and global semantic structure information of the brain glioma image based on the image patch embedding layer and the feature extraction network layer; and can capture the anatomical structure continuity between adjacent magnetic resonance slice images based on the deep learning neural network model, thereby improving the accuracy of the classification result of the brain glioma image.

[0062] In one embodiment, after step S10, it includes step S101, which is specifically as follows:

[0063] S101: Perform a preprocessing operation on the brain glioma image to obtain the preprocessed brain glioma image; wherein, the preprocessing operation includes but is not limited to performing gray-scale normalization processing, denoising processing, and registration and segmentation processing on the brain glioma image.

[0064] In the embodiment of the present application, considering that the quality of the brain glioma image directly affects the accuracy of the subsequent classification result, therefore, the brain glioma image can be preprocessed to improve the quality of the brain glioma image.

[0065] Specifically, for the sampling gray-scale normalization method, gray-scale normalization processing is performed on glioma brain images, which can effectively reduce the problem of inconsistent image gray scale caused by scanning equipment, parameter settings, or patient individual differences, thereby improving the stability of subsequent feature extraction. By using non-local means or wavelet transform methods to denoise glioma brain images, noise interference can be reduced, the visual quality of the images can be improved, and the feature extraction ability of deep learning models can also be enhanced. Based on brain atlases or deep learning models, multi-modal magnetic resonance sequence images (such as T1-weighted images, T2-weighted images, FLAIR images, and contrast-enhanced images, etc.) are aligned, and key regions such as the tumor core area and edema zone are segmented. Using registration technology, different-modal magnetic resonance sequence images are aligned to the same spatial coordinate system to facilitate subsequent multi-modal feature fusion.

[0066] In one embodiment, the feature extraction network layer includes a first feature extraction block. The first feature extraction block includes a first normalization layer, a first multi-head self-attention layer, a second normalization layer, and a first feed-forward neural network layer. Step S50 includes steps S501 to S504, which are specifically as follows:

[0067] S501: Input the first vector into the first normalization layer and the first multi-head self-attention layer in sequence to obtain a second vector;

[0068] S502: Add the second vector to the first vector to obtain a third vector;

[0069] S503: Input the third vector into the second normalization layer and the first feed-forward neural network layer in sequence to obtain a fourth vector;

[0070] S504: Add the fourth vector to the third vector to obtain a first feature vector.

[0071] Among them, the normalization layer (Layer Normalization) is used to normalize the activation values of all neurons in this layer. Specifically, the activation values of all neurons in this layer are converted into a distribution with a mean of 0 and a standard deviation of 1, and then the result is scaled and offset.

[0072] Among them, the multi-head self-attention layer (Multi-Head Self-Attention) includes a multi-head self-attention mechanism. The multi-head self-attention mechanism allows the model to simultaneously focus on different parts of the sequence when processing sequence data, so as to capture the complex relationships within the sequence. Specifically, the multi-head self-attention mechanism maps the input sequence into multiple different representation spaces, and then calculates the attention weights between these representations respectively.

[0073] Among them, the Feedforward Neural Network (FFN) is a neural network in which each neuron is arranged in layers. Each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer, without feedback between layers. The feedforward neural network usually consists of an input layer, one or more hidden layers, and an output layer.

[0074] In the embodiment of the present application, the representation of the first feature vector is as follows:

[0075] Z′ = MHSA(LayerNorm(Z)) + Z

[0076] Z″ = FFN(LayerNorm(Z′)) + Z′

[0077] Among them, Z′ represents the third vector, MHSA(LayerNorm(Z)) represents the second vector, Z″ represents the first feature vector, and FFN(LayerNorm(Z′)) represents the fourth vector.

[0078] Through the combined cooperation of the first normalization layer, the first multi-head self-attention layer, the second normalization layer, and the first feedforward neural network layer, accurate semantic feature information can be extracted from the first vector, improving the accuracy of the first feature vector.

[0079] In one embodiment, the deep learning neural network model includes an encoder and a decoder. Before step S60, it includes steps S601 to S604, which are specifically as follows:

[0080] S601: Obtain a number of sample first feature vectors.

[0081] In the embodiment of the present application, the sample first feature vector can be obtained based on the DinoV2 model, and the process can refer to the aforementioned steps S10 to S50, which will not be elaborated here.

[0082] S602: Mask a preset number of the sample first feature vectors among the number of sample first feature vectors, and input the unmasked sample first feature vectors into the encoder to obtain sample second feature vectors.

[0083] Among them, the preset number can be set according to actual needs.

[0084] In the embodiment of the present application, a part of the first feature vectors of the samples are randomly masked, and only the unmasked first feature vectors of the samples are retained as the input of the encoder. The encoder uses multiple layers of Transformer Blocks to encode the unmasked first feature vectors of the samples to obtain the second feature vectors of the samples. Among them, the Transformer Block consists of a multi-head self-attention mechanism (Multi-Head Self-Attention, abbreviated as MHSA) and a feed-forward neural network (Feed-Forward Network, FFN).

[0085] S603: Input the preset mask position vector and the second feature vector of the sample into the decoder to obtain the reconstructed third sample feature vector; among them, the mask position vector is used to indicate the information of the first feature vector of the sample processed by the masking process.

[0086] Among them, the mask position vector can indicate which first feature vectors of the samples are processed by the masking process.

[0087] In the embodiment of the present application, the decoder also uses multiple layers of Transformer Blocks to reconstruct the features at the mask position through the self-attention mechanism to obtain the reconstructed third sample feature vector.

[0088] S604: Train the encoder and the decoder according to the reconstructed third sample feature vector, a preset number of first feature vectors of the samples, and a preset loss function to obtain a trained deep learning neural network model.

[0089] Among them, the preset loss function includes but is not limited to the root mean square error loss function and the cross-entropy loss function.

[0090] In the embodiment of the present application, the root mean square error loss function is used to calculate the value of the loss function according to the reconstructed third sample feature vector and a preset number of first feature vectors of the samples, and optimize the network weight parameters of the encoder and the decoder according to the value of the loss function to obtain a trained deep learning neural network model.

[0091] In the process of training the MAE model in the embodiment of the present application, by randomly masking the input data and reconstructing the original information, the MAE model is forced to learn the context semantic representation of the data, which is convenient for subsequent application of the MAE model to the correlation modeling of the inter-layer features of magnetic resonance slice images, and uses its powerful reconstruction ability to capture the anatomical structure continuity between adjacent magnetic resonance slice images, thereby improving the accuracy of glioma image classification results.

[0092] In one embodiment, the deep learning neural network model includes an encoder, and step S60 includes steps S61 to S63, which are specifically as follows:

[0093] S61: Set a second position encoding vector for each layer of magnetic resonance slice images; wherein, the second position encoding vector is used to indicate the position information of the magnetic resonance slice image in the brain glioma image.

[0094] In the embodiments of the present application, by obtaining the horizontal and vertical position information of each layer of magnetic resonance slice images in the brain glioma image, and using the 1D position encoding method, the corresponding second position encoding vector is generated.

[0095] S62: Add the preset second classification marker vector, each first feature vector, and each second position encoding vector to obtain a fifth vector.

[0096] Among them, the preset second classification marker vector is a learnable embedding vector used to aggregate global features.

[0097] In the embodiments of the present application, the representation of the fifth vector is as follows:

[0098] Q = [q [CLS] ; q1; q2;...; q N + E″ pos

[0099] Wherein, Q represents the fifth vector, q [CLS] represents the preset second classification marker vector, q1, q2,..., q N respectively represent the 1st, 2nd,..., Nth first feature vectors, and E′ pos represents the second position encoding vector.

[0100] S63: Through the encoder, perform feature extraction on the fifth vector to obtain a second feature vector.

[0101] In the embodiments of the present application, input the fifth vector into the encoder to obtain a second feature vector.

[0102] In the embodiments of the present application, the encoder performs feature extraction on the fifth vector to capture the anatomical structure continuity between adjacent magnetic resonance slice images, improving the accuracy of the second feature vector.

[0103] In one embodiment, the encoder includes a second feature extraction block, and the second feature extraction block includes a third normalization layer, a second multi-head self-attention layer, a fourth normalization layer, and a second feed-forward neural network layer. Step S63 includes steps S631 to S633, which are specifically as follows:

[0104] S631: Input the fifth vector into the third normalization layer and the second multi-head self-attention layer in sequence to obtain a sixth vector;

[0105] S632: Add the sixth vector to the fifth vector to obtain a seventh vector;

[0106] S633: Input the seventh vector into the fourth normalization layer and the second feed-forward neural network layer in sequence to obtain an eighth vector;

[0107] S634: Add the eighth vector and the seventh vector to obtain a second feature vector.

[0108] In the embodiments of the present application, the second feature vector is represented as follows:

[0109] Q′ = MHSA(LayerNorm(Q)) + Q

[0110] Q″ = FFN(LayerNorm(Q′)) + Q′

[0111] Wherein, Q′ represents the seventh vector, MHSA(LayerNorm(Q)) represents the sixth vector, Q″ represents the second feature vector, and FFN(LayerNorm(Q′)) represents the eighth vector.

[0112] Through the combined cooperation of the third normalization layer, the second multi-head self-attention layer, the fourth normalization layer, and the second feed-forward neural network layer, the anatomical structure continuity between adjacent magnetic resonance slice images can be accurately extracted from the fifth vector, improving the accuracy of the second feature vector.

[0113] The following is an apparatus embodiment of the present application, which can be used to execute the content of the method in the embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the content of the method in the embodiments of the present application.

[0114] Please refer to Figure 2 , which shows a schematic structural diagram of a classification apparatus for brain glioma images provided in the embodiments of the present application. The classification apparatus 8 for brain glioma images provided in the embodiments of the present application includes:

[0115] An image acquisition module 81, configured to acquire brain glioma images; the brain glioma images include several magnetic resonance slice images;

[0116] An embedded vector acquisition module 82, configured to divide each magnetic resonance slice image into several image patches through an image patch embedding layer, and map each image patch to a corresponding embedded vector;

[0117] A position encoding vector acquisition module 83, configured to set a first position encoding vector for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image;

[0118] A first vector acquisition module 84, configured to add a preset first classification marker vector, each embedded vector, and each first position encoding vector to obtain a first vector;

[0119] The first eigenvector obtaining module 85 is configured to extract features from the first vector through a feature extraction network layer to obtain a first eigenvector;

[0120] The second eigenvector obtaining module 86 is configured to obtain a second eigenvector according to the first eigenvector corresponding to each magnetic resonance slice image and a deep learning neural network model;

[0121] The classification result obtaining module 87 is configured to obtain a classification result of a brain glioma image according to the second eigenvector and a classification network model.

[0122] It should be noted that when the classification device for brain glioma images provided in the above embodiments executes the classification method for brain glioma images, only the above-mentioned division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the classification device for brain glioma images and the classification method for brain glioma images provided in the above embodiments belong to the same concept. The implementation process is detailed in the method embodiments and will not be repeated here.

[0123] The following is an embodiment of the device of the present application, which can be used to execute the content of the method in the embodiments of the present application. For details not disclosed in the embodiments of the device of the present application, please refer to the content of the method in the embodiments of the present application.

[0124] Please refer to Figure 3 , the present application also provides an electronic device 300. The electronic device can specifically be a computer, a mobile phone, a tablet computer, etc. In an exemplary embodiment of the present application, the electronic device 300 is a computer, and the computer may include: at least one processor 301, at least one memory 302, at least one display, at least one network interface 303, a user interface 304, and at least one communication bus 305.

[0125] Among them, the user interface 304 is mainly used to provide an input interface for the user to obtain data input by the user. Optionally, the user interface may further include a standard wired interface and a wireless interface.

[0126] Among them, the network interface 303 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0127] Among them, the communication bus 305 is used to implement connection communication between these components.

[0128] Among them, the processor 301 may include one or more processing cores. The processor connects various parts within the entire electronic device through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory, and by invoking data stored in the memory, the processor performs various functions of the electronic device and processes data. Optionally, the processor may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed in the display layer; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor and may be implemented separately by a single chip.

[0129] Among them, the memory 302 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store the data involved in the above-mentioned various method embodiments. Optionally, the memory may also be at least one storage device located far from the aforementioned processor. As Figure 3 shown, the memory as a computer storage medium may include an operating system, a network communication module, a user interface module, and an operation application program.

[0130] The processor may be used to call the application program of the classification method of brain glioma images stored in the memory and specifically execute the method steps of the above-mentioned embodiments. The specific execution process may refer to the specific description shown in the embodiments and will not be elaborated here.

[0131] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent in such a process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of another identical element in the process, method, commodity or device comprising the element.

[0132] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A classification method for glioma images of the brain, characterized in that, It includes the following steps: Obtain a brain glioma image; the brain glioma image includes a number of magnetic resonance slice images; Through the image patch embedding layer, each layer of the magnetic resonance slice image is segmented into a number of image patches, and each image patch is mapped to a corresponding embedding vector; Set a first position encoding vector for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image; Add a preset first classification marker vector, each embedding vector, and each first position encoding vector to obtain a first vector; Through the feature extraction network layer, perform feature extraction on the first vector to obtain a first feature vector; According to the first feature vector corresponding to each layer of the magnetic resonance slice image and the deep learning neural network model, obtain a second feature vector; According to the second feature vector and the classification network model, obtain the classification result of the brain glioma image.

2. The classification method of the brain glioma image according to claim 1, wherein: The feature extraction network layer includes a first feature extraction block, and the first feature extraction block includes a first normalization layer, a first multi-head self-attention layer, a second normalization layer, and a first feed-forward neural network layer; The step of performing feature extraction on the first vector through the feature extraction network layer to obtain a first feature vector includes: Input the first vector into the first normalization layer and the first multi-head self-attention layer in sequence to obtain a second vector; Add the second vector to the first vector to obtain a third vector; Input the third vector into the second normalization layer and the first feed-forward neural network layer in sequence to obtain a fourth vector; Add the fourth vector to the third vector to obtain a first feature vector.

3. The classification method of the brain glioma image according to claim 1, wherein: The deep learning neural network model includes an encoder; The step of obtaining a second feature vector according to the first feature vector corresponding to each layer of the magnetic resonance slice image and the deep learning neural network model includes: Set a second position encoding vector for each layer of the magnetic resonance slice image; wherein, the second position encoding vector is used to indicate the position information of the magnetic resonance slice image in the brain glioma image; Add a preset second classification marker vector, each first feature vector, and each second position encoding vector to obtain a fifth vector; Through the encoder, perform feature extraction on the fifth vector to obtain a second feature vector.

4. The classification method of the brain glioma image according to claim 3, wherein: The encoder includes a second feature extraction block, and the second feature extraction block includes a third normalization layer, a second multi-head self-attention layer, a fourth normalization layer, and a second feed-forward neural network layer; The step of performing feature extraction on the fifth vector through the encoder to obtain a second feature vector includes: Input the fifth vector into the third normalization layer and the second multi-head self-attention layer in sequence to obtain a sixth vector; Add the sixth vector and the fifth vector to obtain a seventh vector; Input the seventh vector into the fourth normalization layer and the second feed-forward neural network layer in sequence to obtain an eighth vector; Add the eighth vector and the seventh vector to obtain a second feature vector.

5. The classification method for glioma brain image according to any one of claims 1 to 4, characterized in that: The deep learning neural network model includes an encoder and a decoder; Before the step of obtaining the second feature vector according to the first feature vector corresponding to each layer of the magnetic resonance slice image and the deep learning neural network model, it includes: Obtain a plurality of sample first feature vectors; Perform masking processing on a preset number of the sample first feature vectors among the plurality of sample first feature vectors, and input the unmasked sample first feature vectors into the encoder to obtain sample second feature vectors; Input the preset masking position vector and the sample second feature vectors into the decoder to obtain a reconstructed third sample feature vector; wherein, the masking position vector is used to indicate the information of the masked sample first feature vectors; Train the encoder and the decoder according to the reconstructed third sample feature vector, the preset number of sample first feature vectors, and a preset loss function to obtain a trained deep learning neural network model.

6. The classification method for glioma brain image according to any one of claims 1 to 4, characterized in that: After the step of obtaining the glioma brain image, it includes: Perform a preprocessing operation on the glioma brain image to obtain a preprocessed glioma brain image; wherein, the preprocessing operation includes but is not limited to performing gray-scale normalization processing, denoising processing, and registration and segmentation processing on the glioma brain image.

7. A classification device for glioma images of the brain, characterized in that, It includes: An image acquisition module, configured to acquire a glioma brain image; the glioma brain image includes a plurality of layers of magnetic resonance slice images; An embedded vector acquisition module, configured to divide each layer of the magnetic resonance slice image into a plurality of image patches through an image patch embedding layer, and map each image patch to a corresponding embedded vector; A position encoding vector acquisition module, configured to set a first position encoding vector for each image patch; wherein, the first position encoding vector is used to indicate the position information of each image patch in the magnetic resonance slice image; A first vector acquisition module, configured to add a preset first classification marker vector, each embedded vector, and each first position encoding vector to obtain a first vector; A first feature vector acquisition module, configured to perform feature extraction on the first vector through the feature extraction network layer to obtain a first feature vector; A second feature vector acquisition module, configured to obtain a second feature vector according to the first feature vector corresponding to each layer of the magnetic resonance slice image and the deep learning neural network model; A classification result obtaining module, configured to obtain a classification result of the glioma brain image according to the second feature vector and a classification network model.

8. An electronic device, characterized in that, It includes: A processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the steps of the classification method of the glioma brain image according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the classification method of the glioma brain image according to any one of claims 1 to 6.

Images