Glioma type identification method, model training method, device, and equipment

Abstract

The present disclosure provides a type identification method, model training method, device, and equipment for gliomas, which involve image processing technology, including: obtaining a histopathological image of a glioma; inputting the histopathological image into a preset recognition model, extracting each feature vector of each target block in the glioma, processing each feature vector based on the first channel to obtain first classification information and a target vector corresponding to each type, processing the target vector based on the second channel to obtain second classification information corresponding to each type; processing the first classification information and the second classification information to obtain a target classification result. In this solution, the first classification information and the target vector corresponding to each type can be obtained through the first channel; the target vector can be processed through the second channel to obtain the second classification information corresponding to each type; and the classification result can be obtained based on the first and second classification information. The accuracy of typing can be improved to a certain extent.

Description

Technical Field

[0001] The present disclosure relates to image processing technology, and in particular to a glioma type recognition method, model training method, device, and equipment. Background Art

[0002] Glioma is the most common primary brain tumor and the most common intracranial malignant tumor. According to its histological morphological characteristics, glioma is divided into three types: astrocytoma, oligodendroglioma and ependymoma. Accurate classification of glioma is very important.

[0003] In the prior art, the classification of gliomas mainly relies on pathologists observing tissue pathology images and confirming the classification based on histological morphological characteristics.

[0004] However, the accuracy of manual glioma classification needs to be improved. Summary of the invention

[0005] The present invention provides a glioma type identification method, model training method, device, and equipment, which can improve the accuracy of glioma classification to a certain extent.

[0006] According to a first aspect of the present disclosure, a method for identifying a type of glioma is provided, the method comprising:

[0007] Obtain histopathological images of gliomas;

[0008] The tissue pathology image is input into a preset recognition model, and each feature vector of each target block in the glioma is extracted based on the feature extraction module in the recognition model, and each feature vector is processed based on the first channel in the recognition model to obtain first classification information corresponding to each type and a target vector corresponding to each type, and the target vector is processed based on the second channel in the recognition model to obtain second classification information corresponding to each type; the first classification information and the second classification information are processed based on the integration module in the recognition model to obtain a target classification result of the glioma in the tissue pathology image;

[0009] The recognition model is trained using a training data set, wherein the training data set includes multiple groups of training data, and the training data includes histopathological images of gliomas and annotated types of gliomas in the histopathological images.

[0010] According to a second aspect of the present disclosure, a model training method is provided, the method comprising:

[0011] Acquire a training data set; wherein the training data set includes multiple groups of training data, and the training data includes histopathological images of gliomas and annotated types of the gliomas;

[0012] The histopathological image of the glioma is input into a preset model to obtain the predicted type of the glioma in the histopathological image; according to the labeled type and predicted type of the glioma, the parameters in the preset model are optimized to obtain a recognition model; wherein the recognition model is used to identify the type of glioma.

[0013] According to a third aspect of the present disclosure, a device for identifying a type of glioma is provided, the device comprising:

[0014] An acquisition unit, used for acquiring a tissue pathology image of a glioma;

[0015] A recognition unit, used for inputting the tissue pathology image into a preset recognition model, extracting each feature vector of each target block in the glioma based on a feature extraction module in the recognition model, processing each feature vector based on a first channel in the recognition model to obtain first classification information corresponding to each type and a target vector corresponding to each type, processing the target vector based on a second channel in the recognition model to obtain second classification information corresponding to each type; processing the first classification information and the second classification information based on an integration module in the recognition model to obtain a target classification result of the glioma in the tissue pathology image;

[0016] The recognition model is trained using a training data set, wherein the training data set includes multiple groups of training data, and the training data includes histopathological images of gliomas and annotated types of gliomas in the histopathological images.

[0017] According to a fourth aspect of the present disclosure, a model training device is provided, the device comprising:

[0018] An acquisition unit, configured to acquire a training data set; wherein the training data set includes multiple sets of training data, and the training data includes a histopathological image of a glioma and an annotated type of the glioma;

[0019] A training unit is used to input the tissue pathology image of the glioma into a preset model to obtain the predicted type of the glioma in the tissue pathology image; according to the labeled type and predicted type of the glioma, the parameters in the preset model are optimized to obtain a recognition model; wherein the recognition model is used to identify the type of glioma.

[0020] According to a fifth aspect of the present disclosure, an electronic device is provided, including a memory and a processor; wherein:

[0021] The memory is used to store computer programs;

[0022] The processor is used to read the computer program stored in the memory and execute the method as described in the first aspect and the second aspect according to the computer program in the memory.

[0023] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and the second aspect is implemented.

[0024] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method described in the first aspect and the second aspect is implemented.

[0025] The type identification method, model training method, device and equipment of glioma provided by the present disclosure include: obtaining a histopathological image of glioma; inputting the histopathological image into a preset recognition model, extracting each feature vector of each target block in the glioma based on the feature extraction module in the recognition model, processing each feature vector based on the first channel in the recognition model, obtaining the first classification information corresponding to each type and the target vector corresponding to each type, processing the target vector based on the second channel in the recognition model, obtaining the second classification information corresponding to each type; processing the first classification information and the second classification information based on the integration module in the recognition model, obtaining the target classification result of the glioma in the histopathological image; wherein the recognition model is obtained by training using a training data set, and the training data set includes multiple groups of training data, and the training data includes the histopathological image of the glioma, and the annotated type of the glioma in the histopathological image. In this scheme, the first classification information corresponding to each type and the target vector corresponding to each type can be obtained through the first channel in the recognition model; the target vector is processed through the second channel to obtain the second classification information corresponding to each type; and the classification result is obtained according to the first classification information and the second classification information. It can improve the accuracy of glioma classification to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a method for identifying the type of glioma according to an exemplary embodiment of the present disclosure;

[0027] Figure 2 A schematic diagram of a flow chart of a method for identifying a type of glioma according to another exemplary embodiment of the present disclosure;

[0028] Figure 3 A schematic diagram of a feature vector fusion process shown in an exemplary embodiment of the present disclosure;

[0029] Figure 4 A flowchart of a model training method according to an exemplary embodiment of the present disclosure is shown;

[0030] Figure 5 This is a schematic diagram of a glioma type identification process according to an exemplary embodiment of the present disclosure;

[0031] Figure 6 This is a structural diagram of a device for identifying the type of glioma according to an exemplary embodiment of the present disclosure;

[0032] Figure 7 A structural diagram of a model training device shown as an exemplary embodiment of the present disclosure;

[0033] Figure 8 The present invention is a structural diagram of an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0034] Glioma is the most common primary brain tumor and the most common intracranial malignant tumor. Glioma accounts for about 30% of all brain and central nervous system tumors and 80% of all malignant brain tumors. According to its histological morphological characteristics, glioma is divided into three types: astrocytoma, oligodendroglioma and ependymoma. Accurate classification of glioma is very important. In the prior art, the classification of glioma mainly relies on pathologists to observe histopathological images and confirm the classification through histological morphological characteristics.

[0035] However, the existing technology still has the following problems: 1) There are certain similarities between different types of gliomas, and the manual method is highly subjective and the accuracy needs to be improved; 2) The classification of oligodendroglioma depends on the isocitrate dehydrogenase (IDH) gene mutation status and chromosome 1p / 19q co-deletion status, which are obtained through gene sequencing; it is costly and inefficient.

[0036] In order to solve the above technical problems, in the solution provided by the present disclosure, the first classification information corresponding to each type and the target vector corresponding to each type can be obtained through the first channel in the recognition model; the target vector is processed through the second channel to obtain the second classification information corresponding to each type; and the classification result is obtained based on the first classification information and the second classification information. The accuracy and efficiency of glioma classification can be improved to a certain extent, and a higher accuracy can be obtained without obtaining molecular information such as IDH gene mutation status through gene sequencing.

[0037] Figure 1 The figure is a flowchart of a method for identifying the type of glioma according to an exemplary embodiment of the present disclosure.

[0038] like Figure 1 As shown, the method for identifying the type of glioma provided in this embodiment includes:

[0039] Step 101, obtaining a histopathological image of a glioma.

[0040] The method provided in the present disclosure may be performed by an electronic device with computing capability, such as a computer, etc. The electronic device is capable of acquiring histopathological images of gliomas.

[0041] The histopathological image of glioma is a full scan image of histopathological image of glioma. Specifically, the tissue slice of glioma can be scanned by a scanner to obtain an image of the tissue slice of glioma under the scanner, and the image can be digitized to obtain the histopathological image of glioma. The scanner can be an optical microscope.

[0042] Step 102, input the tissue pathology image into a preset recognition model, extract each feature vector of each target block in the glioma based on the feature extraction module in the recognition model, process each feature vector based on the first channel in the recognition model to obtain first classification information corresponding to each type and a target vector corresponding to each type, process the target vector based on the second channel in the recognition model to obtain second classification information corresponding to each type; process the first classification information and the second classification information based on the integration module in the recognition model to obtain a target classification result of the glioma in the tissue pathology image; wherein the recognition model is trained using a training data set, the training data set includes multiple groups of training data, and the training data includes a tissue pathology image of the glioma and annotated types of gliomas in the tissue pathology image.

[0043] Specifically, the tissue pathology image input into the preset recognition model can be first divided into multiple target blocks of equal size, and the feature extraction module in the recognition model is used to extract the feature vectors of each target block in the glioma. Specifically, a convolutional neural network (CNN) can be used to extract the feature vectors of each target block.

[0044] Specifically, the first channel in the recognition model is used to process each feature vector to obtain the first classification information corresponding to each type and the target vector corresponding to each type. The recognition model can recognize multiple types, and after the first channel processes each feature vector, the first classification information corresponding to each type can be obtained. The first classification information can characterize the probability of the type to which the glioma belongs. Specifically, the feature vector corresponding to the first classification information of each type can be determined as the target vector; there can be multiple target vectors.

[0045] Specifically, the target vector corresponding to each type can be input into the second channel, and the target vector can be processed based on the second channel to obtain the second classification information corresponding to each type. The second classification information can represent the probability of the type to which the glioma belongs.

[0046] Specifically, the integration module can obtain the target classification result of glioma based on the first classification information and the second classification information corresponding to each type.

[0047] Furthermore, the recognition model is pre-trained using a training data set, wherein the training data set includes multiple sets of training data, and the training data includes histopathological images of gliomas and annotated types of gliomas in the histopathological images.

[0048] Specifically, the recognition model obtained by training the labeled categories can recognize the same types as those included in the labeled categories. Specifically, the labeled categories can include multiple types, for example, astrocytoma, oligodendroglioma, and ependymoma.

[0049] The type identification method of glioma provided by the present disclosure includes: obtaining a histopathological image of glioma; inputting the histopathological image into a preset recognition model, extracting each feature vector of each target block in the glioma based on a feature extraction module in the recognition model, processing each feature vector based on the first channel in the recognition model, obtaining first classification information corresponding to each type and a target vector corresponding to each type, processing the target vector based on the second channel in the recognition model, obtaining second classification information corresponding to each type; processing the first classification information and the second classification information based on the integration module in the recognition model, obtaining the target classification result of the glioma in the histopathological image; wherein the recognition model is obtained by training using a training data set, and the training data set includes multiple sets of training data, and the training data includes the histopathological image of the glioma, and the annotated type of the glioma in the histopathological image. In the method adopted by the present disclosure, the first classification information corresponding to each type and the target vector corresponding to each type can be obtained through the first channel in the recognition model; the target vector is processed through the second channel to obtain the second classification information corresponding to each type; and the classification result is obtained according to the first classification information and the second classification information. It can improve the accuracy and efficiency of glioma classification to a certain extent, and can achieve a high accuracy without obtaining molecular information such as IDH gene mutation status through gene sequencing.

[0050] Figure 2 The figure is a flowchart of a method for identifying the type of glioma according to another exemplary embodiment of the present disclosure.

[0051] like Figure 2As shown, the method for identifying the type of glioma provided in this embodiment includes:

[0052] Step 201, obtaining a histopathological image of a glioma.

[0053] Specifically, the principle and implementation of step 201 are similar to those of step 101 and will not be described in detail.

[0054] Step 202, input the tissue pathology image into a preset recognition model, pre-process the tissue pathology image, and obtain a first image of the glioma; wherein the recognition model is trained using a training data set, the training data set includes multiple groups of training data, the training data includes the tissue pathology image of the glioma, and the annotated type of the glioma in the tissue pathology image.

[0055] Specifically, the resolution of the complete glioma tissue pathology full scan image is extremely high, which can reach 10 11 At the pixel level, due to the limitation of computation, it is unrealistic to directly use such images as the input of the neural network. Therefore, this scheme can first divide the tissue pathology image into blocks of fixed size, and then analyze and process these blocks.

[0056] Considering that there are a lot of blank areas and fragmented tissues in pathological sections, these areas are not very meaningful for type recognition. In order to further reduce the amount of calculation, this scheme can first extract the main tissue part in the tissue pathology image, and then extract the blocks in the main tissue part, which can reduce the interference of meaningless samples on the model.

[0057] Specifically, the recognition model is pre-trained using a training data set, wherein the training data set includes multiple sets of training data, and the training data includes histopathological images of gliomas and annotated types of gliomas in the histopathological images.

[0058] Specifically, the recognition model obtained by training the labeled categories can recognize the same types as those included in the labeled categories. Specifically, the labeled categories can include multiple types, for example, astrocytoma, oligodendroglioma, and ependymoma.

[0059] In one achievable manner, the tissue pathology image is compressed according to a preset ratio value to obtain a compressed tissue pathology image.

[0060] Specifically, the preset ratio value may be, for example, 1 / 16. Specifically, the histopathological image may be compressed to 1 / 16 of the original value to obtain a compressed histopathological image.

[0061] A second image of the glioma is determined according to the compressed tissue pathology image, the preset hue value, and the preset saturation value.

[0062] Specifically, the histopathological image may be an image obtained by a histopathological section stained with hematoxylin-eosin. The histopathological section stained with hematoxylin-eosin shows that the cell nucleus is stained with blue-purple and the cytoplasm is stained with different degrees of red. In this embodiment, the compressed histopathological image may be converted from the RGB space to the HSV space so as to screen the stained tissue area by hue and saturation.

[0063] The preset hue value range may be [0, 20] or [160, 360]; the preset saturation value may be 30. In the HSV space, the hue value may be in the range of [0, 20] or [160, 360] and the saturation value may be above 30, and only pixels meeting the standard may be retained to obtain a second image of the glioma.

[0064] Among them, the RGB color model is a color standard in the industry. It obtains various colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them on each other. RGB represents the colors of the three channels of red, green, and blue. This standard covers almost all colors that can be perceived by human vision and is one of the most widely used color systems at present.

[0065] Among them, HSV (Hue Saturation Value) is a color space created by AR Smith in 1978 based on the intuitive characteristics of color, also known as the Hexcone Model. The color parameters in this model are: Hue (Hue, H), Saturation (Saturation, S), Value (Value, V).

[0066] The second image is binarized to obtain a binarized image; and the binarized image is median filtered to obtain the first image.

[0067] Among them, image binarization is to set the grayscale value of the pixel points on the image to 0 or 255, that is, to make the entire image present an obvious black and white effect.

[0068] Specifically, median filtering is performed on the binarized image to remove isolated noise points and eliminate fine and fragmented tissues.

[0069] Step 203: Determine each target block in the glioma according to the first image and the preset sliding window.

[0070] Specifically, a preset sliding window may be used to slide on the first image to determine each target block in the glioma.

[0071] In one achievable manner, a preset sliding window is used to slide on the first image to obtain a plurality of first to-be-selected image blocks.

[0072] Specifically, the preset sliding window is preset according to actual conditions. The preset sliding window includes a preset sliding window size and a preset step size. The preset sliding window size may be 64*64; the preset step size may be 64.

[0073] If the pixel value of the first to-be-selected image block is greater than the preset pixel value, the first to-be-selected image block is determined to be the second to-be-selected image block.

[0074] The preset pixel value is a pixel value preset according to actual conditions.

[0075] Optionally, the second image can be used as a mask and a preset sliding window can be used to slide on the mask to obtain multiple first candidate blocks. If the pixel value of the first candidate block is greater than the preset pixel value, the first candidate block is determined to be the second candidate block.

[0076] Specifically, the second block to be selected is the main tissue part of the screened glioma.

[0077] The region in the tissue pathology image corresponding to the second image block to be selected is determined as the target image block.

[0078] Specifically, the position of the second block to be selected in the second image is mapped back to the histopathological image to obtain a region in the histopathological image corresponding to the second block to be selected. The block formed by the region is the target block.

[0079] Step 204: extract the features of the target image block using a feature extraction module to obtain a feature vector of the target image block.

[0080] Specifically, the feature vector of each target tile can be extracted using a CNN network pre-trained on ImageNet.

[0081] ImageNet is the name of a computer vision system recognition project and an image recognition database. It is built by simulating the human recognition system.

[0082] Step 205, input the feature vector into the fully connected layer corresponding to each type in the first channel to obtain intermediate classification information corresponding to the feature vector and each type.

[0083] The recognition model may be a model created based on CNN. The first channel included in the recognition model may include multiple fully connected layers. The fully connected layer may process each input feature vector to obtain intermediate classification information corresponding to the feature vector and the type.

[0084] Among them, the intermediate classification information can represent the probability of the type of glioma.

[0085] Specifically, glioma may include multiple types, so each feature vector may correspond to multiple intermediate classification information.

[0086] Step 206: determine the first classification information corresponding to the type in the intermediate classification information corresponding to the type.

[0087] Specifically, among the multiple intermediate classification information corresponding to the type, the most representative one can be extracted as the first classification information corresponding to the type. For example, the intermediate classification information representing the maximum probability of the type can be determined as the first classification information.

[0088] In one implementable manner, the intermediate classification information of the type is used to characterize the probability value that the glioma belongs to the type; and the probability value with the largest probability of characterizing that the glioma belongs to the type is determined as the first classification information of the type.

[0089] Specifically, if the intermediate classification information of the type represents the probability value of glioma belonging to the type; then the probability value with the largest probability of representing that glioma belongs to the type is determined as the first classification information of the type.

[0090] Step 207: determine the feature vector corresponding to the first classification information as a target vector corresponding to the type.

[0091] Specifically, the feature vector corresponding to the intermediate classification information as the first classification information is determined as the target vector corresponding to the type.

[0092] Specifically, each type has a corresponding target vector, and the target vector corresponding to each type may be different.

[0093] Step 208: fuse the first eigenvector and each second eigenvector to obtain a first fused vector corresponding to the first eigenvector; wherein the second eigenvector is a eigenvector adjacent to the first eigenvector.

[0094] Specifically, multiple tiles can be obtained by presetting a sliding window, and the multiple tiles can be sorted according to the time sequence of tile acquisition. Each tile corresponds to a feature vector. The feature vectors can be sorted according to the order of the tiles. Each feature vector is adjacent to its previous and next feature vectors. The first feature vector is adjacent to the second and last feature vectors. The last feature vector is adjacent to the first and second to last feature vectors.

[0095] In one implementable manner, a first attention weight between the first eigenvector and itself and second attention weights between the first eigenvector and each second eigenvector are determined based on the first eigenvector and each second eigenvector.

[0096] Specifically, the second channel may include three fully connected layers, and each feature vector passes through the three fully connected layers to obtain three vectors; the first attention weight and the second attention weight can be determined by the three vectors corresponding to each feature vector.

[0097] Specifically, the formula for the first attention weight is as follows:

[0098]

[0099] Q 0 =x 0 *W Q

[0100] K 0 =x 0 *W K

[0101] Among them, W Q , W K , respectively, the two fully connected layers included in the second channel, so that W Q , W K ∈R 1024*256 ;x 0 is the first eigenvector, which is a d-dimensional vector, and we can set d=1024; d k As a hyperparameter, we can let d k =64; T means transpose; Q 0 For x 0 By W Q The obtained vector can be used to make Q 0 ∈R 256 ; K 0 For x 0 By W K The obtained vector can be used to make K 0 ∈R 256 .

[0102] The formula for the second attention weight is as follows:

[0103]

[0104] Q 0 =x 0 *W Q

[0105] K 1 =x 1 *W K Among them, W Q , W K , respectively, the two fully connected layers included in the second channel, so that W Q , W K ∈R1024 *256 ;x 0 is the first eigenvector, which is a d-dimensional vector; x 1 is the second eigenvector, which is a d-dimensional vector, and we can set d=1024; d k As a hyperparameter, we can let d k =64; T means transpose; Q 0 For x 0 By W Q The obtained vector can be used to make Q 0 ∈R 256 ; K 1 For x 1 By W K The obtained vector can be used to make K 1 ∈R 256 .

[0106] A first product is determined according to the first eigenvector and the first attention weight, and a second product is determined according to the second eigenvector and the second attention weight corresponding to the second eigenvector; and a sum of the first product and each second product is determined as a first fusion vector of the first eigenvector.

[0107] Specifically, the formula of the first fusion vector is as follows:

[0108]

[0109] Q 0 =x 0 *W Q ;Q 1 =x 1 *W Q ;Q 2 =x 2 *W Q

[0110] K 0 =x 0 *W K ; K 1 =x 1 *W K ; K 2 =x 2 *W K

[0111] V 0 =x 0 *W V ; V 1 =x 1 *W V ; V 2 =x 2 *W V

[0112] Among them, W Q , W K , W V The three fully connected layers included in the second channel can make W Q , W K , W V ∈R 1024*256 ;x 0 is the first eigenvector, which is a d-dimensional vector; x 1 and x 2 The second eigenvector is a d-dimensional vector, and d can be set to 1024; d k As a hyperparameter, we can let d k =64; T means transpose; softmax means normalization algorithm; Q 0 For x 0 By W Q The obtained vector can be used to make Q 0 ∈R 256 ; K 0 For x 0 By W K The obtained vector can be used to make K 0 ∈R 256 ; V 0 For x 0 By W V The obtained vector can be used to make V 0 ∈R 256 ;Q 1 For x 1 By W Q The obtained vector can be used to make Q 1 ∈R 256 ; K 1 For x 1 By W K The obtained vector can be used to make K 1 ∈R 256 ; V 1 For x 1 By W V The obtained vector can be used to make V 0 ∈R 256 ;Q 2 For x 2 By W Q The obtained vector can be used to make Q 2 ∈R 256 ; K 2 For x 2 By W K The obtained vector can be used to make K 2 ∈R 256 ; V 2 For x 2 By W VThe obtained vector can be used to make V 2 ∈R 256 .

[0113] Step 209: fuse the target vector and each first fused vector to obtain a second fused vector corresponding to the target vector.

[0114] Specifically, each type corresponds to a target vector, and each target vector is fused with each first fusion vector to obtain a second fusion vector corresponding to the target vector.

[0115] In one achievable manner, third attention weights between the target vector and the first fusion vectors are determined based on the target vector and the first fusion vectors.

[0116] Specifically, the formula for the third attention weight is as follows:

[0117]

[0118] Q 4 =x 4 *W Q

[0119] K 5 =x 5 *W K

[0120] Among them, W Q , W K , respectively, the two fully connected layers included in the second channel, so that W Q , W K ∈R 1024*256 ;x 4 is the target vector, which is a d-dimensional vector; x 5 is the first fused vector, which is a d-dimensional vector, and d can be set to 1024; d k As a hyperparameter, we can let d k =64; T means transpose; Q 4 For x 4 By W Q The obtained vector can be used to make Q 0 ∈R 256 ; K 5 For x 5 By W K The obtained vector can be used to make K 5 ∈R 256 .

[0121] A third product is determined according to the first fusion vector and a third attention weight corresponding to the first fusion vector; and a sum of the third products is determined as a second fusion vector of the target vector.

[0122] Specifically, the formula of the second fusion vector is as follows:

[0123]

[0124] Q 4 =x 4 *W Q ; K m =x m *W K ; V m =x m *W V

[0125] Among them, W Q , W K , W V The three fully connected layers included in the second channel can be made W Q , W K , W V ∈R 1024*256 ;x 4 is the target vector, which is a d-dimensional vector; x m represents the mth first fused vector, which is a d-dimensional vector, and d can be set to 1024; d k As a hyperparameter, we can let d k =64; n represents the total number of the first fusion vectors; T represents transposition; softmax represents the normalization algorithm; Q 4 For x 4 By W Q The obtained vector can be used to make Q 4 ∈R 256 ; K m For x m By W K The obtained vector can be used to make K m ∈R 256 ; V m For x m By W V The obtained vector can be used to make V m ∈R 256 .

[0126] Specifically, the feature vector fusion method provided in this embodiment is as follows: Figure 3 As shown, the circle in the outer circle represents the feature vector, and the circle at the center represents the target vector. Each feature vector is first fused with two adjacent feature vectors to obtain a first fused vector; the target vector is then fused with all the first fused vectors to obtain a second fused vector.

[0127] Step 210: Determine second classification information corresponding to each type according to each second fusion vector corresponding to each target vector.

[0128] Specifically, each type has a corresponding target vector, and each target vector corresponds to a second fusion vector, so each type has a corresponding second fusion vector. The second classification information of the type can be determined using the second fusion vector corresponding to the type.

[0129] The second classification information can be used to characterize the probability of the type to which the glioma belongs.

[0130] Step 211: The first classification information is a first probability value, and the second classification information is a second probability value; the total probability corresponding to each type is determined according to the first probability value and the second probability value corresponding to each type.

[0131] Specifically, the total probability formula corresponding to each type is as follows:

[0132]

[0133] Among them, i represents the i-th type; K represents the total number of types; represents the first probability value of the i-th type; represents the second probability value of the i-th type; c i Represents the total probability value of the i-th type.

[0134] Step 212, determining the target classification result in each category according to the total probability of each category.

[0135] Specifically, the type with the largest total probability among all types is determined as the target classification result.

[0136] Figure 4 A flowchart of a model training method according to an exemplary embodiment of the present disclosure.

[0137] like Figure 4 As shown, the model training method provided in this embodiment includes:

[0138] Step 401, obtaining a training data set; wherein the training data set includes multiple groups of training data, and the training data includes histopathological images of gliomas and annotated types of gliomas.

[0139] Specifically, the training data set may be historical glioma tissue pathology images of Hospital X.

[0140] Step 402, input the histopathological image of the glioma into the preset model to obtain the predicted type of the glioma in the histopathological image; according to the labeled type and predicted type of the glioma, optimize the parameters in the preset model to obtain the recognition model; wherein the recognition model is used to identify the type of glioma.

[0141] Specifically, the recognition model obtained by training the labeled categories can recognize the same types as those included in the labeled categories. Specifically, the labeled categories can include multiple types, for example, astrocytoma, oligodendroglioma, and ependymoma.

[0142] Figure 5 This is a schematic diagram of a glioma type identification process according to an exemplary embodiment of the present disclosure.

[0143] like Figure 5 As shown, a tissue section of a glioma can be scanned by a scanner to obtain a histopathological image of the glioma, and then a plurality of blocks can be extracted from the histopathological image of the glioma; the blocks can be converted into feature vectors using a CNN method;

[0144] Using the first channel to process each feature vector and perform type prediction, obtaining first classification information corresponding to each type and a target vector corresponding to each type;

[0145] Using the second channel to process each eigenvector, each eigenvector is fused with its two adjacent eigenvectors to obtain a first fused vector corresponding to each eigenvector;

[0146] Using the second channel to fuse the target vector with the plurality of first fused vectors to obtain a second fused vector; determining second classification information corresponding to each type according to the plurality of second fused vectors;

[0147] The total probability corresponding to each type is determined according to the first classification information and the second classification information; and the target classification result is determined in each type according to the total probability of each type.

[0148] Figure 6 The present invention is a structural diagram of a device for identifying the type of glioma according to an exemplary embodiment of the present invention.

[0149] like Figure 6 As shown, the glioma type identification device 600 provided by the present disclosure includes:

[0150] An acquisition unit 610 is used to acquire a histopathological image of a glioma;

[0151] The recognition unit 620 is used to input the tissue pathology image into a preset recognition model, extract each feature vector of each target block in the glioma based on the feature extraction module in the recognition model, process each feature vector based on the first channel in the recognition model to obtain first classification information corresponding to each type and a target vector corresponding to each type, process the target vector based on the second channel in the recognition model to obtain second classification information corresponding to each type; process the first classification information and the second classification information based on the integration module in the recognition model to obtain a target classification result of the glioma in the tissue pathology image;

[0152] The recognition model is trained using a training data set, which includes multiple groups of training data. The training data include histopathological images of gliomas and labeled types of gliomas in the histopathological images.

[0153] The recognition unit 620 is specifically used to input the feature vector into the fully connected layer corresponding to each type in the first channel to obtain intermediate classification information corresponding to the feature vector and each type;

[0154] determining first classification information corresponding to the type in the intermediate classification information corresponding to the type;

[0155] The feature vector corresponding to the first classification information is determined as a target vector corresponding to the type.

[0156] The identification unit 620 is specifically used for the intermediate classification information of the type to characterize the probability value of the glioma belonging to the type; and the probability value with the largest probability of characterizing that the glioma belongs to the type is determined as the first classification information of the type.

[0157] The identification unit 620 is specifically used to fuse the first feature vector and each second feature vector to obtain a first fused vector corresponding to the first feature vector; wherein the second feature vector is a feature vector adjacent to the first feature vector;

[0158] Fusing the target vector and each of the first fused vectors to obtain a second fused vector corresponding to the target vector;

[0159] According to each second fusion vector corresponding to each target vector, second classification information corresponding to each type is determined.

[0160] The identification unit 620 is specifically configured to determine, according to the first eigenvector and each second eigenvector, a first attention weight between the first eigenvector and itself, and each second attention weight between the first eigenvector and each second eigenvector;

[0161] A first product is determined according to the first eigenvector and the first attention weight, and a second product is determined according to the second eigenvector and the second attention weight corresponding to the second eigenvector; and a sum of the first product and each second product is determined as a first fusion vector of the first eigenvector.

[0162] The identification unit 620 is specifically configured to determine, according to the target vector and each first fusion vector, a third attention weight between the target vector and itself, and each fourth attention weight between the target vector and each first fusion vector;

[0163] A third product is determined according to the target vector and the third attention weight, and a fourth product is determined according to the first fusion vector and the fourth attention weight corresponding to the first fusion vector; and a sum of the third product and each fourth product is determined as a second fusion vector of the target vector.

[0164] The identification unit 620 is specifically configured to determine the total probability corresponding to each type according to the first probability value and the second probability value corresponding to each type when the first classification information is a first probability value and the second classification information is a second probability value;

[0165] According to the total probability of each type, the target classification result is determined in each type.

[0166] The recognition unit 620 is specifically used to pre-process the tissue pathology image to obtain a first image of the glioma;

[0167] Determine each target block in the glioma according to the first image and the preset sliding window;

[0168] The feature extraction module is used to extract the features of the target block and obtain the feature vector of the target block.

[0169] The identification unit 620 is specifically used to compress the tissue pathology image according to a preset ratio value to obtain a compressed tissue pathology image;

[0170] Determining a second image of the glioma according to the compressed tissue pathology image, a preset hue value, and a preset saturation value;

[0171] The second image is binarized to obtain a binarized image; and the binarized image is median filtered to obtain the first image.

[0172] The identification unit 620 is specifically configured to slide a preset sliding window on the first image to obtain a plurality of first to-be-selected image blocks;

[0173] If the pixel value of the image block to be selected is greater than the preset pixel value, determining the first image block to be selected as the second image block to be selected;

[0174] The region in the tissue pathology image corresponding to the second image block to be selected is determined as the target image block.

[0175] Figure 7 The figure is a structural diagram of a model training device according to an exemplary embodiment of the present disclosure.

[0176] like Figure 7 As shown, the model training device 700 provided by the present disclosure includes:

[0177] An acquisition unit 710 is used to acquire a training data set; wherein the training data set includes multiple sets of training data, and the training data includes histopathological images of gliomas and annotated types of gliomas;

[0178] The training unit 720 is used to input the histopathological image of the glioma into a preset model to obtain the predicted type of the glioma in the histopathological image; according to the labeled type and predicted type of the glioma, the parameters in the preset model are optimized to obtain the recognition model; wherein the recognition model is used to identify the type of glioma.

[0179] Figure 8 The present invention is a structural diagram of an electronic device according to an exemplary embodiment of the present invention.

[0180] like Figure 8 As shown, the electronic device provided in this embodiment includes:

[0181] Memory 801;

[0182] Processor 802; and

[0183] Computer programs;

[0184] The computer program is stored in the memory 801 and is configured to be executed by the processor 802 to implement any of the above methods.

[0185] This embodiment also provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement any of the above methods.

[0186] This embodiment also provides a computer program product, including a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0187] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying the type of glioma, characterized in that: The method comprises: Obtain histopathological images of gliomas; The tissue pathology image is input into a preset recognition model, and each feature vector of each target block in the glioma is extracted based on the feature extraction module in the recognition model, and each feature vector is processed based on the first channel in the recognition model to obtain first classification information corresponding to each type and a target vector corresponding to each type, wherein the target vector is a feature vector corresponding to the first classification information of each type, and there are multiple target vectors, and the target vector is processed based on the second channel in the recognition model to obtain second classification information corresponding to each type; the first classification information and the second classification information are processed based on the integration module in the recognition model to obtain a target classification result of the glioma in the tissue pathology image; Wherein, the recognition model is obtained by training using a training data set, wherein the training data set includes multiple sets of training data, and the training data includes a histopathological image of a glioma, and annotated types of gliomas in the histopathological image; The processing of each of the feature vectors based on the first channel in the recognition model to obtain first classification information corresponding to each type and a target vector corresponding to each type includes: Inputting the feature vector into the fully connected layer corresponding to each type in the first channel to obtain intermediate classification information corresponding to the feature vector and each type; determining first classification information corresponding to the type in the intermediate classification information corresponding to the type; Determine the feature vector corresponding to the first classification information as a target vector corresponding to the type; The intermediate classification information of the type is used to represent the probability value of the glioma belonging to the type; The determining the first classification information corresponding to the type in the intermediate classification information corresponding to the type includes: Determine the probability value for characterizing that the glioma has the highest probability of belonging to the type as the first classification information of the type; The processing of the target vector based on the second channel in the recognition model to obtain second classification information corresponding to each type includes: Fusing the first eigenvector and each second eigenvector to obtain a first fused vector corresponding to the first eigenvector; wherein the second eigenvector is a eigenvector adjacent to the first eigenvector; Fusing the target vector and each of the first fused vectors to obtain a second fused vector corresponding to the target vector; The second classification information corresponding to each type is determined according to each of the second fusion vectors corresponding to each of the target vectors.

2. The method according to claim 1, characterized in that The fusing the first eigenvector and each second eigenvector to obtain a first fused vector corresponding to the first eigenvector includes: Determine, according to the first feature vector and each of the second feature vectors, a first attention weight between the first feature vector and itself, and each second attention weight between the first feature vector and each of the second feature vectors; A first product is determined according to the first eigenvector and the first attention weight, and a second product is determined according to the second eigenvector and a second attention weight corresponding to the second eigenvector; and a sum of the first product and each of the second products is determined as a first fusion vector of the first eigenvector.

3. The method according to claim 1, characterized in that The fusing the target vector and each of the first fused vectors to obtain a second fused vector corresponding to the target vector includes: Determining third attention weights between the target vector and each of the first fusion vectors according to the target vector and each of the first fusion vectors; A third product is determined according to the first fusion vector and a third attention weight corresponding to the first fusion vector; and a sum of the third products is determined as a second fusion vector of the target vector.

4. The method according to claim 1, characterized in that: in, The first classification information is a first probability value, and the second classification information is a second probability value; The step of processing the first classification information and the second classification information based on the integration module in the recognition model to obtain a target classification result of the glioma in the tissue pathology image includes: Determine a total probability corresponding to each of the types according to the first probability value and the second probability value corresponding to each of the types; The target classification result is determined in each of the types according to the total probability of each of the types.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a training data set; wherein the training data set includes multiple groups of training data, and the training data includes histopathological images of gliomas and annotated types of the gliomas; The histopathological image of the glioma is input into a preset model to obtain the predicted type of the glioma in the histopathological image; according to the labeled type and predicted type of the glioma, the parameters in the preset model are optimized to obtain a recognition model; wherein the recognition model is used to identify the type of glioma.

6. An electronic device, characterized in that: comprising a memory and a processor; wherein, The memory is used to store computer programs; The processor is used to read the computer program stored in the memory and execute the method according to any one of claims 1 to 5 according to the computer program in the memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method described in any one of claims 1 to 5 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method described in any one of claims 1 to 5 is implemented.

Images