Tumor image classification and identification method, device and equipment

Abstract

The invention provides a tumor image classification and recognition method, device and equipment, and relates to the technical field of medical image recognition. According to the method, the difference degree between different types of tumor images is enhanced by combining the images of the central region and the peritumoral region of the tumor and by means of the tumor image of the peritumoral region. According to the method, multi-modal images such as diffusion weighted imaging (DWI), liver enhanced scanning (HBP) and T2 weighted imaging (T2WI) are comprehensively analyzed, image omics features such as first-order statistical features, shape features and texture features and depth features such as basic shape features, local shape features and global features are extracted, fusion, screening and dimension reduction are carried out, and finally, the image quality is improved. According to the method, the inherent features of each type of tumor images are reserved, the difference degree between different types of tumors is ensured, the complexity of feature vectors is reduced, and the interference of useless information is reduced. The classification and recognition model obtained through training can improve the accuracy of tumor image classification and recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image recognition, and in particular to a method, device and equipment for classifying and recognizing tumor images. Background Art

[0002] In the context of the rapid development of medical imaging technology, the classification and identification of tumor types relies on high-precision medical imaging recognition technology. However, the low accuracy of tumor image classification and recognition has become a key technical issue that restricts the further development of medical imaging technology.

[0003] The image features of tumor lesions are highly similar between different individuals and at different stages of the same individual. This similarity makes it difficult for traditional image classification algorithms to accurately distinguish different types of tumors, resulting in a significant reduction in classification accuracy. In addition, existing medical image recognition technologies mainly rely on single-modality image analysis methods, such as diffusion-weighted imaging (DWI), liver enhancement scanning (HBP), and T2-weighted imaging (T2WI). These single-modality image information often cannot fully reflect the internal structure and boundary changes of tumors, especially their complex and variable heterogeneous characteristics. At the same time, the interference of multiple factors such as image noise, resolution differences, and individual differences among patients further aggravates the difficulty of tumor image recognition, making the recognition results susceptible to errors and difficult to ensure accuracy. Summary of the invention

[0004] The present invention provides a tumor image classification and recognition method, device and equipment, which can improve the accuracy of tumor image classification and recognition.

[0005] In a first aspect, the present invention provides a method for classifying and identifying tumor images, the method comprising: obtaining multimodal images of central areas and peritumoral areas of various types of tumors, and the tumor types corresponding to each multimodal image; performing feature extraction and fusion on the multimodal images to obtain fusion features of each multimodal image, the fusion features including imaging genomics features and depth features, the imaging genomics features including first-order statistical features, shape features and texture features; the depth features including basic shape features, local shape features and global features; based on the fusion features of each multimodal image, performing feature screening and dimensionality reduction to obtain key features of each multimodal image; based on the key features of each multimodal image and the tumor types corresponding to each multimodal image, generating multiple training samples; based on the multiple training samples, training a classification and recognition model.

[0006] In a second aspect, an embodiment of the present invention provides a tumor image classification and identification device, the device comprising: a communication module, a feature extraction module, a feature screening module and a model training module, the communication module obtains multimodal images of the central area and the peritumoral area of ​​each type of tumor, and the tumor type corresponding to each multimodal image; the feature extraction module extracts and fuses features of the multimodal images to obtain fusion features of each multimodal image, the fusion features include imaging genomics features and depth features, the imaging genomics features include first-order statistical features, shape features and texture features; the depth features include basic shape features, local shape features and global features; the feature screening module performs feature screening and dimensionality reduction based on the fusion features of each multimodal image to obtain the key features of each multimodal image; the model training module generates multiple training samples based on the key features of each multimodal image and the tumor type corresponding to each multimodal image; based on the multiple training samples, a classification and identification model is trained.

[0007] In a possible implementation, the classification and identification device further includes a classification evaluation module; the classification evaluation module classifies and identifies the multimodal image to be identified based on the classification evaluation model to obtain a classification and identification result; the classification and identification result includes a tumor type.

[0008] In a possible implementation, the classification and recognition device also includes a preprocessing module; the preprocessing module resamples each multimodal image to obtain a resampled multimodal image, and the resampled multimodal image is an image with the same image size and consistent pixel spacing in each dimension; the resampled multimodal image is subjected to image grayscale normalization processing to obtain a preprocessed multimodal image.

[0009] In a possible implementation, the feature extraction module is specifically used to extract features based on each multimodal image to obtain imaging features; the first-order statistical features include the mean, median, standard deviation, skewness and kurtosis of the image pixels; the shape features include the volume, surface area, compactness and sphericity of the tumor in the image; the texture features include the grayscale co-occurrence matrix, the grayscale length matrix, the grayscale structure matrix, the domain grayscale difference matrix and the grayscale dependency matrix; the residual network is used to extract deep features from each multimodal image, the basic shape features are used to describe the contour and internal structure of the tumor, the local shape features are used to describe the detailed information of the local area of ​​the tumor, and the global features are used to describe the overall morphology and diffusion of the tumor; based on the imaging features and deep features of each multimodal image, feature splicing and fusion are performed to obtain the fusion features of each multimodal image.

[0010] In a possible implementation, the feature screening module is specifically used to perform a Levene variance consistency test based on the fusion features of each multimodal image and the tumor type corresponding to each multimodal image to obtain a test result; the test result includes a Levene statistic between multiple fusion features corresponding to each tumor type, and the Levene statistic is used to characterize the degree of difference between the fusion features corresponding to each tumor type; based on the test result, multiple fusion features corresponding to each tumor type are screened to obtain multiple fusion features corresponding to each tumor type after screening; for any fusion feature among the multiple fusion features corresponding to each tumor type after screening, a random forest classifier is used to evaluate the importance of each feature in the fusion feature; based on the importance of each feature in the fusion feature, the fusion feature is feature eliminated, and the retained feature is standardized to obtain the standard feature corresponding to the fusion feature; for the multiple standard features of each tumor type, norm regularization is applied to obtain the key features of each multimodal image.

[0011] In a possible implementation, the model training module is specifically used to perform the following steps: step one, based on the linear regression model and the sigmoid logic function, construct a logistic regression model; step two, based on the training set in multiple training samples, train the logistic regression model to obtain the trained logistic regression model; step three, based on the test set in multiple training samples, test the trained logistic regression model to obtain the tested logistic regression model and the loss function value; step four, if the loss function value meets the set conditions, terminate the iterative process and determine the tested logistic regression model as a classification recognition model; if the loss function value does not meet the set conditions, repeat steps two to four until the loss function value meets the set conditions.

[0012] In a possible implementation, the classification evaluation module is also used to split the multimodal image to obtain a single-modal image corresponding to each multimodal image, and the single-modal image includes a DWI image, an HBP image, and a T2WI image; based on the multimodal image and the single-modal image corresponding to each multimodal image, a plurality of test samples are generated, and the test samples include a multimodal sample, a DWI sample, an HBP sample, and a T2WI sample; based on the plurality of test samples and the classification recognition model, evaluation indicators of each type of test sample are determined; the evaluation indicators include accuracy, F1 score, and area under the ROC curve; based on the evaluation indicators of each type of test sample, the imaging modality with the best classification effect is determined.

[0013] In a possible implementation, the classification and evaluation module is further used to classify the peritumoral areas in various types of modal images based on multiple test samples, and obtain test samples of various peritumoral ranges corresponding to various types of modal images; for any type of modal image, based on the classification and recognition model, and test samples of various peritumoral ranges of this type of modal image, tests are performed to obtain evaluation indicators of this type of modal image in various peritumoral ranges; based on the evaluation indicators of various peritumoral ranges of various types of modal images, the peritumoral range with the best classification effect under various types of modal images is determined.

[0014] In a possible implementation, the feature extraction module crops the multimodal image of the peritumoral area based on the peritumoral range with the best classification effect to obtain the cropped multimodal image of the peritumoral area; based on the cropped multimodal image of the peritumoral area and the multimodal image of the central area, feature extraction and fusion are performed to obtain updated fusion features; the feature screening module performs feature screening and dimensionality reduction based on the updated fusion features of each multimodal image to obtain updated key features of each multimodal image; the model training module generates multiple updated training samples based on the updated key features of each multimodal image and the tumor type corresponding to each multimodal image; based on the multiple updated training samples, the classification and recognition model is retrained to obtain an updated classification and recognition model.

[0015] In a third aspect, an embodiment of the present invention provides a tumor image classification and recognition device, the electronic device comprising a memory and a processor, the memory storing a computer program, the processor being used to call and run the computer program stored in the memory to execute the method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein when the computer program is executed by a processor, the steps of the method described in the first aspect and any possible implementation method of the first aspect are implemented.

[0017] The present invention provides a classification and recognition method, device and equipment for tumor images. The present invention enhances the difference between different types of tumor images by combining images of the central area of ​​the tumor and the peritumoral area with the tumor images of the peritumoral area. The present invention comprehensively analyzes multimodal images such as diffusion weighted imaging (DWI), liver enhancement scanning (HBP) and T2 weighted imaging (T2WI), extracts imaging genomics features such as first-order statistical features, shape features and texture features, as well as deep features such as basic shape features, local shape features and global features, and performs fusion screening and dimensionality reduction, which not only retains the inherent features of each type of tumor image, ensures the difference between different types of tumors, but also reduces the complexity of feature vectors and reduces the interference of useless information. The classification and recognition model obtained by training can more accurately recognize different types of tumor images, solves the technical problems of high similarity between different types of tumor images and limited features of single modality images, and improves the accuracy of tumor image classification and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 It is a structural schematic diagram of a tumor image classification and recognition device provided by an embodiment of the present invention;

[0020] Figure 2 It is a flow chart of a tumor image classification and recognition method provided by an embodiment of the present invention;

[0021] Figure 3 is a flow chart of another tumor image classification and recognition method provided by an embodiment of the present invention;

[0022] Figure 4 It is a structural schematic diagram of a tumor image classification and recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0024] In the description of the present invention, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" and "plurality" refer to two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different.

[0025] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0026] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include other steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following will be described through specific embodiments in conjunction with the accompanying drawings of the present invention.

[0028] As described in the background technology, medical image recognition currently has the technical problem of low accuracy.

[0029] To solve this technical problem, Figure 1 As shown, an embodiment of the present invention provides a tumor image classification and recognition device. The classification and recognition device includes a communication module, a feature extraction module, a feature screening module and a model training module.

[0030] based on Figure 1 The classification identification device shown, such as Figure 2 and Figure 3 As shown, an embodiment of the present invention provides a method for classifying and identifying tumor images, which includes steps S101-S105.

[0031] S101. The communication module obtains multimodal images of the central region and the peritumoral region of each type of tumor, and the tumor type corresponding to each multimodal image.

[0032] Exemplarily, the embodiment of the present invention can collect multimodal MRI images of patients with intrahepatic cholangiocarcinoma (ICC) and metastases from multiple hospitals, and the multimodal images include diffusion weighted imaging (DWI), liver enhancement scan (HBP) and T2 weighted imaging (T2WI). To ensure the consistency of image quality, the embodiment of the present invention can use standardized tools to calibrate the scanning parameters. Subsequently, the multimodal MRI images are filtered using the Scipy library and PyRadiomics software. The tumor area in the image is annotated in detail, and the tumor is expanded in combination with the scipy.ndimage library to obtain peritumoral areas with an outward expansion of 3mm, 5mm, 10mm, 15mm, and 20mm, respectively. Finally, a series of mathematical operations are performed on the multimodal MRI images using the Numpy library to comprehensively extract the imaging genomics features of the tumor.

[0033] For example, specific operations include square operation (enhancing high signal areas), square root operation (reducing contrast and highlighting details), logarithmic operation (enhancing contrast in low signal areas), exponential operation (enhancing contrast in high signal areas), Gaussian Laplace operator (edge ​​detection) and wavelet transform (multi-scale analysis).

[0034] In order to further optimize the processing process, the embodiment of the present invention can pre-process the multimodal image, resample the pixels of the tumor and peritumoral area images to a fixed size, and normalize the image grayscale. We set the following parameters, see steps S201-S202 for details.

[0035] Label: Set to 1 to specify the tumor region (ROI).

[0036] Interpolator: Smoothly resamples the image using cubic interpolation.

[0037] Resampling pixel spacing: Set to [1.0,1.0,1.0] to ensure that the pixel spacing in each dimension is consistent.

[0038] Normalization: Eliminate brightness and contrast differences and improve feature extraction consistency.

[0039] Grayscale width: Set to 5 to refine the histogram analysis.

[0040] S102: The feature extraction module extracts and fuses features of the multimodal images to obtain fusion features of each multimodal image.

[0041] In the embodiment of the present application, the fusion features include imaging genomics features and deep features. The imaging genomics features include first-order statistical features, shape features and texture features; the deep features include basic shape features, local shape features and global features.

[0042] As a possible implementation manner, step S102 may be specifically implemented as steps S1021 - S1023 .

[0043] S1021. Perform feature extraction based on each multimodal image to obtain radiomics features.

[0044] In some embodiments, the first-order statistical features include the mean, median, standard deviation, skewness and kurtosis of image pixels; the shape features include the volume, surface area, compactness and sphericity of the tumor in the image; the texture features include the grayscale co-occurrence matrix, grayscale length matrix, grayscale structure matrix, domain grayscale difference matrix and grayscale dependence matrix.

[0045] It should be noted that the embodiment of the present invention can extract three types of radiomics features from the multimodal MRI images of the tumor and peritumoral area respectively through the PyRadiomics software package: first-order statistical features (306), shape features (51) and texture features (1275).

[0046] Among them, the first-order statistical features: describe the pixel intensity distribution, such as mean, median, standard deviation, skewness, kurtosis, etc.

[0047] Shape features: quantify the geometry of the tumor ROI, such as volume, surface area, compactness, sphericity, etc.

[0048] Texture features: Grayscale co-occurrence matrix: 408 features, describing the spatial relationship of grayscale distribution of the image; Grayscale length matrix: 272 features, describing the spatial distribution of grayscale length; Grayscale structure matrix: 272 features, describing the structure of grayscale area; Neighborhood grayscale difference matrix: 85 features, describing the grayscale difference within the neighborhood; Grayscale dependency matrix: 238 features, describing the grayscale dependency relationship.

[0049] S1022. Use a residual network to extract deep features from each multimodal image.

[0050] In some embodiments, basic shape features are used to describe the outline and internal structure of a tumor, local shape features are used to describe detailed information of a local area of ​​the tumor, and global features are used to describe the overall morphology and spread of the tumor.

[0051] It should be noted that the embodiment of the present invention can extract three deep features from the multimodal MRI images of the tumor area and the peritumoral area respectively by using the residual network (Residual Network, ResNet): basic shape features, local shape features and global features. First, the ResNet34 network is pre-trained using the public data set ImageNet to improve its generalization ability and feature extraction efficiency. The pre-trained ResNet34 network automatically extracts deep features from multimodal MRI images. The ResNet34 network consists of a 34-layer deep structure. The initial convolution layer uses a 7*7 convolution kernel and a stride of 2, followed by a maximum pooling layer. The core is 16 residual blocks containing two 3*3 convolution layers. Finally, a global average pooling layer is used to integrate the features, and the fully connected layer outputs the classification results. The feature output of the second to last fully connected layer of the ResNet34 network is used as the deep learning feature. The specific features include:

[0052] Basic shape features: Based on the angles of basic shapes such as edges and textures, detailed descriptions of tumor contours and internal structures.

[0053] Local shape features: Quantify the shape, size and relationship of the tumor with surrounding tissues, and provide detailed information of the local area.

[0054] Global features: Based on the smoothness of the tumor edge and the degree of infiltration, the overall morphology and spread of the tumor are described from a global perspective.

[0055] S1023. Based on the imaging omics features and depth features of each multimodal image, feature splicing and fusion are performed to obtain fusion features of each multimodal image.

[0056] For example, in the embodiment of the present invention, the concatenate function of NumPy can be used to stitch together the above-mentioned imaging features inside the tumor, the depth features inside the tumor, the imaging features of the peritumoral area, and the depth features of the peritumoral area to obtain the stitched fusion feature X = {X1, X2, ...X i ,…,X n}, where n represents the number of eigenvectors, X i ={x i1 ,x i2 ,…x ij ,…,x im} T represents the i-th feature, x ij represents the value of the ith feature of the jth sample, and m represents the number of samples. The tumor classification result is used as a label based on the pathological information, denoted as Y = {Y1, Y2, ..., Y m}.

[0057] S103, the feature screening module performs feature screening and dimensionality reduction based on the fusion features of each multimodal image to obtain the key features of each multimodal image.

[0058] As a possible implementation manner, step S103 may be specifically implemented as steps S1031 - S1035 .

[0059] S1031. Based on the fusion features of each multimodal image and the tumor type corresponding to each multimodal image, a Levene variance consistency test is performed to obtain a test result.

[0060] In some embodiments, the test result includes a Levene statistic between a plurality of fusion features corresponding to each tumor type, and the Levene statistic is used to characterize the degree of difference between the fusion features corresponding to each tumor type.

[0061] S1032. Based on the test results, multiple fusion features corresponding to each tumor type are screened to obtain multiple screened fusion features corresponding to each tumor type.

[0062] S1033. For any fusion feature among the multiple fusion features corresponding to each tumor type after screening, use a random forest classifier to evaluate the importance of each feature in the fusion feature;

[0063] S1034. Based on the importance of each feature in the fused feature, the fused feature is removed, and the retained features are standardized to obtain a standard feature corresponding to the fused feature.

[0064] S1035. Apply norm regularization to multiple standard features of each tumor type to obtain key features of each multimodal image.

[0065] It should be noted that in order to obtain the best fusion features to meet the classification task, we take the following feature selection and dimensionality reduction steps:

[0066] First, the Levene test was used to evaluate the variance consistency of ICC features and metastasis features, and the features with significant statistical differences between the groups (P < 0.05) were screened out. Independent sample T tests were performed on these features to evaluate whether their mean differences were statistically significant.

[0067] Secondly, recursive feature elimination (RFE) is performed on the features selected by the independent sample T test, and the importance of each feature is evaluated using a random forest classifier. In each iteration, the least important feature is removed, and this process is repeated until the q most discriminative features are retained. The StandardScaler method is used, that is, the q features selected by RFE are standardized using the following formula to eliminate the differences in dimensions and numerical ranges.

[0068]

[0069] Among them, X i ′ is the i-th feature after standardization, X i is the i-th feature before normalization, is the mean of q features, std(X i ) is the variance of q features.

[0070] After standardization, we get the standard feature X′={X1′,X2′,…,X q ′}.

[0071] Finally, L0 norm regularization is applied to perform sparse selection on the above q features. Specifically, L0 regularization minimizes the following objective function:

[0072]

[0073] Among them, X′ is the standard feature, Y is the tumor category label, is the prediction error of the model, w0 represents the weight parameter w={w1,w2,…,w q} T The zero norm of represents the number of non-zero elements in the vector w, and λ is the regularization parameter. L0 norm regularization is an NP-hard problem. The present invention adopts a regularized smoothed L0 norm algorithm to solve the above objective function, and the iterative formula is as follows.

[0074] w k+1 =w k -X′ H (X′X′ H +λ -1 I q ) -1 (X′w k -Y);

[0075] And the regularization parameter λ of the kth step can be given by

[0076]

[0077] in, represents the first trial solution of the inner loop. k is the weight parameter of the kth iteration, w k+1 is the weight parameter for the k+1th iteration, w k-1 is the weight parameter for the k-1th iteration, X′ H is the conjugate professional matrix of the standard feature X′, λ -1 is the inverse of the regularization parameter, I q represents the identity matrix of order q, and Y is the tumor category label.

[0078] When the algorithm meets the iteration stop condition, the features with weight parameters of 0 are eliminated, and all features with non-0 weight parameters are retained, that is, the features selected by the regularized smoothing L0 norm algorithm.

[0079] S104: The model training module generates multiple training samples based on the key features of each multimodal image and the tumor type corresponding to each multimodal image.

[0080] S105: The model training module trains a classification recognition model based on multiple training samples.

[0081] As a possible implementation manner, step S105 can be specifically implemented as steps 1 to 4.

[0082] Step 1: Build a logistic regression model based on the linear regression model and the sigmoid logistic function;

[0083] Step 2: training the logistic regression model based on the training set in the multiple training samples to obtain a trained logistic regression model;

[0084] Step 3: Testing the trained logistic regression model based on the test set in the multiple training samples to obtain the tested logistic regression model and the loss function value;

[0085] Step 4: If the loss function value meets the set conditions, the iteration process is terminated and the tested logistic regression model is determined as the classification recognition model; if the loss function value does not meet the set conditions, repeat steps 2 to 4 until the loss function value meets the set conditions.

[0086] It should be noted that the embodiment of the present invention can input the final screened features into a logical function (such as a sigmoid function) to obtain the probability P that the sample belongs to ICC. When P ≥ 0.5, the sample is classified as ICC; when P < 0.5, the sample is classified as a metastatic tumor. The goal of the logistic regression model is to find the optimal model parameter ω so that the difference between the model's prediction result and the actual label is minimized. The maximum likelihood estimation or cross entropy loss function is usually used as the optimization target. The loss function used in this experiment is as follows.

[0087]

[0088] Among them, p j Y represents the probability that the jth sample belongs to ICC. j is the tumor category label of the jth sample. m is the number of samples. L(ω) is the loss function value; ω is the model parameter.

[0089] The classification model was evaluated using a five-fold cross-validation method with stratified sampling to ensure that the training set and test set were divided into 80% and 20% ratios, to ensure consistent category distribution, avoid category imbalance problems, and improve the fairness and consistency of model evaluation. The logistic regression model was fine-tuned using the GridSearchCV algorithm to construct a parameter grid with different regularization strengths (parameter C ranges from 0.01 to 10, with an interval of one order of magnitude). The impact of different configurations on model performance was comprehensively evaluated by systematically traversing parameter combinations. GridSearchCV internally uses three-fold cross-validation to ensure the fairness and accuracy of parameter evaluation. Finally, the best performing parameter configuration was selected to optimize the model classification effect and improve prediction accuracy and generalization ability.

[0090] The present invention uses accuracy (Accuracy), F1 score (F1 Score) and area under the ROC curve (AUC) as the main indicators for evaluating the classification model of ICC and metastasis.

[0091] The present invention provides a classification and recognition method for tumor images, which enhances the difference between different types of tumor images by combining images of the central area of ​​the tumor and the peritumoral area with the help of the tumor images of the peritumoral area. The present invention comprehensively analyzes multimodal images such as diffusion weighted imaging (DWI), liver enhancement scanning (HBP) and T2 weighted imaging (T2WI), extracts imaging genomics features such as first-order statistical features, shape features and texture features, as well as deep features such as basic shape features, local shape features and global features, and performs fusion screening and dimensionality reduction, which not only retains the inherent features of each type of tumor image, ensures the difference between different types of tumors, but also reduces the complexity of feature vectors and reduces the interference of useless information. The classification and recognition model obtained by training can more accurately identify different types of tumor images, solves the technical problems of high similarity between different types of tumor images and limited features of single modality images, and improves the accuracy of tumor image classification and recognition.

[0092] Optionally, the classification identification device also includes a classification evaluation module.

[0093] Correspondingly, the tumor image classification and recognition method provided by the embodiment of the present invention further includes step S106.

[0094] S106: The classification evaluation module classifies and identifies the multimodal image to be identified based on the classification evaluation model to obtain a classification identification result.

[0095] In some embodiments, the classification identification result includes a tumor type.

[0096] In this way, the present invention can classify and identify the multimodal images to be identified that are acquired in real time through the classification evaluation model obtained through training, obtain classification and identification results, and improve the recognition accuracy of tumor images.

[0097] Optionally, the classification and identification device also includes a preprocessing module.

[0098] Accordingly, the tumor image classification and recognition method provided by the embodiment of the present invention further includes steps S201 - S202 before step S102 .

[0099] S201 . A preprocessing module resamples each multimodal image to obtain a resampled multimodal image.

[0100] In some embodiments, the resampled multimodal images are images with the same image size and consistent pixel spacing in each dimension.

[0101] S202 , performing grayscale normalization processing on the resampled multimodal image to obtain a preprocessed multimodal image.

[0102] In this way, the embodiment of the present invention can pre-process the multimodal image, optimize the processing process, facilitate multimodal image recognition, and improve the accuracy of tumor image recognition.

[0103] Optionally, the tumor image classification and recognition method provided by the embodiment of the present invention further includes steps S301-S304 after step S105.

[0104] S301, a classification and evaluation module splits the multimodal image to obtain a single-modal image corresponding to each multimodal image.

[0105] In some embodiments, the single modality image includes a DWI image, a HBP image, and a T2WI image.

[0106] S302: The classification evaluation module generates a plurality of test samples based on the multimodal image and the unimodal image corresponding to each multimodal image.

[0107] In some embodiments, the test samples include multimodality samples, DWI samples, HBP samples, and T2WI samples.

[0108] S303: The classification evaluation module determines evaluation indicators of various types of test samples based on multiple test samples and a classification recognition model.

[0109] In some embodiments, the evaluation metrics include accuracy, F1 score, and area under the ROC curve.

[0110] S304: The classification evaluation module determines the image modality with the best classification effect based on the evaluation indicators of various test samples.

[0111] Exemplarily, as shown in Table 1, an embodiment of the present invention provides an experimental data table of ICC and metastasis classification model based on single-modality tumor features. Based on the ICC values ​​and metastasis classification results of three single-modality tumor features of DWI, HBP, and T2WI, it is found that the DWI modality exhibits the best classification performance among all imaging modalities.

[0112] Table 1

[0113] Evaluation indicators Accuracy F1 Score AUC DWI 78.30％ 75.79％ 85.91％ HBP 69.81％ 65.22％ 80.50％ T2WI 76.89％ 72.63％ 81.60％

[0114] Optionally, the tumor image classification and recognition method provided by the embodiment of the present invention further includes steps S401-S403 after step S105.

[0115] S401 . The classification evaluation module classifies the peritumoral regions in various types of modality images based on multiple test samples to obtain test samples of various peritumoral regions corresponding to various types of modality images.

[0116] S402: For any type of modality image, the classification evaluation module performs a test based on the classification recognition model and test samples of the modality image in various peritumoral ranges to obtain evaluation indicators of the modality image in various peritumoral ranges.

[0117] S403, the classification evaluation module determines the best peritumoral range for classification in each type of modality image based on the evaluation indicators of various peritumoral ranges in each type of modality image.

[0118] Tables 2, 3, and 4 show the ICC values ​​and metastasis classification model test results based on the tumor peritumoral features of DWI, HBP, and T2WI images with an expansion of 3mm, 5mm, 10mm, 15mm, and 20mm, respectively. The study found that among all imaging modalities, the tumor peritumoral range of 5mm showed the best classification performance.

[0119] Table 2

[0120] Evaluation indicators Accuracy F1 Score AUC DWI_3mm 74.06％ 70.27％ 78.57％ DWI_5mm 77.36％ 75.00％ 87.35％ DWI_10mm 74.06％ 70.27％ 79.22％ DWI_15mm 78.30％ 75.53％ 85.05％ DWI_20mm 45.75％ 62.30％ 63.70％

[0121] The Delong test was used to compare the model performance based on the combination of peritumoral features of DWI, HBP and T2WI images with an outward expansion of 3mm, 5mm, 10mm, 15mm and 20mm. The results showed that there was a statistically significant difference between the 5mm peritumoral range and other peritumoral ranges.

[0122] Table 3

[0123] Evaluation indicators Accuracy F1 Score AUC HBP_3mm 71.23％ 65.92％ 77.27％ HBP_5mm 75.94％ 73.02％ 83.04％ HBP_10mm 70.75％ 67.71％ 79.61％ HBP_15mm 72.17％ 67.76％ 75.76％ HBP_20mm 75.00％ 70.72％ 79.16％

[0124] The experimental results of the ICC and metastasis classification model based on the fusion features of tumor and optimal peritumoral features are shown in Table 5. By comparing the experimental results of Tables 1 to 5, it can be seen that the use of the fusion features of tumor and peritumoral features is better than the use of tumor features or peritumoral features alone to distinguish ICC and metastasis, and DWI images plus 5mm peritumoral features perform best among all tumor and peritumoral fusion features, especially in the AUC index (88.47%). HBP and T2WI modalities also show good classification performance, but slightly lower than DWI modality.

[0125] Table 4

[0126] Evaluation indicators Accuracy F1 Score AUC T2WI_3mm 71.70％ 67.03％ 79.95％ T2WI_5mm 71.70％ 67.03％ 81.31％ T2WI_10mm 75.94％ 71.51％ 80.66％ T2WI_15mm 46.23％ 62.75％ 69.02％ T2WI_20mm 64.62％ 55.62％ 59.09％

[0127] The classification performance of tumor and 5mm peritumor features using the three modalities of DWI, HBP and T2WI was significantly improved, and the accuracy, F1 score and AUC all reached the highest values.

[0128] Table 5

[0129] Evaluation indicators Accuracy F1 Score AUC DWI_5mm 78.30％ 75.27％ 88.47％ HBP_5mm 76.89％ 74.07％ 83.80％ T2WI_5mm 77.36％ 72.73％ 81.70％ DWI_HBP_T2WI_5mm 84.43％ 82.72％ 90.80％

[0130] Through the above embodiments, it can be clearly seen that the present invention provides an innovative ICC and metastasis classification method. This method not only integrates the imaging genomics features and the advanced features extracted by deep learning, but also comprehensively considers the characteristics of the two areas of tumor and peritumor. This joint analysis not only improves the ability to recognize the complex internal structure of the tumor, but also enhances the understanding of the relevant characteristics of the peripheral tissue, thereby significantly improving the accuracy and robustness of the classification. The experimental results show that this method is superior to the traditional single feature analysis method in predictive performance, and provides a powerful tool for the accurate diagnosis of ICC and metastases.

[0131] Optionally, the tumor image classification and recognition method provided by the embodiment of the present invention further includes steps S501-S505 after step S105.

[0132] S501 . The feature extraction module crops the multimodal image of the peritumoral region based on the peritumoral range with the best classification effect to obtain a cropped multimodal image of the peritumoral region.

[0133] S502: The feature extraction module extracts and fuses features based on the cropped multimodal image of the peritumoral area and the multimodal image of the central area to obtain updated fused features.

[0134] S503 , the feature screening module performs feature screening and dimension reduction based on the updated fusion features of each multimodal image to obtain updated key features of each multimodal image.

[0135] S504: The model training module generates a plurality of updated training samples based on the updated key features of each multimodal image and the tumor type corresponding to each multimodal image.

[0136] S505: The model training module retrains the classification recognition model based on multiple updated training samples to obtain an updated classification recognition model.

[0137] In this way, the embodiment of the present invention can use the multimodal image with the best classification effect and the best peritumoral area to update the classification and recognition model, improve the recognition accuracy of the classification and recognition model, and further improve the accuracy of tumor image classification and recognition.

[0138] Figure 4 Schematic diagram of a classification and identification device provided by an embodiment of the present invention. Figure 4 As shown, the classification identification device 600 includes: a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, the steps in the above-mentioned method embodiments are implemented, for example Figure 2 Alternatively, when the processor 601 executes the computer program 603, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 1 The functions of the communication module 11, feature extraction module 12, feature screening module 13 and model training module 14 are shown.

[0139] Exemplarily, the computer program 603 may be divided into one or more modules / units, which are stored in the memory 602 and executed by the processor 601 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 603 in the classification recognition device 600. For example, the computer program 603 may be divided into Figure 3 The shown modules include a communication module 11, a feature extraction module 12, a feature screening module 13 and a model training module 14.

[0140] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0141] The memory 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the classification and identification device 600. The memory 602 may also be an external storage device of the classification and identification device 600, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the classification and identification device 600. Further, the memory 602 may also include both an internal storage unit of the classification and identification device 600 and an external storage device. The memory 602 is used to store the computer program and other programs and data required by the terminal. The memory 602 may also be used to temporarily store data that has been output or is to be output.

[0142] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A tumor image classification and recognition method, characterized in that: include: Obtain multimodal images of the central area and peritumoral area of ​​each type of tumor, as well as the tumor type corresponding to each multimodal image; Extracting and fusing features of the multimodal images to obtain fusion features of each multimodal image, wherein the fusion features include imaging omics features and depth features, and the imaging omics features include first-order statistical features, shape features, and texture features; Deep features include basic shape features, local shape features and global features; Based on the fusion features of the multimodal images, feature screening and dimensionality reduction are performed to obtain key features of the multimodal images; Based on the key features of each multimodal image and the tumor type corresponding to each multimodal image, a plurality of training samples are generated; based on the plurality of training samples, a classification recognition model is trained.

2. A tumor image classification and recognition device, characterized in that: include: The communication module obtains multimodal images of the central area and the peritumoral area of ​​each type of tumor, as well as the tumor type corresponding to each multimodal image; The feature extraction module extracts and fuses features of the multimodal images to obtain fusion features of each multimodal image, wherein the fusion features include imaging omics features and depth features, wherein the imaging omics features include first-order statistical features, shape features, and texture features; and the depth features include basic shape features, local shape features, and global features; The feature screening module performs feature screening and dimension reduction based on the fusion features of each multimodal image to obtain the key features of each multimodal image; The model training module generates a plurality of training samples based on the key features of each multimodal image and the tumor type corresponding to each multimodal image; and trains a classification recognition model based on the plurality of training samples.

3. The tumor image classification and recognition device according to claim 2, characterized in that: The classification and identification device also includes a preprocessing module; The preprocessing module resamples the multimodal images to obtain resampled multimodal images, wherein the resampled multimodal images are images with the same image size and consistent pixel spacing in each dimension; The resampled multimodal image is subjected to image grayscale normalization processing to obtain a preprocessed multimodal image.

4. The tumor image classification and recognition device according to claim 2, characterized in that: The feature extraction module is specifically used to perform feature extraction based on each multimodal image to obtain imaging features; the first-order statistical features include the mean, median, standard deviation, skewness and kurtosis of image pixels; the shape features include the volume, surface area, compactness and sphericity of the tumor in the image; the texture features include grayscale co-occurrence matrix, grayscale length matrix, grayscale structure matrix, domain grayscale difference matrix and grayscale dependency matrix; a residual network is used to extract deep features from each multimodal image, the basic shape features are used to describe the contour and internal structure of the tumor, the local shape features are used to describe detailed information of the local area of ​​the tumor, and the global features are used to describe the overall morphology and diffusion of the tumor; based on the imaging features and deep features of each multimodal image, feature splicing and fusion are performed to obtain the fusion features of each multimodal image.

5. The tumor image classification and recognition device according to claim 2, characterized in that: The feature screening module is specifically used to perform a Levene variance consistency test based on the fusion features of each multimodal image and the tumor type corresponding to each multimodal image to obtain a test result; the test result includes a Levene statistic between multiple fusion features corresponding to each tumor type, and the Levene statistic is used to characterize the degree of difference between the fusion features corresponding to each tumor type; based on the test result, the multiple fusion features corresponding to each tumor type are screened to obtain multiple fusion features corresponding to each tumor type after screening; For any fusion feature among the multiple fusion features corresponding to each tumor type after screening, a random forest classifier is used to evaluate the importance of each feature in the fusion feature; based on the importance of each feature in the fusion feature, the fusion feature is removed, and the retained features are standardized to obtain the standard feature corresponding to the fusion feature; Norm regularization is applied to multiple standard features of each tumor type to obtain key features of each multimodal image.

6. The tumor image classification and recognition device according to claim 2, characterized in that: The model training module is specifically used to perform the following steps: Step 1: Build a logistic regression model based on the linear regression model and the sigmoid logistic function; Step 2: training the logistic regression model based on the training set in the multiple training samples to obtain a trained logistic regression model; Step 3, testing the trained logistic regression model based on the test set in the multiple training samples to obtain the tested logistic regression model and the loss function value; Step 4: If the loss function value meets the set conditions, the iterative process is terminated and the tested logistic regression model is determined as the classification recognition model; if the loss function value does not meet the set conditions, steps 2 to 4 are repeated until the loss function value meets the set conditions.

7. The tumor image classification and recognition device according to claim 2, characterized in that: The classification evaluation module is also used to split the multimodal image to obtain a single-modal image corresponding to each multimodal image, wherein the single-modal image includes a DWI image, an HBP image, and a T2WI image; based on the multimodal image and the single-modal image corresponding to each multimodal image, a plurality of test samples are generated, wherein the test samples include a multimodal sample, a DWI sample, an HBP sample, and a T2WI sample; based on the plurality of test samples and the classification recognition model, evaluation indicators of each type of test sample are determined; the evaluation indicators include accuracy, F1 score, and area under the ROC curve; based on the evaluation indicators of each type of test sample, the imaging modality with the best classification effect is determined.

8. The tumor image classification and recognition device according to claim 7, characterized in that: The classification and evaluation module is further used to classify the multiple test samples according to the range of the peritumoral area in each type of modality image, so as to obtain test samples of multiple peritumoral ranges corresponding to each type of modality image; For any type of modality image, based on the classification and recognition model and test samples of various peritumoral ranges of the modality image, a test is performed to obtain evaluation indicators of the modality image in various peritumoral ranges; Based on the evaluation indicators of various peritumoral ranges in various modal images, the peritumoral range with the best classification effect in various modal images is determined.

9. The tumor image classification and recognition device according to any one of claims 2 to 8, characterized in that: The feature extraction module crops the multimodal image of the peritumoral region based on the peritumoral range with the best classification effect to obtain a cropped multimodal image of the peritumoral region; performs feature extraction and fusion based on the cropped multimodal image of the peritumoral region and the multimodal image of the central region to obtain an updated fusion feature; The feature screening module performs feature screening and dimension reduction based on the updated fusion features of each multimodal image to obtain updated key features of each multimodal image; The model training module generates multiple updated training samples based on the updated key features of each multimodal image and the tumor type corresponding to each multimodal image; based on the multiple updated training samples, the classification and recognition model is retrained to obtain an updated classification and recognition model.

10. A tumor image classification and recognition device, characterized in that: The classification identification device includes a memory and a processor, the memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to perform the following method: Acquire multimodal images of the central area and peritumoral area of ​​each type of tumor, and the tumor type corresponding to each multimodal image; perform feature extraction and fusion on the multimodal images to obtain fusion features of each multimodal image, wherein the fusion features include imaging omics features and depth features, and the imaging omics features include first-order statistical features, shape features, and texture features; Deep features include basic shape features, local shape features and global features; Based on the fusion features of the multimodal images, feature screening and dimensionality reduction are performed to obtain key features of the multimodal images; Based on the key features of each multimodal image and the tumor type corresponding to each multimodal image, a plurality of training samples are generated; based on the plurality of training samples, a classification recognition model is trained.

Images