Breast tumor image processing method, device, medium and equipment

Abstract

The present invention discloses a method, apparatus, medium, and device for processing breast tumor images, relating to the field of image processing technology. The method comprises: constructing a network model comprising a feature extraction module, a splicing layer, and a classification layer connected in sequence, wherein the feature extraction module comprises a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map the breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; and the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label. The resulting end-to-end breast tumor image classification model avoids the gradual amplification of errors caused by segmentation in each step.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, medium and equipment for processing breast tumor images. Background Art

[0002] Ultrasound imaging can noninvasively reveal the characteristics of breast tumors and, in turn, reveal their connection to breast tumor pathology. Based on ultrasound imaging technology, some studies have proposed machine learning and deep learning methods based on radiological features to accurately predict breast tumor pathology. Currently, some studies have attempted to fuse shallow and deep features and construct machine learning classifiers to predict breast tumor pathology.

[0003] In the prior art, Chinese patent CN113870194B discloses a breast tumor ultrasound image processing device that fuses deep features and shallow LBP features. This invention extracts deep features of images based on a deep learning network, which can describe the high-level complex features of tumor images. It extracts features based on rotationally invariant local binary patterns, which can describe the low-level simple features of tumor images. The high-level complex features and low-level simple features are fused and input into an SVM classifier for breast tumor ultrasound image classification.

[0004] However, the image classification results of the above method rely on the deep features, shallow features, the method of fusing deep and shallow features, and the adaptability of the fused features to the SVM classifier output by the deep learning network. Once this step-by-step processing method is faced with unknown breast tumor ultrasound images that are different from the distribution of the training data, the deep feature deviations generated will be gradually amplified through each step, resulting in a sharp drop in classification accuracy. Summary of the Invention

[0005] Based on this, in order to solve the technical problems in the prior art, the present invention provides a breast tumor image processing method, device, medium and equipment.

[0006] The present invention provides a breast tumor image processing method, comprising:

[0007] Constructing a network model comprising a feature extraction module, a splicing layer, and a classification layer connected in sequence, wherein the feature extraction module comprises a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label;

[0008] Breast tumor images and corresponding true classification labels are collected to construct a data set, the breast tumor images in the data set are input into the network model, minimizing the error between the predicted classification label output by the network model and the true classification label is used as an optimization goal, first and second trainable weight coefficients in the network model are optimized iteratively to obtain an optimal first and second trainable weight coefficients that optimize the optimization goal, and a breast tumor image classification model is obtained based on the optimal first and second trainable weight coefficients;

[0009] Breast tumor images are processed using a breast tumor image classification model.

[0010] Furthermore, the deep feature extraction branch is formed by a DenseNet169 network and three fully connected layers connected to the output end of the DenseNet169 network; the DenseNet169 network maps the breast tumor image into an initial deep feature vector with a dimension of 1664 through a combination of its Dense Blocks and TransitionLayers; the three fully connected layers weight and reduce the dimension of the initial deep feature vector with a dimension of 1664, and output a deep feature vector with a dimension of 64.

[0011] Furthermore, before the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image, an image segmentation algorithm or a target labeling algorithm is used to extract a breast tumor region in the breast tumor image;

[0012] After extracting the breast tumor region in the breast tumor image, the shallow feature extraction branch uses a non-training feature extraction method to calculate the sphericity, regional entropy, perimeter and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4.

[0013] Furthermore, the output end of the shallow feature extraction branch is connected to four consecutive fully connected layers, and the four consecutive fully connected layers remap the shallow feature vector with a dimension of 4 to a new shallow feature vector with a dimension of 64 by introducing a third trainable weight coefficient.

[0014] Furthermore, the classification layer is a CNN network, and the CNN network maps the fusion features into image classification labels through convolutional layers, pooling layers and fully connected layers.

[0015] The present invention provides a breast tumor image processing device, comprising:

[0016] A model construction module is used to construct a network model including a feature extraction module, a splicing layer, and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label;

[0017] A model training module is configured to collect breast tumor images and corresponding true classification labels to construct a data set, input the breast tumor images in the data set into the network model, minimize the error between the predicted classification labels output by the network model and the true classification labels as an optimization goal, iteratively optimize the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain an optimal first trainable weight coefficient and an optimal second trainable weight coefficient that optimize the optimization goal, and obtain a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient;

[0018] The image processing module is used to process breast tumor images using a breast tumor image classification model.

[0019] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned breast tumor image processing method is implemented.

[0020] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned breast tumor image processing method when executing the program.

[0021] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0022] In the breast tumor image processing method provided by the present invention, the shallow feature branch uses a non-training feature extraction method to directly obtain low-level feature vectors from the breast tumor image without participating in model training. Even when encountering an unknown breast tumor ultrasound image with a distribution different from that of the training data, it will provide prior information of the unknown breast tumor ultrasound image to the classification layer. During the training process, based on the prior information provided by the low-level feature vector, the first trainable weight coefficient of the deep feature extraction branch in the network model and the second trainable weight coefficient of the classification layer are synchronously adjusted through a joint optimization method rather than being updated independently. This synchronous adjustment ensures that the weights of the two parts work together during the training process, thereby avoiding the degradation of the overall model performance due to overfitting or underfitting of a certain part. The obtained end-to-end breast tumor image classification model avoids the gradual amplification of errors caused by the segmentation of each step. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1 A schematic flow chart of a breast tumor image processing method provided by the present invention;

[0025] Figure 2 A schematic diagram of the network model structure provided by the present invention;

[0026] Figure 3 This is a network structure diagram of the DenseNet169 model provided by the present invention;

[0027] Figure 4 A schematic diagram of the shallow feature verification results provided by the present invention;

[0028] Figure 5 This is the ROC curve diagram of the CNN fusion model provided by the present invention.

[0029] Figure 6 This is a flowchart for constructing the breast tumor image classification model provided by the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0031] Ultrasound imaging can noninvasively reveal the characteristics of breast tumors and, in turn, reveal their connection to breast tumor pathology. Based on ultrasound imaging technology, several studies have proposed various effective methods for accurately predicting breast tumor pathology, primarily including machine learning and deep learning methods based on radiological features. Related research results have shown that these methods can accurately predict tumor pathology. However, both methods have their own shortcomings. Machine learning methods based on radiological features rely on manually extracted features and manual labeling of tumor regions, revealing superficial tumor information but failing to capture deeper information. Deep learning methods, on the other hand, can automatically learn deep features of breast tumors to reveal deeper information, but this method loses superficial information. The two types of features are complementary, and their fusion can improve breast tumor diagnostic performance. Currently, some studies have attempted to fuse superficial and deep features to construct machine learning classifiers to predict breast tumor pathology. Results have shown that feature fusion methods can effectively improve tumor prediction performance. Unlike these traditional feature fusion methods, whether incorporating shallow breast tumor feature information into CNN fusion models can improve the model's predictive performance remains unknown. Furthermore, the independence of deep and shallow feature extraction, fusion, and classification in existing techniques results in weak generalization. Deep learning models often require extensive training data to capture complex features, while shallow feature extraction (such as LBP) relies more on local information and is susceptible to interference when working with ultrasound images with high noise or complex backgrounds. Because features from different sources may not be effectively aligned or coordinated during the fusion process, the classifier's prediction accuracy for new, unseen samples decreases, resulting in insufficient generalization. Specifically, if the fused features do not fully account for the differences between deep and shallow features, or if more complex models (such as attention-based fusion) are not used to enhance feature complementarity, then even if the classifier achieves good results during training, its performance will be significantly compromised in practice, especially for the classification of unknown or complex samples.

[0032] Based on this, in order to improve the diagnostic performance of breast tumors, the present invention first extracts the radiological features of the breast tumor area based on breast ultrasound data, and uses a feature selection algorithm to screen the optimal shallow features. Subsequently, 8 pre-trained CNN models are transferred for learning to predict the pathological information of breast tumors, from which the optimal CNN model is determined and used as the backbone network of the subsequent CNN fusion model. Based on the determined OSFs and the optimal CNN model, the present invention proposes a CNN fusion model that integrates OSFs, which includes two parallel channel modules, namely a shallow feature channel module and a deep feature channel module. Unlike previous feature fusion methods, this method integrates the shallow feature information of breast tumors into the CNN fusion model to achieve the fusion of deep features and shallow features, thereby improving the diagnostic performance of breast tumors.

[0033] To facilitate the understanding of the readers, the English abbreviations appearing in this invention are explained as follows:

[0034] Superficial features: The full English name is optimal superficial features, abbreviated as OSFs in this article.

[0035] Convolutional neural network: The full name in English is convolutional neural network, which is abbreviated as CNN in this article.

[0036] Logistic regression: The full English name is logistic regression, which is abbreviated as LR in this article.

[0037] Support vector machine: The full English name is support vector machine, which is abbreviated as SVM in this invention.

[0038] Random forest: The full English name is random forest, abbreviated as RF in this article.

[0039] Accuracy: The full English name is Accuracy, which is abbreviated as ACC in this article.

[0040] Receiver operating characteristic: The full English name is receiver operating characteristic, abbreviated as ROC in this invention.

[0041] Area under the ROC curve: The full English name is area under the ROC curve, which is abbreviated as AUC in the present invention.

[0042] Example 1

[0043] Figure 1 The flow chart of the breast tumor image processing method of this embodiment is shown. Figure 1 The breast tumor image processing method is described in detail, and specifically comprises the following steps:

[0044] S1: Construct a network model including a feature extraction module, a splicing layer and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map the breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label.

[0045] Figure 2 The network model structure diagram constructed in this embodiment is shown. Figure 2 Describe the network model in detail, including:

[0046] S101: Deep feature extraction branch.

[0047] The deep feature extraction branch consists of a DenseNet169 network and three fully connected layers connected to the output end of the DenseNet169 network; the DenseNet169 network maps breast tumor images into an initial deep feature vector with a dimension of 1664 through a combination of its Dense Blocks and Transition Layers; the three fully connected layers weight and reduce the dimension of the initial deep feature vector with a dimension of 1664, and output a deep feature vector with a dimension of 64.

[0048] Eight pre-trained CNN models were used to directly predict the benign and malignant nature of breast tumors, including ResNet18, VGG16, VGG19, DenseNet121, DenseNet169, EffcientNetB3, EffcientNetB4, and GoogleNet. Tumor images were first pre-processed by resizing them to 224x224 pixels and normalizing their pixel values, with each color channel standardized using a given mean and standard deviation. Each of the eight CNN models was pre-trained using the ImageNet dataset and then fine-tuned using ultrasound tumor images. To ensure fairness, all models used the following hyperparameters: learning rate = 0.0001, batch size = 16, maximum epochs = 30, and the Adam optimizer. Table 1 lists the prediction performance of the eight CNN models. The results show that among all CNN models, the DenseNet169 model has the best prediction performance (ACC=0.9231, AUC=0.9521). This proves that the DenseNet169 model has advantages as the backbone network of the subsequent fusion model. The network structure of this model is as follows Figure 3 shown.

[0049] Table 1 Prediction performance of traditional CNN model

[0050] Models ACC AUC Sensitivity Specificity DenseNet169 0.9231 0.9521 0.9048 0.9318 DenseNet121 0.9128 0.9473 0.8571 0.9394 VGG16 0.8923 0.9451 0.8413 0.9167 GoogleNet 0.8872 0.9407 0.8254 0.9167 VGG19 0.8821 0.9350 0.7302 0.9545 EffcientNetB4 0.8821 0.9295 0.7937 0.9242 ResNet18 0.8769 0.9427 0.7302 0.9470 EffcientNetB3 0.8769 0.9352 0.7937 0.9167

[0051] S102: Shallow feature extraction branch.

[0052] Before the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector from the breast tumor image, an image segmentation algorithm or an object labeling algorithm is used to extract the breast tumor region from the breast tumor image. After extracting the breast tumor region from the breast tumor image, the shallow feature extraction branch uses the non-training feature extraction method to calculate the sphericity, regional entropy, perimeter, and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4. Furthermore, four consecutive fully connected layers can be connected to the output of the shallow feature extraction branch to introduce a third trainable weight coefficient, thereby remapping the shallow feature vector with a dimension of 4 into a new shallow feature vector with a dimension of 64.

[0053] Based on the original ultrasound image and the tumor region, the signal intensity of the tumor region was extracted. The publicly available Python toolkit Pyradiomics was used to extract 102 radiomic features, including 9 morphological features, 18 statistical features, and 75 texture features. All features were normalized using the Z-score method to a mean of 0 and a standard deviation of 1. Based on the Gini coefficient, a five-fold cross-validation method was used to determine the importance score of each feature, and the OSFs were determined based on the ranking. Figure 4 The classification performance of the model with different numbers of features is shown, and the results show that the number of OSFs is 4. Finally, four important features are determined, namely sphericity, regional entropy, perimeter and elongation, which can be calculated using formulas (1)-(4) respectively.

[0054]

[0055] Where R is the radius of the tumor area and P is the perimeter.

[0056]

[0057] Where p(i,j) is the element in the i-th row and j-th column of the normalized region matrix, and ∈ is an arbitrarily small positive number.

[0058]

[0059] Among them, a i with b i is the vertex of the i-th line in the perimeter grid.

[0060]

[0061] Among them, λ max and λ sec are the lengths of the first and second principal component axes.

[0062] The four OSFs obtained above will serve as the input for the subsequent fusion model. Based on the four OSFs obtained, three classification models, LR, SVM, and RF, are constructed in sequence to predict the benign and malignant nature of breast tumors. The main parameters involved in the three classification models are adjusted to determine the optimal parameters to achieve the best predictive performance. The main parameters of LR include the inverse of the regularization coefficient C, the penalty term penalty, and the optimization algorithm solver, whose value ranges are {0.001, 0.01, 0.1, 1, 10, 100, 1000}, {L1, L2, None}, and {liblinear, lbfgs}, respectively. Based on five-fold cross-validation, the optimal parameters of LR are determined to be C = 1, penalty = L1, and solver = liblinear. The main parameters of SVM include the penalty coefficient C and the parameter gamma of the radial basis function, whose ranges are {0.001, 0.1, 10, 1000} and {0.001, 0.1, 10, 1000}, respectively. Based on five-fold cross-validation, the optimal parameters for SVM are C = 10.0 and gamma = 0.1. The main parameter for RF is the number of trees, n_estimators, which ranges from [10, 200] with a step size of 5. Based on five-fold cross-validation, the optimal parameter for RF is n_estimators = 15.

[0063] Once the model's optimal parameters are determined, it can be used to predict the benign and malignant nature of breast tumors. The model's predictive performance in this regard is evaluated using the following metrics: Accuracy (ACC), Area Under the Circumference (AUC), Sensitivity, and Specificity. Table 2 lists the predictive performance of three traditional classifiers for this purpose. The results show that all three classifiers achieve relatively good predictive performance. The SVM's predictive performance (ACC = 0.9692, AUC = 0.9919) significantly outperforms the other two classifiers.

[0064] Table 2 Diagnostic performance of machine learning classifiers

[0065] Methods ACC AUC Sensitivity Specificity Support Vector Machine 0.9692 0.9919 0.9048 1.0000 RF 0.9436 0.9701 0.9048 0.9621 LR 0.9026 0.9781 0.8571 0.9242

[0066] S103: Splicing layer.

[0067] Based on the four OSFs identified above and the optimal CNN model DenseNet169, this paper proposes a CNN fusion method integrating OSFs, including model A ( Figure 2 left) and model B ( Figure 2 to improve the model’s performance in tumor diagnosis. Figure 2 The network structures of the two models are shown. Both models have two parallel channel modules, including a deep feature channel module and a shallow feature channel module.

[0068] For model A, the Densenet169 model is first used to automatically extract 1664 features from the tumor image. 64 deep features are output through three fully connected layers, with the number of output nodes being 832, 416, and 64, respectively. The 64 deep features and the four shallow features are then concatenated according to the following formula (5) to obtain 68 features:

[0069] fused A =concat(x,y) (5)

[0070] Among them, x = {x1, x2, x3, x4} refers to the 4 optimal shallow features, and y = {y1, y2, ..., y64} refers to 64 deep features.

[0071] Finally, three fully connected layers are built, with the number of output nodes being 832, 416, and 2 respectively, and the softmax function is used to convert them into a probability matrix to obtain the prediction results.

[0072] Unlike Model A, Model B further learns four OSFs. This model builds four fully connected layers for the shallow feature pathway module, outputting 64 shallow features. The number of nodes in the output layer is 1664, 832, 416, and 64, respectively. The 64 deep features and 64 shallow features are then concatenated using the following formula (6) to obtain 128 features. The rest of the structure of Model B is similar to that of Model A and will not be repeated here.

[0073] fused B =concat(s 64 ,y) (6)

[0074] Among them, s 64 =Learn(x), refers to the new shallow features learned.

[0075] S104: Classification layer.

[0076] Table 3 and Figure 5 Both demonstrated the predictive performance of the proposed CNN fusion model in tumor diagnosis. The results showed that the predictive performance of the CNN fusion model was superior to that of traditional machine learning methods or deep learning methods. The above findings indicate that the proposed CNN fusion method is an effective method for breast ultrasound tumor diagnosis. In addition, compared with model A (ACC = 0.9795, AUC = 0.9924), model B (ACC = 0.9846, AUC = 0.9983) has better predictive performance in tumor diagnosis. One possible reason is that OSFs are further learned and converted into new features that are more relevant to the benign and malignant nature of the tumor, thereby achieving better diagnostic performance.

[0077] Table 3 Prediction performance of CNN fusion model integrating OSFs

[0078]

[0079]

[0080] S2: Collect breast tumor images and corresponding true classification labels to construct a data set, input the breast tumor images in the data set into the network model, and take minimizing the error between the predicted classification label and the true classification label output by the network model as the optimization goal, and optimize the first trainable weight coefficient and the second trainable weight coefficient in the network model iteratively to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtain a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient.

[0081] Public data on breast cancer patients, including ultrasound imaging data and pathological information, were collected, and a complete data set that met the requirements of this experiment was screened out. The present invention uses a public data set, which includes a total of 780 breast ultrasound images with an average image size of 500×500 pixels. These images can be divided into three categories: normal people, benign masses, and malignant masses. Normal people do not have mass areas. For this reason, we selected ultrasound images of benign masses and malignant masses as the final samples, with the numbers of the two being 437 and 210 respectively, for a total of 647 samples. All samples were randomly divided into a training set (N=452) and a test set (N=195) in a ratio of 7:3.

[0082] S3: Use the breast tumor image classification model to process breast tumor images.

[0083] The present invention proposes a CNN fusion method that integrates OSFs, and uses this method to predict the benign and malignant nature of breast ultrasound tumors, in order to explore the clinical value of this method in the diagnosis of breast ultrasound tumors. The steps include: collecting ultrasound imaging data and corresponding clinical data of breast cancer patients; determining the tumor area, and extracting the radiological features of the tumor area, screening it using a feature selection method to determine OSFs, and constructing a machine learning model to predict the benign and malignant nature of the tumor; using several commonly used CNNs to directly predict the benign and malignant nature of the tumor, in order to determine the optimal CNN model, and use it as the backbone network of the subsequent fusion method; based on the determined OSFs and the optimal CNN model, a CNN fusion model that integrates OSFs is proposed to predict the benign and malignant nature of breast tumors. The model includes two parallel channel modules, namely a shallow feature channel module and a deep feature channel module. The construction process of the breast tumor image classification model is as follows: Figure 6The present invention integrates shallow feature information of breast tumors into the CNN model, so that shallow features and deep features complement each other, thereby improving the prediction performance of breast tumors and assisting clinical diagnosis and treatment.

[0084] The above is a breast tumor image processing method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding breast tumor image processing device, including:

[0085] A model construction module is used to construct a network model including a feature extraction module, a splicing layer and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map the breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label.

[0086] The model training module is used to collect breast tumor images and corresponding true classification labels to construct a data set, input the breast tumor images in the data set into the network model, minimize the error between the predicted classification label output by the network model and the true classification label as the optimization goal, and iteratively optimize the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that optimize the optimization goal, and obtain a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient.

[0087] The image processing module is used to process breast tumor images using a breast tumor image classification model.

[0088] The specific definition of the breast tumor image processing device can be found in the definition of the breast tumor image processing method above and will not be repeated here. The various modules in the above-mentioned breast tumor image processing device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0089] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provided is a breast tumor image processing method.

[0090] The present invention also provides a computer device structure. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provided is a breast tumor image processing method.

[0091] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0092] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A breast tumor image processing method, characterized in that: include: Construct a network model including sequentially connecting feature extraction module, splicing layer and classification layer, the feature The extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch The branch uses the first trainable weight coefficient to map the breast tumor image into a deep feature vector; the shallow The layer feature extraction branch uses a non-training feature extraction method to extract low-level feature vectors of breast tumor images; The splicing layer is used to combine the deep feature vector output by the deep feature extraction branch with the shallow feature extraction branch The output low-level feature vectors are concatenated into fusion features; The classification layer uses the second trainable weight coefficient to Fuse feature maps into image classification labels; Before the shallow feature extraction branch uses the non-training feature extraction method to extract the low-level feature vector of the breast tumor image, an image segmentation algorithm or an object labeling algorithm is used to extract the breast tumor region in the breast tumor image; after extracting the breast tumor region in the breast tumor image, the shallow feature extraction branch uses the non-training feature extraction method to calculate the sphericity, regional entropy, perimeter and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4; The output end of the shallow feature extraction branch is connected to four consecutive fully connected layers, and the four consecutive fully connected layers remap the shallow feature vector with a dimension of 4 to a new shallow feature vector with a dimension of 64 by introducing a third trainable weight coefficient; Collect breast tumor images and the corresponding true classification labels to construct a dataset. The adenoma image is input into the network model, and the predicted classification label and the true classification label output by the network model are compared. Minimizing the class label error is the optimization goal, and the first trainable weight coefficient in the network model is The optimization iteration is performed with the second trainable weight coefficient to obtain the optimal first trainable weight coefficient that makes the optimization target optimal. The training weight coefficient and the best second trainable weight coefficient are based on the best first trainable weight coefficient and the best The second trainable weight coefficient obtains a breast tumor image classification model; Breast tumor images are processed using a breast tumor image classification model.

2. The breast tumor image processing method according to claim 1, wherein: The deep feature extraction branch is formed by a DenseNet169 network and three fully connected layers connected to the output end of the DenseNet169 network; the DenseNet169 network maps the breast tumor image into an initial deep feature vector with a dimension of 1664 through a combination of its Dense Blocks and Transition Layers; the three fully connected layers weight and reduce the dimension of the initial deep feature vector with a dimension of 1664, and output a deep feature vector with a dimension of 64.

3. The breast tumor image processing method according to claim 1, wherein: The classification layer is a CNN network, which maps the fused features into image classification labels through convolutional layers, pooling layers and fully connected layers.

4. A breast tumor image processing device, characterized in that: include: Model building module, used to build a model including sequentially connected feature extraction modules, splicing layers and classification layers The network model comprises a feature extraction module including a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; The classification layer uses a second trainable weight coefficient to map the fused features into image classification labels; before the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image, an image segmentation algorithm or a target labeling algorithm is used to extract a breast tumor region in the breast tumor image; after extracting the breast tumor region in the breast tumor image, the shallow feature extraction branch uses a non-training feature extraction method to calculate the sphericity, regional entropy, perimeter and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4; the output end of the shallow feature extraction branch is connected to four consecutive fully connected layers, and the four consecutive fully connected layers remap the shallow feature vector with a dimension of 4 to a new shallow feature vector with a dimension of 64 by introducing a third trainable weight coefficient; Model training module, used to collect breast tumor images and corresponding true classification labels to build data The breast tumor images in the dataset are input into the network model, and the predicted The optimization goal is to minimize the error between the measured classification label and the true classification label. The first trainable weight coefficient and the second trainable weight coefficient are optimized iteratively to obtain the optimization target The optimal first trainable weight coefficient and the optimal second trainable weight coefficient are based on the optimal first trainable weight coefficient. The breast tumor image classification model is obtained by training the weight coefficient and the optimal second trainable weight coefficient; The image processing module is used to process breast tumor images using a breast tumor image classification model.

5. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the breast tumor image processing method according to any one of claims 1 to 3 is implemented.

6. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for processing breast tumor images according to any one of claims 1 to 3 is implemented.

Images