Breast tumor image processing method and device, medium and equipment

Abstract

The invention discloses a breast tumor image processing method and device, a medium and equipment, and relates to the technical field of image processing. Comprising the steps that a network model comprising a feature extraction module, a splicing layer and a classification layer which are sequentially connected is constructed, and the feature extraction module comprises a deep-layer feature extraction branch and a shallow-layer feature extraction branch which are parallel; the deep feature extraction branch maps the breast tumor image into a deep feature vector by using a first trainable weight coefficient; the shallow feature extraction branch extracts low-level feature vectors of the breast tumor image by using a non-training feature extraction method; the splicing layer is used for splicing a depth feature vector output by the deep feature extraction branch and a low-level feature vector output by the shallow feature extraction branch into a fusion feature; the classification layer maps the fusion feature into an image classification tag using a second trainable weight coefficient. The obtained end-to-end breast tumor image classification model avoids gradual error amplification caused by segmentation in each step.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, medium and equipment for processing breast tumor images. Background Art

[0002] Ultrasound imaging can reveal the characteristics of breast tumors non-invasively, and further reveal the relationship between them and breast tumor pathology information. Based on ultrasound imaging technology, some studies have proposed machine learning and deep learning methods based on radiological features to achieve accurate prediction of breast tumor pathology information. At present, some studies have attempted to fuse shallow features with deep features and build machine learning classifiers to predict breast tumor pathology information.

[0003] In the prior art, Chinese patent CN113870194B discloses a breast tumor ultrasound image processing device that fuses deep features and shallow LBP features. The invention extracts deep features of images based on a deep learning network, which can describe high-level complex features of tumor images. It extracts features based on rotationally invariant local binary patterns, which can describe low-level simple features of tumor images. The high-level complex features and low-level simple features are fused and input into a SVM classifier for breast tumor ultrasound image classification.

[0004] However, the image classification results of the above method depend on the deep features, shallow features, the method of fusing deep and shallow features, and the adaptability of the fused features to the SVM classifier output by the deep learning network. Once this step-by-step processing method is faced with unknown breast tumor ultrasound images that are different from the distribution of the training data, the deep feature deviations generated will be gradually amplified through each step, resulting in a sharp drop in classification accuracy. Summary of the invention

[0005] Based on this, in order to solve the technical problems in the prior art, the present invention provides a breast tumor image processing method, device, medium and equipment.

[0006] The present invention provides a breast tumor image processing method, comprising:

[0007] Constructing a network model including sequentially connecting a feature extraction module, a splicing layer and a classification layer, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label;

[0008] Collecting breast tumor images and corresponding true classification labels to construct a data set, inputting the breast tumor images in the data set into the network model, taking minimizing the error between the predicted classification label and the true classification label output by the network model as the optimization goal, optimizing and iterating the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtaining a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient;

[0009] Breast tumor images are processed using a breast tumor image classification model.

[0010] Furthermore, the deep feature extraction branch is formed by a DenseNet169 network and three fully connected layers connected to the output end of the DenseNet169 network; the DenseNet169 network maps the breast tumor image into an initial deep feature vector with a dimension of 1664 through a combination of its Dense Blocks and TransitionLayers; the three fully connected layers weight and reduce the dimension of the initial deep feature vector with a dimension of 1664, and output a deep feature vector with a dimension of 64.

[0011] Further, before the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image, an image segmentation algorithm or a target labeling algorithm is used to extract a breast tumor region in the breast tumor image;

[0012] After extracting the breast tumor region in the breast tumor image, the shallow feature extraction branch uses a non-training feature extraction method to calculate the sphericity, regional entropy, perimeter and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4.

[0013] Furthermore, the output end of the shallow feature extraction branch is connected to four consecutive fully connected layers, and the four consecutive fully connected layers remap the shallow feature vector with a dimension of 4 to a new shallow feature vector with a dimension of 64 by introducing a third trainable weight coefficient.

[0014] Furthermore, the classification layer is a CNN network, and the CNN network maps the fused features into image classification labels through a convolutional layer, a pooling layer and a fully connected layer.

[0015] The present invention provides a breast tumor image processing device, comprising:

[0016] A model construction module is used to construct a network model including a feature extraction module, a splicing layer and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label;

[0017] A model training module, used for collecting breast tumor images and corresponding true classification labels to construct a data set, inputting the breast tumor images in the data set into the network model, taking minimizing the error between the predicted classification label and the true classification label output by the network model as the optimization goal, optimizing and iterating the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtaining a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient;

[0018] The image processing module is used to process breast tumor images using a breast tumor image classification model.

[0019] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned breast tumor image processing method is implemented.

[0020] The present invention provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned breast tumor image processing method when executing the program.

[0021] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0022] In the breast tumor image processing method provided by the present invention, the shallow feature branch uses a non-training feature extraction method to directly obtain a low-level feature vector from the breast tumor image without participating in the training of the model. Even when encountering an unknown breast tumor ultrasound image that is different from the distribution of the training data, the prior information of the unknown breast tumor ultrasound image is provided to the classification layer. During the training process, based on the prior information provided by the low-level feature vector, the first trainable weight coefficient of the deep feature extraction branch in the network model and the second trainable weight coefficient of the classification layer are synchronously adjusted by a joint optimization method instead of being updated independently. This synchronous adjustment can ensure that the weights of the two parts work together during the training process, thereby avoiding the degradation of the performance of the entire model due to overfitting or underfitting of a certain part. The end-to-end breast tumor image classification model obtained avoids the gradual amplification of the error caused by the segmentation of each step. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1 A schematic diagram of a process flow of a breast tumor image processing method provided by the present invention;

[0025] Figure 2 A schematic diagram of the network model structure provided by the present invention;

[0026] Figure 3 The network structure diagram of the DenseNet169 model provided by the present invention;

[0027] Figure 4 A schematic diagram of shallow feature verification results provided by the present invention;

[0028] Figure 5 This is the ROC curve diagram of the CNN fusion model provided by the present invention.

[0029] Figure 6 This is a flow chart for constructing the breast tumor image classification model provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Ultrasound imaging can reveal the characteristics of breast tumors non-invasively, and then reveal the relationship between them and the pathological information of breast tumors. Based on ultrasound imaging technology, some studies have proposed a variety of effective methods to achieve accurate prediction of breast tumor pathological information, mainly including machine learning and deep learning methods based on radiological features. Related research results show that the above methods can accurately predict the pathological information of tumors. However, both of the above methods have their own shortcomings. The machine learning method based on radiological features is based on manually extracted features. It relies on manual labeling of the tumor area and can reveal the shallow information of the tumor, but cannot obtain the deep information of the tumor. The deep learning method can automatically learn the deep features of breast tumors to reveal the deep information of the tumor, but this method loses the shallow information. The two types of features have a certain complementarity, and the fusion of the two can improve the diagnostic performance of breast tumors. At present, some studies have tried to fuse shallow features and deep features, and construct machine learning classifiers to predict the pathological information of breast tumors. The results show that the feature fusion method can effectively improve the prediction performance of tumors. Unlike these traditional feature fusion methods, it is still unknown whether incorporating shallow feature information of breast tumors into the CNN fusion model can improve the prediction performance of the model. In addition, due to the independence of deep and shallow feature extraction, fusion, and classification in the existing technology, the generalization ability of this method is weak. Deep learning models often require a large amount of training data to capture complex features, while shallow feature extraction (such as LBP) relies more on local information and is easily disturbed when facing ultrasound images with high noise or complex background. Since features from different sources may not be effectively aligned or coordinated during the fusion process, the classifier's prediction accuracy for new and unseen samples decreases, that is, the generalization ability is insufficient. Specifically, if the fused features do not fully consider the differences between deep and shallow features, or fail to enhance the complementarity between features through more complex models (such as fusion methods based on attention mechanisms), then even if the classifier can achieve good results during training, in practical applications, especially for the classification of unknown or complex samples, the performance of the model will be greatly reduced.

[0032] Based on this, in order to improve the diagnostic performance of breast tumors, the present invention first extracts the radiological features of the breast tumor area based on breast ultrasound data, and uses a feature selection algorithm to screen the optimal shallow features. Subsequently, 8 pre-trained CNN models are transferred and learned to predict the pathological information of breast tumors, from which the optimal CNN model is determined and used as the backbone network of the subsequent CNN fusion model. Based on the determined OSFs and the optimal CNN model, the present invention proposes a CNN fusion model integrating OSFs, which includes two parallel channel modules, namely a shallow feature channel module and a deep feature channel module. Different from previous feature fusion methods, this method integrates the shallow feature information of breast tumors into the CNN fusion model to achieve the fusion of deep features and shallow features, thereby improving the diagnostic performance of breast tumors.

[0033] In order to facilitate the understanding of the readers, the English abbreviations appearing in the present invention are explained as follows:

[0034] Superficial features: The full English name is optimal superficial features, abbreviated as OSFs in this article.

[0035] Convolutional neural network: The full name in English is convolutional neural network, which is abbreviated as CNN in this article.

[0036] Logistic regression: The full English name is logistic regression, which is abbreviated as LR in this article.

[0037] Support vector machine: The full English name is support vector machine, which is abbreviated as SVM in the present invention.

[0038] Random forest: The full English name is random forest, which is abbreviated as RF in this invention.

[0039] Accuracy: The full English name is Accuracy, which is abbreviated as ACC in the present invention.

[0040] Receiver operating characteristic: The full English name is receiver operating characteristic, abbreviated as ROC in the present invention.

[0041] Area under the ROC curve: The full English name is area under the ROC curve, which is abbreviated as AUC in the present invention.

[0042] Example 1

[0043] Figure 1 The process of the breast tumor image processing method of this embodiment is shown. Figure 1 The breast tumor image processing method is described in detail, and specifically comprises the following steps:

[0044] S1: Construct a network model including a feature extraction module, a splicing layer and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature to an image classification label.

[0045] Figure 2 The schematic diagram of the network model structure constructed in this embodiment is shown below. Figure 2 The network model is described in detail, including:

[0046] S101: Deep feature extraction branch.

[0047] The deep feature extraction branch is formed by a DenseNet169 network and three fully connected layers connected to the output end of the DenseNet169 network; the DenseNet169 network maps the breast tumor image into an initial deep feature vector with a dimension of 1664 through a combination of its Dense Blocks and Transition Layers; the three fully connected layers weight and reduce the dimension of the initial deep feature vector with a dimension of 1664, and output a deep feature vector with a dimension of 64.

[0048] Eight pre-trained CNN models were used to directly predict the benign and malignant nature of breast tumors, including ResNet18, VGG16, VGG19, DenseNet121, DenseNet169, EffcientNetB3, EffcientNetB4, and GoogleNet models. First, the tumor images were preprocessed, the image size was adjusted to 224x224, and the pixel values ​​of the images were normalized, and each color channel was standardized using the given mean and standard deviation. The ImageNet dataset was used to pre-train the above eight CNN models in turn, and then the pre-trained models were fine-tuned using ultrasound tumor images. To ensure fairness, the hyperparameters of all models were: learning rate = 0.0001, batch size = 16, maximum number of epochs = 30, and the optimizer was Adam. Table 1 lists the prediction performance of the eight CNN models. The results show that among all CNN models, the DenseNet169 model has the best prediction performance (ACC = 0.9231, AUC = 0.9521). This proves that the DenseNet169 model has advantages as the backbone network of the subsequent fusion model. The network structure of this model is as follows Figure 3 shown.

[0049] Table 1 Prediction performance of traditional CNN models

[0050] Models ACC AUC Sensitivity Specificity DenseNet169 0.9231 0.9521 0.9048 0.9318 DenseNet121 0.9128 0.9473 0.8571 0.9394 VGG16 0.8923 0.9451 0.8413 0.9167 GoogleNet 0.8872 0.9407 0.8254 0.9167 VGG19 0.8821 0.9350 0.7302 0.9545 EffcientNetB4 0.8821 0.9295 0.7937 0.9242 ResNet18 0.8769 0.9427 0.7302 0.9470 EffcientNetB3 0.8769 0.9352 0.7937 0.9167

[0051] S102: Shallow feature extraction branch.

[0052] Before the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of a breast tumor image, an image segmentation algorithm or a target labeling algorithm is used to extract a breast tumor region in the breast tumor image; after extracting the breast tumor region in the breast tumor image, the shallow feature extraction branch uses a non-training feature extraction method to calculate the sphericity, regional entropy, perimeter, and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4. Furthermore, four consecutive fully connected layers can be connected at the output end of the shallow feature extraction branch to introduce a third trainable weight coefficient to remap the shallow feature vector with a dimension of 4 to a new shallow feature vector with a dimension of 64.

[0053] Based on the original ultrasound image and the tumor area, the signal intensity of the tumor area was extracted. The public Python toolkit Pyradiomics was used to extract 102 radiological features, including 9 morphological features, 18 statistical features, and 75 texture features. The Z-score method was used to standardize all features so that their mean was 0 and their standard deviation was 1. Based on the Gini coefficient, the five-fold cross-validation method was used to determine the importance score of each feature, and the OSFs were determined based on the ranking. Figure 4 The classification performance of the model with different numbers of features is shown, and the results show that the number of OSFs is 4. Finally, four important features are determined, namely sphericity, regional entropy, perimeter and elongation, which can be calculated using formulas (1)-(4) respectively.

[0054]

[0055] Among them, R is the radius of the tumor area surface and P is the perimeter.

[0056]

[0057] Where p(i,j) is the element in the i-th row and j-th column of the normalized region matrix, and ∈ is an arbitrarily small positive number.

[0058]

[0059] Among them, a i With b i is the vertex of the i-th line in the perimeter grid.

[0060]

[0061] Among them, λ max With λ sec is the length of the first and second principal component axes.

[0062] The four OSFs obtained above will be used as the input of the subsequent fusion model. Based on the four OSFs obtained, three classification models, LR, SVM and RF, are constructed in turn to predict the benign and malignant nature of breast tumors. The main parameters involved in the three classification models are adjusted to determine the optimal parameters to achieve the best prediction performance. The main parameters of LR include the inverse of the regularization coefficient C, the penalty term penalty and the optimization algorithm solver, and their value ranges are {0.001, 0.01, 0.1, 1, 10, 100, 1000}, {L1, L2, None} and {liblinear, lbfgs}, respectively. Based on five-fold cross validation, the optimal parameters of LR are determined to be C=1, penalty=L1, and solver=liblinear. The main parameters of SVM include the penalty coefficient C and the parameter gamma of the radial basis function, and their ranges are {0.001, 0.1, 10, 1000} and {0.001, 0.1, 10, 1000}, respectively. Based on the five-fold cross-validation method, the optimal parameters of SVM are determined to be C = 10.0 and gamma = 0.1. The main parameter of RF is the number of trees n_estimators, which has a value range of [10,200] and a step size of 5. Based on the five-fold cross-validation method, the optimal parameter of RF is determined to be n_estimators = 15.

[0063] Once the optimal parameters of the model are determined, it can be used to predict the benign and malignant nature of breast tumors. The four indicators of ACC, AUC, sensitivity and specificity are used to evaluate the prediction performance of the model in the benign and malignant nature of tumors. Table 2 lists the prediction performance of three traditional classifiers in the benign and malignant nature of tumors. The results show that all three classifiers have achieved relatively good prediction performance. Among them, the prediction performance of SVM (ACC = 0.9692, AUC = 0.9919) is significantly better than that of the other two classifiers.

[0064] Table 2 Diagnostic performance of machine learning classifiers

[0065] Methods ACC AUC Sensitivity Specificity SVM 0.9692 0.9919 0.9048 1.0000 RF 0.9436 0.9701 0.9048 0.9621 LR 0.9026 0.9781 0.8571 0.9242

[0066] S103: stitching layer.

[0067] Based on the four OSFs determined above and the optimal CNN model DenseNet169, the present invention proposes a CNN fusion method integrating OSFs, including model A ( Figure 2 left) and model B ( Figure 2 to improve the performance of the model in tumor diagnosis. Figure 2 The network structures of the two models are shown. Both of them have two parallel channel modules, including a deep feature channel module and a shallow feature channel module.

[0068] For model A, we first use the Densenet169 model to automatically extract 1664 features from the tumor image, and output 64 deep features through three fully connected layers, with the number of output nodes being 832, 416, and 64, respectively. Then, we concatenate the 64 deep features and the 4 shallow features according to the following formula (5) to obtain 68 features:

[0069] fused A =concat(x,y) (5)

[0070] Among them, x = {x1, x2, x3, x4} refers to the 4 best shallow features, and y = {y1, y2, ..., y64} refers to 64 deep features.

[0071] Finally, three fully connected layers are built, with the number of output nodes being 832, 416, and 2 respectively, and the softmax function is used to convert them into a probability matrix to obtain the prediction results.

[0072] Unlike model A, model B further learns 4 OSFs. This model builds four fully connected layers to output 64 shallow features for the shallow feature pathway module, and the number of output layer nodes is 1664, 832, 416, and 64 respectively. Then, the following formula (6) is used to concatenate 64 deep features and 64 shallow features to obtain 128 features. The other structures of model B are similar to those of model A and will not be repeated here.

[0073] fused B =concat(s 64 ,y) (6)

[0074] Among them, s 64 =Learn(x), refers to the new shallow features learned.

[0075] S104: Classification layer.

[0076] Table 3 and Figure 5 All of them demonstrated the predictive performance of the proposed CNN fusion model in tumor diagnosis. The results showed that the predictive performance of the CNN fusion model was better than that of the traditional machine learning method or deep learning method. The above findings indicate that the proposed CNN fusion method is an effective method for breast ultrasound tumor diagnosis. In addition, compared with model A (ACC = 0.9795, AUC = 0.9924), model B (ACC = 0.9846, AUC = 0.9983) has better predictive performance in tumor diagnosis. One possible reason is that OSFs are further learned and converted into new features that are more relevant to the benign and malignant nature of the tumor, thereby obtaining better diagnostic performance.

[0077] Table 3 Prediction performance of CNN fusion model integrating OSFs

[0078]

[0079]

[0080] S2: Collect breast tumor images and corresponding true classification labels to construct a data set, input the breast tumor images in the data set into the network model, take minimizing the error between the predicted classification label and the true classification label output by the network model as the optimization goal, and optimize the first trainable weight coefficient and the second trainable weight coefficient in the network model iteratively to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtain a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient.

[0081] Collect the data of public breast cancer patients, including ultrasound image data and pathological information, and select the complete data set that meets the requirements of this experiment. The present invention uses a public data set, which includes a total of 780 breast ultrasound images, with an average image size of 500×500 pixels. These images can be divided into three categories: normal people, benign masses and malignant masses. Normal people have no mass area. For this reason, we selected ultrasound images of benign masses and malignant masses as the final samples, with the numbers of the two being 437 and 210 respectively, and the total number of samples being 647. All samples were randomly divided into a training set (N=452) and a test set (N=195) in a ratio of 7:3.

[0082] S3: Use the breast tumor image classification model to process breast tumor images.

[0083] The present invention proposes a CNN fusion method integrating OSFs, and uses the method to predict the benign and malignant nature of breast ultrasound tumors, in order to explore the clinical value of the method in breast ultrasound tumor diagnosis. The steps include: collecting ultrasound imaging data and corresponding clinical data of breast cancer patients; determining the tumor area, and extracting the radiological features of the tumor area, screening it using a feature selection method to determine OSFs, and constructing a machine learning model to predict the benign and malignant nature of the tumor; using several commonly used CNNs to directly predict the benign and malignant nature of the tumor, in order to determine the optimal CNN model, and use it as the backbone network for subsequent fusion methods; based on the determined OSFs and the optimal CNN model, a CNN fusion model integrating OSFs is proposed to predict the benign and malignant nature of breast tumors. The model includes two parallel channel modules, namely a shallow feature channel module and a deep feature channel module. The construction process of the breast tumor image classification model is as follows. Figure 6The present invention integrates the shallow feature information of breast tumors into the CNN model, so that the shallow features and deep features complement each other, thereby improving the prediction performance of breast tumors and playing the role of assisting clinical diagnosis and treatment.

[0084] The above is a breast tumor image processing method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding breast tumor image processing device, including:

[0085] A model building module is used to build a network model including a feature extraction module, a splicing layer and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fused feature; the classification layer uses a second trainable weight coefficient to map the fused feature into an image classification label.

[0086] A model training module is used to collect breast tumor images and corresponding true classification labels to construct a data set, input the breast tumor images in the data set into a network model, minimize the error between the predicted classification label and the true classification label output by the network model as the optimization goal, optimize and iterate the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtain a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient.

[0087] The image processing module is used to process breast tumor images using a breast tumor image classification model.

[0088] For the specific definition of the breast tumor image processing device, please refer to the definition of the breast tumor image processing method above, which will not be repeated here. Each module in the above-mentioned breast tumor image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0089] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A breast tumor image processing method is provided.

[0090] The present invention also provides a computer device structure. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A breast tumor image processing method is provided.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0092] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for processing breast tumor images, characterized in that: include: Constructing a network model including sequentially connecting a feature extraction module, a splicing layer and a classification layer, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fusion feature; The classification layer maps the fused features to image classification labels using a second trainable weight coefficient; Collecting breast tumor images and corresponding true classification labels to construct a data set, inputting the breast tumor images in the data set into the network model, taking minimizing the error between the predicted classification label and the true classification label output by the network model as the optimization goal, optimizing and iterating the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtaining a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient; Breast tumor images are processed using a breast tumor image classification model.

2. The breast tumor image processing method according to claim 1, characterized in that: The deep feature extraction branch is formed by a DenseNet169 network and three fully connected layers connected to the output end of the DenseNet169 network; the DenseNet169 network maps the breast tumor image into an initial deep feature vector with a dimension of 1664 through a combination of its Dense Blocks and Transition Layers; the three fully connected layers weight and reduce the dimension of the initial deep feature vector with a dimension of 1664, and output a deep feature vector with a dimension of 64.

3. The breast tumor image processing method according to claim 1, characterized in that: Before the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image, an image segmentation algorithm or a target labeling algorithm is used to extract a breast tumor region in the breast tumor image; After extracting the breast tumor region in the breast tumor image, the shallow feature extraction branch uses a non-training feature extraction method to calculate the sphericity, regional entropy, perimeter and elongation of the breast tumor region to obtain a shallow feature vector with a dimension of 4.

4. The breast tumor image processing method according to claim 3, characterized in that: The output end of the shallow feature extraction branch is connected to four consecutive fully connected layers, and the four consecutive fully connected layers remap the shallow feature vector with a dimension of 4 to a new shallow feature vector with a dimension of 64 by introducing a third trainable weight coefficient.

5. The breast tumor image processing method according to claim 1, characterized in that: The classification layer is a CNN network, which maps fused features into image classification labels through convolutional layers, pooling layers and fully connected layers.

6. A breast tumor image processing device, characterized in that: include: A model construction module is used to construct a network model including a feature extraction module, a splicing layer and a classification layer connected in sequence, wherein the feature extraction module includes a parallel deep feature extraction branch and a shallow feature extraction branch; the deep feature extraction branch uses a first trainable weight coefficient to map a breast tumor image into a deep feature vector; the shallow feature extraction branch uses a non-training feature extraction method to extract a low-level feature vector of the breast tumor image; the splicing layer is used to splice the deep feature vector output by the deep feature extraction branch and the low-level feature vector output by the shallow feature extraction branch into a fusion feature; The classification layer maps the fused features to image classification labels using a second trainable weight coefficient; A model training module, used for collecting breast tumor images and corresponding true classification labels to construct a data set, inputting the breast tumor images in the data set into the network model, taking minimizing the error between the predicted classification label and the true classification label output by the network model as the optimization goal, optimizing and iterating the first trainable weight coefficient and the second trainable weight coefficient in the network model to obtain the optimal first trainable weight coefficient and the optimal second trainable weight coefficient that make the optimization goal optimal, and obtaining a breast tumor image classification model based on the optimal first trainable weight coefficient and the optimal second trainable weight coefficient; The image processing module is used to process breast tumor images using a breast tumor image classification model.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images