Method, device and medium for classifying benign and malignant breast tumors based on ultrasound images

Abstract

The present invention provides a method, device and medium for classifying benign and malignant breast tumors based on ultrasound images. The method comprises: preprocessing an original ultrasound image of a breast tumor with a classification label to obtain a preprocessed image; obtaining a region of interest in the preprocessed image; performing three types of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively; performing dimensionality reduction processing on the deep residual network feature vector and the texture feature vector respectively, and then performing feature fusion on the two to obtain fused vector data; using a support vector machine classifier to learn a certain amount of fused vector data, and using a naive Bayes classifier to learn the morphological feature vector, and obtaining a tumor classification model by weighting the two classifiers; inputting the breast tumor ultrasound image to be classified into the breast tumor classification model to obtain a classification result. The method of the present invention can realize more accurate automatic classification of breast tumor ultrasound images.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a method, device and medium for classifying benign and malignant breast tumors based on ultrasonic images. Background Art

[0002] Cancer is one of the most important diseases threatening human health worldwide. According to statistics from the World Health Organization, the incidence of cancer has been on the rise. Early detection, early diagnosis and early treatment are the key to improving cancer cure rates and reducing mortality. Ultrasound imaging technology is widely used in the detection of breast tumors due to its versatility, safety and high sensitivity. It has been recognized as a universal and effective screening and diagnostic tool for benign and malignant breast tumors. In clinical practice, it mainly includes thyroid ultrasound, breast ultrasound, prostate ultrasound, cardiac ultrasound and kidney ultrasound.

[0003] At present, ultrasound is the preferred imaging examination and preoperative evaluation method for diagnosing breast tumors. However, the correct interpretation of breast ultrasound images requires doctors to have high technical and clinical experience. In the process of ultrasound diagnosis, manual analysis of images is time-consuming and subjective. The clinical experience and knowledge level of different radiologists have a great impact on the diagnostic results, which can easily lead to misdiagnosis, missed diagnosis or inconsistent diagnostic results. In addition, some patients who cannot be directly diagnosed by imaging results need to undergo a biopsy, but a biopsy is an invasive examination, and most of the examination results are benign, which will undoubtedly cause great physical and psychological harm to the patient. At the same time, the current imaging method of ultrasound images has the disadvantages of large speckle noise and low contrast, resulting in low distinction between the foreground and background of the image, which also increases the difficulty of diagnosis for doctors.

[0004] Automatic breast tumor computer-aided diagnosis (CAD) technology can detect information from ultrasound images that cannot be obtained by the human eye, such as texture information, grayscale quantization, edge information, etc., and provide doctors with reliable auxiliary diagnosis opinions. Therefore, the CAD method can not only improve the accuracy and objectivity of breast tumor detection in ultrasound images, but also help to relieve the workload of doctors, reduce misdiagnosis and missed diagnosis, and reduce the biopsy rate and pain for patients. In recent years, the automatic classification of breast tumor ultrasound images using computer-aided diagnosis methods has also received widespread attention. However, the current imaging method of ultrasound images will result in low distinction between the image background and foreground, high and close grayscale, and it is difficult to distinguish the differences between similar grayscale areas in ultrasound images. Most studies have not paid attention to the problem of low distinction between image background and foreground, and only studied single-class features or directly merged two types of features, which easily ignores important breast tumor information, resulting in inaccurate automatic classification of breast tumor ultrasound images. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method, device and medium for classifying benign and malignant breast tumors based on ultrasound images. By combining the depth features and texture features of breast tumor ultrasound images and weighted scoring of multiple classifiers, more accurate automatic classification of breast tumor ultrasound images can be achieved.

[0006] In a first aspect, the present invention provides a method for classifying benign and malignant breast tumors based on ultrasound images, the method comprising:

[0007] Step 10: preprocessing the original breast tumor ultrasound image with classification labels to obtain a preprocessed image, wherein the classification labels include malignant breast tumor and benign breast tumor;

[0008] Step 20, obtaining a region of interest in the preprocessed image;

[0009] Step 30, performing three types of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively;

[0010] Step 40: perform dimensionality reduction processing on the deep residual network feature vector, perform dimensionality reduction processing on the texture feature vector, and then perform feature fusion on the residual network feature vector and the texture feature vector after the dimensionality reduction processing to obtain fused vector data;

[0011] Step 50: using a support vector machine classifier to learn a certain amount of fusion vector data, and using a naive Bayes classifier to learn the morphological feature vector, and using a maximum random search method to weight the two classifiers to obtain a tumor classification model;

[0012] Step 60: Input the breast tumor ultrasound image to be classified into a breast tumor classification model to obtain a classification result.

[0013] Furthermore, the preprocessing includes removing text and personal information around the ultrasound image, reducing speckle noise and enhancing contrast.

[0014] Furthermore, the region of interest in step 20 is obtained by: the doctor designates a region in the pre-processed image as the region of interest.

[0015] Furthermore, the three processes in step 30 are specifically as follows:

[0016] The first processing method is to extract texture features from the ultrasound image, wherein the texture features include image local binary pattern (LBP), directional gradient histogram (HOG) and gray level co-occurrence matrix (GLCM), and then perform normalization operation to obtain a texture feature vector;

[0017] The second processing method is to extract deep residual neural network features from the region of interest, wherein the deep residual neural network features include extracting deep learning features from the pooling layer Pool5 using Resnet101, and then performing a normalization operation to obtain a deep learning feature vector;

[0018] The third processing method: extracting morphological features of the region of interest, wherein the morphological features include compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, and minimum rectangular frame area surrounding the breast tumor, and then normalizing the morphological features to obtain a morphological feature vector.

[0019] Furthermore, the compactness is used to measure the similarity between the shape of the breast tumor and its fitting circle. The calculation formula of the compactness C is as follows: Among them, A n represents the breast tumor area, P n represents the circumference of breast tumor, r represents the radius of the fitted circle;

[0020] The ellipse compactness is the ratio of the circumference of the fitted ellipse to the circumference of the original breast tumor contour. The calculation formula of the ellipse compactness EC is as follows: Among them, a represents the semi-major axis of the fitted ellipse, b represents the semi-minor axis of the fitted ellipse, and P n represents the circumference of breast tumor;

[0021] The radial distance spectrum is to quantify the degree of roughness of the edge of the breast tumor by statistically analyzing the radial distance from each point on the edge of the breast tumor to the center of the breast tumor. The calculation formula of the radial distance is as follows: Where (x1, y1) is the coordinate of the edge point of the breast tumor, and (x0, y0) is the coordinate of the center point of the breast tumor;

[0022] The convexity is a function that measures the ratio of the convex hull to the original contour shape of the breast tumor. The calculation formula of the convexity is as follows: Where P c is the perimeter of the convex hull, P n is the circumference of the breast tumor;

[0023] The solidity is used to describe the convexity or concavity of the shape of a breast tumor. The calculation formula of the solidity is as follows: Among them A n is the breast tumor area, A c is the area of ​​the convex hull;

[0024] The elongation is the ratio between the length and width of the bounding box around the lesion, and the calculation formula of the elongation is as follows: Where Wn is the width of the breast tumor, L n is the length of the breast tumor;

[0025] The compactness is the ratio of the breast tumor area to the square of the perimeter. The calculation formula of the compactness is as follows: Where P n is the circumference of breast tumor, A n is the breast tumor area;

[0026] The rectangularity is defined as the ratio between the area of ​​the breast tumor and the area of ​​the minimum bounding rectangular frame of the breast tumor. The calculation formula of the rectangularity is as follows: Among them A n is the breast tumor area, A r is the area of ​​the smallest rectangular frame of the breast tumor;

[0027] The circularity is the ratio of the breast tumor area to the square of the minimum convex hull perimeter of the breast tumor boundary. The circularity calculation formula is as follows: Among them A n is the breast tumor area, P c is the perimeter of the convex hull;

[0028] The breast tumor perimeter is the number of pixels within the breast tumor boundary, and the calculation formula of the breast tumor perimeter is as follows: Among them, X i , …, X n are the boundary coordinates of the breast tumor;

[0029] The breast tumor area is the number of pixels within the breast tumor region.

[0030] Furthermore, the region of interest is a breast tumor morphological outline manually extracted by a doctor; by obtaining the region of interest specified by the doctor, the suspicious area can be processed accurately, and the consumption of computing resources caused by invalid data processing is reduced.

[0031] Furthermore, the step 40 further comprises:

[0032] Step 41: Obtain the m-dimensional texture feature vectors of n ultrasound images to form a texture feature matrix X m×n , find the mean of each m-dimensional texture feature vector in the texture feature matrix, and then subtract the corresponding mean from all m-dimensional texture feature vectors to obtain the texture feature matrix after removing the average value;

[0033] Step 42, calculating the covariance matrix of the texture feature matrix after removing the average value, diagonalizing the eigenvalues ​​and eigenvectors of the covariance matrix so that the elements other than the diagonal are 0, and the energy is attributed to the main direction;

[0034] Step 43, sort the eigenvectors according to the size of the eigenvalues, and retain the largest first ω eigenvalues;

[0035] Step 44: convert the data in the texture feature matrix into a new space constructed by the feature vector to obtain n texture feature vectors after dimensionality reduction.

[0036] Step 45: Obtain the m-dimensional deep residual network feature vectors of n ultrasound images to form a deep learning feature matrix X m×n , find the mean of each m-dimensional deep learning feature vector in the deep learning feature matrix, and then subtract the corresponding mean from all m-dimensional deep learning feature vectors to obtain the deep learning feature matrix after removing the average value;

[0037] Step 46: Calculate the covariance matrix of the deep learning feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction;

[0038] Step 47, sort the eigenvectors according to the size of the eigenvalues, and retain the largest first ω eigenvalues;

[0039] Step 48: convert the data in the deep learning feature matrix into a new space constructed by the feature vector to obtain n deep learning feature vectors after dimensionality reduction;

[0040] Step 49: Cascade the deep learning features after dimensionality reduction and the texture features after dimensionality reduction to obtain a fusion feature after the deep learning features and the shallow texture are cascaded.

[0041] In a second aspect, the present invention provides a device for classifying benign and malignant breast tumors based on ultrasound images, the device comprising:

[0042] A preprocessing module, used for preprocessing an original breast tumor ultrasound image with classification labels to obtain a preprocessed image, wherein the classification labels include malignant breast tumor and benign breast tumor;

[0043] ROI acquisition module, used to obtain the region of interest in the preprocessed image;

[0044] A vector extraction module, used for performing three kinds of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively;

[0045] A vector fusion module is used to perform dimensionality reduction processing on the deep residual network feature vector and the texture feature vector, and then perform feature fusion on the residual network feature vector and the texture feature vector after dimensionality reduction processing to obtain fused vector data;

[0046] A learning module, for learning a certain amount of fusion vector data using a support vector machine classifier, and learning a morphological feature vector using a naive Bayes classifier, and obtaining a tumor classification model by weighting the two classifiers using a maximum random search method; and

[0047] The classification module is used to input the breast tumor ultrasound image to be classified into the breast tumor classification model to obtain the classification result.

[0048] Furthermore, the preprocessing includes removing text and personal information around the ultrasound image, reducing speckle noise and enhancing contrast.

[0049] Furthermore, the three processes in the vector acquisition module are specifically as follows:

[0050] The first processing method is to extract texture features from the ultrasound image, wherein the texture features include image local binary pattern (LBP), directional gradient histogram (HOG) and gray level co-occurrence matrix (GLCM), and then perform normalization operation to obtain a texture feature vector;

[0051] The second processing method is to extract deep residual neural network features from the region of interest, wherein the deep residual neural network features include extracting deep learning features from the pooling layer Pool5 using Resnet101, and then performing a normalization operation to obtain a deep learning feature vector;

[0052] The third processing method: extracting morphological features of the region of interest, wherein the morphological features include compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, and minimum rectangular frame area surrounding the breast tumor, and then normalizing the morphological features to obtain a morphological feature vector.

[0053] Furthermore, the vector fusion module is specifically:

[0054] Get the m-dimensional texture feature vectors of n ultrasound images to form a texture feature matrix X m×n , find the mean of each m-dimensional texture feature vector in the texture feature matrix, and then subtract the corresponding mean from all m-dimensional texture feature vectors to obtain the texture feature matrix after removing the average value;

[0055] Calculate the covariance matrix of the texture feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction;

[0056] Sort the eigenvectors by eigenvalue and keep the largest first ω eigenvalues;

[0057] The data in the texture feature matrix is ​​converted into a new space constructed by the feature vector to obtain n texture feature vectors after dimensionality reduction.

[0058] Get the m-dimensional deep residual network feature vectors of n ultrasound images to form a deep learning feature matrix X m×n , find the mean of each m-dimensional deep learning feature vector in the deep learning feature matrix, and then subtract the corresponding mean from all m-dimensional deep learning feature vectors to obtain the deep learning feature matrix after removing the average value;

[0059] Calculate the covariance matrix of the deep learning feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction;

[0060] Sort the eigenvectors by eigenvalue and keep the largest first ω eigenvalues;

[0061] Convert the data in the deep learning feature matrix to the new space constructed by the feature vector to obtain n deep learning feature vectors after dimensionality reduction;

[0062] The deep learning features after dimensionality reduction and the texture features after dimensionality reduction are cascaded to obtain the fusion features of the deep learning features and the shallow texture cascade.

[0063] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0064] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0065] 1. By reducing the dimension of texture features and fusing them with deep learning features, the characteristics of breast tumor ultrasound images can be learned more comprehensively;

[0066] 2. From the perspective of geometric morphology, the calculation formulas related to the morphological characteristics of breast tumors are listed, so that the classification model can better learn the morphological characteristics of breast tumors;

[0067] 3. It can effectively improve the accuracy and robustness of ultrasound image prediction of benign and malignant breast tumors, help doctors improve the accuracy of diagnosis of benign and malignant breast tumors, reduce misdiagnosis and missed diagnosis due to subjective factors, and improve the diagnostic efficiency of doctors.

[0068] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0070] Figure 1 is a flow chart of the method of the present invention;

[0071] Figure 2 It is a structural schematic diagram of the second device of the present invention;

[0072] Figure 3 It is a structural schematic diagram of the medium of the present invention;

[0073] Figure 4 A flowchart of a method according to an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of ultrasound image preprocessing in one embodiment of the present invention;

[0075] Figure 6 Schematic diagram of the outline of a breast tumor in an ultrasound image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The embodiment of the present application provides a method, device and medium for classifying breast tumor ultrasound images based on deep features and multiple shallow features combined with multiple classifiers. Since the edge brightness of breast tumors is always close to that of normal tissues, special attention is paid to the grayscale difference close to the intensity of normal tissues. By normalizing the feature vector and reducing the dimension of the texture feature, the texture feature vector and the deep residual network feature vector are effectively fused, which improves the accuracy of breast tumor ultrasound image classification to a certain extent and provides doctors with a more accurate diagnostic reference.

[0077] In order to better understand the technical solution of the present invention, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0078] Embodiment 1

[0079] This embodiment provides a method for classifying benign and malignant breast tumors based on ultrasound images. Figure 1 As shown, the method includes:

[0080] Step 10: preprocessing the original breast tumor ultrasound image with classification labels to obtain a preprocessed image, wherein the classification labels include malignant breast tumor and benign breast tumor;

[0081] Step 20, obtaining a region of interest in the preprocessed image;

[0082] Step 30, performing three types of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively;

[0083] Step 40: perform dimensionality reduction processing on the deep residual network feature vector, perform dimensionality reduction processing on the texture feature vector, and then perform feature fusion on the residual network feature vector and the texture feature vector after the dimensionality reduction processing to obtain fused vector data;

[0084] Step 50: using a support vector machine classifier to learn a certain amount of fusion vector data, and using a naive Bayes classifier to learn the morphological feature vector, and using a maximum random search method to weight the two classifiers to obtain a tumor classification model;

[0085] Step 60: Input the breast tumor ultrasound image to be classified into a breast tumor classification model to obtain a classification result.

[0086] Preferably, the preprocessing includes removing text and personal information around the ultrasound image, reducing speckle noise and enhancing contrast.

[0087] like Figure 5 As shown, in a specific embodiment, the “removing text and personal information around the ultrasound image” specifically includes: using a morphological mask to remove text around the ultrasound image, the patient's name, personal privacy and other information. The mask for detecting the right edge of the ultrasound image is shown as follows:

[0088]

[0089] The morphological mask is convolved with the original image. The convolution result is the largest at the right edge of the original ultrasound image. The column with the maximum value is taken out and the right edge is obtained by screening. The left, upper and lower edges can be obtained in the same way.

[0090] The “reducing speckle noise” is specifically: using speckle suppression anisotropic diffusion (SRAD) filtering to suppress the local texture detail changes caused by speckle noise while maintaining the main breast tumor morphological information, and eliminate the mixed noise in the process of ultrasound image digitization, wherein the diffusion coefficient and edge detection operator in the filtering are:

[0091]

[0092]

[0093]

[0094] Where c(i,j,t) is the diffusion coefficient, is the gradient operator, q(i,j,t) is the edge detection operator, q0(t) is the diffusion threshold at time t, var is the variance of the grayscale in a uniform area when the number of iterations is t, is the mean grayscale value in a uniform area when the number of iterations is t.

[0095] The "contrast enhancement" is specifically as follows: using the histogram equalization method, the grayscale probability density function and the cumulative probability distribution function are calculated according to the image grayscale value, and then the cumulative probability distribution function is normalized to the original grayscale value range to obtain the grayscale conversion function. This method adjusts the grayscale histogram of the original breast ultrasound image from a relatively dense interval to a uniform distribution within the full grayscale range, thereby enhancing the overall contrast of the image.

[0096] Preferably, the region of interest (ROI) in step 20 is obtained by: the doctor specifies a region in the pre-processed image as the region of interest. Figure 6 By obtaining the region of interest specified by the doctor, the suspicious lesion area can be processed accurately, and the consumption of computing resources caused by invalid data processing can be reduced.

[0097] Preferably, the three processes in step 30 are specifically as follows:

[0098] The first processing: extract texture features from the ultrasound image, the texture features include the local binary pattern (LBP), histogram of oriented gradients (HOG) and gray-level co-occurrence matrix (GLCM), and then perform normalization to obtain a texture feature vector; the LBP operator is defined as a 3×3 window. Assume that the coordinates of the center pixel of the window are (x, y), compare the grayscale values ​​of the eight surrounding neighborhood pixels with the grayscale value of the center pixel. If the value is greater than the center pixel, the pixel is marked as 1; otherwise, the pixel is marked as 0. Finally, an ordered 8-bit binary number (i.e., a 256-bit decimal) is generated to reflect the texture information of the local area. The specific calculation formula is: Among them, s(x) is the sign function, i c is the gray value of the center pixel, i p is the gray value of the neighborhood pixel. When extracting GLCM features, the gray level is set to 64. The distance between pixels is adjusted within the range of [1,10], and the relationship between pixels with a certain distance is calculated from four directions (0, 45, 90, 135). Finally, 40 different matrices are obtained from each image. There are 14 specific parameters for texture feature extraction based on GLCM. This embodiment selects four texture features: energy, contrast, correlation and homogeneity. Normalization is the use of discrete normalization to linearly transform the original data and map the data values ​​between them. The specific calculation method is as follows: Among them, min is the minimum value and max is the maximum value.

[0099] The second processing method is to extract deep residual neural network features from the region of interest, wherein the deep residual neural network features include extracting deep learning features from the pooling layer Pool5 using Resnet101, and then performing a normalization operation to obtain a deep learning feature vector;

[0100] The third processing method: extracting morphological features of the region of interest, wherein the morphological features include compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, and minimum rectangular frame area surrounding the breast tumor, and then normalizing the morphological features to obtain a morphological feature vector.

[0101] In this embodiment, the doctor first manually extracts the morphological contour of the breast tumor lesion in the ultrasound image (such as Figure 6 As shown), by obtaining the morphological contour extracted by the doctor, thirteen breast tumor morphological characteristics are extracted according to the morphological contour, including compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, and minimum rectangular frame area surrounding the breast tumor.

[0102] The compactness is used to measure the similarity between the shape of the breast tumor and its fitting circle. The calculation formula of the compactness C is as follows: Among them, A n represents the breast tumor area, P n represents the circumference of breast tumor, r represents the radius of the fitted circle;

[0103] The ellipse compactness is the ratio of the circumference of the fitted ellipse to the circumference of the original breast tumor contour. The calculation formula of the ellipse compactness EC is as follows: Among them, a represents the semi-major axis of the fitted ellipse, b represents the semi-minor axis of the fitted ellipse, and P n represents the circumference of breast tumor;

[0104] The radial distance spectrum is to quantify the degree of roughness of the edge of the breast tumor by statistically analyzing the radial distance from each point on the edge of the breast tumor to the center of the breast tumor. The calculation formula of the radial distance is as follows: Where (x1, y1) is the coordinate of the edge point of the breast tumor, and (x0, y0) is the coordinate of the center point of the breast tumor;

[0105] The convexity is a function that measures the ratio of the convex hull to the original contour shape of the breast tumor. The calculation formula of the convexity is as follows: Where P c is the perimeter of the convex hull, P n is the circumference of the breast tumor;

[0106] The solidity is used to describe the convexity or concavity of the shape of a breast tumor. The calculation formula of the solidity is as follows: Among them A n is the breast tumor area, A c is the area of ​​the convex hull;

[0107] The elongation is the ratio between the length and width of the bounding box around the lesion, and the calculation formula of the elongation is as follows: Where W n is the width of the breast tumor, L n is the length of the breast tumor;

[0108] The compactness is the ratio of the breast tumor area to the square of the perimeter. The calculation formula of the compactness is as follows: Where P n is the circumference of breast tumor, A n is the breast tumor area;

[0109] The rectangularity is defined as the ratio between the area of ​​the breast tumor and the area of ​​the minimum bounding rectangular frame of the breast tumor. The calculation formula of the rectangularity is as follows: Among them A n is the breast tumor area, A r is the area of ​​the smallest rectangular frame of the breast tumor;

[0110] The roundness is the ratio of the breast tumor area to the square of the minimum convex hull perimeter of the breast tumor boundary. The calculation formula of the roundness is as follows: Among them A n is the breast tumor area, P c is the perimeter of the convex hull;

[0111] The breast tumor perimeter is the number of pixels within the breast tumor boundary, and the calculation formula of the breast tumor perimeter is as follows: Among them, X i , …, X n are the boundary coordinates of the breast tumor;

[0112] The breast tumor area is the space occupied by the breast tumor on the plane. The area here is defined as the number of pixels within the breast tumor region.

[0113] Preferably, in this embodiment, the method used for dimensionality reduction of texture features and neural network features is principal component analysis (PCA). PCA is a technique for analyzing and simplifying data sets. It is usually used to reduce the dimension of a data set while keeping the features that contribute most to variance in a data set. Here, PCA is used to reduce the dimension of the texture feature vector matrix, while retaining the most effective feature vectors. First, for a given sample space Xm×n , calculate the covariance matrix, the eigenvalues ​​and eigenvectors of the matrix; finally, arrange the eigenvectors into a matrix from top to bottom in rows according to the size of the corresponding eigenvalues, and the matrix composed of the first ω rows is the data after dimensionality reduction. Specifically, the step 40 further includes:

[0114] Step 41: Obtain the m-dimensional texture feature vectors of n ultrasound images to form a texture feature matrix X m×n , find the mean of each m-dimensional texture feature vector in the texture feature matrix, and then subtract the corresponding mean from all m-dimensional texture feature vectors to obtain the texture feature matrix after removing the average value;

[0115] Step 42, calculating the covariance matrix of the texture feature matrix after removing the average value, diagonalizing the eigenvalues ​​and eigenvectors of the covariance matrix so that the elements other than the diagonal are 0, and the energy is attributed to the main direction;

[0116] Step 43: Sort the eigenvectors by eigenvalue, and keep the largest first ω eigenvalues ​​(ω <m);

[0117] Step 44: convert the data in the texture feature matrix into a new space constructed by the feature vector to obtain n texture feature vectors after dimensionality reduction.

[0118] Step 45: Obtain the m-dimensional deep residual network feature vectors of n ultrasound images to form a deep learning feature matrix X m×n , find the mean of each m-dimensional deep learning feature vector in the deep learning feature matrix, and then subtract the corresponding mean from all m-dimensional deep learning feature vectors to obtain the deep learning feature matrix after removing the average value;

[0119] Step 46: Calculate the covariance matrix of the deep learning feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction;

[0120] Step 47: Sort the eigenvectors by eigenvalue, and keep the largest first ω eigenvalues ​​(ω <m);

[0121] Step 48: convert the data in the deep learning feature matrix into a new space constructed by the feature vector to obtain n deep learning feature vectors after dimensionality reduction;

[0122] Step 49, cascade the deep learning features after dimensionality reduction and the texture features after dimensionality reduction to obtain the fusion features after the deep learning features and the shallow texture cascade. In this embodiment, the processed texture features are linearly fused with the corresponding neural network features, that is, the texture feature vector and the neural network feature vector of the same breast tumor ultrasound image are connected in series, so that the next step classifier learns the breast tumor features.

[0123] In this embodiment, step 50 uses a classifier to learn and train the fused vector data after the linear fusion of the texture features of the breast tumor ultrasound image and the neural network features. The support vector machine (SVM) algorithm can be used to learn the fused vector data, or the naive Bayes (NB) algorithm can be used to learn the fused vector data. Of course, it is not limited to this algorithm, and other machine learning algorithms can also be used, such as: linear discriminant analysis (LDA), K nearest neighbor (KNN), etc. SVM classifier is a generalized linear classifier that performs binary classification on data in a supervised learning manner. Its decision boundary is to solve the maximum margin hyperplane for the learning sample, and the kernel function is used for nonlinear classification. The input data is mapped to a high-dimensional space to construct the optimal separation hyperplane.

[0124] In this embodiment, the breast tumor ultrasound image to be classified is input into the model obtained through training to obtain a predicted benign or malignant classification result of the breast tumor. The processing steps of the original breast tumor ultrasound image to be classified are similar to the processing steps of the original breast tumor ultrasound image with classification labels mentioned above, but it does not carry a classification label. Instead, the breast tumor ultrasound image to be classified is obtained after processing, and then input into the breast tumor classification model, and classified by the classifier to obtain a classification label, i.e., malignant or benign.

[0125] The breast tumor ultrasound image to be classified can be obtained by the following steps: preprocessing the original breast tumor ultrasound image to be classified, then obtaining the region of interest, extracting texture features from the region of interest, normalizing the texture features, and then reducing the dimension of the normalized texture features to obtain a texture feature vector; extracting neural network features from the region of interest, normalizing the neural network features, and then reducing the dimension of the normalized neural network features to obtain a neural network feature vector; finally, feature fusion of the texture feature vector and the neural network feature vector. Extracting morphological features from the region of interest, and then normalizing the morphological features to obtain a morphological feature vector.

[0126] In this specific embodiment, a total of 1061 ultrasound images (472 benign and 589 malignant) of breast tumor ultrasound images selected by experts were selected for training and testing of the classification model, including 848 images in the training set and 213 images in the test set. The final test showed that the accuracy of benign and malignant breast tumor classification was 89.172%, which was 8.372%, 16.562% and 5.733% higher than the classification accuracy of learning texture features, morphological features or deep learning features alone, respectively, and the classification speed was increased by an average of 8 times.

[0127] It can be understood that the classification of breast tumor ultrasound images is mainly determined by neural network features, texture features and morphological features. The key is to linearly fuse the deep learning features after dimensionality reduction and the texture features after dimensionality reduction so that the classifier can learn more breast tumor information. In this embodiment, breast tumor ultrasound images are classified. The image classification method of the present invention is not limited to the breast. Other ultrasound images, such as thyroid, can also adopt the ultrasound image breast tumor classification method based on deep features and multiple shallow features combined with multiple classifiers and feature fusion provided by the present invention.

[0128] Based on the same inventive concept, the present application also provides a device corresponding to the method in Example 1, see Example 2 for details.

[0129] Embodiment 2

[0130] This embodiment provides a device for classifying benign and malignant breast tumors based on ultrasound images. Figure 2 As shown, the device comprises:

[0131] A preprocessing module, used for preprocessing an original breast tumor ultrasound image with classification labels to obtain a preprocessed image, wherein the classification labels include malignant breast tumor and benign breast tumor;

[0132] ROI acquisition module, used to obtain the region of interest in the preprocessed image;

[0133] A vector extraction module, used for performing three kinds of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively;

[0134] A vector fusion module is used to perform dimensionality reduction processing on the deep residual network feature vector and the texture feature vector, and then perform feature fusion on the residual network feature vector and the texture feature vector after dimensionality reduction processing to obtain fused vector data;

[0135] A learning module, for learning a certain amount of fusion vector data using a support vector machine classifier, and learning a morphological feature vector using a naive Bayes classifier, and obtaining a tumor classification model by weighting the two classifiers using a maximum random search method; and

[0136] The classification module is used to input the breast tumor ultrasound image to be classified into the breast tumor classification model to obtain the classification result.

[0137] Preferably, the preprocessing includes removing text and personal information around the ultrasound image, reducing speckle noise and enhancing contrast.

[0138] like Figure 5As shown, in a specific embodiment, the “removing text and personal information around the ultrasound image” specifically includes: using a morphological mask to remove text around the ultrasound image, the patient's name, personal privacy and other information. The mask for detecting the right edge of the ultrasound image is shown as follows:

[0139]

[0140] The morphological mask is convolved with the original image. The convolution result is the largest at the right edge of the original ultrasound image. The column with the maximum value is taken out and the right edge is obtained by screening. The left, upper and lower edges can be obtained in the same way.

[0141] The “reducing speckle noise” is specifically: using speckle suppression anisotropic diffusion (SRAD) filtering to suppress the local texture detail changes caused by speckle noise while maintaining the main breast tumor morphological information, and eliminate the mixed noise in the process of ultrasound image digitization, wherein the diffusion coefficient and edge detection operator in the filtering are:

[0142]

[0143]

[0144]

[0145] Where c(i,j,t) is the diffusion coefficient, is the gradient operator, q(i,j,t) is the edge detection operator, q0(t) is the diffusion threshold at time t, var is the variance of the grayscale in a uniform area when the number of iterations is t, is the mean grayscale value in a uniform area when the number of iterations is t.

[0146] The "contrast enhancement" is specifically as follows: using the histogram equalization method, the grayscale probability density function and the cumulative probability distribution function are calculated according to the image grayscale value, and then the cumulative probability distribution function is normalized to the original grayscale value range to obtain the grayscale conversion function. This method adjusts the grayscale histogram of the original breast ultrasound image from a relatively dense interval to a uniform distribution within the full grayscale range, thereby enhancing the overall contrast of the image.

[0147] Preferably, the region of interest (ROI) is obtained by: the doctor specifies a region in the pre-processed image as the region of interest. Figure 6 By obtaining the region of interest specified by the doctor, the suspicious lesion area can be processed accurately, and the consumption of computing resources caused by invalid data processing can be reduced.

[0148] Preferably, the three processes in the vector acquisition module are specifically as follows:

[0149] The first processing: extract texture features from the ultrasound image, the texture features include the local binary pattern (LBP), histogram of oriented gradients (HOG) and gray-level co-occurrence matrix (GLCM), and then perform normalization to obtain a texture feature vector; the LBP operator is defined as a 3×3 window. Assume that the coordinates of the center pixel of the window are (x, y), compare the grayscale values ​​of the eight surrounding neighborhood pixels with the grayscale value of the center pixel. If the value is greater than the center pixel, the pixel is marked as 1; otherwise, the pixel is marked as 0. Finally, an ordered 8-bit binary number (i.e., a 256-bit decimal) is generated to reflect the texture information of the local area. The specific calculation formula is: Among them, s(x) is the sign function, i c is the gray value of the center pixel, i p is the gray value of the neighborhood pixel. When extracting GLCM features, the gray level is set to 64. The distance between pixels is adjusted within the range of [1,10], and the relationship between pixels with a certain distance is calculated from four directions (0, 45, 90, 135). Finally, 40 different matrices are obtained from each image. There are 14 specific parameters for texture feature extraction based on GLCM. This embodiment selects four texture features: energy, contrast, correlation and homogeneity. Normalization is the use of discrete normalization to linearly transform the original data and map the data values ​​between them. The specific calculation method is as follows: Among them, min is the minimum value and max is the maximum value.

[0150] The second processing method is to extract deep residual neural network features from the region of interest, wherein the deep residual neural network features include extracting deep learning features from the pooling layer Pool5 using Resnet101, and then performing a normalization operation to obtain a deep learning feature vector;

[0151] The third processing method: extracting morphological features of the region of interest, wherein the morphological features include compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, and minimum rectangular frame area surrounding the breast tumor, and then normalizing the morphological features to obtain a morphological feature vector.

[0152] In this embodiment, the doctor first manually extracts the morphological contour of the breast tumor lesion in the ultrasound image (such as Figure 6As shown), by obtaining the morphological contour extracted by the doctor, thirteen breast tumor morphological characteristics are extracted according to the morphological contour, including compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, and minimum rectangular frame area surrounding the breast tumor.

[0153] The compactness is used to measure the similarity between the shape of the breast tumor and its fitting circle. The calculation formula of the compactness C is as follows: Among them, A n represents the breast tumor area, P n represents the circumference of breast tumor, r represents the radius of the fitted circle;

[0154] The ellipse compactness is the ratio of the circumference of the fitted ellipse to the circumference of the original breast tumor contour. The calculation formula of the ellipse compactness EC is as follows: Among them, a represents the semi-major axis of the fitted ellipse, b represents the semi-minor axis of the fitted ellipse, and P n represents the circumference of breast tumor;

[0155] The radial distance spectrum is to quantify the degree of roughness of the edge of the breast tumor by statistically analyzing the radial distance from each point on the edge of the breast tumor to the center of the breast tumor. The calculation formula of the radial distance is as follows: Where (x1, y1) is the coordinate of the edge point of the breast tumor, and (x0, y0) is the coordinate of the center point of the breast tumor;

[0156] The convexity is a function that measures the ratio of the convex hull to the original contour shape of the breast tumor. The calculation formula of the convexity is as follows: Where P c is the perimeter of the convex hull, P n is the circumference of the breast tumor;

[0157] The solidity is used to describe the convexity or concavity of the shape of a breast tumor. The calculation formula of the solidity is as follows: Among them A n is the breast tumor area, A c is the area of ​​the convex hull;

[0158] The elongation is the ratio between the length and width of the bounding box around the lesion, and the calculation formula of the elongation is as follows: Where W n is the width of the breast tumor, L n is the length of the breast tumor;

[0159] The compactness is the ratio of the breast tumor area to the square of the perimeter. The calculation formula of the compactness is as follows: Where P nis the circumference of breast tumor, A n is the breast tumor area;

[0160] The rectangularity is defined as the ratio between the area of ​​the breast tumor and the area of ​​the minimum bounding rectangular frame of the breast tumor. The calculation formula of the rectangularity is as follows: Among them A n is the breast tumor area, A r is the area of ​​the smallest rectangular frame of the breast tumor;

[0161] The roundness is the ratio of the breast tumor area to the square of the minimum convex hull perimeter of the breast tumor boundary. The calculation formula of the roundness is as follows: Among them A n is the breast tumor area, P c is the perimeter of the convex hull;

[0162] The breast tumor perimeter is the number of pixels within the breast tumor boundary, and the calculation formula of the breast tumor perimeter is as follows: Among them, X i , …, X n are the boundary coordinates of the breast tumor.

[0163] The breast tumor area is the space occupied by the breast tumor on the plane. The area here is defined as the number of pixels within the breast tumor region.

[0164] Preferably, in this embodiment, the method used for dimensionality reduction of texture features and neural network features is principal component analysis (PCA). PCA is a technique for analyzing and simplifying data sets. It is usually used to reduce the dimension of a data set while keeping the features that contribute most to variance in a data set. Here, PCA is used to reduce the dimension of the texture feature vector matrix, while retaining the most effective feature vectors. First, for a given sample space X m×n , calculate the covariance matrix, the eigenvalues ​​and eigenvectors of the matrix; finally, arrange the eigenvectors into a matrix from top to bottom according to the size of the corresponding eigenvalues, and the matrix composed of the first ω rows is the data after dimensionality reduction. Specifically, the vector fusion module is specifically:

[0165] Get the m-dimensional texture feature vectors of n ultrasound images to form a texture feature matrix X m×n , find the mean of each m-dimensional texture feature vector in the texture feature matrix, and then subtract the corresponding mean from all m-dimensional texture feature vectors to obtain the texture feature matrix after removing the average value;

[0166] Calculate the covariance matrix of the texture feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction;

[0167] Sort the eigenvectors by eigenvalue and keep the largest first ω eigenvalues ​​(ω <m);

[0168] The data in the texture feature matrix is ​​converted into a new space constructed by the feature vector to obtain n texture feature vectors after dimensionality reduction.

[0169] Get the m-dimensional deep residual network feature vectors of n ultrasound images to form a deep learning feature matrix X m×n , find the mean of each m-dimensional deep learning feature vector in the deep learning feature matrix, and then subtract the corresponding mean from all m-dimensional deep learning feature vectors to obtain the deep learning feature matrix after removing the average value;

[0170] Calculate the covariance matrix of the deep learning feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction;

[0171] Sort the eigenvectors by eigenvalue and keep the largest first ω eigenvalues ​​(ω <m);

[0172] Convert the data in the deep learning feature matrix to the new space constructed by the feature vector to obtain n deep learning feature vectors after dimensionality reduction;

[0173] The deep learning features after dimensionality reduction and the texture features after dimensionality reduction are cascaded to obtain the fusion features after the deep learning features and the shallow texture cascade. In this embodiment, the processed texture features are linearly fused with the corresponding neural network features, that is, the texture feature vector and the neural network feature vector of the same breast tumor ultrasound image are connected in series so that the next step classifier can learn the breast tumor features.

[0174] In this embodiment, the fusion vector data after linear fusion of breast tumor ultrasound image texture features and neural network features is trained by classifier learning. The support vector machine (SVM) algorithm can be used to learn the fusion vector data, or the naive Bayes (NB) algorithm can be used to learn the fusion vector data. Of course, it is not limited to this algorithm, and other machine learning algorithms can also be used, such as: linear discriminant analysis (LDA), K nearest neighbor (KNN), etc. SVM classifier is a generalized linear classifier that performs binary classification on data in a supervised learning manner. Its decision boundary is to solve the maximum margin hyperplane for the learning sample, and the kernel function is used for nonlinear classification. The input data is mapped to a high-dimensional space to construct the optimal separation hyperplane.

[0175] like Figure 4As shown, in this embodiment, the breast tumor ultrasound image to be classified is input into the model obtained through training to obtain a predicted benign or malignant classification result of the breast tumor. The processing steps of the original breast tumor ultrasound image to be classified are similar to the processing steps of the original breast tumor ultrasound image with classification labels mentioned above, but it does not carry a classification label. Instead, the breast tumor ultrasound image to be classified is obtained after processing, and then input into the breast tumor classification model. It is classified by two types of classifiers, and a score is obtained respectively. The classification result is obtained after weighting, and the classification label, i.e., malignant or benign, is obtained.

[0176] The breast tumor ultrasound image to be classified can be obtained by the following steps: preprocessing the original breast tumor ultrasound image to be classified, then obtaining the region of interest, extracting texture features from the region of interest, normalizing the texture features, and then reducing the dimension of the normalized texture features to obtain a texture feature vector; extracting neural network features from the region of interest, normalizing the neural network features, and then reducing the dimension of the normalized neural network features to obtain a neural network feature vector; finally, feature fusion of the texture feature vector and the neural network feature vector. Extracting morphological features from the region of interest, and then normalizing the morphological features to obtain a morphological feature vector.

[0177] Since the device introduced in the second embodiment of the present invention is a device used to implement the method of the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the device based on the method introduced in the first embodiment of the present invention, so it is not described here in detail. All devices used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0178] Based on the same inventive concept, the present application provides a storage medium corresponding to Example 1, see Example 3 for details.

[0179] Embodiment 3

[0180] This embodiment provides a computer-readable storage medium, such as Figure 3 As shown, a computer program is stored thereon, and when the computer program is executed by a processor, any implementation method in Example 1 can be implemented.

[0181] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0183] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0185] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages: by combining deep features and multiple shallow features with multiple classifiers for the designated area of ​​interest of the ultrasound image of the breast tumor, and by reducing the dimension of the texture features and fusing them with the morphological features, the features in the ultrasound image of the breast tumor can be extracted more comprehensively, and from the perspective of geometric morphology, calculation formulas related to the morphological features of the breast tumor are listed, so that the classification model can better learn the morphological features of the breast tumor; it can effectively improve the accuracy and robustness of the prediction of the benign and malignant nature of the breast tumor in the ultrasound image, help doctors improve the accuracy of the diagnosis of the benign and malignant nature of the breast tumor, reduce the misdiagnosis and missed diagnosis caused by subjective factors, and at the same time improve the diagnostic efficiency of doctors.

[0186] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for classifying benign and malignant breast tumors based on ultrasound images, characterized in that: The method comprises: Step 10: preprocessing the original breast tumor ultrasound image with classification labels to obtain a preprocessed image, wherein the classification labels include malignant breast tumor and benign breast tumor; Step 20, obtaining a region of interest in the preprocessed image; Step 30, performing three types of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively; Step 40: perform dimensionality reduction processing on the deep residual network feature vector, perform dimensionality reduction processing on the texture feature vector, and then perform feature fusion on the residual network feature vector and the texture feature vector after the dimensionality reduction processing to obtain fused vector data; Step 50: using a support vector machine classifier to learn a certain amount of fusion vector data, and using a naive Bayes classifier to learn the morphological feature vector, and using a maximum random search method to weight the two classifiers to obtain a tumor classification model; Step 60, inputting the breast tumor ultrasound image to be classified into the breast tumor classification model to obtain a classification result; The three processes in step 30 are specifically as follows: The first processing method is to extract texture features from the ultrasound image, wherein the texture features include image local binary patterns, directional gradient histograms, and gray-level co-occurrence matrices, and then perform normalization operations to obtain texture feature vectors; The second processing method is to extract deep residual neural network features from the region of interest, wherein the deep residual network features include extracting deep learning features from the pooling layer Pool5 using Resnet101, and then performing a normalization operation to obtain a deep residual network feature vector; The third processing method: extracting morphological features from the region of interest, the morphological features including compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, minimum rectangular frame area surrounding the breast tumor, and then normalizing the morphological features to obtain a morphological feature vector; The step 40 further comprises: Step 41: Obtain the m-dimensional texture feature vectors of n ultrasound images to form a texture feature matrix X m×n , find the mean of each m-dimensional texture feature vector in the texture feature matrix, and then subtract the corresponding mean from all m-dimensional texture feature vectors to obtain the texture feature matrix after removing the average value; Step 42, calculating the covariance matrix of the texture feature matrix after removing the average value, diagonalizing the eigenvalues ​​and eigenvectors of the covariance matrix so that the elements other than the diagonal are 0, and the energy is attributed to the main direction; Step 43, sort the eigenvectors according to the size of the eigenvalues, and retain the largest first ω eigenvalues; Step 44, converting the data in the texture feature matrix into a new space constructed by the feature vector to obtain n texture feature vectors after dimensionality reduction; Step 45: Obtain the m-dimensional deep residual network feature vectors of n ultrasound images to form a deep learning feature matrix X m×n , find the mean of each m-dimensional deep learning feature vector in the deep learning feature matrix, and then subtract the corresponding mean from all m-dimensional deep learning feature vectors to obtain the deep learning feature matrix after removing the average value; Step 46: Calculate the covariance matrix of the deep learning feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction; Step 47, sort the eigenvectors according to the size of the eigenvalues, and retain the largest first ω eigenvalues; Step 48: convert the data in the deep learning feature matrix into a new space constructed by the feature vector to obtain n deep learning feature vectors after dimensionality reduction; Step 49: Cascade the deep learning features after dimensionality reduction and the texture features after dimensionality reduction to obtain a fusion feature after the deep learning features and the shallow texture are cascaded.

2. The method for classifying benign and malignant breast tumors based on ultrasound images according to claim 1, characterized in that: The preprocessing includes removing text and personal information around the ultrasound image, reducing speckle noise and enhancing contrast.

3. The method for classifying benign and malignant breast tumors based on ultrasound images according to claim 1, characterized in that: The compactness is used to measure the similarity between the shape of the breast tumor and its fitting circle. The calculation formula of the compactness C is as follows: Among them, A n represents the breast tumor area, P n represents the circumference of breast tumor, r represents the radius of the fitted circle; The ellipse compactness is the ratio of the circumference of the fitted ellipse to the circumference of the original breast tumor contour. The calculation formula of the ellipse compactness EC is as follows: Where a represents the semi-major axis of the fitted ellipse, and b represents the semi-minor axis of the fitted ellipse; The radial distance spectrum is to quantify the degree of roughness of the edge of the breast tumor by statistically analyzing the radial distance from each point on the edge of the breast tumor to the center of the breast tumor. The calculation formula of the radial distance is as follows: Where (x1, y1) is the coordinate of the edge point of the breast tumor, and (x0, y0) is the coordinate of the center point of the breast tumor; The convexity is a function that measures the ratio of the convex hull to the original contour shape of the breast tumor. The calculation formula of the convexity is as follows: Where P c is the perimeter of the convex hull; The solidity is used to describe the convexity or concavity of the shape of a breast tumor. The calculation formula of the solidity is as follows: Among them A c is the area of ​​the convex hull; The elongation is the ratio between the length and width of the bounding box around the lesion, and the calculation formula of the elongation is as follows: Where W n is the width of the breast tumor, L n is the length of the breast tumor; The compactness is the ratio of the breast tumor area to the square of the perimeter. The calculation formula of the compactness is as follows: The rectangularity is defined as the ratio between the area of ​​the breast tumor and the area of ​​the minimum bounding rectangular frame of the breast tumor. The calculation formula of the rectangularity is as follows: Among them A r is the area of ​​the smallest rectangular frame of the breast tumor; The circularity is the ratio of the breast tumor area to the square of the minimum convex hull perimeter of the breast tumor boundary. The circularity calculation formula is as follows: The breast tumor perimeter is the number of pixels within the breast tumor boundary, and the calculation formula of the breast tumor perimeter is as follows: Among them, X i , …, X n are the boundary coordinates of the breast tumor; The breast tumor area is the number of pixels within the breast tumor region.

4. A device for classifying benign and malignant breast tumors based on ultrasound images, characterized in that: The device comprises: A preprocessing module, used for preprocessing an original breast tumor ultrasound image with classification labels to obtain a preprocessed image, wherein the classification labels include malignant breast tumor and benign breast tumor; ROI acquisition module, used to obtain the region of interest in the preprocessed image; The vector extraction module is used to perform three kinds of processing on the region of interest to obtain a deep residual network feature vector, a texture feature vector and a morphological feature vector respectively; the three kinds of processing in the vector extraction module are as follows: The first processing method is to extract texture features from the ultrasound image, wherein the texture features include image local binary patterns, directional gradient histograms, and gray-level co-occurrence matrices, and then perform normalization operations to obtain texture feature vectors; The second processing method is to extract deep residual neural network features from the region of interest, wherein the deep residual network features include extracting deep learning features from the pooling layer Pool5 using Resnet101, and then performing a normalization operation to obtain a deep residual network feature vector; The third processing method: extracting morphological features from the region of interest, the morphological features including compactness, ellipse compactness, radial distance spectrum parameters, convexity, solidity, elongation, compactness, rectangularity, roundness, breast tumor perimeter, breast tumor area, minimum rectangular frame perimeter surrounding the breast tumor, minimum rectangular frame area surrounding the breast tumor, and then normalizing the morphological features to obtain a morphological feature vector; The vector fusion module is used to perform dimensionality reduction processing on the deep residual network feature vector and the texture feature vector, and then perform feature fusion on the residual network feature vector and the texture feature vector after dimensionality reduction processing to obtain fused vector data; the vector fusion module is specifically: Get the m-dimensional texture feature vectors of n ultrasound images to form a texture feature matrix X m×n , find the mean of each m-dimensional texture feature vector in the texture feature matrix, and then subtract the corresponding mean from all m-dimensional texture feature vectors to obtain the texture feature matrix after removing the average value; Calculate the covariance matrix of the texture feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction; Sort the eigenvectors by eigenvalue and keep the largest first ω eigenvalues; The data in the texture feature matrix is ​​converted into a new space constructed by the feature vector to obtain n texture feature vectors after dimensionality reduction; Get the m-dimensional deep residual network feature vectors of n ultrasound images to form a deep learning feature matrix X m×n , find the mean of each m-dimensional deep learning feature vector in the deep learning feature matrix, and then subtract the corresponding mean from all m-dimensional deep learning feature vectors to obtain the deep learning feature matrix after removing the average value; Calculate the covariance matrix of the deep learning feature matrix after removing the average value, diagonalize the eigenvalues ​​and eigenvectors of the covariance matrix so that all elements except the diagonal are 0, and attribute the energy to the main direction; Sort the eigenvectors by eigenvalue and keep the largest first ω eigenvalues; Convert the data in the deep learning feature matrix to the new space constructed by the feature vector to obtain n deep learning feature vectors after dimensionality reduction; Cascading the deep learning features after dimensionality reduction and the texture features after dimensionality reduction to obtain the fusion features after the deep learning features and the shallow texture cascade; A learning module, for learning a certain amount of fusion vector data using a support vector machine classifier, and learning a morphological feature vector using a naive Bayes classifier, and obtaining a tumor classification model by weighting the two classifiers using a maximum random search method; and The classification module is used to input the breast tumor ultrasound image to be classified into the breast tumor classification model to obtain the classification result.

5. The device for classifying benign and malignant breast tumors based on ultrasound images according to claim 4, characterized in that: The preprocessing includes removing text and personal information around the ultrasound image, reducing speckle noise and enhancing contrast.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images