A microwave breast tumor classification method based on deep learning

Abstract

The present invention discloses a microwave breast tumor classification method based on deep learning, which is used to solve breast tumor localization by constructing a domain adversarial neural network, re-designating it as a soft classification problem, and redefining the loss function. Compared with a real-valued multi-layer perceptron, the domain adversarial neural network uses only 250 target domain data to adapt the neural network to a new domain with significantly different data distribution, and obtains reasonable results for unknown data in laboratory experiments.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a microwave breast tumor classification method based on deep learning. Background Art

[0002] In breast tumor detection, determining the size and location of the tumor is crucial for preventive diagnosis. Therefore, a fast and accurate method for tumor localization and size estimation is very important. Although magnetic resonance imaging (MRI) and computed tomography (CT) serve contemporary medical needs, they are expensive, bulky and heavy, making them unsuitable for early diagnosis of breast tumors.

[0003] Electromagnetic imaging has great potential as a complement to existing imaging modalities. Electromagnetic imaging uses an antenna array around the breast to measure transmission and reflection coefficients. These complex-valued parameters can be processed using various techniques, such as tomography and radar-based techniques, to complete imaging-based detection, localization, and classification. Tomography requires forward and inverse solvers, which take a long time to calculate tens of thousands of unknowns, and also requires accurate electromagnetic simulation tools and expensive computing hardware. On the other hand, radar-based techniques are difficult to implement in heterogeneous tissues and usually cannot provide classification. Machine learning can make up for these shortcomings and provide almost real-time results. Deep convolutional neural networks are a very good choice for handling such highly nonlinear and complex tasks. However, a major obstacle is the amount of data required to train the network. The technology is still in its infancy and has not yet been put into use. It requires the use of data in a simulated environment. Although simulations require huge computing power, they are still a very practical solution for obtaining basic training data. However, results based on full simulations may encounter difficulties in practical applications. Summary of the invention

[0004] The present invention overcomes the deficiencies of the prior art and aims to solve the following technical problems: the difficulty that may be encountered in practical applications of a network based on complete simulation training, and the technical problems of large amount of calculation and inability to perform real-time imaging in traditional methods.

[0005] The technical solution adopted by the present invention to solve the technical problem is: constructing a microwave breast tumor classification method based on deep learning, including:

[0006] Breast tumor cases are created by placing tumors of different sizes at random locations in healthy breasts, and synthetic data sets are created by finite difference time domain simulation of the created breast tumor cases to form a source domain data set;

[0007] Multiple printed monopole antennas are arranged at equal angles in an outer ring of a first diameter to form a transmitter array, a corresponding number of printed monopole antennas are arranged at equal angles in an inner ring of a second diameter to form a receiver array, and a uniform breast model of a third diameter is placed at the center of the ring; a dual-port vector network analyzer is used to collect dual-base data to obtain a real tensor target domain data set;

[0008] Build a domain adversarial neural network, establish a mapping between the source domain dataset and the target domain dataset, and train the constructed domain adversarial neural network by establishing the mapped source domain dataset and target domain dataset until the output result is consistent with the breast tumor type actually represented by the dataset;

[0009] The real-time microwave breast tumor image is input into the trained domain adversarial neural network, and the output result is used as the breast tumor classification result.

[0010] The domain adversarial neural network consists of three parts: feature extractor, label predictor and domain classifier. The feature extractor maps the data to the feature space, so that the label predictor can distinguish the label of the source domain data, and the domain classifier cannot distinguish whether the data comes from the source domain or the target domain.

[0011] Domain adversarial neural networks can only classify source domain data sets. In order to achieve the classification task of target domain data sets, the domain adversarial neural network must regard the target domain data as source domain data. There are two tasks in the training phase. The first is to accurately classify the source domain data set and minimize the label prediction error. The second task is to confuse the source domain data set and the target domain data set to maximize the domain classification error.

[0012] Among them, the domain classifier has two loss functions: one is binary cross entropy, and the other is confusion domain classifier; the label predictor and domain classifier compete with each other during the training process and finally achieve a balance between image label prediction loss and domain classification loss.

[0013] Among them, a soft classification method is used to discretize the breast tumor area into 41 units, creating a 41-dimensional vector, each entry corresponds to a cell and contains the amount of tumor area located in that cell, which is then normalized to create a valid distribution.

[0014] Among them, Kullback-Lerbler (KL) divergence is used as the accuracy metric for soft classification problems, which measures the additional information gained due to the divergence from one distribution to another.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The present invention provides a microwave breast tumor classification method based on domain adversarial neural network. When there is an offset between the training distribution and the test distribution, the method tries its best to map between the source domain and the target domain so that the classifier learned in the source domain can also be applied to the target domain. Compared with the traditional method for solving the microwave breast electromagnetic inverse scattering problem, the method can realize real-time classification with small calculation amount. Compared with the previous neural network, the method can apply 2D breast tumor simulation classification in a 3D experimental environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0018] Figure 1 It is a flowchart of a microwave breast tumor classification method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described in addition are only used to explain the present invention, but are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without making creative work will fall within the scope of protection of the present invention.

[0020] See attached Figure 1 The present invention provides a microwave breast tumor classification method based on deep learning, comprising:

[0021] Breast tumor cases are created by placing tumors of different sizes at random locations in healthy breasts, and synthetic data sets are created by finite difference time domain simulation of the created breast tumor cases to form a source domain data set;

[0022] Multiple printed monopole antennas are arranged at equal angles in an outer ring of a first diameter to form a transmitter array, a corresponding number of printed monopole antennas are arranged at equal angles in an inner ring of a second diameter to form a receiver array, and a uniform breast model of a third diameter is placed at the center of the ring; a dual-port vector network analyzer is used to collect dual-base data to obtain a real tensor target domain data set;

[0023] Build a domain adversarial neural network, establish a mapping between the source domain dataset and the target domain dataset, and train the constructed domain adversarial neural network by establishing the mapped source domain dataset and target domain dataset until the output result is consistent with the breast tumor type actually represented by the dataset;

[0024] The real-time microwave breast tumor image is input into the trained domain adversarial neural network, and the output result is used as the breast tumor classification result.

[0025] The domain adversarial neural network consists of three parts: feature extractor, label predictor and domain classifier. The feature extractor maps the data to the feature space, so that the label predictor can distinguish the label of the source domain data, and the domain classifier cannot distinguish whether the data comes from the source domain or the target domain.

[0026] Domain adversarial neural networks can only classify source domain data sets. In order to achieve the classification task of target domain data sets, the domain adversarial neural network must regard the target domain data as source domain data. There are two tasks in the training phase. The first is to accurately classify the source domain data set and minimize the label prediction error. The second task is to confuse the source domain data set and the target domain data set to maximize the domain classification error.

[0027] This paper uses a domain adaptation technique to bridge the gap between applying deep neural networks to synthetic and real data in electromagnetics. The technique allows relying on computer simulated data while adapting it to real-world environments. The proposed method establishes a mapping between the source and target domains so that when combined with a learned mapping between the domains, a classifier learned in the source domain can also be applied to the target domain.

[0028] Breast tumor cases were created by placing tumors of different sizes at different locations in the healthy breast. A synthetic dataset was created using finite-difference time-domain simulations of 9,000 breast cases with tumors, which constituted the source domain dataset.

[0029] The target domain dataset is obtained from real experiments. 16 printed monopole antennas are placed at equal angles in an outer ring with a diameter of 500 mm and an inner ring with a diameter of 300 mm to form a transmitter array and a receiver array. A uniform breast model with a diameter of 100 mm can be placed in the center of the ring. A two-port vector network analyzer is used to collect bistatic data to obtain the target domain dataset.

[0030] In the present invention, instead of minimizing the classifier's trust in the correct domain label, the trust in the wrong label is maximized. In this setting, a soft classification method is used to discretize the breast tumor region into 41 cells, create a 41-dimensional vector, each entry corresponds to a cell and contains how much of the tumor region is located in that cell, and then normalize to create a valid distribution.

[0031] Among them, Kullback-Lerbler (KL) divergence is used as the accuracy metric for soft classification problems, which measures the additional information gained due to the divergence from one distribution to another.

[0032] Specifically, in order to create synthetic scattered field data for training the neural network, a magnetic resonance imaging-derived numerical breast model was first created by converting each pixel value in the tomographic slice of the MRI breast image into a corresponding complex-valued dielectric constant. As the base model, a total of 9 magnetic resonance imaging-derived breast models of four different types were used.

[0033] These healthy breast models come from an online repository developed and maintained by the Computational Electromagnetism Laboratory at the University of Wisconsin. Each model consists of a 3D grid of cubic voxels, where each voxel is 0.5mm×0.5mm×0.5mm. The breast model includes a skin layer approximately 1.5 mm thick, a subcutaneous fat layer 1.5 cm thick at the bottom of the breast, and a muscular chest wall 0.5 cm thick. It was created from 50 2D slices extracted from 9 different real 3D breast models, providing 450 different 2D breast slices.

[0034] In order to generate a sufficient amount of data with various tumors to train the neural network, the dielectric constant values ​​at the pixels where the tumors were present in the 9 breast models were first set to the dielectric constant values ​​of the fibroglandular tissue. Then, for 50 2D slices of each breast model, 1000 models were generated in which one or two tumors occupied the fibroglandular area. 500 models had one tumor and the other 500 models consisted of two tumors. The generated tumors had a maximum diameter of approximately 1.1-1.5 cm and were randomly located in the fibroglandular area. Therefore, the dataset consists of 9000 breast models, each with 500 single and double tumor images.

[0035] In terms of real data, a homogeneous medium model was used.

[0036] When constructing the homogeneous breast model, the breast was treated as a complete fat layer with a tumor inside. The materials and mixing ratios of the homogeneous breast model, tumor, and skin are listed in Table 1. For the homogeneous breast model, sodium chloride, polyethylene powder, xanthan gum, sodium dehydroacetate monohydrate, and distilled water were used. Polyethylene powder was used to adjust the dielectric constant, and sodium chloride was used to adjust the conductivity. Agar was used to maintain the shape of the model by preventing the separation of water content, xanthan gum was used as a thickener, and sodium dehydroacetate monohydrate was used as a preservative. Basically, the materials that change the dielectric properties of the model are sodium chloride, polyethylene powder, agar, and distilled water, which are the main components of the model. The radius of the breast model is 50 mm, and the tumor diameter is 10 mm, located at a random point inside the breast.

[0037] The dielectric constant of fat is 12.58 and the conductivity is 0.141S / m. The dielectric constant of tumor is 57.37. Tumor has a higher dielectric constant due to its higher water content. A 3 mm surface layer was also prepared, and the dielectric properties of the surface material were measured to be 23.44. The dielectric constant of the muscle-like material placed under the breast model for analyzing body effects was close to 44.

[0038]

[0039] Table 1 List of materials and mixing ratios used in uniform phantoms

[0040] The experimental setup uses printed monopole antennas. The outer ring where the transmitter is located has a diameter of 500 mm. 16 printed monopole antennas are placed at equal angles. The inner ring where the receiver is located has a diameter of 300 mm. There are also 16 printed monopole antennas placed at equal angles. A uniform breast model with a diameter of 100 mm can be placed in the center. A two-port vector network analyzer is used to collect bistatic data. The synthetic data of the array is obtained by manually switching the receiver to each of the 16 positions. Measurement process: Connect the transmitter and receiver to port 1 and port 2 of the vector network analyzer respectively. Change the receiving antenna of port 2. After a complete scan of 15 positions, move the transmitting antenna to the next position and repeat the process.

[0041] The entire data set forms a tensor of size 250 × 16 × 16 × 2 (2 represents the real and imaginary parts of the complex signal). At an operating frequency of 3 GHz, the data set is a 16 × 16 complex-valued S-parameter matrix containing the transmission and reflection coefficients of a multi-statically operated 16-channel microwave array consisting of dielectric-loaded waveguides.

[0042] In electromagnetics, accurately modeling reality with simulation is not simple or easy to achieve, because there are countless complex factors that need to be considered based on transmission lines and physical theory. Because a deep convolutional network trained on simulated data may fail on real data. In computer vision, when tested on images from different datasets, a neural network trained on one dataset may perform drastically worse on another dataset, even if the two datasets have the same objects. Domain adaptation algorithms are dedicated to solving this problem.

[0043] Learning a discriminative classifier or other predictor in the presence of a shift between the training and test distributions is called domain adaptation. The proposed domain adversarial neural network establishes a mapping between the source and target domains so that when combined with the learned mapping between the domains, the classifier learned in the source domain can also be applied to the target domain.

[0044] In the most common domain adaptation scenario, the first domain has more labeled data (called the source domain), while the domain of interest has less data (called the target domain). In our setting, the source domain is synthetic simulation data, and the target domain is the real scene in the laboratory. A domain adversarial neural network (DANN) is adopted. The first part is the DANN network; the second part is to redefine the loss function; the third part is to use soft binning as a classifier; the fourth part is the accuracy metric for soft classification problems.

[0045] The basic structure of DANN consists of three parts: Feature Extractor (FE), Label Predictor (LP) and Domain Classifier (DC).

[0046] Feature mapper Gf: maps the data into feature space so that Gy can distinguish the label of the source domain data, but Gd cannot distinguish whether the data comes from the source domain or the target domain.

[0047] Label predictor Gy: Classifies the source domain data in the feature space and identifies the correct label as much as possible.

[0048] Domain classifier Gd: performs domain classification on the data in the feature space, trying to identify which domain the data comes from.

[0049] In DANN training, the network input is a source domain dataset with image classification labels and a target domain dataset without image classification labels, as well as domain classification labels of the source and target domain datasets. DANN can only classify the source domain dataset. To achieve the classification task of the target domain dataset, DANN must treat the target domain data as source domain data. There are two tasks in the training phase. The first is to accurately classify the source domain dataset and minimize the label prediction error; the second task is to confuse the source domain dataset and the target domain dataset to maximize the domain classification error. The label predictor and the domain classifier compete with each other during the training process and finally achieve a balance between the image label prediction loss and the domain classification loss.

[0050] The input of DANN is x∈X, where X represents the image input space, and the classification label y∈Y, where Y (Y={1,2,3,...,k}) represents the classification label space. There are two data distributions in DANN: source domain data distribution S(x,y) and target domain data distribution Τ(x,y). The goal of DANN is to accurately predict the classification label of the input image in the target domain.

[0051] The training samples are {x1,x2,...,x N} are from the marginal distribution S(x) and Τ(x) of the source domain and the target domain respectively. At the same time, define d iis the domain label of the i-th training sample, where d i ∈{0,1}. If d i =0 then x i ~S(x), otherwise d i =1 then x i ~T(x).

[0052] During the training phase, the image input x of DANN first passes through the feature extraction network f = G f (x;θ f ) is converted into a D-dimensional feature vector, that is, f∈R D DANN is divided into two branches, namely the label prediction network G y (x;θ y ) and domain classification network G d (x;θ d ). The feature vector corresponding to the source domain data input will pass through G y (x;θ y ) to obtain the corresponding label prediction result. Whether it is the feature vector of the source domain input or the target domain input, it will pass through G d (x;θ d ) to obtain the domain classification result for each input.

[0053] The SGD algorithm is used to optimize the DANN model parameters. The gradient update formula of the DANN model parameters is as follows:

[0054]

[0055]

[0056]

[0057] As can be seen from the following two formulas, the inputs of the domain classifier and the label predictor both come from the feature extractor, but the goal of the domain classifier is to maximize the domain classification loss and confuse the target domain data with the source domain data, but the goal of the label predictor is to minimize the loss of label prediction.

[0058] The learning rate also changes with the iteration process. The transformation formula is as follows

[0059]

[0060] Where μ0 is the initial learning rate, which is 0.01. p represents the relative value of the iteration process, that is, the ratio of the current number of iterations to the total number of iterations. α and β are hyperparameters, α = 10, β = 0.75.

[0061] Redefine the loss function

[0062] In this invention, instead of minimizing the classifier's trust in the correct domain labels, we maximize the trust in the wrong labels. In this setting, the domain classifier will have two loss functions. The first is the commonly used binary cross entropy, which tries to make the classifier better. The second loss function tries to confuse the domain classifier.

[0063]

[0064] N is the batch size, h(x i ) is the input x i The classifier output, y i is the corresponding label.

[0065] For the feature extractor, sigmoid is used as the activation function and the output is

[0066] G f (x; W, b) = sigm(Wx+b)

[0067] For the label predictor, softmax is used as the activation function and the output is

[0068] G f (G f (x); V,c) = softmax(VG f (x)+c)

[0069] When a data point (x i ,y i ), with negative log-likelihood as the loss function, the loss of the label predictor is

[0070]

[0071] Therefore, in the source domain, the optimization goal of training is

[0072]

[0073] in, represents the label prediction loss of the i-th sample, R(W,b) is a regularizer, λ is an artificially set regularization parameter, and λ·R(W,b) is used to prevent overfitting of the neural network.

[0074] Since the confounding loss formulation is structurally adversarial, it eliminates the need for gradient reversal layers commonly used in DANNs.

[0075] Then the total loss function to be minimized becomes:

[0076] Loss total =Loss LP source+loss LP target+λLossconfusion

[0077] The domain classifier loss is completely separated from the overall loss, so there can be a separate optimizer operating on its weights.

[0078] According to the actual situation, it is reformulated as a soft classification problem. To achieve this, the imaging domain is discretized into 41 cells, which provides sufficient resolution for positioning. For each breast tumor case, a breast tumor region is constructed according to its parameters. Then, a 41-dimensional vector is created, each entry corresponds to a cell and contains how much of the tumor region is located in that cell. Normalization is then required to create a valid distribution. This technique is called "soft binning".

[0079] If we simply use the categorical cross entropy loss itself as a performance metric, it does not reflect the true performance. This makes it difficult to tell whether the neural network is performing well when observing the loss curve. To solve this problem, the Kullback-Lerbler (KL) divergence is used, which measures the additional information gained due to the divergence from one distribution to another.

[0080] KL(P||Q)=CXε(P||Q)-H(P)

[0081] CXε is the categorical cross entropy and H is the entropy measure. From an information theory point of view, KL will measure external information instead of measuring the new entropy of CXε. This subtraction of entropy precisely eliminates the variable bias of the cross entropy, so when the neural network achieves zero loss in an optimal way, instead of returning CXε(P||Q)=CXε(Q||Q)=CXε(P||P), it is equal to the entropy H(P) and H(Q). The former depends on the current data point and is non-zero in most cases. From an optimization point of view, this change will not have any impact, although the entropy is instance-dependent, it is constant relative to the parameters of the neural network.

[0082] In all cases, the 16×16 matrix is ​​reshaped into a 1D vector of dimension 105; the neural network receives the stacked real and imaginary parts of the S matrix,

[0083] In domain adaptation, the ratio of training set and test set in the source domain dataset is 1:10, and 80% of the target domain is used for training and 20% for testing. The batch size is set to 32. The adversarial effect is gradually introduced into the training procedure to avoid confusing the label predictor and the domain classifier. This is achieved by a training schedule that makes λ grow exponentially from 0 to 1.0.

[0084] Compared with the traditional real-valued multilayer perceptron, the input is the absolute value of the S parameter, the 16×16 matrix is ​​reshaped into a 105-dimensional one-dimensional vector (the S parameter has a symmetric property and its redundant values ​​are discarded), and the neural network receives the real part of the S matrix.

[0085] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A microwave breast tumor classification method based on deep learning, characterized in that: include: Breast tumor cases are created by placing tumors of different sizes at random locations in healthy breasts, and synthetic data sets are created by finite difference time domain simulation of the created breast tumor cases to form a source domain data set; Creating a synthetic data set by converting each pixel value in a tomographic slice of an MRI breast image into a corresponding complex-valued dielectric constant to create a magnetic resonance imaging-derived numerical breast model, wherein the dielectric constant value at a pixel where a tumor exists in the breast model is set to the dielectric constant value of fibroglandular tissue; Multiple printed monopole antennas are arranged at equal angles in an outer ring of a first diameter to form a transmitter array, a corresponding number of printed monopole antennas are arranged at equal angles in an inner ring of a second diameter to form a receiver array, and a uniform breast model of a third diameter is placed at the center of the ring; a dual-port vector network analyzer is used to collect dual-base data to obtain a real tensor target domain data set; Build a domain adversarial neural network, establish a mapping between the source domain dataset and the target domain dataset, and train the constructed domain adversarial neural network by establishing the mapped source domain dataset and target domain dataset until the output result is consistent with the breast tumor type actually represented by the dataset; The real-time microwave breast tumor image is input into the trained domain adversarial neural network, and the output result is used as the breast tumor classification result.

2. The microwave breast tumor classification method based on deep learning according to claim 1 is characterized in that: The domain adversarial neural network consists of three parts: feature extractor, label predictor and domain classifier. The feature extractor maps the data to the feature space, so that the label predictor can distinguish the label of the source domain data, and the domain classifier cannot distinguish whether the data comes from the source domain or the target domain. Domain adversarial neural networks can only classify source domain data sets. In order to achieve the classification task of target domain data sets, the domain adversarial neural network must regard the target domain data as source domain data. There are two tasks in the training phase. The first is to accurately classify the source domain data set and minimize the label prediction error. The second task is to confuse the source domain data set and the target domain data set to maximize the domain classification error.

3. The microwave breast tumor classification method based on deep learning according to claim 2 is characterized in that: The domain classifier has two loss functions: one is binary cross entropy and the other is confusion domain classifier; the label predictor and domain classifier compete with each other during the training process and finally achieve a balance between image label prediction loss and domain classification loss.

4. The microwave breast tumor classification method based on deep learning according to claim 3 is characterized in that: Using a soft classification method, the breast tumor region is discretized into 41 cells, creating a 41-dimensional vector, where each entry corresponds to a cell and contains how much of the tumor region lies in that cell, and then normalized to create a valid distribution.

5. The microwave breast tumor classification method based on deep learning according to claim 4 is characterized in that: We use the Kullback-Lerbler (KL) divergence as the accuracy metric for soft classification problems, which measures the additional information gained due to the divergence from one distribution to another.

Images