High-quality real-time microwave thermoacoustic imaging method and device based on deep learning

Abstract

The application discloses a high-quality real-time microwave thermoacoustic imaging method and device based on deep learning, and the method is as follows: a COMSOL software is used to obtain a heat absorption distribution map and import the heat absorption distribution map into MATLAB software, and a microwave thermoacoustic reconstruction image and a corresponding label image are obtained; normalization processing and data expansion are performed, and the data are divided into a training set and a verification set according to a set proportion; a microwave thermoacoustic image is collected, and after normalization processing, the microwave thermoacoustic image is set as a test set; the training set is used to train a full-dense generative adversarial neural network model, and after the training is completed, the verification set is used to verify the imaging quality of the model; the weight and bias values after verification and the imaging quality of the model after verification are saved; the test set is used to test the model after verification; the model after testing is imported into a microwave thermoacoustic real-time imaging device, and the microwave thermoacoustic real-time imaging device drives the model to generate a high-quality image. The application can effectively improve the'splitting' artifact problem existing in microwave thermoacoustic imaging and improve the imaging quality.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of microwave thermoacoustic imaging, and particularly relates to a high-quality real-time microwave thermoacoustic imaging method and device based on deep learning. BACKGROUND

[0002] As a new type of non-destructive physical medical imaging method, microwave thermoacoustic imaging combines the advantages of high contrast of microwave imaging and high resolution of ultrasonic imaging, and has rapidly developed in many biomedical fields in the past few decades. Microwave thermoacoustic imaging uses microwave as an excitation source and ultrasonic as an information carrier, and realizes non-destructive and centimeter-depth high-resolution imaging of biological tissues through the transformation of microwave-to-ultrasonic energy transmission.

[0003] Existing image reconstruction methods, including filtered back-projection algorithm and delay-and-sum algorithm, are computationally efficient and robust to implement, but they require sufficiently high spatial and temporal sampling rates of acoustic signals to obtain high-quality sample images, resulting in huge time consumption, and multiple mechanical scans also cause discomfort to the human body. If the ultrasonic transducer array is combined with a multi-channel acquisition system, although the mechanical scanning is eliminated and the total scanning time is reduced, the cost of the entire system is greatly increased. If the number of measurements is significantly reduced, the image quality caused by the filtered back-projection algorithm and the delay-and-sum algorithm will inevitably decrease, and serious artifacts and blurring may occur due to information loss. Therefore, under the premise of not sacrificing imaging quality, the technology of using sparse data for microwave thermoacoustic real-time imaging has important significance for reducing system cost and scanning time, alleviating the discomfort and panic of patients, and promoting the practicality of microwave thermoacoustic imaging in clinical applications.

[0004] A microwave thermoacoustic imaging method based on compressed sensing is proposed in the prior art, which can be used to solve the problem of sparse data, and high-quality images can be obtained with fewer measurements and sparser data sampling; however, the establishment of a structured dictionary as prior knowledge and the solution of the optimization problem related thereto bring huge computational burden. In recent years, with the great success of deep learning methods in the field of medical imaging, compared with traditional microwave thermoacoustic imaging methods, the output images obtained by deep learning methods have better quality and artifact suppression capability, and the fully dense generative adversarial neural network is one of the most promising methods produced by the combination of image processing technology and deep learning technology in recent years. SUMMARY

[0005] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and provide a high-quality real-time microwave thermoacoustic imaging method and device based on deep learning, which can generate microwave thermoacoustic images in real time by quickly scanning the sample, and improve the quality of microwave thermoacoustic imaging, reduce image noise and artifacts, and reduce labor costs by using a full dense generative adversarial neural network model.

[0006] In order to achieve the above purpose, the present application provides a high-quality real-time microwave thermoacoustic imaging method based on deep learning, comprising the following steps:

[0007] The COMSOL software is used to simulate the heat absorption distribution of the biological tissue model under microwave irradiation, and the heat absorption distribution graphs in different directions are obtained;

[0008] The heat absorption distribution graphs are imported into the MATLAB software to obtain multiple microwave thermoacoustic reconstruction images and corresponding label images;

[0009] The microwave thermoacoustic reconstruction images and the corresponding label images are normalized and data augmented, and are divided into a training set and a validation set according to a set proportion;

[0010] The microwave thermoacoustic real-time imaging device is used to collect microwave thermoacoustic images, which are normalized and set as a test set;

[0011] The training set is used to train the full dense generative adversarial neural network model, and the imaging quality of the model is verified after the training is completed using the validation set;

[0012] The weight and bias values of the full dense generative adversarial neural network model after verification and the imaging quality of the model after verification are saved;

[0013] The test set is used to test the full dense generative adversarial neural network model after verification;

[0014] The full dense generative adversarial neural network model after testing is imported into the microwave thermoacoustic real-time imaging device, and the full dense generative adversarial neural network model is driven by the microwave thermoacoustic real-time imaging device to generate high-quality images.

[0015] As a preferred technical solution, the COMSOL software simulates the heat absorption distribution of biological tissue models with different diameters and thicknesses under 3GHz microwave irradiation and coupling oil environment, takes different tomographic depths, and then changes the polarization direction of the electric field to obtain heat absorption distribution graphs in different directions;

[0016] The microwave thermoacoustic reconstruction image is a cross-sectional image with noise and artifacts;

[0017] The label image is a corresponding microwave thermoacoustic reconstruction image without noise and artifacts.

[0018] As a preferred technical solution, each pixel value in the microwave thermoacoustic reconstructed image and the corresponding label image is normalized by dividing by 255;

[0019] The normalized microwave thermoacoustic reconstructed image and the corresponding label image are subjected to image augmentation operation using translation, rotation, stretching, horizontal flipping and patch processing methods; the size of the uniformly augmented image is made consistent in width and height;

[0020] The images are divided according to the ratio of training set:validation set=4:1 to obtain the training set and the validation set.

[0021] As a preferred technical solution, the microwave thermoacoustic image is reconstructed by using a microwave thermoacoustic real-time imaging device under the irradiation of 3GHz microwave and coupling oil environment, and the microwave thermoacoustic image is collected; the microwave thermoacoustic image is a cross-sectional image;

[0022] The collected microwave thermoacoustic image is subjected to normalization processing, and the image is uniformly set to a certain size, which is set as a test set.

[0023] As a preferred technical solution, the fully dense generative adversarial neural network model comprises a generator and a discriminator; the generator comprises an encoder and a decoder;

[0024] The fully dense generative adversarial neural network model comprises convolution layers with a convolution kernel size of 3x3, batch normalization layers, ReLU activation function layers, deconvolution layers with a convolution kernel size of 2x2, convolution layers with a convolution kernel size of 1x1, Sigmoid activation function layers, flattening layers and fully connected layers arranged in sequence.

[0025] As a preferred technical solution, the encoder comprises 21 convolution layers with a convolution kernel size of 3x3, 20 convolution layers with a convolution kernel size of 1x1 and 4 maximum pooling layers with a convolution kernel size of 2x2; the decoder comprises 20 convolution layers with a convolution kernel size of 3x3, 17 convolution layers with a convolution kernel size of 1x1 and 4 deconvolution layers with a convolution kernel size of 2x2;

[0026] The discriminator comprises 5 convolution layers with a convolution kernel size of 4x4.

[0027] As a preferred technical solution, the fully dense generative adversarial neural network model is trained, specifically:

[0028] The parameters of the fully dense generative adversarial neural network model are initialized, and an initial learning rate is set;

[0029] The microwave thermoacoustic reconstructed image with noise artifacts in the training set is input into the generator for forward propagation;

[0030] The output image of the generator and the label image without noise and artifact in the training set are input into the discriminator for forward propagation;

[0031] The mean square error loss function is used to measure the error between the output value of the generator and the label target value, and the cross-entropy loss function is used to measure the error between the output value of the discriminator and the label target value.

[0032] According to the loss function, the adaptive moment estimation optimizer is used to calculate the gradient of the loss function, and the parameters of the fully dense generative adversarial neural network model are updated by back propagation until the loss function converges or the maximum iteration number is reached.

[0033] As a preferred technical solution, the evaluation index of the model imaging quality is the structural similarity SSIM, which is used to measure the consistency of the brightness, contrast and structure of the output image X and the target image Y, and the formula is:

[0034]

[0035] Wherein, μ X is the average value of the output image X, μ Y is the average value of the target image Y, σ X is the standard deviation of the output image X, σ Y is the standard deviation of the target image Y, σ XY is the covariance of the output image X and the target image Y, C1 and C2 are positive constants.

[0036] On the other hand, the application also provides a high-quality real-time microwave thermoacoustic imaging system based on deep learning, which is applied to the high-quality real-time microwave thermoacoustic imaging method based on deep learning, and includes an image simulation module, an image acquisition module, an image preprocessing module, a model obtaining module and a real-time imaging module.

[0037] The image simulation module is used to simulate the heat absorption distribution of a biological tissue model under microwave irradiation using COMSOL software to obtain heat absorption distribution maps in different directions; the heat absorption distribution maps are imported into MATLAB software to obtain multiple microwave thermoacoustic reconstruction images and corresponding label images.

[0038] The image acquisition module is used to acquire microwave thermoacoustic images using a microwave thermoacoustic real-time imaging device.

[0039] The image preprocessing module is used to normalize and data augment the microwave thermoacoustic reconstruction images and corresponding label images, and divide them into a training set and a validation set according to a set proportion; at the same time, the microwave thermoacoustic images are normalized and set as a test set.

[0040] The model obtaining module is configured to train the full dense generative adversarial neural network model by using a training set, and verify the imaging quality of the model by using a verification set after the training is completed; save the weight and bias values of the full dense generative adversarial neural network model after verification, and the imaging quality of the model after verification is completed; and test the full dense generative adversarial neural network model after verification by using a test set.

[0041] The real-time imaging module is configured to import the full dense generative adversarial neural network model after testing into a microwave thermoacoustic real-time imaging device, and drive the full dense generative adversarial neural network model to generate high-quality images by the microwave thermoacoustic real-time imaging device.

[0042] In still another aspect, the present application provides a computer-readable storage medium storing a program, wherein the program is executed by a processor to implement the deep learning-based high-quality real-time microwave thermoacoustic imaging method described above.

[0043] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0044] 1. The deep learning-based high-quality real-time microwave thermoacoustic imaging method improves the resolution of imaging, and only needs to input the collected microwave thermoacoustic images into the neural network model after simple preprocessing to obtain high-quality images. The end-to-end image reconstruction method reduces the cost of human participation, and the reconstructed image has high resolution and removes artifacts and noise in the case of sparse data, overcoming the serious artifacts and blurring caused by information loss in the traditional image reconstruction method in the case of sparse data.

[0045] 2. The deep learning-based high-quality real-time microwave thermoacoustic imaging method can effectively improve the "splitting" artifact phenomenon that occurs when using linearly polarized antenna microwave thermoacoustic image reconstruction, and make the circularly polarized antenna microwave thermoacoustic image reconstruction thermal signal distribution more uniform.

[0046] 3. The present application uses a full dense generative adversarial neural network model to realize noise reduction and artifact removal of microwave thermoacoustic reconstructed images without human interaction and intervention, reducing the cost of artificial labor; and based on deep learning and full dense generative adversarial neural network model, real-time high-quality microwave thermoacoustic imaging is realized; only sparse data is needed to reconstruct the microwave thermoacoustic image, reducing the scanning cost and reducing the discomfort and nausea of the human body in the case of multiple scans.

[0047] 4. The present application can be easily extended to other applications of microwave thermoacoustic imaging mode, has compact structure and is convenient to operate, and has the potential to benefit a series of practical applications, such as tumor imaging, nondestructive testing, security inspection and treatment guidance, etc., and has strong applicability and wide popularization range. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description only represent some embodiments of the present application, and all other drawings obtained by those skilled in the art without creative effort based on these drawings also belong to the protection scope of the present application.

[0049] Figure 1 The flow chart of the high-quality real-time microwave thermoacoustic imaging method based on deep learning in the embodiments of the present application;

[0050] Figure 2 The training flow chart of the fully dense generative adversarial neural network model in the embodiments of the present application;

[0051] Figure 3 The result schematic diagram of the thermoacoustic imaging in the embodiments of the present application;

[0052] Figure 4 The structure diagram of the high-quality real-time microwave thermoacoustic imaging system based on deep learning in the embodiments of the present application;

[0053] Figure 5 The structure schematic diagram of the computer readable storage medium in the embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only represent some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort also belong to the protection scope of the present application.

[0055] In the present application, the phrase "embodiments" means that the specific features, structures or characteristics described in combination with the embodiments can be contained in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. The person skilled in the art explicitly and implicitly understands that the embodiments described in the present application can be combined with other embodiments.

[0056] The high-quality real-time microwave thermoacoustic imaging method based on deep learning in the present application improves the resolution of the image. After simple preprocessing of the collected microwave thermoacoustic image, the high-quality image can be obtained by inputting the image into the neural network model. The end-to-end image reconstruction method reduces the cost of manual participation. The reconstructed image in the case of sparse data has high resolution in addition to artifacts and noise. For specific process, please refer to Figure 1A high-quality real-time microwave thermoacoustic imaging method based on deep learning comprises the following steps:

[0057] S1, using COMSOL software to simulate the heat absorption distribution of a biological tissue model under microwave irradiation, obtaining heat absorption distribution graphs in different directions;

[0058] S2, importing the heat absorption distribution graph into MATLAB software to obtain multiple microwave thermoacoustic reconstruction images and corresponding label images;

[0059] In this embodiment, COMSOL software is used to simulate the heat absorption distribution of biological tissue models with different diameters and thicknesses under 3GHz microwave irradiation and coupling oil environment, different tomographic depths are taken, and then the polarization direction of the electric field is changed to obtain heat absorption distribution graphs in different directions. Then the heat absorption distribution graph is exported and imported into MATLAB software to obtain multiple microwave thermoacoustic reconstruction images with artifacts and corresponding label images without artifacts and noise. The microwave thermoacoustic reconstruction image is a cross-sectional image with noise and artifacts.

[0060] S3, normalizing and data augmenting the microwave thermoacoustic reconstruction images and corresponding label images, and dividing them into a training set and a validation set according to a set proportion;

[0061] In this embodiment, first, the pixel value in each microwave thermoacoustic reconstruction image and corresponding label image is normalized by dividing by 255. In order to improve the diversity of the microwave thermoacoustic reconstruction image samples, the normalized microwave thermoacoustic reconstruction images and corresponding label images are subjected to image augmentation operations using translation, rotation, stretching, horizontal flipping and block processing methods. The size of the augmented images is unified to make the image width and height consistent. The images are divided according to the proportion of training set:validation set=4:1 to obtain the training set and the validation set.

[0062] In this embodiment, after translation, rotation, stretching, horizontal flipping and block processing, image augmentation operations are performed, a total of 500 microwave thermoacoustic reconstruction images with noise and artifacts and 500 corresponding label images without artifacts and noise, each with a size of 256x256, are obtained, a total of 500 pairs. According to the proportion of 4:1, 400 pairs are used as the training set and 100 pairs are used as the validation set.

[0063] S4, using a microwave thermoacoustic real-time imaging device to collect microwave thermoacoustic images, and setting the normalized images as a test set;

[0064] Similarly, the microwave thermoacoustic real-time imaging device also reconstructs microwave thermoacoustic images of biological tissues under 3GHz microwave irradiation and coupling oil environment, and collects microwave thermoacoustic images. The microwave thermoacoustic image is also a cross-sectional image.

[0065] Then the collected microwave thermoacoustic images are normalized, and the images are set to a certain size, which is the test set.

[0066] In this embodiment, 100 microwave thermoacoustic images are obtained, and the image size is unified to 256x256.

[0067] S5, using the training set to train the full dense generative adversarial neural network model, and after training, using the validation set to verify the imaging quality of the model;

[0068] Specifically, the full dense generative adversarial neural network model includes a generator and a discriminator; the generator further includes an encoder and a decoder; the full dense generative adversarial neural network model includes a convolution layer with a convolution kernel size of 3x3, a batch normalization layer, a ReLU activation function layer, a deconvolution layer with a convolution kernel size of 2x2, a convolution layer with a convolution kernel size of 1x1, a Sigmoid activation function layer, a flattening layer and a fully connected layer arranged in sequence.

[0069] More specifically, the encoder includes 21 convolution layers with a convolution kernel size of 3x3, 20 convolution layers with a convolution kernel size of 1x1, and 4 maximum pooling layers with a convolution kernel size of 2x2; the decoder includes 20 convolution layers with a convolution kernel size of 3x3, 17 convolution layers with a convolution kernel size of 1x1, and 4 deconvolution layers with a convolution kernel size of 2x2; the discriminator includes 5 convolution layers with a convolution kernel size of 4x4.

[0070] The following table shows the structure of each component in the full dense generative adversarial neural network model in this embodiment:

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] In this embodiment, Pycharm is used as an integrated development environment to debug the full dense generative adversarial neural network model, and Python language and TensorFlow2.6 framework are used to build the full dense generative adversarial neural network model, and NVIDIA GeForce RTX 3060 is used for GPU acceleration, and the specific training steps are as shown in Figure 2

[0078] ​S51, initialize the parameters of the full dense generative adversarial neural network model, and set the initial learning rate;

[0079] S52, input the microwave thermoacoustic reconstructed image with noise artifacts in the training set into the generator for forward propagation;

[0080] S53, input the output image of the generator and the label image without noise and artifacts in the training set into the discriminator for forward propagation;

[0081] S54, use the mean square error loss function to measure the error between the output value of the generator and the label target value, which is used as the basis for updating the parameters of the generator by back propagation; and use the cross-entropy loss function to measure the error between the output value of the discriminator and the label target value, which is used as the basis for updating the parameters of the discriminator by back propagation;

[0082] S55, according to the loss function, use the adaptive moment estimation optimizer to calculate the gradient of the loss function, and update the parameters of the full dense generative adversarial neural network model by back propagation until the loss function converges or the maximum number of iterations is reached.

[0083] In this embodiment, the adaptive moment estimation optimizer is used to find the global optimal point parameters, and the one-dimensional and two-dimensional moment estimations of the gradients of each layer are used to dynamically adjust the learning rate of each parameter, so that the learning rate has a certain range in each iteration learning process, ensuring that the parameters are relatively stable; after multiple update iterations, the final ideal situation is to make the discriminator unable to distinguish whether the sample comes from the output of the generator or the real output without noise and artifacts, that is, the final sample discrimination probability is 0.5.

[0084] In this embodiment, the purpose of the convolutional layer is to extract the features of the image; the purpose of the pooling layer is to select features, reduce the number of features, reduce the number of parameters, and speed up the training; the purpose of the deconvolutional layer is to restore the small size high-dimensional feature map and restore the image size, and restore the image details; the purpose of merging is to fuse multi-scale features, which can combine high-dimensional features and low-dimensional features.

[0085] In this embodiment, 400 microwave thermoacoustic reconstructed images with a size of 256x256 and 400 label images with a size of 256x256 corresponding to them in the training set are input into the constructed network model, the learning rate is set to 1e-4, and after training and parameter adjustment, the network model outputs high-quality images without noise and artifacts.

[0086] S6, save the weights and bias values of the full dense generative adversarial neural network model after verification, and the imaging quality of the verified model;

[0087] In this embodiment, structural similarity (SSIM) is used as an evaluation index to evaluate the imaging quality of the model, which is used to measure the consistency of the output image X and the target image Y in brightness, contrast and structure, and the formula is:

[0088]

[0089] Wherein, μ X is the average value of the output image X, μ Y is the average value of the target image Y, σ X is the standard deviation of the output image X, σ Y is the standard deviation of the target image Y, σ XY is the covariance of the output image X and the target image Y, C1 and C2 are positive constants.

[0090] S7, using the test set to test the full dense generative adversarial neural network model after verification;

[0091] As shown in Figure 3 , which is the test result of some three groups of images in this embodiment, the left half is the original microwave thermoacoustic image with noise for testing, and the right half is the high-quality microwave thermoacoustic image without noise output by the full dense generative adversarial network model.

[0092] S8, the full dense generative adversarial neural network model after testing is imported into the microwave thermoacoustic real-time imaging device, and the full dense generative adversarial neural network model is driven by the microwave thermoacoustic real-time imaging device to generate high-quality images.

[0093] Specifically, after the microwave thermoacoustic real-time imaging device performs default one frame of microwave thermoacoustic image reconstruction, the weight and bias value of the full dense generative adversarial neural network model are loaded, and the frame of microwave thermoacoustic reconstruction image is imported into the full dense generative adversarial neural network model, the model is called to process the value corresponding to each pixel point of the microwave thermoacoustic reconstruction image, remove background noise, edge artifacts and splitting artifacts, compensate for the image information loss caused by uneven heat absorption distribution, generate a frame of high-quality reconstruction image, and finally generate a high-quality real-time microwave thermoacoustic reconstruction image.

[0094] It should be noted that for the above-mentioned method embodiments, in order to simplify the description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other order or simultaneously.

[0095] Based on the same idea as the high-quality real-time microwave thermoacoustic imaging method based on deep learning in the above embodiment, the application also provides a high-quality real-time microwave thermoacoustic imaging system based on deep learning, which can be used to perform the high-quality real-time microwave thermoacoustic imaging method based on deep learning described above. For the convenience of description, in the structural schematic diagram of the embodiment of the high-quality real-time microwave thermoacoustic imaging system based on deep learning, only the parts related to the embodiments of the application are shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, which can include more or fewer components than the illustrated, or combine certain components, or different component arrangements.

[0096] As shown in Figure 4 Another embodiment of the application provides a high-quality real-time microwave thermoacoustic imaging system based on deep learning, which includes an image simulation module, an image acquisition module, an image preprocessing module, a model obtaining module and a real-time imaging module.

[0097] The image simulation module is used to simulate the heat absorption distribution of a biological tissue model under microwave irradiation using COMSOL software to obtain heat absorption distribution maps in different directions; the heat absorption distribution maps are imported into MATLAB software to obtain a plurality of microwave thermoacoustic reconstruction images and corresponding label images.

[0098] The image acquisition module is used to acquire microwave thermoacoustic images using a microwave thermoacoustic real-time imaging device.

[0099] The image preprocessing module is used to normalize and data augment the microwave thermoacoustic reconstruction images and corresponding label images, and divide them into a training set and a validation set according to a set proportion; and normalize the microwave thermoacoustic images and set them as a test set.

[0100] The model obtaining module is used to train a fully dense generative adversarial neural network model using the training set, verify the imaging quality of the model using the validation set after the training is completed, save the weight, bias value of the fully dense generative adversarial neural network model after verification and the imaging quality of the model after verification is completed, and test the fully dense generative adversarial neural network model after verification using the test set.

[0101] The real-time imaging module is used to import the fully dense generative adversarial neural network model after testing into the microwave thermoacoustic real-time imaging device, and drive the fully dense generative adversarial neural network model to generate high-quality images through the microwave thermoacoustic real-time imaging device.

[0102] It should be noted that the high-quality real-time microwave thermoacoustic imaging system based on deep learning of the present application corresponds to the high-quality real-time microwave thermoacoustic imaging method based on deep learning of the present application, and the technical features and advantages described in the above embodiment of the high-quality real-time microwave thermoacoustic imaging method based on deep learning are applicable to the embodiment of the high-quality real-time microwave thermoacoustic imaging system based on deep learning, and the specific content can be referred to the description in the method embodiment of the present application. Therefore, it is declared here.

[0103] In addition, in the embodiment of the high-quality real-time microwave thermoacoustic imaging system based on deep learning of the above embodiment, the logical division of each program module is only an example, and in actual application, the above function allocation can be completed by different program modules according to the needs, for example, considering the configuration requirements of the corresponding hardware or the convenience of software implementation. The internal structure of the high-quality real-time microwave thermoacoustic imaging system based on deep learning is divided into different program modules to complete all or part of the functions described above.

[0104] As shown in Figure 5 In one embodiment, a computer readable storage medium is provided, which stores a program in the memory, and the program is executed by the processor to realize the high-quality real-time microwave thermoacoustic imaging method based on deep learning, specifically:

[0105] The COMSOL software is used to simulate the heat absorption distribution of the biological tissue model under microwave irradiation, and heat absorption distribution graphs in different directions are obtained;

[0106] The heat absorption distribution graph is imported into the MATLAB software to obtain multiple microwave thermoacoustic reconstruction images and corresponding label images;

[0107] The microwave thermoacoustic reconstruction images and the corresponding label images are normalized and data augmented, and are divided into a training set and a validation set according to a set proportion;

[0108] The microwave thermoacoustic real-time imaging device is used to collect microwave thermoacoustic images, which are normalized and set as a test set;

[0109] The training set is used to train the full dense generative adversarial neural network model, and after the training is completed, the validation set is used to verify the imaging quality of the model;

[0110] The weight and bias values of the full dense generative adversarial neural network model after verification are saved, and the imaging quality of the verified model is saved;

[0111] The test set is used to test the full dense generative adversarial neural network model after verification;

[0112] The full dense generative adversarial neural network model after the test is imported into the microwave thermoacoustic real-time imaging device, and the full dense generative adversarial neural network model is driven by the microwave thermoacoustic real-time imaging device to generate high-quality images.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiment methods. Among them, any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0114] The technical features of the above embodiments can be combined in any way. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0115] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods and shall be within the scope of protection of the present application.

Claims

1. A high-quality real-time microwave thermoacoustic imaging method based on deep learning, characterized in that, Includes the following steps: The thermal absorption distribution of a biological tissue model under microwave irradiation was simulated using COMSOL software, and thermal absorption distribution maps in different directions were obtained. Import the thermal absorption distribution map into MATLAB software to obtain multiple microwave thermoacoustic reconstruction images and corresponding label images; The microwave thermoacoustic reconstructed images and their corresponding labeled images are normalized and augmented, and then divided into training and validation sets according to a set ratio. Microwave thermal acoustic images were acquired using a real-time microwave thermal acoustic imaging device, and after normalization, they were set as the test set. The fully dense generative adversarial neural network model was trained using a training set, and the imaging quality of the model was verified using a validation set after training. Save the weights, bias values, and image quality of the fully dense generative adversarial neural network model after validation; The validated fully dense generative adversarial neural network model was tested using a test set; After testing, the fully dense generative adversarial neural network model is imported into the microwave thermoacoustic real-time imaging device, which then drives the fully dense generative adversarial neural network model to generate high-quality images. The fully dense generative adversarial neural network model includes a generator and a discriminator; the generator includes an encoder and a decoder. The fully dense generative adversarial neural network model includes, in sequence, a convolutional layer with a kernel size of 3×3, a batch normalization layer, a ReLU activation function layer, a deconvolutional layer with a kernel size of 2×2, a convolutional layer with a kernel size of 1×1, a Sigmoid activation function layer, a flattening layer, and a fully connected layer. The encoder includes 21 convolutional layers with a kernel size of 3×3, 20 convolutional layers with a kernel size of 1×1, and 4 max pooling layers with a kernel size of 2×2; the decoder includes 20 convolutional layers with a kernel size of 3×3, 17 convolutional layers with a kernel size of 1×1, and 4 deconvolutional layers with a kernel size of 2×2. The discriminator comprises five convolutional layers with a kernel size of 4×4.

2. The high-quality real-time microwave thermoacoustic imaging method based on deep learning according to claim 1, characterized in that, The COMSOL software simulated the thermal absorption distribution of biological tissue models with different diameters and thicknesses under 3GHz microwave irradiation and coupling oil environment. Different tomographic depths were taken, and then the electric field polarization direction was changed to obtain thermal absorption distribution maps in different directions. The microwave thermoacoustic reconstructed image is a cross-sectional image with noise and artifacts; The labeled image is a corresponding noise-free and artifact-free microwave thermoacoustic reconstruction image.

3. The high-quality real-time microwave thermoacoustic imaging method based on deep learning according to claim 1, characterized in that, Normalize the value of each pixel in the microwave thermoacoustic reconstructed image and the corresponding labeled image by dividing it by 255. Image augmentation operations were performed on the normalized microwave thermoacoustic reconstructed image and the corresponding label image using translation, rotation, stretching, horizontal flipping, and block processing methods. The size of the amplified images is standardized to ensure that the image width and height are consistent. The images are divided into training and validation sets in a ratio of 4:

1.

4. The high-quality real-time microwave thermoacoustic imaging method based on deep learning according to claim 1, characterized in that, Microwave thermoacoustic images were reconstructed from biological tissues using a real-time microwave thermoacoustic imaging device under 3GHz microwave irradiation and coupling oil conditions. The microwave thermoacoustic image is a cross-sectional image; The acquired microwave thermoacoustic images were normalized to a set size and used as the test set.

5. The high-quality real-time microwave thermoacoustic imaging method based on deep learning according to claim 1, characterized in that, The training of the fully dense generative adversarial neural network model specifically involves: Initialize the parameters of the fully dense generative adversarial neural network model and set the initial learning rate; The microwave thermoacoustic reconstructed images with noise artifacts in the training set are input into the generator for forward propagation. The generator's output image and the noise-free, artifact-free label images from the training set are input into the discriminator for forward propagation. The mean squared error loss function is used to measure the error between the generator's output value and the target label value; the cross-entropy loss function is used to measure the error between the discriminator's output value and the target label value. Based on the loss function, the gradient of the loss function is calculated using an adaptive moment estimator optimizer, and the parameters of the fully dense generative adversarial neural network model are updated by backpropagation until the loss function converges or the maximum number of iterations is reached.

6. The high-quality real-time microwave thermoacoustic imaging method based on deep learning according to claim 5, characterized in that, The evaluation metric for the imaging quality of the model is structural similarity (SSIM), which measures the degree of consistency between the brightness, contrast, and structure of the output image X and the target image Y. The formula is as follows: ， in, μ X The average value of the output image X. μ Y The average value of Y in the target image. σ X To output the standard deviation of image X, σ Y Let Y be the standard deviation of the target image. σ XY To output the covariance between the output image X and the target image Y, C 1 and C 2 is a positive constant.

7. A high-quality real-time microwave thermal imaging system based on deep learning, characterized in that, The high-quality real-time microwave thermoacoustic imaging method based on deep learning, applied to any one of claims 1-6, includes an image simulation module, an image acquisition module, an image preprocessing module, a model acquisition module, and a real-time imaging module; The image simulation module is used to simulate the thermal absorption distribution of a biological tissue model under microwave irradiation using COMSOL software, and obtain thermal absorption distribution maps in different directions; the thermal absorption distribution maps are then imported into MATLAB software to obtain multiple microwave thermoacoustic reconstructed images and corresponding label images; The image acquisition module is used to acquire microwave thermal images using a real-time microwave thermal imaging device. The image preprocessing module is used to normalize and augment the microwave thermoacoustic reconstructed images and their corresponding label images, and divide them into training and validation sets according to a set ratio; at the same time, the microwave thermoacoustic images are normalized and set as the test set. The model acquisition module is used to train a fully dense generative adversarial neural network model using a training set, and after training, to verify the imaging quality of the model using a validation set; it saves the weights, bias values, and the image quality of the fully dense generative adversarial neural network model after verification; and it uses a test set to test the verified fully dense generative adversarial neural network model. The real-time imaging module is used to import the fully dense generative adversarial neural network model after testing into the microwave thermoacoustic real-time imaging device, and drive the fully dense generative adversarial neural network model to generate high-quality images through the microwave thermoacoustic real-time imaging device.

8. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the high-quality real-time microwave thermoacoustic imaging method based on deep learning as described in any one of claims 1-6.

Images