A method and system for removing CBCT metal artifacts based on unpaired learning

Abstract

The embodiment of the application provides a kind of CBCT metal artifact removal method and system based on unpaired learning, belong to computer oral cavity repair field.The method includes obtaining artifact data set and clean data set, artifact data set and clean data set are unpaired data set;Artifact data set and clean data set are processed, and the training data set after processing is obtained;Metal artifact removal model is trained using training data set, and initial metal artifact removal model is obtained;The original data corresponding to the predicted artifact removal data output by initial metal artifact removal model is obtained by restoring processing;The initial metal artifact removal model is optimized using predicted artifact removal data and original data, and target metal artifact removal model is obtained;The CT image to be removed metal artifact is input into target metal artifact removal model, and target CT image after metal artifact removal is obtained, and the practicability and value and model artifact removal effect of the application are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer dental prosthetics, and in particular, to a CBCT metal artifact removal method and system based on unpaired learning. BACKGROUND

[0002] Oral and maxillofacial cone beam CT (CBCT) scanning is a medical imaging technique used in radiology to obtain detailed oral images for diagnostic purposes. Unfortunately, artifacts caused by metal implants appear as dark and bright streaks in CBCT, greatly reducing image quality and the accuracy of CT values, severely affecting clinical diagnosis, so reducing metal artifacts (MAR) is an urgent problem in the CBCT imaging process.

[0003] Traditional projection data correction, iterative reconstruction and other methods have limitations, limiting their large-scale application in clinical medicine. Therefore, reducing metal artifacts in CBCT images has so far been a research hotspot and difficulty in the field of medical imaging.

[0004] In recent years, deep learning (DL) has made great progress in image processing and pattern recognition. For example, convolutional neural networks (CNNs) have been applied to low-dose CT reconstruction and artifact reduction in medical imaging. U-Net is a fully convolutional network with many applications in biomedical image segmentation, such as brain image segmentation and liver image segmentation. Methods of MAR based on DL technology have also been widely studied, which generally use deep convolutional neural networks to map CBCT images with metal artifacts end-to-end to artifact-free images to remove metal artifacts and restore the original anatomical structure to improve the accuracy of clinical diagnosis.

[0005] Currently, many teams use image conversion based on the generative adversarial network (GAN) framework to build a generator with feature transfer capability to establish a mapping from the input image to the target image, so that the generated image has the characteristics of the target image; then use a discriminator to distinguish between the generated image and the target image for adversarial training to complete complex image conversion.

[0006] However, it is difficult to collect paired noisy and clean images in most real-world applications, so most teams use simulated data to learn the task of artifact images to artifact-free images. However, simulated data cannot represent real artifact data, making the learning model perform poorly on real artifact data.

[0007] Therefore, how to solve the above problems is currently an urgent problem to be solved. SUMMARY

[0008] The present application provides a CBCT metal artifact removal method and system based on unpaired learning, aiming to improve the above problems.

[0009] In a first aspect, the present application provides a CBCT metal artifact removal method based on unpaired learning, which comprises:

[0010] Obtaining an artifact data set and a clean data set, wherein the artifact data set and the clean data set are unpaired data sets;

[0011] Performing data processing on the artifact data set and the clean data set to obtain a processed training data set;

[0012] Training a metal artifact removal model using the training data set to obtain an initial metal artifact removal model;

[0013] Performing restoration processing on the predicted artifact removal data output by the initial metal artifact removal model to obtain the original data corresponding to the predicted artifact removal data;

[0014] Optimizing the initial metal artifact removal model using the predicted artifact removal data and the original data to obtain a target metal artifact removal model;

[0015] Inputting a CT image to be removed of metal artifacts into the target metal artifact removal model to obtain a target CT image after metal artifact removal.

[0016] In a possible embodiment, the data processing on the artifact data set and the clean data set to obtain a processed training data set comprises:

[0017] Converting each three-dimensional CBCT image in the artifact data set and the clean data set into a plurality of two-dimensional image data;

[0018] Fixing each two-dimensional image data to a preset size to obtain a plurality of new two-dimensional image data, each new two-dimensional image data being single-channel data;

[0019] Separating the width and height of each new two-dimensional image data to obtain multi-channel two-dimensional image data;

[0020] Expanding the multi-channel two-dimensional image data using a padding operation to obtain expanded two-dimensional image data;

[0021] Normalizing the expanded two-dimensional image data using a k-sigma transformation formula to obtain a processed training data set.

[0022] In a possible embodiment, the converting the artifact data set and each three-dimensional CBCT image in the clean data set into a plurality of two-dimensional image data includes:

[0023] According to a CT value range, the two-dimensional data of each layer of each three-dimensional CBCT image in the artifact data set and the clean data set is intercepted to obtain a plurality of two-dimensional image data.

[0024] In a possible embodiment, the separating the width and the height of each new two-dimensional image data to obtain multi-channel two-dimensional image data includes:

[0025] The width and the height of the multi-channel two-dimensional image data are separated into 1 / 2 of the original to generate four-dimensional data.

[0026] The four-dimensional data is merged and converted into 4-channel two-dimensional image data.

[0027] In a possible embodiment, the expanding the multi-channel two-dimensional image data by using a padding operation to obtain expanded two-dimensional image data includes:

[0028] According to a size padding algorithm, the 4-channel two-dimensional image data is edge-padded to obtain expanded two-dimensional image data.

[0029] In a possible embodiment, the k and sigma satisfy:

[0030]

[0031] Wherein, x and f respectively represent an input artifact image and an output artifact-removed image, k and sigma 2 are parameters obtained by using a linear equation to fit the artifact images in the training data set.

[0032] In a possible embodiment, the initial metal artifact removal model includes a generator module, an image blurring module, and a discriminator module; wherein the generator module, the image blurring module, and the discriminator module are sequentially connected.

[0033] The generator module receives the training data set as input and generates a predicted artifact-removed image correspondingly;

[0034] The image blurring module performs sharpening processing on the predicted artifact-removed image and the clean image in the training data set to generate a sharpened image;

[0035] The discriminator module outputs a predicted probability value according to the sharpened image.

[0036] In a possible embodiment, the initial metal artifact reduction model comprises a loss function calculation module, the loss function calculation module comprises a generator loss function submodule and a discriminator loss function submodule, and the loss function calculation module adopts the following loss function:

[0037]

[0038] wherein x and z respectively represent data in a clean data set P data(x) and an artifact data set P noise(z) in the training data set, D(x) represents a return value of data x inputting a discriminator loss function submodule D, and G(z) represents a return value of data z inputting a generator loss function submodule G.

[0039] In a possible embodiment, when the generator loss function submodule is fixed, the model of the discriminator loss function submodule is trained, and the loss function of the discriminator loss function submodule is as follows:

[0040]

[0041] When the discriminator loss function submodule is fixed, the model of the generator loss function submodule is trained, and the loss function of the generator loss function submodule is as follows:

[0042]

[0043] In a second aspect, the application provides a CBCT metal artifact reduction system based on unpaired learning, and the system comprises:

[0044] a data acquisition unit configured to acquire an artifact data set and a clean data set, wherein the artifact data set and the clean data set are unpaired data sets;

[0045] a data processing unit configured to perform data processing on the artifact data set and the clean data set to obtain a processed training data set;

[0046] a model training unit configured to train a metal artifact reduction model by using the training data set to obtain an initial metal artifact reduction model;

[0047] a data restoration unit configured to perform restoration processing on predicted artifact reduction data output by the initial metal artifact reduction model to obtain original data corresponding to the predicted artifact reduction data;

[0048] a model optimization unit configured to optimize the initial metal artifact reduction model by using the predicted artifact reduction data and the original data to obtain a target metal artifact reduction model;

[0049] A metal artifact removal unit is configured to input a CT image to be removed of metal artifacts into the target metal artifact removal model to obtain a target CT image removed of metal artifacts.

[0050] The CBCT metal artifact removal method and system based on unpaired learning provided by the application obtain an artifact data set and a clean data set, the artifact data set and the clean data set are unpaired data sets; the artifact data set and the clean data set are processed to obtain a processed training data set; the training data set is used to train a metal artifact removal model to obtain an initial metal artifact removal model; the predicted artifact removal data output by the initial metal artifact removal model is restored to obtain original data corresponding to the predicted artifact removal data; the initial metal artifact removal model is optimized by using the predicted artifact removal data and the original data to obtain a target metal artifact removal model; a CT image to be removed of metal artifacts is input into the target metal artifact removal model to obtain a target CT image removed of metal artifacts, thereby training an end-to-end metal artifact removal model from unpaired artifact images and clean images, and improving the practicability and value of the method. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0052] Figure 1 A structural schematic diagram of an electronic device provided by the first embodiment of the application;

[0053] Figure 2 A flowchart of a CBCT metal artifact removal method based on unpaired learning provided by the second embodiment of the application;

[0054] Figure 3 For Figure 2 A model schematic diagram of a metal artifact removal model in a CBCT metal artifact removal method based on unpaired learning shown in the first embodiment of the application;

[0055] Figure 4 A functional module schematic diagram of a CBCT metal artifact removal system based on unpaired learning provided by the third embodiment of the application. DETAILED DESCRIPTION

[0056] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0057] First embodiment:

[0058] Figure 1 A structural schematic diagram of an electronic device provided by the embodiments of the present application can be used to describe an example of the electronic device 100 for implementing the CBCT metal artifact removal method based on unpaired learning of the embodiments of the present application. Figure 1 A structural schematic diagram of an electronic device as shown in

[0059] A structural schematic diagram of an electronic device as shown in Figure 1 The electronic device 100 includes one or more processors 102, one or more storage devices 104, an input device 106, and an output device 108, which are interconnected through a bus system and / or other forms of connection mechanism (not shown). It should be noted that Figure 1 The components and structures of the electronic device 100 shown are only exemplary and are not limiting, and the electronic device can have Figure 1 some components shown, and can also have Figure 1 other components and structures not shown.

[0060] The processor 102 can be a central processing unit (CPU) or other forms of processing units with data processing and / or instruction execution capabilities, and can control other components in the electronic device 100 to perform desired functions.

[0061] It should be understood that the processor 102 in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0062] The storage device 104 can include one or more computer program products, which can include various forms of computer-readable storage media.

[0063] It should be understood that the storage device 104 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0064] Among them, one or more computer program instructions can be stored on the computer readable storage medium, and the processor 102 can run the program instructions to realize the client functions (implemented by the processor) in the embodiments of the present application described below and / or other desired functions. Various application programs and various data, such as various data used and / or generated by the application programs, etc. can also be stored in the computer readable storage medium.

[0065] The input device 106 can be a device used by a user to input instructions, and can include one or more of a keyboard, a mouse, a microphone, a touch screen, etc.

[0066] Second embodiment:

[0067] Reference Figure 2A flowchart of a CBCT metal artifact removal method based on unpaired learning is shown, which specifically comprises the following steps:

[0068] Step S201, obtaining an artifact data set and a clean data set.

[0069] Among them, the artifact data set and the clean data set are unpaired data sets.

[0070] Optionally, the unpaired artifact data set and the clean data set each contain CBCT images of multiple sizes, such as 650x650x435, 800x800x500, etc. Among them, the artifact data set generally contains CBCT imaging noise, bone artifacts, metal artifacts, etc. The clean data set needs to meet the conditions of low noise, no artifacts or few artifacts, clear details, etc. This embodiment uses diverse data as the training set of the model to improve the practicality and generalization of the model.

[0071] That is, the images in the clean data set are clean images, and the images in the artifact data set are artifact images.

[0072] Step S202, data processing is performed on the artifact data set and the clean data set to obtain a processed training data set.

[0073] As an implementation manner, step S202 comprises: converting each three-dimensional CBCT image in the artifact data set and the clean data set into a plurality of two-dimensional image data; fixing each two-dimensional image data to a preset size to obtain a plurality of new two-dimensional image data, each new two-dimensional image data being single-channel data; separating the width and height of each new two-dimensional image data to obtain multi-channel two-dimensional image data; expanding the multi-channel two-dimensional image data using a padding operation to obtain expanded two-dimensional image data; and performing normalization processing on the expanded two-dimensional image data using a k-sigma transformation formula to obtain a processed training data set.

[0074] Optionally, the conversion of each three-dimensional CBCT image in the artifact data set and the clean data set into a plurality of two-dimensional image data comprises: performing an intercepting operation on the two-dimensional data of each layer of each three-dimensional CBCT image in the artifact data set and the clean data set according to the CT value range to obtain a plurality of two-dimensional image data.

[0075] Specifically, the size of the fixed unpaired artifacts and clean data can fix the multi-size data and perform normalization operation so that the data can be trained in parallel. The embodiment first acquires the unpaired artifacts and clean data respectively, and can perform intercept operation on the two-dimensional data of each layer (i.e. the third dimension) according to the CT value range (considering that the CT value is large due to the existence of metal, the CT value range is adjusted to-1000-5500) to reduce the influence of extremely small or extremely large CT values on image quality. Further, the size of the input two-dimensional data can be fixed to the preset size (default is (768, 768)) according to the preset size. Further, the data range can be adjusted to 0-1 by using normalization operation, so as to be processed by subsequent data normalization operation.

[0076] Optionally, the separating the width and height of each of the new two-dimensional image data to obtain multi-channel two-dimensional image data comprises: separating the width and height of the multi-channel two-dimensional image data into 1 / 2 of the original to generate four-dimensional data; and merging the four-dimensional data and converting the four-dimensional data into 4-channel two-dimensional image data.

[0077] In the embodiment, the width and height of the data are first separated into 1 / 2 of the original, and the two-dimensional data is converted into four-dimensional data, i.e. (H, W)→(H / 2, 2, W / 2, 2), wherein H and W represent the width and height of the data respectively; further, the data can be merged and converted into 4-channel data, i.e. (H / 2, 2, W / 2, 2)→(H / 2, W / 2, 4); further, the data dimensions can be exchanged, i.e. (H / 2, W / 2, 4)→(4, H / 2, W / 2), so as to be trained by the subsequent model.

[0078] Optionally, the expanding the multi-channel two-dimensional image data by using padding operation to obtain expanded two-dimensional image data comprises: performing data edge padding on the 4-channel two-dimensional image data according to a size padding algorithm to obtain expanded two-dimensional image data.

[0079] In the embodiment, the data is expanded by using 0 padding operation to protect the edge information of the image. In the embodiment, the data edge padding is performed according to the size padding formula, and the calculation formula is as follows:

[0080] p h =(s-h%s) / 2

[0081] wherein p h represents the size of the data width direction that needs to be filled, s represents the specified multiple size (default is 32), and the size of the data height direction that needs to be filled p w can be calculated in the same way.

[0082] Optionally, the k-sigma satisfies:

[0083]

[0084] where x and f represent the input artifact image and the output de-artifact image respectively, k and σ 2 are parameters fitted using linear equation on the artifact image in the training dataset. Specifically, the embodiment first calculates the mean value E(x) of each pixel position on a certain artifact image; further, the pixel values with the same mean value are collected to obtain the variance Var(x); further, in the graph with E(x) as the horizontal axis and Var(x) as the vertical axis, the linear equation fitting is used to obtain the k and σ 2 .

[0085] Preferably, after normalizing the data, the data range is adjusted to 0-255 through the inverse normalization operation, so as to facilitate subsequent model training.

[0086] Step S203, training the metal artifact reduction model using the training dataset to obtain an initial metal artifact reduction model.

[0087] wherein, as shown in Figure 3 the initial metal artifact reduction model includes a generator module, an image blurring module and a discriminator module; wherein the generator module, the image blurring module and the discriminator module are connected in sequence; the generator module receives the input training dataset and generates a predicted de-artifact image correspondingly; the image blurring module sharpens the predicted de-artifact image and the clean image in the training dataset to generate a sharpened image; and the discriminator module outputs a predicted probability value according to the sharpened image.

[0088] Specifically, the generator module generates a predicted de-artifact image according to the preprocessed data, and the generator module can include a nonlinear non-activation module, an encoder module and a decoder module.

[0089] Optionally, the encoder module and the decoder module are both composed of multiple nonlinear non-activation modules, and the encoder module and the decoder module are connected, and the multiple nonlinear non-activation modules inside are connected in sequence, so that the output of the previous module is the input of the next module.

[0090] Further, in the present embodiment, the non-linear activation-free module first normalizes the de-normalized artifact data and clean data through layer normalization operation to standardize, so as to accelerate the convergence of the model and improve the performance of the model; then the channel number is adjusted to be twice the original by using 1x1 standard convolution and 3x3 channel-wise convolution, so as to reduce the model calculation amount; then the channel control unit can separate and multiply the channels of the input feature map, so that the channel becomes 1 / 2 of the original, and a simplified pixel-by-pixel multiplication method is used instead of a non-linear activation function; then the channel attention module can generate weight parameters corresponding to the channel number through adaptive average pooling and 1x1 standard convolution, and perform multiplication operation with the input feature map, so as to improve the learning performance of the feature in a simplified attention calculation manner; then the input feature map of the non-linear activation-free module is added through 1x1 standard convolution; finally, layer normalization, 1x1 standard convolution, channel control unit and 1x1 standard convolution are used to deepen the depth of the model.

[0091] Further, the encoder module can convert the input channel number (default 4) to 16 through 1x1 standard convolution; further, four-layer encoding structure can be composed according to the non-linear activation-free module, and the channel numbers are 16, 32, 64 and 128 respectively, wherein the first and second layers contain one non-linear activation-free module, and the third and fourth layers contain two non-linear activation-free modules, and each layer of encoding structure is down-sampled through 3x3 convolution with a step of 2. Further, the intermediate layer is composed of one non-linear activation-free module, and the channel number is 256, so as to improve the feature extraction capability.

[0092] Further, the decoder module can be composed of four-layer decoding structure according to the non-linear activation-free module, and the channel numbers are 128, 64, 32 and 16 respectively, and each layer only contains one non-linear activation-free module, wherein each layer of decoding structure is up-sampled through 3x3 deconvolution with a step of 2, and the feature map of each layer of decoding structure is added to the corresponding feature map of the encoding structure; further, the channel number is converted to the input channel number of the encoder module through 1x1 standard convolution, that is, the output channel number of 1x1 standard convolution is 4.

[0093] In the present embodiment, the input of the image blurring module is the predicted artifact-removed image and the clean image, the sharpening technology uses 3x3 as the kernel size for calculating the local mean, and then the sharpened image and the original image are used as two groups of input channels of the discriminator module.

[0094] In the embodiment, the image blurring module is called twice, and the inputs are the image with predicted artifacts and the clean image, and the corresponding sharpened images are generated; the corresponding input and output images of the image blurring module are combined together as the input of the discriminator module. The original image refers to the input image of the discriminator module each time the image blurring module is called, that is, the original image includes the image with predicted artifacts and the clean image.

[0095] Further, the discriminator module can take the combined image of the image blurring module as the input of the discriminator to learn the feature information of the artifacts and the clean image. The embodiment uses a four-layer convolutional structure, and the convolution kernel is 5x5, 5x5, 5x5, and 3x3, respectively, the step is 4, 4, 4, and 2, respectively, and the channel number is 64, 128, 256, and 512, respectively. Further, one probability value can be output through two fully connected layers and a Sigmoid function, and the parameters of the first fully connected layer can be calculated as ((3x3x512), 1024) and the parameters of the second fully connected layer can be calculated as (1024, 1) according to the data size in the training data set (in the embodiment, the width and height of the preprocessed data are 400 and 400, respectively).

[0096] Optionally, the initial metal artifact reduction model comprises a loss function calculation module, the loss function calculation module comprises a generator loss function submodule and a discriminator loss function submodule, and the loss function calculation module adopts the following loss function:

[0097]

[0098] wherein x and z respectively represent data in the clean data set P data(x) and the artifact data set P noise(z) in the training data set, D(x) represents the return value of the data x input to the discriminator loss function submodule D, and G(z) represents the return value of the data z input to the generator loss function submodule G.

[0099] Further, when the generator loss function submodule is fixed, the discriminator loss function submodule is trained, and the loss function of the discriminator loss function submodule is as follows:

[0100]

[0101] That is, when the input is the real clean data, the discriminator loss function submodule needs to return a high score; when the input is the artifact data, the discriminator loss function submodule needs to return a low score. That is, the training process of the discriminator loss function submodule is the process of taking the maximum value of L GAN .

[0102] Further, when the discriminator loss function submodule is fixed, the generator loss function submodule is trained, and the loss function of the generator loss function submodule is as follows:

[0103]

[0104] That is, the output of the generator loss function submodule is the de-artifact prediction map, and when the discriminator loss function submodule outputs the return value, the corresponding label needs to be equal to 1, and then min G L GAN effect is achieved.

[0105] It should be understood that E in the above formula represents the expected function.

[0106] In step S204, the predicted de-artifact data output by the initial de-metal artifact model is restored to obtain the original data corresponding to the predicted de-artifact data.

[0107] Optionally, the restoration processing includes inverse normalization data, removing padding operations, restoring the width and height of the data, restoring the original size of the data, and converting the data type and saving.

[0108] Specifically, before inverse normalization data, the model output data range is adjusted to 0-1 through normalization operation; further, the normalized data is inverse normalized by using the inverse k-sigma transformation formula, so as to restore the data size subsequently. In this embodiment, the inverse k-sigma transformation formula is as follows:

[0109]

[0110] More specifically, the data expansion is removed, and p h and p w The 0 padding size of the outer edge of the data after inverse k-sigma is removed.

[0111] More specifically, the width and height of the data are restored, and the 4-channel data is restored to single-channel data. In this embodiment, first, the data dimensions are exchanged, i.e. (4, H / 2, W / 2)→(H / 2, W / 2, 4); further, the data can be split and converted to four-dimensional data, i.e. (H / 2, W / 2, 4)→(H / 2, 2, W / 2, 2); further, the width and height of the restored data can be converted to two-dimensional data, i.e. (H / 2, 2, W / 2, 2)→(H, W).

[0112] More specifically, restoring the original size of the data can use the size readjustment technology to adjust the size of the restored data to the original size of the data; further, the inverse normalization operation can be used to adjust the range of the readjusted data to 0-255.

[0113] More specifically, the converted data type is saved, and the re-adjusted grayscale image can be converted into a CBCT image according to a CT value range (i.e., the CT value range is -1000-5500) and saved as D ICOM data.

[0114] It should be noted that, since the input data is pre-processed in the present application, the output data needs to be de-processed to restore the size of the original data and to be visualized, so as to facilitate the comparison between the data after the subsequent contrast learning and the original data, and to facilitate the optimization of the model in step S205.

[0115] In step S205, the initial metal artifact reduction model is optimized by using the predicted artifact reduction data and the original data to obtain a target metal artifact reduction model.

[0116] In step S206, a CT image to be removed of metal artifacts is input into the target metal artifact reduction model to obtain a target CT image after metal artifact reduction.

[0117] In summary, the present application has the following beneficial effects:

[0118] (1) The present application aims to solve the problem of difficult collection of artifact and non-artifact paired data, and trains an end-to-end metal artifact reduction model from non-paired artifact images and clean images through non-paired learning technology, thereby improving the practicality and value of the method.

[0119] (2) The present application aims to solve the problem of high complexity and inference efficiency requirement of the model structure, and uses a nonlinear non-activation network to reduce the model parameter amount and improve the artifact reduction effect of the model, while the average inference efficiency of a single CBCT image reaches 54ms, basically meeting the actual application requirements.

[0120] (3) The present application aims to solve the problem of high difficulty in training a GAN model, and proposes an image blurring method to make the GAN model focus on the feature information in the image and accelerate the convergence of the model.

[0121] Third embodiment:

[0122] Referring to Figure 4 A CBCT metal artifact removal system based on non-paired learning is shown in the drawing, which comprises a data acquisition unit 510, a data processing unit 520, a model training unit 530, a data restoration unit 540, a model optimization unit 550 and a metal artifact removal unit 560. The specific functions of each unit are as follows:

[0123] The data acquisition unit 510 is used to acquire an artifact data set and a clean data set, wherein the artifact data set and the clean data set are non-paired data sets.

[0124] a data processing unit 520, configured to perform data processing on the artifact data set and the clean data set to obtain a processed training data set;

[0125] a model training unit 530, configured to train a metal artifact reduction model by using the training data set to obtain an initial metal artifact reduction model;

[0126] a data restoring unit 540, configured to perform restoring processing on predicted artifact reduction data output by the initial metal artifact reduction model to obtain original data corresponding to the predicted artifact reduction data;

[0127] a model optimizing unit 550, configured to optimize the initial metal artifact reduction model by using the predicted artifact reduction data and the original data to obtain a target metal artifact reduction model;

[0128] a metal artifact removing unit 560, configured to input a CT image to be removed of metal artifacts into the target metal artifact reduction model to obtain a target CT image after metal artifact reduction.

[0129] It should be noted that the specific functions of the above units can refer to the description of the method embodiments, and will not be described here.

[0130] Further, the embodiment also provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processing device, the steps of any one of the CBCT metal artifact removal methods based on unpaired learning provided in the above embodiment two are executed.

[0131] The computer program product of the CBCT metal artifact removal method and system based on unpaired learning provided in the embodiment of the application includes a computer readable storage medium storing program codes. The instructions included in the program codes can be used to execute the method described in the foregoing method embodiments. The specific implementation can be referred to the method embodiments, and will not be described here.

[0132] It should be noted that the above-described embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented using software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0133] It should be understood that the term "and / or" herein merely describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents that the associated objects before and after it are in an "or" relationship, but it can also represent an "and / or" relationship, which can be understood according to the context before and after it.

[0134] In this application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0135] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0136] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0137] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0138] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0139] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. can be located in one place or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0140] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0141] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

Claims

1. A non-paired learning based CBCT metal artifact removal method, characterized in that, The method comprises: obtaining an artifact data set and a clean data set, the artifact data set and the clean data set being non-paired data sets; data processing is performed on the artifact data set and the clean data set to obtain a processed training data set; training a metal artifact reduction model using the training data set to obtain an initial metal artifact reduction model; performing restoration processing on the predicted artifact reduction data output by the initial metal artifact reduction model to obtain original data corresponding to the predicted artifact reduction data, the restoration processing including inverse normalization processing, removal of padding operations, restoration of the width and height of the data, restoration of the original size of the data, and conversion of the data type; optimizing the initial metal artifact reduction model using the predicted artifact reduction data and the original data to obtain a target metal artifact reduction model; inputting a CT image to be removed of metal artifacts into the target metal artifact reduction model to obtain a target CT image after metal artifact reduction; wherein the initial metal artifact reduction model comprises a generator module, an image blurring module, and a discriminator module; wherein the generator module, the image blurring module, and the discriminator module are connected in sequence; the generator module receives the input training data set and generates a predicted artifact reduction image accordingly; the image blurring module sharpens the predicted artifact reduction image and the clean image in the training data set to generate a sharpened image; the discriminator module outputs a predicted probability value based on the sharpened image; the initial metal artifact reduction model includes a loss function calculation module, which includes a generator loss function submodule and a discriminator loss function submodule, and the loss function calculation module uses the following loss function: ； wherein, and respectively represent data in a clean data set and an artifact data set in the training data set, respectively represent data represent the return value of the input discriminator loss function submodule , represent the return value of the input generator loss function submodule , represent the expected operation on data x in the clean data set , represent the expected operation on data z in the artifact data set , .​ when the generator loss function submodule is fixed, the discriminator loss function submodule is trained, and the loss function of the discriminator loss function submodule is as follows: ； when the discriminator loss function submodule is fixed, the generator loss function submodule is trained, and the loss function of the generator loss function submodule is as follows: 。 2. The method of claim 1, wherein, the data processing of the artifact data set and the clean data set to obtain a processed training data set comprises: convert each three-dimensional CBCT image in the artifact data set and the clean data set into a plurality of two-dimensional image data; fix each two-dimensional image data to a predetermined size to obtain a plurality of new two-dimensional image data, each new two-dimensional image data being single-channel data; separate the width and height of each new two-dimensional image data to obtain multi-channel two-dimensional image data; use padding operations to expand the multi-channel two-dimensional image data to obtain expanded two-dimensional image data; use the k-sigma transformation formula to normalize the expanded two-dimensional image data to obtain a processed training data set.

3. The method of claim 2, wherein, the conversion of each three-dimensional CBCT image in the artifact data set and the clean data set into a plurality of two-dimensional image data comprises: According to the CT value range, the two-dimensional data of each layer of each three-dimensional CBCT image in the artifact data set and the clean data set is intercepted to obtain a plurality of two-dimensional image data.

4. The method of claim 2, wherein, The width and height of each of the new two-dimensional image data are separated to obtain multi-channel two-dimensional image data, including: The width and height of the multi-channel two-dimensional image data are separated to 1 / 2 of the original to generate four-dimensional data; The four-dimensional data is merged and converted into four-channel two-dimensional image data.

5. The method of claim 4, wherein, The multi-channel two-dimensional image data is expanded by using a padding operation to obtain expanded two-dimensional image data, including: According to the size padding algorithm, the data edge of the four-channel two-dimensional image data is padded to obtain the expanded two-dimensional image data.

6. The method of claim 5, wherein, The k-sigma satisfies: ； wherein, and represents an input artifact image and an output de-artifacted image, respectively, and are parameters obtained using linear equation fitting on artifact images in the training dataset.

7. A non-paired learning based CBCT metal artifact removal system, characterized in that, The system comprises: A data acquisition unit configured to acquire an artifact data set and a clean data set, wherein the artifact data set and the clean data set are non-paired data sets; A data processing unit configured to perform data processing on the artifact data set and the clean data set to obtain a processed training data set; A model training unit configured to train a metal artifact reduction model using the training data set to obtain an initial metal artifact reduction model; A data restoration unit configured to perform restoration processing on predicted artifact reduction data output by the initial metal artifact reduction model to obtain original data corresponding to the predicted artifact reduction data, wherein the restoration processing includes inverse normalization processing, removal of padding operation, recovery of data width and height, recovery of data original size, and conversion of data type; A model optimization unit configured to optimize the initial metal artifact reduction model using the predicted artifact reduction data and the original data to obtain a target metal artifact reduction model; A metal artifact removal unit configured to input a CT image to be removed of metal artifacts into the target metal artifact reduction model to obtain a target CT image after metal artifact reduction; The initial metal artifact reduction model comprises a generator module, an image blurring module, and a discriminator module; wherein the generator module, the image blurring module, and the discriminator module are connected in sequence; The generator module receives the training data set as input and generates a predicted artifact reduction image accordingly; The image blurring module performs sharpening processing on the predicted artifact reduction image and a clean image in the training data set to generate a sharpened image; The discriminator module outputs a prediction probability value according to the sharpened image; The initial metal artifact reduction model comprises a loss function calculation module, which comprises a generator loss function submodule and a discriminator loss function submodule, and the loss function calculation module adopts the following loss function: ； wherein, and respectively represent data in a clean data set and an artifact data set in the training data set, represents data input into a discriminator loss function submodule , represents data input into a generator loss function submodule , represents an expected operation on data x in a clean data set , represents an expected operation on data z in an artifact data set .​ When the generator loss function submodule is fixed, the discriminator loss function submodule is model trained, and the loss function of the discriminator loss function submodule is as follows: ； When the discriminator loss function submodule is fixed, the generator loss function submodule is model trained, and the loss function of the generator loss function submodule is as follows: 。

Images