Oral cavity CT metal artifact removal model training method and device, electronic equipment and storage medium

Abstract

The invention discloses a training method and device of an oral cavity CT metal artifact removal model, electronic equipment and a storage medium. The method comprises the following steps: determining an image containing a metal artifact feature based on an oral CT slice image with a metal artifact and a slice down-sampling image; performing feature extraction on the image containing the metal artifact features through a multi-head self-attention network to obtain the metal artifact features; image reconstruction is carried out to obtain a clean slice image after metal artifacts are removed; and completing model training based on the clean slice image after the metal artifacts are removed and the real clean slice image. According to the scheme, the prior structure of the metal artifact is obtained by determining the image containing the metal artifact feature, a traditional interpolation method does not need to be used for reconstructing a prior image and extracting a metal mask any more, the model data processing efficiency is improved, in addition, the multi-head self-attention network can capture the physical regularity and rotation invariance of the artifact, and the processing efficiency is improved. And the extraction precision of the metal artifact features is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a training method, device, electronic device, and storage medium for an oral CT metal artifact removal model. Background Art

[0002] Metal implants (such as dental fillings, artificial joints, or fixators) have extremely high density and will significantly absorb X-rays in computed tomography (CT). Since the X-rays absorbed by the metal exceed the dynamic range of the CT detector, artifacts (such as streak-like, radial, or shadow artifacts) are generated.

[0003] In existing deep learning-based methods for removing metal artifacts, before using neural networks for learning, traditional interpolation methods are often used to reconstruct prior images and extract metal masks, and then input them into the neural network, which takes a lot of time. Moreover, when dealing with extremely complex metal artifacts, artifact residues or overcorrection may still occur, resulting in low accuracy of metal artifact removal. Summary of the Invention

[0004] The present invention provides a training method, device, electronic device, and storage medium for an oral CT metal artifact removal model, so as to improve the model data processing efficiency and the accuracy of metal artifact removal.

[0005] According to an aspect of the present invention, a training method for an oral CT metal artifact removal model is provided, including:

[0006] Obtaining an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts;

[0007] Determining an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image;

[0008] Extracting features of the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features;

[0009] Performing image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts;

[0010] Determining a model loss based on the clean slice image after removing metal artifacts and a real clean slice image, and updating network parameters based on the model loss until a model training stop condition is met, to obtain an oral CT metal artifact removal model.

[0011] According to another aspect of the present invention, there is provided a training device for an oral CT metal artifact removal model, including:

[0012] A slice image acquisition module, configured to acquire an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts;

[0013] An image determination module containing metal artifact features, configured to determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image;

[0014] A multi-head self-attention mechanism feature extraction module, configured to perform feature extraction on the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features;

[0015] A slice image reconstruction module, configured to perform image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts;

[0016] A network parameter update module, configured to determine a model loss based on the clean slice image after removing metal artifacts and a real clean slice image, and update network parameters based on the model loss until a model training stop condition is met, to obtain an oral CT metal artifact removal model.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device includes:

[0018] At least one processor;

[0019] And a memory communicatively connected to the at least one processor;

[0020] Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the oral CT metal artifact removal model according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, there is provided a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the training method of the oral CT metal artifact removal model according to any embodiment of the present invention when executed by a processor.

[0022] The technical solution of the embodiment of the present invention is as follows: by obtaining an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts, then determining an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image, and then using a multi-head self-attention network to extract features from the image containing metal artifact features to obtain metal artifact features, and then performing image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts; determining a model loss based on the clean slice image after removing metal artifacts and the real clean slice image, and updating network parameters based on the model loss until the model training stop condition is met, to obtain an oral CT metal artifact removal model. The above technical solution determines an image containing metal artifact features, obtains the prior structure of metal artifacts, no longer needs to use traditional interpolation methods to reconstruct the prior image and extract the metal mask, improves the model data processing efficiency. In addition, the multi-head self-attention network can capture the physical regularity and rotational invariance of artifacts, thereby improving the extraction accuracy of metal artifact features, further improving the accuracy of image reconstruction, and further improving the accuracy of removing metal artifacts of the oral CT metal artifact removal model.

[0023] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Brief Description of the Drawings

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 is a flowchart of a method for training an oral CT metal artifact removal model according to Embodiment 1 of the present invention;

[0026] Figure 2 is a flowchart of a method for training an oral CT metal artifact removal model according to Embodiment 2 of the present invention;

[0027] Figure 3 is a flowchart of a method for training an oral CT metal artifact removal model according to Embodiment 3 of the present invention;

[0028] Figure 4 is a flowchart of a method for removing metal artifacts according to an embodiment of the present invention;

[0029] Figure 5 It is a comparison result diagram of models provided according to an embodiment of the present invention;

[0030] Figure 6 It is a schematic structural diagram of a training device for an oral CT metal artifact removal model provided according to Embodiment 4 of the present invention;

[0031] Figure 7 It is a schematic structural diagram of an electronic device for implementing the training method of the oral CT metal artifact removal model of the embodiment of the present invention. Detailed implementation manners

[0032] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices. The acquisition, storage, use, processing, etc. of data in the technical solution of this application all comply with the relevant regulations of national laws and regulations.

[0034] Embodiment 1

[0035] Figure 1 It is a flowchart of a training method for an oral CT metal artifact removal model provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of removing metal artifacts in medical images. This method can be executed by a training device for an oral CT metal artifact removal model. The training device for an oral CT metal artifact removal model can be implemented in the form of hardware and / or software, and the training device for an oral CT metal artifact removal model can be configured in a terminal and / or a server. As Figure 1 shown, the method includes:

[0036] S110. Obtain an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts.

[0037] Among them, the oral CT slice image with metal artifacts refers to the slice in the oral CT image with metal artifacts. The downsampled slice image refers to the image obtained by downsampling the oral CT slice image with metal artifacts.

[0038] Exemplarily, the oral CT slice image with metal artifacts can be read from a preset storage path of an electronic device, and then the oral CT slice image with metal artifacts is downsampled to obtain a downsampled slice image.

[0039] S120. Determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image.

[0040] Among them, the image containing metal artifact features can be used as the prior structure of the metal artifact and input into the neural network. It should be emphasized that by determining the image containing metal artifact features, the prior structure of the metal artifact is obtained, and there is no longer a need to use traditional interpolation methods to reconstruct the prior image and extract the metal mask, reducing the image processing time and thus improving the model data processing efficiency.

[0041] In some alternative embodiments, determining an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image includes: taking the difference between the oral CT slice image with metal artifacts and the downsampled slice image as the image containing metal artifact features.

[0042] Exemplarily, the calculation formula for the image containing metal artifact features is:

[0043] AR = ΔCT = CT real -CT downsampled ;

[0044] Among them, AR represents the image containing metal artifact features, CT real represents the oral CT slice image with metal artifacts, and CT downsampled represents the downsampled slice image.

[0045] S130. Extract features from the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features.

[0046] Among them, the multi-head self-attention network can capture the physical regularity and rotational invariance of artifacts in the image. It should be noted that by introducing the multi-head self-attention network, global feature extraction in different directions of metal artifacts can be achieved, thereby capturing the physical regularity and rotational invariance of artifacts in the image, improving the extraction accuracy of metal artifact features, further improving the accuracy of image reconstruction, and further improving the accuracy of metal artifact removal in the oral CT metal artifact removal model.

[0047] In some alternative embodiments, feature extraction is performed on an image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features, including: feature extraction is performed on an image containing metal artifact features through a multi-head self-attention network based on convolution to obtain metal artifact features.

[0048] Among them, the convolution can be two-dimensional convolution or convolution of other dimensions, which is not specifically limited here.

[0049] It should be noted that the calculations of Q (query), K (key), and V (value) in the self-attention mechanism are performed through convolution, and the weight sharing mechanism of the convolution kernel enables the model to have better generalization ability and computational efficiency in space.

[0050] Exemplarily, an image containing metal artifact features can be upsampled through transposed convolution, and the image size can be kept unchanged through parameters s and p:

[0051] H out =s·(H in -1)+k h -2p;

[0052] W out =s·(W in -1)+k w -2p;

[0053] Among them, H in and W in respectively represent the height and width of the image containing metal artifact features, H out and W out respectively represent the height and width of the upsampled image containing metal artifact features, k h and k w respectively represent the height and width of the convolution kernel, s represents the stride, and p represents the padding.

[0054] Furthermore, based on the multi-head self-attention mechanism of two-dimensional convolution, the upsampled image containing metal artifact features is processed, and the formula is as follows:

[0055]

[0056] Among them, b represents the bias of the two-dimensional convolution, w(i, j) represents the weight matrix of the convolution kernel, x(m + i, n + j) represents the pixel value of the input feature map x at the position (m + i, n + j), and the input feature map x is an upsampled image containing metal artifact features.

[0057] Furthermore, the subsequent processing steps are as follows:

[0058]

[0059] Output = Attention weights ·V;

[0060] Among them, d k represents the feature dimension. To prevent gradient vanishing or explosion and ensure the stability of softmax, is used as the scaling factor. Attention weights represents the attention weight, and Output represents the output of the multi-head self-attention.

[0061] Furthermore, a 1×1 convolution is used to perform channel fusion on the multi-head attention output to obtain metal artifact features. The calculation formula is as follows:

[0062] Output final = w o ·Output + b o ;

[0063] Among them, Output final represents the metal artifact features, and w o and b o represent the weight and bias of the 1×1 convolution respectively.

[0064] S140. Based on the oral CT slice image with metal artifacts and the metal artifact features, perform image reconstruction to obtain a clean slice image after removing the metal artifacts.

[0065] Exemplarily, the oral CT slice image with metal artifacts and the metal artifact features are input into the reconstruction network to obtain a clean slice image after removing the metal artifacts.

[0066] S150. Based on the clean slice image after removing the metal artifacts and the real clean slice image, determine the model loss, and update the network parameters based on the model loss until the model training stop condition is satisfied to obtain an oral CT metal artifact removal model.

[0067] In the embodiments of the present invention, the loss function for determining the model loss can be the mean squared error loss function or other custom loss functions, which are not specifically limited herein. The network parameters can include the weight parameters of the downsampling network, the multi-head self-attention network, and the reconstruction network, etc.

[0068] Exemplarily, the loss function for determining the model loss can be composed of the weighted mean squared error and the absolute error, and the corresponding formula is as follows:

[0069]

[0070] where, L MSE represents the mean squared error loss, L SAE represents the absolute error loss, N represents the number of samples, yi represents the true clean slice image of the i-th sample, and y′ i represents the clean slice image of the i-th sample after removing the metal artifact.

[0071] The reconstruction network can include multiple stages. In each stage, the clean slice image after removing the metal artifact and the metal artifact features are dynamically updated. Combining the cross-stage learning and the filter sharing mechanism, global elimination and correction of the artifact can be achieved. The cross-stage learning occurs in the reconstruction network and can be expressed as:

[0072]

[0073] where, X represents the image reconstruction result after cross-stage learning, X i represents the image reconstruction result of the previous stage, X i+1 represents the reconstruction result of the current stage; λ i+1 represents the step size parameter, which is used to control the weights of the reconstruction result of the current stage and the reconstruction result of the previous stage, represents the scaling factor, and the reconstruction result is the clean slice image after removing the metal artifact.

[0074] The technical solution of the embodiment of the present invention is to obtain an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts, and then determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image. Then, through a multi-head self-attention network, feature extraction is performed on the image containing metal artifact features to obtain metal artifact features. Then, image reconstruction is performed based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts. Determine the model loss based on the clean slice image after removing metal artifacts and the real clean slice image, and update the network parameters based on the model loss until the model training stop condition is met, and obtain an oral CT metal artifact removal model. The above technical solution determines an image containing metal artifact features, obtains the prior structure of metal artifacts, and no longer needs to use traditional interpolation methods to reconstruct the prior image and extract the metal mask, improving the model data processing efficiency. In addition, the multi-head self-attention network can capture the physical regularity and rotational invariance of the artifacts, thereby improving the extraction accuracy of metal artifact features, further improving the accuracy of image reconstruction, and further improving the accuracy of removing metal artifacts of the oral CT metal artifact removal model.

[0075] Embodiment 2

[0076] Figure 2 It is a flowchart of a training method for an oral CT metal artifact removal model provided by Embodiment 2 of the present invention. The method of this embodiment can be combined with various optional solutions in the training method of the oral CT metal artifact removal model provided in the above embodiment. The training method for the oral CT metal artifact removal model provided in this embodiment is further optimized. Optionally, the obtaining of the oral CT slice image with metal artifacts and the downsampled slice image corresponding to the oral CT slice image with metal artifacts includes: obtaining an original oral computed tomography image; determining whether there are slices with metal artifacts in the original oral computed tomography image; in the case where there are slices with metal artifacts in the original oral computed tomography image, downsample the oral CT slice image with metal artifacts to obtain the downsampled slice image corresponding to the oral CT slice image with metal artifacts.

[0077] As Figure 2 shown, the method includes:

[0078] S210. Obtain an original oral computed tomography image.

[0079] Among them, the original oral computed tomography image refers to the original oral CT image after scanning and reconstruction by a CT device.

[0080] Exemplarily, the electronic device may receive the original oral computed tomography (CT) image sent by the CT device.

[0081] S220. Determine whether there is a slice with metal artifacts in the original oral CT image.

[0082] S230. In the case where there is a slice with metal artifacts in the original oral CT image, downsample the oral CT slice image with metal artifacts to obtain a downsampled slice image corresponding to the oral CT slice image with metal artifacts.

[0083] In an embodiment of the present invention, downsampling can be implemented using a two-dimensional convolutional network, and the specific formula is as follows:

[0084] CT downsample = F.conv2d(CT real );

[0085] Wherein, CT real represents the oral CT slice image with metal artifacts, CT downsampled represents the downsampled slice image, and F.conv2d(·) represents the downsampling process. The initial parameters of the two-dimensional convolutional network can be a convolutional kernel with a size of 3×3, an input channel of 1, an output channel of 32, a stride of 1, and zero padding of 1 pixel at the edges to ensure that the input and output sizes remain unchanged.

[0086] It should be noted that by determining whether there is a slice with metal artifacts in the original oral CT image, the slices with metal artifacts can be screened out for subsequent processing, and the slices without metal artifacts are not processed, thereby reducing the data processing amount of the model and improving the data processing speed of the model.

[0087] Specifically, the gray value of each slice in the original oral CT image can be determined. In the case where the gray value of the slice is greater than the preset gray threshold, it is determined that the slice has metal artifacts; in the case where the gray value of the slice is not greater than the preset gray threshold, it is determined that the slice does not have metal artifacts.

[0088] Wherein, the preset gray threshold is a gray value threshold set by the user according to experiments. For example, the preset gray threshold can be 2000 HU or 3000 HU, etc., and no specific limitation is made here.

[0089] S240. Determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image.

[0090] S250. Extract features from the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features.

[0091] S260. Based on the oral CT slice image with metal artifacts and the metal artifact features, perform image reconstruction to obtain a clean slice image after removing the metal artifacts.

[0092] S270. Determine the model loss based on the clean slice image after removing the metal artifacts and the real clean slice image, and update the network parameters based on the model loss until the model training stop condition is met to obtain an oral CT metal artifact removal model.

[0093] The technical solution of the embodiment of the present invention can screen out the slices with metal artifacts in the original oral computed tomography image for subsequent processing, and do not process the slices without metal artifacts, thereby reducing the data processing volume of the model and improving the data processing speed of the model.

[0094] Embodiment III

[0095] Figure 3 As shown in the flowchart of a method for training an oral CT metal artifact removal model provided by Embodiment III of the present invention, the method of this embodiment can be combined with various optional solutions in the method for training an oral CT metal artifact removal model provided in the above embodiment. The method for training an oral CT metal artifact removal model provided in this embodiment is further optimized. Optionally, the performing image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing the metal artifacts includes: separating metal features from the oral CT slice image with metal artifacts based on the metal artifact features to obtain a clean slice image after removing the metal artifacts.

[0096] As Figure 3 shown, the method includes:

[0097] S310. Obtain an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts.

[0098] S320. Determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image.

[0099] S330. Extract features from the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features.

[0100] S340. Based on the metal artifact features, perform metal feature separation on the oral CT slice image with metal artifacts to obtain a clean slice image after removing the metal artifacts.

[0101] It should be noted that by performing metal feature separation processing on the oral CT slice image with metal artifacts, the metal artifact features can be removed from the oral CT slice image with metal artifacts, thereby obtaining a clean slice image after removing the metal artifacts, realizing the precise reconstruction of the slice image.

[0102] Optionally, based on the metal artifact features, perform metal feature separation on the oral CT slice image with metal artifacts to obtain a clean slice image after removing the metal artifacts, including: subtracting the metal artifact features from the oral CT slice image with metal artifacts to obtain a clean slice image after removing the metal artifacts.

[0103] Exemplarily, the formula for determining the clean slice image after removing the metal artifacts can be:

[0104] CT clean = CT real - AF;

[0105] where CT clean represents the clean slice image after removing the metal artifacts, CT real represents the oral CT slice image with metal artifacts, and AF represents the metal artifact features.

[0106] S350. Determine the model loss based on the clean slice image after removing the metal artifacts and the true clean slice image, and update the network parameters based on the model loss until the model training stop condition is met, to obtain an oral CT metal artifact removal model.

[0107] The technical solution of the embodiment of the present invention can remove the metal artifact features from the oral CT slice image with metal artifacts by performing metal feature separation processing on the oral CT slice image with metal artifacts, thereby obtaining a clean slice image after removing the metal artifacts, realizing the precise reconstruction of the slice image.

[0108] Exemplarily, Figure 4It is a flowchart of a method for removing metal artifacts according to an embodiment of the present invention. After obtaining an oral CT metal artifact removal model, the method further includes: inputting a CT image to be subjected to metal artifact removal into the trained oral CT metal artifact removal model to obtain a clean CT image. Among them, the oral CT metal artifact removal model may include a two-dimensional convolutional network, a multi-head self-attention network, and a reconstruction network. The two-dimensional convolutional network is used to downsample the oral CT slice image with metal artifacts. The multi-head self-attention network is used to extract features from the image containing metal artifact features. The reconstruction network is used to perform image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features. The reconstruction network may include multiple stages, and each stage includes an artifact optimization network and an artifact removal network. Among them, the artifact optimization network is used to optimize the image containing metal artifact features, and the artifact removal network is used to separate the metal features from the oral CT slice image with metal artifacts based on the metal artifact features to obtain a clean slice image after removing the metal artifacts. Each stage also includes a multi-head self-attention network, and the multi-head self-attention network is used as a filter to extract metal artifact features from the image.

[0109] Figure 5 It is a graph showing the comparison results of models according to an embodiment of the present invention. After obtaining the oral CT metal artifact removal model, the oral CT metal artifact removal model of the embodiment of the present invention is compared with the metal artifact removal models in the prior art. As Figure 5 shown, the PSNR (Peak Signal-to-Noise Ratio) / SSIM (Structural SIMilarity) scores of the oral CT metal artifact removal model (Ours) of the embodiment of the present invention are better than those of other existing models. The existing metal artifact removal models for comparison include the ACDNet model, the OSCNet model, and the InDuDoNet model. Ground Truth represents the real clean oral CT image, and Input represents the input CT image to be subjected to metal artifact removal.

[0110] In some alternative embodiments, after obtaining the oral CT metal artifact removal model, the method further includes: accelerating and optimizing the oral CT metal artifact removal model through a deep learning inference engine (TensorRT).

[0111] Specifically, through TensorRT's dynamic computation graph generation, multi-precision support, and layer fusion technologies, the efficient inference and deployment of the oral CT metal artifact removal model are realized, greatly improving the inference speed and resource utilization rate of the oral CT metal artifact removal model, meeting the real-time requirements for metal artifact removal, and at the same time ensuring the accuracy and robustness of the oral CT metal artifact removal model.

[0112] Exemplarily, the comparison table before and after TensorRT accelerates and optimizes the oral CT metal artifact removal model is shown in Table 1. s represents seconds, and the time in Table 1 is the inference time of a single CT slice.

[0113] Table 1

[0114] Graphics card Before TensorRT acceleration After TensorRT acceleration RTX 2060 1.1s 0.23s RTX 3060 laptop 1s 0.2s RTX 4090 0.1s 0.02s

[0115] Example 4

[0116] Figure 6 The following is a schematic structural diagram of a training device for an oral CT metal artifact removal model provided in Example 4 of the present invention. As Figure 6 shown, the device includes:

[0117] A slice image acquisition module 410, configured to acquire an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts;

[0118] An image determination module 420 including metal artifact features, configured to determine an image including metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image;

[0119] A multi-head self-attention mechanism feature extraction module 430, configured to perform feature extraction on the image including metal artifact features through a multi-head self-attention network to obtain metal artifact features;

[0120] A slice image reconstruction module 440, configured to perform image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts;

[0121] A network parameter update module 450, configured to determine a model loss based on the clean slice image after removing metal artifacts and a true clean slice image, and update network parameters based on the model loss until a model training stop condition is met, to obtain an oral CT metal artifact removal model.

[0122] The technical solution of the embodiment of the present invention is as follows: by obtaining an oral CT slice image with metal artifacts and a subsampled slice image corresponding to the oral CT slice image with metal artifacts, and then determining an image containing metal artifact features based on the oral CT slice image with metal artifacts and the subsampled slice image, and then using a multi-head self-attention network to extract features from the image containing metal artifact features to obtain metal artifact features, and then performing image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts; determining a model loss based on the clean slice image after removing metal artifacts and the real clean slice image, and updating network parameters based on the model loss until the model training stop condition is met to obtain an oral CT metal artifact removal model. The above technical solution determines an image containing metal artifact features, obtains the prior structure of metal artifacts, no longer needs to use traditional interpolation methods to reconstruct prior images and extract metal masks, improves the model data processing efficiency. In addition, the multi-head self-attention network can capture the physical regularity and rotational invariance of artifacts, thereby improving the extraction accuracy of metal artifact features, further improving the accuracy of image reconstruction, and further improving the accuracy of removing metal artifacts of the oral CT metal artifact removal model.

[0123] In some alternative embodiments, the slice image acquisition module 410 includes:

[0124] An original oral computed tomography image acquisition unit for acquiring an original oral computed tomography image;

[0125] A metal artifact slice judgment unit for judging whether there is a slice with metal artifacts in the original oral computed tomography image;

[0126] A metal artifact slice image subsampling unit for subsampling the oral CT slice image with metal artifacts to obtain a subsampled slice image corresponding to the oral CT slice image with metal artifacts when there is a slice with metal artifacts in the original oral computed tomography image.

[0127] In some alternative embodiments, the metal artifact slice judgment unit may specifically be used for:

[0128] Determining the gray value of each slice in the original oral computed tomography image;

[0129] When the gray value of the slice is greater than a preset gray threshold, determining that the slice is a slice with metal artifacts;

[0130] When the gray value of the slice is not greater than the preset gray threshold, determining that the slice is a slice without metal artifacts.

[0131] In some alternative embodiments, the image determination module 420 including metal artifact features may specifically be configured to:

[0132] Use the difference between the oral CT slice image with metal artifacts and the downsampled slice image as the image including metal artifact features.

[0133] In some alternative embodiments, the multi-head self-attention mechanism feature extraction module 430 may specifically be configured to:

[0134] Extract features from the image including metal artifact features through a multi-head self-attention network based on Fourier series expansion to obtain metal artifact features.

[0135] In some alternative embodiments, the slice image reconstruction module 440 includes:

[0136] A metal feature separation unit configured to separate metal features from the oral CT slice image with metal artifacts based on the metal artifact features to obtain a clean slice image after removing the metal artifacts.

[0137] In some alternative embodiments, the metal feature separation unit may specifically be configured to:

[0138] Subtract the metal artifact features from the oral CT slice image with metal artifacts to obtain a clean slice image after removing the metal artifacts.

[0139] The training device of the oral CT metal artifact removal model provided by the embodiments of the present invention can execute the training method of the oral CT metal artifact removal model provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0140] Embodiment Five

[0141] Figure 7 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0142] Such as Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The I / O interface 15 is also connected to the bus 14.

[0143] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0144] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the training method of the oral CT metal artifact removal model, and this method includes:

[0145] Obtain an oral CT slice image with metal artifacts and a downsampled slice image corresponding to the oral CT slice image with metal artifacts;

[0146] Determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the downsampled slice image;

[0147] Extract features from the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features;

[0148] Perform image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing metal artifacts;

[0149] Determine the model loss based on the clean slice image after removing metal artifacts and the true clean slice image, and update the network parameters based on the model loss until the model training stop condition is satisfied, so as to obtain an oral CT metal artifact removal model.

[0150] In some embodiments, the training method of the oral CT metal artifact removal model can be implemented as a computer program, which is tangibly included in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by the processor 11, one or more steps of the training method of the oral CT metal artifact removal model described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the training method of the oral CT metal artifact removal model in any other suitable manner (e.g., by means of firmware).

[0151] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor. The programmable processor can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing devices, such that when the computer programs are executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0156] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0157] It should be understood that various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0158] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A training method for an oral CT metal artifact removal model, characterized in that: include: Acquire an oral CT slice image with metal artifacts and a slice down-sampled image corresponding to the oral CT slice image with metal artifacts; Determining an image containing metal artifact features based on the oral CT slice image with metal artifacts and the slice downsampled image; Extracting features of the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features; Reconstructing the image based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing the metal artifacts; The model loss is determined based on the clean slice image after removing the metal artifacts and the real clean slice image, and the network parameters are updated based on the model loss until the model training stop condition is met to obtain the oral CT metal artifact removal model.

2. The method according to claim 1, characterized in that The step of acquiring an oral CT slice image with metal artifacts and a slice down-sampled image corresponding to the oral CT slice image with metal artifacts comprises: Original oral computed tomography images were obtained; Determining whether there is a slice with metal artifacts in the original oral computed tomography image; In the case that there are slices with metal artifacts in the original oral CT slice image, the oral CT slice image with the metal artifacts is downsampled to obtain a slice downsampled image corresponding to the oral CT slice image with the metal artifacts.

3. The method according to claim 2, characterized in that The determining whether there is a slice with metal artifacts in the original oral computed tomography image comprises: Determining the grayscale value of each slice in the original oral computed tomography image; When the grayscale value of the slice is greater than a preset grayscale threshold, determining that the slice is a slice with a metal artifact; When the grayscale value of the slice is not greater than a preset grayscale threshold, the slice is determined to be a slice without metal artifacts.

4. The method according to claim 1, characterized in that The step of determining an image containing metal artifact features based on the oral CT slice image having metal artifacts and the slice down-sampled image comprises: The difference between the oral CT slice image with metal artifacts and the slice down-sampled image is used as an image containing metal artifact features.

5. The method according to claim 1, characterized in that The method of extracting features of the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features includes: The image containing the metal artifact features is subjected to feature extraction through a convolution-based multi-head self-attention network to obtain the metal artifact features.

6. The method according to claim 1, characterized in that The image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after the metal artifacts are removed includes: Based on the metal artifact features, metal feature separation is performed on the oral CT slice image with metal artifacts to obtain a clean slice image after the metal artifacts are removed.

7. The method according to claim 6, characterized in that The method of performing metal feature separation on the oral CT slice image with metal artifacts based on the metal artifact features to obtain a clean slice image after removing the metal artifacts includes: The metal artifact feature is subtracted from the oral CT slice image with the metal artifact to obtain a clean slice image after the metal artifact is removed.

8. A training device for an oral CT metal artifact removal model, characterized in that: include: A slice image acquisition module, used for acquiring an oral CT slice image with metal artifacts and a slice down-sampled image corresponding to the oral CT slice image with metal artifacts; An image determination module containing metal artifact features, used to determine an image containing metal artifact features based on the oral CT slice image with metal artifacts and the slice down-sampled image; A multi-head self-attention mechanism feature extraction module is used to extract features of the image containing metal artifact features through a multi-head self-attention network to obtain metal artifact features; A slice image reconstruction module, used for performing image reconstruction based on the oral CT slice image with metal artifacts and the metal artifact features to obtain a clean slice image after removing the metal artifacts; The network parameter updating module is used to determine the model loss based on the clean slice image after removing the metal artifacts and the real clean slice image, and update the network parameters based on the model loss until the model training stop condition is met to obtain the oral CT metal artifact removal model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method for the oral CT metal artifact removal model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the training method for the oral CT metal artifact removal model according to any one of claims 1 to 7 when executed.

Images