Artifact removal model training method and device, equipment, medium and program product

Abstract

The application discloses a training method and device of an artifact removal model, equipment, a medium and a program product, and relates to the field of machine learning. The method comprises the following steps: acquiring a reference image and an artifact image matched with the image content; inputting the artifact image into a plurality of sample removal models to respectively output artifact removal results corresponding to the artifact image; determining prediction loss values respectively corresponding to the plurality of sample removal models; inputting the plurality of prediction loss values into a sample weight model to output weight parameters respectively corresponding to the plurality of prediction loss values; and training the plurality of sample removal models based on the prediction loss values and the weight parameters to obtain an artifact removal model composed of a plurality of artifact removal sub-models. That is, by determining the weight parameters respectively corresponding to the plurality of sample removal models through the sample weight model, the plurality of sample removal models can be flexibly trained according to the weight parameters in the training process, and the accuracy of the artifact removal model training is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, and in particular to a training method and device of an artifact removal model, equipment, medium and program product. BACKGROUND

[0002] Computed Tomography (CT) refers to that due to different absorption rates and transmission rates of X-rays of different tissues of the human body, different regions with different light and dark degrees are presented in a CT image, which is used to represent different tissue and organ structures. Therefore, the CT image is often applied to assist clinical diagnosis. However, in the CT scanning process, due to the influence of different factors, artifacts will be generated in the CT image. For example, when the oral cavity is scanned by CT, since the denture is implanted in the tooth, the generated CT image will be affected by the denture to generate a strip-shaped shadow, resulting in low image quality and affecting diagnosis.

[0003] In the related art, a dual-domain network (DuDoNet) is used to remove artifacts in a CT image. The dual-domain network is composed of two modules. A CT value processing window is preset to respectively process a chord diagram domain of a chord diagram containing artifacts and process an image domain of a CT image containing artifacts. A repaired chord image and an enhanced CT image are output. Finally, a CT image with removed artifacts is output by using an anti-projection layer.

[0004] However, in the related art, the method for removing a CT image containing artifacts by using a dual-domain network is to normalize the image data by using a single CT value processing window, which ignores the differences between different tissues and organs, resulting in low restoration authenticity, low image accuracy and poor image quality of the removed CT image. SUMMARY

[0005] The embodiments of the present application provide a training method, device, equipment, medium and program product of an artifact removal model, which can improve the accuracy of the output result of the artifact removal model. The technical solution is as follows:

[0006] In one aspect, a training method of an artifact removal model is provided, and the method comprises:

[0007] Obtaining a reference image matched with image content and an artifact image, the reference image is an image generated after scanning a sample detection object without containing an implant, and the artifact image is a reference image containing artifacts, the artifact being a shadow generated by the implant in the scanning process of a sample detection object containing the implant;

[0008] input the artifact image into a plurality of sample removal models, respectively output artifact removal results corresponding to the artifact image, different sample removal models correspond to different preset window ranges, the sample removal model is used for removing artifacts in the artifact image based on the corresponding preset window range;

[0009] Based on the pixel point difference between the artifact removal result and the reference image, determine the prediction loss value corresponding to each of the plurality of sample removal models;

[0010] Input the prediction loss value corresponding to each of the plurality of sample removal models into a sample weight model, and output weight parameters corresponding to each of the plurality of prediction loss values, the weight parameters are used for weight adjustment of parameter update of the sample removal model;

[0011] Based on the prediction loss value and the weight parameter, train the plurality of sample removal models to obtain an artifact removal model composed of a plurality of artifact removal sub-models, the artifact removal sub-model is used for artifact removal of a target image based on the corresponding preset window range.

[0012] On the other hand, an artifact removal model training device is provided, the device comprises:

[0013] The acquisition module is used for acquiring a reference image and an artifact image matched with the image content, the reference image is an image generated after scanning a sample detection object without containing an implant, the artifact image is a reference image containing artifacts, and the artifact is a shadow generated by the implant in the scanning process of a sample detection object containing the implant;

[0014] The input module is used for inputting the artifact image into a plurality of sample removal models, respectively outputting artifact removal results corresponding to the artifact image, different sample removal models correspond to different preset window ranges, the sample removal model is used for removing artifacts in the artifact image based on the corresponding preset window range;

[0015] The determination module is used for determining the prediction loss value corresponding to each of the plurality of sample removal models based on the pixel point difference between the artifact removal result and the reference image;

[0016] The input module is used for inputting the prediction loss value corresponding to each of the plurality of sample removal models into a sample weight model, and outputting weight parameters corresponding to each of the plurality of prediction loss values, the weight parameters are used for weight adjustment of parameter update of the sample removal model;

[0017] The training module is configured to train the plurality of sample removal models based on the prediction loss value and the weight parameter to obtain an artifact removal model composed of a plurality of artifact removal sub-models, wherein each artifact removal sub-model is configured to remove artifacts from a target image based on a corresponding preset window range.

[0018] In another aspect, a computer device is provided, which includes a processor and a memory having stored therein at least one instruction, at least one program, a code set or an instruction set, which is loaded and executed by the processor to implement the training method of the artifact removal model according to any one of the above embodiments.

[0019] In another aspect, a computer readable storage medium is provided, which has stored therein at least one instruction, at least one program, a code set or an instruction set, which is loaded and executed by a processor to implement the training method of the artifact removal model according to any one of the above embodiments.

[0020] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the training method of the artifact removal model according to any one of the above embodiments.

[0021] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0022] The plurality of sample removal models are trained by using the reference image and the artifact image matched in content, wherein the artifact image is input into the plurality of sample removal models to respectively output a plurality of artifact removal results during the training process, the prediction loss value between the plurality of artifact removal results and the reference image is determined, the weight parameter corresponding to each prediction loss value is finally obtained by inputting the prediction loss value into the sample weight model, and the plurality of sample removal models are trained according to the prediction loss value and the weight parameter, so that the artifact removal model containing the plurality of artifact removal sub-models is finally obtained. That is, the plurality of sample removal models corresponding to different preset window ranges are trained by using the weight parameter and the prediction loss value, so that the artifact removal model finally obtained can output the artifact removal image corresponding to different window ranges, meet the artifact removal demand of different images, and improve the artifact removal accuracy of the artifact removal result. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0024] Figure 1 is a schematic diagram of a training method of an artifact removal model provided by an example embodiment of the present application;

[0025] Figure 2 is a schematic diagram of an implementation environment provided by an example embodiment of the present application;

[0026] Figure 3 is a flowchart of a training method of an artifact removal model provided by an example embodiment of the present application;

[0027] Figure 4 is a flowchart of a training method of an artifact removal model provided by another example embodiment of the present application;

[0028] Figure 5 is a schematic diagram of a DICD-Net model provided by another example embodiment of the present application;

[0029] Figure 6 is a schematic diagram of a network structure provided by an example embodiment of the present application;

[0030] Figure 7 is a schematic diagram of a sample weight model network structure provided by an example embodiment of the present application;

[0031] Figure 8 is a schematic diagram of multiple sample removal models provided by an example embodiment of the present application;

[0032] Figure 9 is a schematic diagram of a training method of an artifact removal model provided by another example embodiment of the present application;

[0033] Figure 10 is a schematic diagram of an application process of an artifact removal model provided by an example embodiment of the present application;

[0034] Figure 11 is a schematic diagram of a training method of an artifact removal model provided by an example embodiment of the present application;

[0035] Figure 12 is a schematic diagram of a processing process of an artifact removal model provided by an example embodiment of the present application;

[0036] Figure 13 is a block diagram of a training device structure of an artifact removal model provided by an example embodiment of the present application;

[0037] Figure 14 is a training device structure block diagram of the artifact removal model provided by another exemplary embodiment of the present application;

[0038] Figure 15 is a structural schematic diagram of a server provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0040] First, the terms involved in the embodiments of the present application are briefly introduced.

[0041] Window technology: Window technology is a display technology used to observe normal tissues or lesions of different densities in computed tomography (CT) examination, including window width and window level. Since various tissue structures or lesions have different CT values, when you want to display the details of a specified tissue structure on a CT image, you need to select a window width and window level suitable for observing the specified tissue structure to form a specified window range to obtain the optimal display mode for the specified tissue structure, and generate a gray-scale image corresponding to the CT value of the specified window range.

[0042] Illustratively, please refer to Figure 1 which shows a training method schematic diagram of the artifact removal model provided by an exemplary embodiment of the present application, as shown in Figure 1 , a training image set 100 is obtained, wherein the training image set 100 includes a reference image 101 and an artifact image 102 that match in image content, wherein the reference image 101 and the artifact image 102 belong to a sample image pair, the reference image 101 and the artifact image 102 are both CT images obtained after performing computed tomography (CT) on the abdomen, and the artifact image 102 is an image containing artifacts (abdominal CT image contaminated by artifacts), and the reference image 101 does not contain artifacts (abdominal CT image not contaminated by artifacts).

[0043] The artifact image 102 is input into a plurality of sample removal models 110, and artifact removal results 111 of the artifact image 102 are respectively output, wherein each sample removal model in the plurality of sample removal models 110 corresponds to a different preset window range, and therefore the artifact removal results 111 are implemented as artifact removal images under different preset window ranges (for example, the image 1111 is a CT image under a [-1000, 2000] HU window range, the image 1112 is a CT image under a [-320, 480] HU window range, and the image 1113 is a CT image under a [-160, 240] HU window range). The prediction loss values 112 respectively corresponding to the plurality of sample removal models 110 are determined according to the pixel point differences between the artifact removal results 111 and the reference image 101.

[0044] The prediction loss values 112 are input into a sample weight model 120, and weight parameters 121 respectively corresponding to the prediction loss values 112 are output, wherein the weight parameters 121 are used for weight adjustment of parameter updating of the sample removal model 110. The plurality of sample removal models 110 are trained according to the prediction loss values 112 and the weight parameters 121, and finally an artifact removal model 130 composed of a plurality of artifact removal sub-models is obtained, wherein the artifact removal model 130 is used for artifact removal on an input target image containing artifacts.

[0045] Secondly, the implementation environment involved in the embodiments of the present application is described, and schematically, reference is made to Figure 2 The implementation environment involves a terminal 210 and a server 220, and the terminal 210 and the server 220 are connected through a communication network 230.

[0046] In some embodiments, the terminal 210 sends an artifact removal request to the server 220, wherein the artifact removal request includes a target scan image. In the present embodiment, the target scan image is implemented as a CT image contaminated by metal (that is, a CT image generated in a CT scanning process of a specified part of a human body is affected by metal implanted in the specified part, and metal artifacts are generated). After receiving the artifact removal request sent from the terminal, the server 220 performs artifact removal on the metal artifacts contained in the target scan image, generates an artifact removal result, and feeds back the artifact removal result to the terminal 210.

[0047] The server 220 includes an artifact removal model 221, and the server 220 inputs the target scan image into the artifact removal model 221, and outputs an artifact removal result, wherein the artifact removal result is a CT enhanced image generated after removing an artifact region identified in the target scan image.

[0048] The artifact removal model 221 is obtained by inputting the training artifact image 222 into a plurality of sample removal models 223, outputting a plurality of artifact removal results, determining a plurality of prediction loss values 224 according to the pixel point difference between the artifact removal results and a reference image (an image matching the image content of the artifact image 222 and without artifacts), inputting the prediction loss values 224 into a sample weight model 225, and outputting weight parameters 226 corresponding to the plurality of prediction loss values 224 respectively.

[0049] The terminal 210 can be a mobile phone, a tablet computer, a desktop computer, a portable notebook computer, a smart television, a smart vehicle, and the like. The embodiments of the present application do not limit the form of the terminal 210.

[0050] It should be noted that the server 220 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, and the like.

[0051] The cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0052] In some embodiments, the server 220 can also be implemented as a node in a blockchain system.

[0053] It should be noted that the information (including but not limited to user device information, user personal information, and the like), data (including but not limited to data for analysis, stored data, displayed data, and the like), and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data need to comply with relevant laws, regulations, and standards of relevant countries and regions. For example, the reference images and artifact images used for training and the verification images used for model verification in the present application are obtained under sufficient authorization.

[0054] For illustration purposes, the training method of the artifact removal model provided by the present application is described as follows. Figure 3It shows a flowchart of a training method of an artifact removal model provided by an example embodiment of the present application. The method can be executed by a terminal, a server, or both. In this embodiment, the method is executed by a server, as shown in Figure 3 The method includes the following steps.

[0055] In step 310, a reference image and an artifact image that match image content are obtained.

[0056] The reference image is an image generated after a sample detection object without an implant is scanned, and the artifact image is the reference image containing artifacts. The artifact is a shadow produced by the implant during scanning of a sample detection object containing the implant.

[0057] Illustratively, the reference image refers to a medical image generated after a sample detection object is scanned by a specified scanning technology. Generally, the artifact image is implemented as a grayscale image. The sample detection object is used to represent a specified tissue or organ (e.g., heart, abdomen, chest, lung, etc.).

[0058] In some embodiments, the reference image is a medical image obtained after a sample detection object without an implant is scanned. That is, the reference image is a medical image not affected by the implant.

[0059] In some embodiments, the implant refers to an object containing a metal component and implanted in the detection object, such as at least one of the following implant types: dental implant, pacemaker, stent, etc.

[0060] Illustratively, the specified scanning technology refers to a CT scanning technology. Therefore, the images involved in the embodiments of the present application are all CT images.

[0061] In some embodiments, the image content matching refers to the same content contained in the reference image and the artifact image, such as CT images generated after the same abdomen is scanned by CT. The difference between the artifact image and the reference image is that the artifact image is the reference image containing artifacts. That is, the reference image and the artifact image are implemented as a sample image pair.

[0062] Illustratively, the artifact refers to a shadow (or dark band) produced on the image by an object other than the sample detection object during scanning.

[0063] In step 320, the artifact image is input into a plurality of sample removal models, and artifact removal results corresponding to the artifact image are respectively output.

[0064] Different preset window ranges correspond to different sample removal models.

[0065] In some embodiments, the artifact removal result refers to removing the artifacts contained in the artifact image by the sample removal model, and outputting a scan image corresponding to the preset window range corresponding to the sample removal model, that is, the scan image does not contain artifacts.

[0066] The contrast relationship between the regions presented by the scan image is the same as or different from the contrast relationship between the regions presented by the artifact image, which is not limited.

[0067] Illustratively, the preset window range is used to represent the contrast relationship between the regions in the scan image, for example, the scan image includes region a and region b, under the corresponding preset window range A, the brightness of region a in the scan image is higher than that of region b, and under the corresponding preset window range B, the brightness of region a in the scan image is lower than that of region b. That is, when the same scan image corresponds to different preset window ranges, the contrast between the regions displayed is different, which facilitates targeted viewing of the specified region.

[0068] Optionally, the preset window range corresponding to the sample removal model is a pre-set fixed window range, for example, the preset window range corresponding to the sample removal model A is [-1000, 2000]HU; or the preset window range corresponding to the sample removal model is an adjustable window range set according to actual needs, which is not limited.

[0069] Optionally, the plurality of sample removal models correspond to the same model structure; or the plurality of sample removal models correspond to different model structures, which are not limited.

[0070] Step 330, based on the pixel point difference between the artifact removal result and the reference image, determine the prediction loss value corresponding to each of the plurality of sample removal models.

[0071] Illustratively, the prediction loss value is used to represent the difference between the pixel points between the artifact removal result and the reference image.

[0072] In some embodiments, the preset loss function calculates the distance between the pixel values corresponding to the artifact removal result and the pixel values corresponding to the reference image by the loss function, and the calculated result is taken as the prediction loss value corresponding to each of the plurality of sample removal models.

[0073] Step 340, input the prediction loss value corresponding to each of the plurality of sample removal models into the sample weight model, and output the weight parameter corresponding to each of the plurality of prediction loss values.

[0074] The weight parameter is used for weight adjustment of parameter updating of the sample removal model.

[0075] In some embodiments, the prediction loss values corresponding to the plurality of sample removal models are respectively input into the sample weight model, and a scalar result is output as a weight parameter corresponding to a single prediction loss value.

[0076] Optionally, the weight parameters corresponding to the plurality of prediction loss values output by the sample weight model are different from each other, or the weight parameters corresponding to at least two prediction loss values are the same.

[0077] Illustratively, the weight parameter is used to assign different weights to the prediction loss values in the training process of the sample removal model.

[0078] Optionally, after obtaining the prediction loss values corresponding to the plurality of sample removal models, the plurality of prediction loss values are simultaneously input into the sample weight model, and the weight parameters corresponding to the plurality of prediction loss values are simultaneously output, that is, the weight parameters corresponding to the plurality of prediction loss values are simultaneously obtained, or after obtaining the prediction loss value corresponding to each sample removal model, the prediction loss value is input into the sample weight model, and the weight parameter corresponding to the sample removal model is output, that is, the weight parameters corresponding to the plurality of prediction loss values are sequentially obtained, and the above is not limited.

[0079] At step 350, the sample removal model is trained based on the prediction loss value and the weight parameter, and a plurality of artifact removal sub-models are obtained.

[0080] The artifact removal sub-model is used for artifact removal of a target image based on a corresponding preset window range.

[0081] Illustratively, the first model parameter of the sample removal model is adjusted based on the prediction loss value and the weight parameter, and the artifact removal sub-model is determined based on the adjusted parameter.

[0082] In some embodiments, a single artifact removal sub-model is obtained after training of a single sample removal model, and a plurality of artifact removal sub-models constitute the artifact removal model.

[0083] Optionally, in the process of training the sample removal model based on the prediction loss value and the weight parameter, the training processes of the plurality of sample removal models are simultaneously performed, or the training processes of the plurality of sample removal models are sequentially performed, that is, after the first sample removal model is trained, the second sample removal model is trained, and the above is not limited.

[0084] In summary, the training method of the artifact removal model provided in the embodiments of the present application trains multiple sample removal models using the reference image and the artifact image matched in image content, wherein the artifact image is input into the multiple sample removal models during the training process, and multiple artifact removal results are output respectively, the prediction loss value between the multiple artifact removal results and the reference image is determined, the weight parameters corresponding to each prediction loss value are finally obtained by inputting the prediction loss value into the sample weight model, the multiple sample removal models are trained according to the prediction loss value and the weight parameters, and finally the artifact removal model containing multiple artifact removal sub-models is obtained. By training the multiple sample removal models corresponding to different preset window ranges using the weight parameters and the prediction loss value, the artifact removal model finally trained can output the artifact removal images corresponding to different window ranges, meet the artifact removal requirements of different images, and improve the artifact removal accuracy of the artifact removal results.

[0085] In an optional embodiment, taking the training of a single sample removal model as an example, the training process of the sample removal model is implemented as a multiple-loop iterative training process. For illustration, refer to Figure 4 which shows the flowchart of the training method of the artifact removal model provided in an example embodiment of the present application. The method can be executed by a terminal, a server, or both a terminal and a server. In this embodiment, the method is executed by a server, as shown in Figure 3 that is, step 350 includes steps 351, 352 and 353, and step 340 further includes step 341. The method includes the following steps:

[0086] Step 310: Obtain a reference image and an artifact image matched in image content.

[0087] The reference image is an image generated after scanning a sample detection object without containing an implant, and the artifact image is a reference image containing artifacts. The artifact is a shadow generated by the implant during the scanning of a sample detection object containing an implant.

[0088] In this embodiment, a metal artifact is taken as an example for illustration. The artifact composed of metal is implemented as a strip-shaped artifact.

[0089] For illustration, the reference image is a CT image generated after CT scanning of a sample detection image, and the artifact image is a CT image containing a metal artifact. That is, the current artifact image contains a metal artifact in the image generated after CT scanning due to the existence of metal in the sample detection image.

[0090] Optionally, the artifact image and the reference image are directly obtained from an authorized public dataset; or the reference image is directly obtained from a public dataset, and the artifact image is a reference image containing metal artifacts that is artificially synthesized based on the reference image and in combination with metal mask information corresponding to different metals, and is not limited in this regard.

[0091] In this embodiment, the reference image and the artifact image are implemented as a sample image pair.

[0092] In some embodiments, the reference image and the artifact image correspond to a scan image of the same sample detection object.

[0093] In step 320, the artifact image is input into a plurality of sample removal models to respectively output artifact removal results corresponding to the artifact image.

[0094] In this embodiment, the different sample removal models correspond to different preset window ranges, and the sample removal model is used to remove artifacts in the artifact image based on the corresponding preset window range.

[0095] Illustratively, the preset window range of the sample removal model is a pre-set fixed window range, such as: the preset window range of the sample removal model A is fixed as [-320, 480] HU.

[0096] Illustratively, the different sample removal models correspond to different preset window ranges.

[0097] In some embodiments, the sample removal model is used to remove artifacts in the artifact image, and then the artifact image is adjusted to a display mode corresponding to the preset window range according to the preset window range, and the output result is the artifact removal result, such as: the artifact image is a CT image containing metal artifacts with a window range of [-1000, 2000] HU, before the artifact image is input into the sample removal model (preset window range: [-320, 480] HU), the window range of the artifact image is adjusted to [-320, 480] HU, and then the artifact image is input into the sample removal model, the sample removal model removes the metal artifacts in the artifact image and outputs the image as the artifact removal result.

[0098] Illustratively, the image contents displayed by the artifact removal results corresponding to different preset window ranges remain consistent, and the area contrast of each artifact removal result is different.

[0099] Optionally, the sample removal model in the embodiments of the present application can be implemented as a neural network model such as a Deep Interpretables Convolutional Dictionary Network (DICD-Net), a Convolutional Neural Network (CNN), a U-net network, and the like, and is not limited thereto.

[0100] The following is described taking the sample removal model implemented as the DICD Net as an example.

[0101] For the artifact caused by metal, there is prior knowledge specific to the metal artifact, that is, the metal artifact has a non-local strip structure, which can play a guiding role in parameter learning of the sample removal model. Illustratively, refer to FIG. 6, which shows a DICD-Net model schematic diagram provided by an example embodiment of the present application. Figure 5 Figure 5 As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence.

[0102] As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. Figure 5 As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence.

[0103] The following describes a single Stage in the N iteration removal process.

[0104] Illustratively, as shown in FIG. 6, which shows a network structure schematic diagram provided by an example embodiment of the present application, as shown in FIG. 6, in the current single Stage, Figure 6 As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. Figure 6 As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence. As shown in FIG. 6, the DICD-Net 500 includes N iteration processes, and in any iteration process, the single iteration process is composed of a network and an X network (X-Net) in sequence.

[0105] Illustratively, The network structure of the network 610 can refer to Formula One as follows:

[0106] Formula One: ​​

[0107] wherein, represents the output result of the X network in the n-th iteration removal process, the output result of the network, represents the residual network 630, each residual block in the residual network 630 sequentially comprises a convolutional layer, a batch normalization layer (Batch Normalization), a ReLU layer, a convolutional layer, a batch normalization layer, and a cross-linking layer.

[0108] wherein, Y represents an input artifact image (and the artifact image contains metal artifacts); X represents an artifact removal result (X (n-1) represents an artifact removal result obtained in the n-1-th iteration stage); I represents mask information (Mask) corresponding to non-metal artifacts in the artifact image; represents a mode in which the metal artifact repeatedly appears, which can be understood in the embodiment as a display mode of the metal artifact, is a feature layer, representing a strip-shaped artifact structure of the metal artifact, which can be understood in the embodiment as a feature map (Feature Map) corresponding to the metal artifact; η1 is an update step size of the X network. an update step size of the network.

[0109] Illustratively, the network structure of the X network 620 can refer to the following Formula Two:

[0110] Formula Two:

[0111] wherein, X (n) represents the output result of the X network in the n-th iteration removal process, represents the residual network 640, each residual block in the residual network 640 sequentially comprises a convolutional layer, a batch normalization layer (Batch Normalization), a ReLU layer, a convolutional layer, a batch normalization layer, and a cross-linking layer.

[0112] wherein, Y represents an input artifact image (and the artifact image contains metal artifacts); X represents an artifact removal result (X (n-1) represents an artifact removal result obtained in the n-1-th iteration stage); I represents mask information (Mask) corresponding to non-metal artifacts in the artifact image, represents a mode in which the metal artifact repeatedly appears, which can be understood in the embodiment as a display mode of the metal artifact, is a feature layer, representing a strip-shaped artifact structure of the metal artifact, which can be understood in the embodiment as a feature map (Feature Map) corresponding to the metal artifact, and η2 is an update step size of the X network.

[0113] According to the above formula 1 and formula 2, as shown in formula 3, when the (n-1)th iteration removal stage obtains the (n-1)th feature map X Figure 6 and the artifact removal result (X (n-1) ) obtained by the (n-1)th iteration removal stage are input into the network, the (n)th feature map X obtained by the (n)th iteration removal stage is output. The (n)th feature map X and the artifact removal result (X (n-1) ) obtained by the (n-1)th iteration stage are input into the network, and the artifact removal result (X (n) ) of the (n)th iteration stage is output.

[0114] Step 330: Based on the pixel point difference between the artifact removal result and the reference image, the prediction loss value corresponding to each of the plurality of sample removal models is determined.

[0115] Illustratively, a preset loss function is used to calculate the pixel value difference between the artifact removal result and the reference image, and the calculated result is taken as the prediction loss value corresponding to each of the plurality of sample removal models.

[0116] In this embodiment, the preset loss function is formula 4. Wherein, b represents the sample removal model corresponding to the bth preset window range, and Θ represents the first model parameter of the sample removal model.

[0117] Step 341: The prediction loss obtained by the (s-1)th iteration training is input into the sample weight model obtained by the s th iteration training, and the weight parameter corresponding to the s th iteration training is output.

[0118] In this embodiment, in the process of training the sample removal model, the sample weight model also needs to be trained.

[0119] Next, first, the training process of the sample weight model is described.

[0120] In some embodiments, a verification reference image and a verification artifact image matching the image content are obtained; the verification artifact image is input into the plurality of sample removal models, and a verification removal result corresponding to the verification artifact image is respectively output; based on the pixel point difference between the verification removal result and the verification reference image, a verification loss value corresponding to each of the plurality of sample removal models is determined; and based on the verification loss value, the sample weight model is trained.

[0121] ​Illustratively, the verification reference image is a CT image obtained after CT scanning of a verification detection object, and the verification artifact image is a CT image containing metal artifacts, wherein the verification reference image and the verification artifact image correspond to the same image content (for example, the verification reference image and the verification artifact image are both CT images obtained after CT scanning of the same abdomen, wherein the verification artifact image includes metal artifacts, and the verification reference image does not include artifacts).

[0122] In this embodiment, the verification reference image and the verification artifact image are one image pair in the verification samples.

[0123] In this embodiment, the verification artifact image and the artifact image belong to different CT images.

[0124] Illustratively, the verification artifact image is input into a plurality of sample removal models, and after removing the artifacts in the artifact image by the plurality of sample removal models, an image corresponding to the preset window range of the sample removal model is generated as a verification removal result. Each sample removal model corresponds to one verification removal result.

[0125] Illustratively, a preset loss function is used to calculate the pixel point difference between the plurality of verification removal results and the verification reference image as a verification loss value corresponding to the sample removal model. Each of the plurality of sample removal models corresponds to a plurality of verification loss values.

[0126] In this embodiment, the preset loss function is implemented as The output result of the loss function is used to represent the verification loss value corresponding to the bth sample removal model.

[0127] Illustratively, the second model parameter of the sample weight model is adjusted based on the plurality of verification loss values.

[0128] In some embodiments, during s times of iterative training, the second model parameter of the sample weight model is adjusted based on the verification loss value obtained in the s-1th iteration training, to obtain a sample weight model corresponding to the sth iteration training.

[0129] In this embodiment, the process of training the sample weight model includes N times of iterative training of the sample weight model (corresponding to the N times of iteration process included in the DICD-Net network described above), that is, during the N times of iteration process, the sample weight model and the sample removal model are iteratively updated.

[0130] The scheme implemented in the present application is to train the sample weight model by the verification loss value corresponding to each of the plurality of sample removal models. Taking the process of training the sample weight model for a single verification loss value as an example, the gradient adjustment manner for the second model parameter of the sample weight model can be specifically referred to formula three:

[0131] Formula three:

[0132] Wherein, θ (s) represents the sample weight model corresponding to the s-th iteration training, β is a preset second learning rate, used to represent the update step of training the sample weight model, represents the verification loss value corresponding to the b-th sample removal model.

[0133] Wherein, is used to represent the mapping function about θ in the s-1 iteration process, since the second model parameter (θ) has not been updated in the current s iteration training process, a mapping function about θ is set to represent the mapping relationship between the first model parameter (Θ) corresponding to the sample removal model and the second model parameter (θ) corresponding to the sample weight model. That is, based on the first model parameter obtained by s-1 iteration training, the mapping relationship between the first model parameter and the second model parameter corresponding to the s-1 iteration training is determined; based on the mapping relationship, the verification loss value obtained by s-1 iteration training is determined.

[0134] Illustratively, the mapping function about θ can be specifically referred to formula four:

[0135] Formula four:

[0136]

[0137] Wherein, is used to represent the mapping function about θ in the s-1 iteration training process, α represents a preset first learning rate, is used to represent the network structure of the sample weight model in the s-1 iteration training process, that is, the network mapping function containing θ.

[0138] Step 351, based on the plurality of prediction loss values and the weight parameters corresponding to the plurality of loss values respectively, determine the weighted loss values corresponding to the plurality of sample removal models respectively.

[0139] Illustratively, for the optimization target of the plurality of DICD-Net, the output loss value of the loss function can be minimized by adjusting the model parameters of the plurality of DICD-Net respectively through the preset loss function, that is, the final optimization target of the artifact removal model, therefore, the loss function for the artifact removal model containing the plurality of DICD-Net can refer to formula five for details:

[0140] Formula five:

[0141] Wherein, is the loss sum of the prediction loss values corresponding to the B sample removal models (DICD-Net) respectively in the model, is used to represent the prediction loss value of the bth DICD-Net corresponding to the artifact image, W b is used to represent the weight parameter corresponding to the bth prediction loss value.

[0142] Since the weight parameter corresponding to the sample removal model is obtained through the sample weight model in the embodiment of the present application, therefore, Wherein, represents the network structure of the sample weight model, and the input is the prediction loss value The output is the weight parameter W, and the corresponding second model parameter in the network structure is θ.

[0143] In the embodiment, the is set to a multi-layer perceptron (MLP) network containing a hidden layer, illustratively, please refer to Figure 7 , which shows the network structure diagram of the sample weight model provided by an example embodiment of the present application, as Figure 7 shown, the current display MLP network 700, which includes an input layer 710, a hidden layer 720 and an output layer 730, the input layer is the prediction loss value 711, and the output layer 730 is the weight parameter 731 corresponding to the prediction loss value 711, wherein the hidden layer 720 contains a plurality of neurons, and the number of neurons in the hidden layer 720 is limited according to actual needs.

[0144] In some embodiments, in the s-th iteration training process, the s-th weighted loss value is determined based on the s-1th prediction loss value and the s-th weight parameter.

[0145] Illustratively, the weighted loss value is used to represent the corresponding prediction loss value of the sample removal model after weight adjustment, which can refer to formula six for details:

[0146] Formula six:

[0147] Wherein, a weight parameter corresponding to a predicted loss value, a predicted loss value, a loss sum of weighted loss values corresponding to B sample removal models respectively in a single iteration training process.

[0148] Step 352, adjusting the first model parameters of the plurality of sample removal models respectively based on the weighted loss values corresponding to the plurality of sample removal models respectively, to obtain a plurality of artifact removal sub-models.

[0149] Illustratively, the first model parameters of the plurality of sample removal models are adjusted by gradients based on the plurality of weighted loss values, to obtain the adjusted parameters as the model parameters corresponding to the artifact removal models, and further to obtain the artifact removal models. The specific training process can refer to Formula Seven:

[0150] Formula Seven:

[0151] wherein, Θ (s-1) to represent the first model parameters obtained in the s-1th iteration training process (for the plurality of first model parameters corresponding to the B sample removal models respectively in a single iteration training), to represent that the s-1th first model parameters are adjusted by gradients based on the weighted loss value corresponding to the s th iteration training, that is, the first model parameters obtained by the s-1th iteration training of the sample removal model are adjusted by gradients based on the weighted loss value corresponding to the s th iteration training, to obtain the first model parameters corresponding to the s th iteration training, and to perform the s+1th loop adjustment until the training of the artifact removal model is completed, s≥1 and s is an integer.

[0152] It is worth noting that the training sequence corresponding to the above-mentioned training process of the sample weight model and the sample removal model is Formula Four (parameterizing θ to obtain the mapping function about θ), Formula Three (adjusting the second model parameters of the sample weight model by gradients) and Formula Seven (adjusting the first model parameters of the plurality of sample removal models respectively by gradients).

[0153] In this embodiment, the training target corresponding to Formula Eight is given for the adjustment effect condition of the sample removal model:

[0154] Formula Eight:

[0155] wherein, Θ * to represent the optimal solution (that is, the first parameters meeting the adjustment effect condition) corresponding to the plurality of first model parameters, and the optimal solution of the plurality of first model parameters can be obtained by calculating the minimum value of the weighted loss value.

[0156] Illustratively, for the second model parameter of the sample weight model, we can also give the training target corresponding to formula nine:

[0157] Formula nine:

[0158] Where θ * is used to represent the optimal solution of the second model parameter, which can be obtained by calculating the minimum value of the predicted loss value. When the second model parameter reaches the optimal solution (or infinitely approaches the optimal solution), the second model parameter is used as the final training of the sample weight model.

[0159] It is worth noting that formula eight and formula nine are only training targets set for the sample removal model and the sample weight model. In the actual training process, the training process of the sample removal model and the sample weight model is iteratively trained in the specified order according to formula four, formula three and formula seven.

[0160] Illustratively, after training multiple sample removal models, multiple artifact shadow models corresponding to the multiple sample removal models are generated, that is, a single artifact shadow model corresponds to a single sample removal model.

[0161] Step 353, the multiple artifact removal sub-models are used as the artifact removal model.

[0162] Illustratively, the artifact removal model includes multiple sample removal models that have completed training, that is, the artifact removal model includes multiple artifact removal sub-models.

[0163] In some embodiments, in response to the number of times of cyclic iteration adjustment of the first model parameter reaching the number threshold, the first model parameter obtained by the last adjustment is determined as the first parameter; or; in response to the adjustment effect of the first model parameter meeting the adjustment effect condition, the first model parameter is determined as the first parameter, and the adjustment effect condition is used to represent the limitation requirement on the predicted loss value.

[0164] Illustratively, the model parameter in the artifact removal model is the first parameter, and the training target of the multiple sample removal models in this embodiment is the result obtained by gradient adjustment of the first model parameter corresponding to the multiple sample removal models, which is used as the final first parameter corresponding to the artifact removal model.

[0165] Optionally, the number threshold is a specified number set in advance or set according to the training situation, such as: training 100 times for multiple sample removal models, then 100 times is the number threshold, when the number of cyclic iteration adjustment of the first model parameter of the multiple sample removal models reaches 100 times, the multiple first model parameters obtained by the 100th training are determined as the first parameter.

[0166] Optionally, the adjustment effect condition refers to that when the gradient adjustment is performed on the plurality of sample removal models, the prediction loss value between the artifact removal result output by the artifact image input into the artifact image after the gradient adjustment is performed on the plurality of sample removal models and the reference image meets the adjustment effect condition, the first model parameters of the plurality of sample removal models are determined as the first parameters of the artifact removal network, that is, the first model parameters of this training meet the training target of the sample removal model.

[0167] In summary, the training method of the artifact removal model provided in the embodiments of the present application trains the plurality of sample removal models by using the reference image and the artifact image matched in image content, wherein in the training process, the plurality of artifact removal results are output after the artifact image is input into the plurality of sample removal models, the prediction loss values between the plurality of artifact removal results and the reference image are determined, the weight parameters corresponding to each prediction loss value are finally obtained after the prediction loss values are input into the sample weight model, the plurality of sample removal models are trained according to the prediction loss values and the weight parameters, and finally the artifact removal model containing the plurality of artifact removal sub-models is obtained. By training the plurality of sample removal models corresponding to different preset window ranges by using the weight parameters and the prediction loss values, the artifact removal model finally trained can output the artifact removal images corresponding to different window ranges, meet the artifact removal requirements of different images, and improve the artifact removal accuracy of the artifact removal result.

[0168] In the embodiments, the sample weight model is first trained, the weight parameters are output by inputting the prediction loss values into the sample weight model obtained by training, and the gradient adjustment is performed on the first model parameters of the plurality of sample removal models according to the weight parameters and the plurality of prediction loss values, so that in the iterative training process, the sample weight model and the sample removal model are alternately iteratively trained, the training of the sample weight model can assist the training of the sample removal model, and the accuracy and training effect of the model training are improved.

[0169] In an optional embodiment, the process of window conversion is further included for different preset window ranges, so that the artifact removal model can output the artifact removal result corresponding to the window range of the input artifact image. Figure 3 For example, the steps are described by taking the application of the scheme of the present application to the auxiliary diagnosis field as an example:

[0170] In step 310, the reference image and the artifact image matched in image content are obtained.

[0171] The reference image is an image generated after a sample detection object not containing an implant is scanned, and the artifact image is a reference image containing artifacts. The artifact is a shadow generated by the implant in the scanning process of a sample detection object containing an implant.

[0172] The artifact image is a CT image obtained by image synthesis on the authorized CT image in the public data set obtained in advance.

[0173] In this embodiment, for the acquisition process of the artifact image, first, the CT image in the public data set that is not affected by the metal is used as the reference image, and different types of metal masks are used. According to the data simulation process, the CT image and the metal mask are synthesized to obtain a CT image containing metal artifacts as training data.

[0174] The CT value corresponding to the training data is cropped to obtain a CT image with a window range of [-1000, 2000] HU, and then converted to an attenuation coefficient, which is normalized to the range [0, 1], and finally converted to a CT image in the range [0, 255].

[0175] Each training data is randomly cropped to an image block with a side length of 64x64, and then randomly flipped horizontally and vertically with a probability of 0.5, respectively, for final generation of different artifact images.

[0176] In this embodiment, the reference image and the artifact image are implemented as a sample image pair.

[0177] Illustratively, the plurality of sample removal models correspond to different preset window ranges, respectively, for labeling the contrast relationship displayed by each region of the scan image under different CT values. In this embodiment, three sample removal models are taken as examples for illustration, that is, the three sample removal models included in this embodiment each correspond to a preset window range of [-1000, 2000] HU, [-320, 480] HU, and [-160, 240] HU, which are not limited.

[0178] Illustratively, please refer to Figure 8 which shows a plurality of sample removal model schematic diagrams provided by an example embodiment of the present application, as shown in Figure 8 The current display multi-window sample removal model schematic diagram, wherein the multi-window sample removal model includes three sample removal models 810, 820, and 830 corresponding to different preset window ranges, respectively. The sample removal model is implemented as a DICD-Net structured model, therefore, the sample removal model 810 is a DICD-Net(b=1) corresponding to the first preset window range([-1000, 2000] HU), the sample removal model 820 is a DICD-Net(b=2) corresponding to the second preset window range([-320, 480] HU), and the sample removal model 830 is a DICD-Net(b=3) corresponding to the third preset window range([-160, 240] HU).

[0179] The artifact image 901 is input into the sample removal model 910, and a first prediction region 911 corresponding to the sample removal model 910 is output, where the first prediction region 911 is used to represent a CT image generated by removing the metal artifact in the artifact image 901 by the sample removal model 910 with the first preset window range as a reference.

[0180] Illustratively, the artifact image also corresponds to a preset window range before being input into the sample removal model, and thus, when the preset window range of the artifact image and the preset window range of the input sample removal model belong to different window ranges, the artifact image needs to be window converted before being input into the sample removal model to generate an artifact image consistent with the preset window range corresponding to the sample removal model.

[0181] As shown in FIG. 8, the artifact image 801 and the first prediction region 811 are input into a window conversion layer (Window Layer, marked as W) to perform window conversion, and a first sample conversion result 802 corresponding to the artifact image 801 and a first prediction conversion result 812 corresponding to the first prediction region 811 are obtained, where the window ranges of the first sample conversion result 802 and the first prediction conversion result 812 are consistent with the preset window range of the sample removal model 820. Figure 8 Figure 8 The first prediction region 811 and the first prediction conversion result 812 are input into a channel fusion layer (Channel Concatenation, marked as C), and a first fusion result is obtained. Figure 8 The first fusion result is input into the sample removal model 820, and a second prediction region 821 corresponding to the sample removal model 820 is output.The artifact image 801, the first prediction region 811, and the second prediction region 821 are input into the window conversion layer to perform window conversion, and a second sample conversion result 803 corresponding to the artifact image 801, a second prediction conversion result 813 corresponding to the first prediction region 811, and a third prediction conversion result 822 corresponding to the second prediction region 821 are obtained.

[0182] In some embodiments, a window range corresponding to an i-th sample removal model is determined, where i is a positive integer; and the artifact image and an (i-1)-th artifact removal result are window converted to obtain window conversion results corresponding to the artifact image and the (i-1)-th artifact removal result, respectively, as model inputs of the i-th sample removal model.

[0183] Illustratively, the window conversion process can refer to the following formulas ten to twelve:

[0184] Formula ten: X ori = X curr × (H curr - L curr ) + L curr

[0185] Formula eleven: X clip = Clip (X ori ; [L next , H next ])

[0186] Formula twelve:

[0187] Wherein, X ori is used to represent the original image, X curr and X next respectively represent the corresponding scan images before and after window conversion, [L curr , H curr ] and [L next , H next ] respectively represent the preset window range (L represents the window level, and H represents the window height) corresponding before and after window conversion.

[0188] Step 320, input the artifact image into a plurality of sample removal models, and respectively output the artifact removal results corresponding to the artifact image.

[0189] Wherein, different sample removal models correspond to different preset window ranges, and the sample removal model is used to remove artifacts in the artifact image based on the corresponding preset window range.

[0190] In this embodiment, three artifact removal results are respectively output by three sample removal models with different preset window ranges, and the pixel point distance between the three artifact removal results and the reference image is calculated by a preset loss function respectively, and three prediction loss values are output, which are and

[0191] Step 330, based on the pixel point difference between the artifact removal result and the reference image, determine the prediction loss value corresponding to each of the plurality of sample removal models.

[0192] In this embodiment, the three prediction loss values are respectively input into the sample weight model, and three weight parameters are output, wherein the sample weight model is realized as

[0193] Step 340, input the prediction loss value corresponding to each of the plurality of sample removal models into the sample weight model, and output the weight parameter corresponding to each of the plurality of prediction loss values.

[0194] The weight parameter is used to adjust the weights for updating the parameters of the sample removal model.

[0195] In this embodiment, three sample removal models are trained simultaneously based on three predicted loss values ​​and three corresponding weight parameters to obtain three artifact removal sub-models. These three artifact removal sub-models are then used as the final artifact removal network (Mar Network).

[0196] In this embodiment, during the training process of multiple sample removal models, the training process of multiple sample removal models and sample weight models is carried out iteratively and alternately. That is, the first model parameters of multiple sample removal models and the second model parameters of sample weight models are updated alternately, and the first model parameters are adjusted by the updated second model parameters.

[0197] This is illustrative; please refer to it. Figure 9 It illustrates a schematic diagram of a training method for an artifact removal model provided in an exemplary embodiment of this application, such as... Figure 9 As shown, the artifact image 900 is input into the sample removal model 920 (which contains three models, the number of which is not shown in the figure), and the outputs are artifact removal results 931, 932 and 933. The sample removal model 920 and the sample weight model 940 are trained alternately.

[0198] like Figure 9 As shown, for the s-th iteration of the training process, the first model parameters (Θ) are obtained after the (s-1)-th iteration of the training process. (s-1) ) and second model parameters (θ) (s-1) First, according to Formula 4 above, the mapping function about θ obtained after the (s-1)th iteration of training is determined by the first model parameters obtained after the first iteration of training. Then, according to Formula 3 above, the mapping function about θ is obtained after the (s-1)th iteration training. Determine the validation loss value obtained after the (s-1)th iteration of training, and apply the validation loss value obtained after the (s-1)th iteration of training to θ. (s-1) Perform gradient adjustment to obtain the second model parameters (θ) after the s-th iteration of training. (s) According to θ (s) Determine the sample weight model corresponding to the s-th iteration training. Input the predicted loss value obtained after the (s-1)-th iteration training into the sample weight model corresponding to the s-th iteration training to obtain the weight parameters corresponding to the s-th iteration training. Using Formula 7, determine the weighted loss value corresponding to the s-th iteration training based on the predicted loss value obtained after the (s-1)-th iteration training and the weight parameters corresponding to the s-th iteration training. This weighted loss value is used to adjust Θ. (s-1)Gradient adjustment is performed to obtain the second model parameter (Θ (s) ) corresponding to the s-th iteration training, and the iterative training is performed in this way until the sample removal model training is completed.

[0199] It is worth noting that when the first iteration training is performed, the first model parameter and the second model parameter are both initial values set in advance.

[0200] In some embodiments, a first learning decay rate is obtained, which is a decay form of adjusting the first learning rate according to the number of iterations, and the first learning rate is a pre-set update step for training the plurality of sample removal models; during the training of the plurality of sample removal models, the first learning rate is gradient-decreased based on the first learning decay rate to obtain a target learning rate corresponding to the artifact removal model.

[0201] In this embodiment, the first learning rate for training the sample removal model is set to 2×10 -4 , the first learning decay rate is set to 0.5 per 30 epochs, the total number of epochs for training is 200, the batch size is 16, the sample CT image block size is 64x64, and when the first learning rate is gradient-decreased by the first learning decay rate, if the number of epochs reaches a pre-set number (200), the first learning rate obtained by the last gradient decrease is taken as the target learning rate of the sample removal model.

[0202] In this embodiment, the second learning rate for the sample weight model is set to 1×10 -5 , and the number of neurons corresponding to the hidden layer is 100.

[0203] For illustration, please refer to Figure 10 , which shows a processing process diagram of the artifact removal model provided by an illustrative embodiment of the present application, as shown in Figure 10 , the current processing system includes a front end A1010 (such as a CT scanner), a server 1020 and a front end B (such as a computer terminal or a mobile terminal).

[0204] When the target detection object is scanned by the front end A1010, if the target detection object is implanted with metal, the CT image generated by the current front end A1010 contains a metal artifact area caused by metal.

[0205] The CT image is input into the artifact removal model in the server 1020, the artifact removal model removes the metal artifact area identified from the CT image to generate a CT image without metal artifact in different pre-set window ranges, and feeds it back to the front end B1030 for the doctor to make auxiliary diagnosis.

[0206] In summary, the training method of the artifact removal model provided in the embodiments of the present application trains a plurality of sample removal models using the reference image and the artifact image matched in image content, wherein the artifact image is input into the plurality of sample removal models to output a plurality of artifact removal results, the prediction loss value between the plurality of artifact removal results and the reference image is determined, the weight parameter corresponding to each prediction loss value is obtained by inputting the prediction loss value into the sample weight model, the plurality of sample removal models are trained according to the prediction loss value and the weight parameter, and finally the artifact removal model containing a plurality of artifact removal sub-models is obtained. By training the plurality of sample removal models corresponding to different preset window ranges using the weight parameter and the prediction loss value, the artifact removal model obtained through the final training can output the artifact removal image corresponding to different window ranges, meet the artifact removal requirements of different images, and improve the artifact removal accuracy of the artifact removal result.

[0207] In the embodiments, the training method for the plurality of sample removal models based on a plurality of window ranges can simultaneously train the plurality of sample removal models with different window ranges, and finally obtain the artifact removal model capable of outputting the artifact removal result of different window ranges, thereby better improving the training effect of the model.

[0208] In the embodiments, the images with different window ranges are converted through window conversion, which can better enable the model learning between different window ranges and improve the accuracy and flexibility of model training.

[0209] Please refer to Figure 11 , which shows a training method of an artifact removal model provided in an example embodiment of the present application, as shown in Figure 11 , the method comprises the following steps:

[0210] Step 1110, start.

[0211] Illustratively, the training method of the artifact removal model provided in the present application is executed by a server, and after sending a training request to the server, the server starts to execute the training process of the artifact removal model.

[0212] The server first determines whether the current stage is a training stage or a test stage. If it is a training stage, step 1120 is executed, and if it is a test stage, step 1150 is executed.

[0213] Step 1120, obtain an artifact image.

[0214] In this embodiment, the artifact image refers to a CT image containing metal artifacts, wherein the artifact image is a CT image containing metal artifacts synthesized by obtaining a reference image in a public data set, combining different metal mask information, and through a data simulation process, as training data. The reference image and the artifact image correspond to the same image content, that is, the reference image and the artifact image are realized as a sample image pair.

[0215] Step 1130, the first model parameters of the sample removal model and the second model parameters of the sample weight model are trained in an iterative cycle.

[0216] In this embodiment, the first model parameters corresponding to the three sample removal models are adjusted simultaneously.

[0217] In this embodiment, the artifact image is input into the three sample removal models respectively, and three artifact removal results are output.

[0218] According to the pixel point difference between the three artifact removal results and the reference image, the loss function The prediction loss values corresponding to the three sample removal models are determined.

[0219] The three prediction loss values are input into the sample weight model The weight parameters W1, W2 and W3 corresponding to the three prediction loss values are output.

[0220] According to the prediction loss value and the weight parameter, in the process of cyclic iteration training, in turn according to the above formula four, formula three and formula seven, the sample weight model and the three sample removal models are back propagated, and the second model parameters of the sample weight model and the first model parameters corresponding to the three sample removal models are adjusted in turn.

[0221] When the number of cyclic iteration training for the first model parameter reaches the number threshold, step 1140 is executed, otherwise step 1130 is continued.

[0222] Step 1140, save the trained model.

[0223] After the number of cyclic iteration training for the first model parameter reaches the number threshold, the plurality of first model parameters obtained by the last training are taken as the first parameter, and the artifact removal model is determined, wherein the artifact removal model includes three trained artifact removal sub-models.

[0224] The server stores the trained artifact removal model.

[0225] Step 1150, obtain a test artifact image.

[0226] When the server determines that the current is a test phase, a test artifact image containing metal artifacts is obtained, wherein the test artifact image is implemented as a CT image for effect testing of the trained artifact removal model.

[0227] In step 1160, the trained artifact removal model is loaded.

[0228] After the server obtains the test artifact image, the stored trained artifact removal model is loaded.

[0229] In step 1170, the artifact removal result generated by the artifact removal model is obtained by forward calculation.

[0230] The test artifact image is input into the artifact removal model, and the artifact removal result corresponding to the test artifact image is obtained by forward calculation for the test artifact image. The artifact removal result refers to the CT image in the preset window range generated after removing the artifacts from the test artifact image.

[0231] In step 1180, the CT images in different preset window ranges corresponding to the artifact removal result are output.

[0232] The artifact removal results for three different preset window ranges simultaneously output by the artifact removal model are output.

[0233] In this embodiment, the three artifact removal sub-models finally trained are used as the artifact removal model, that is, after the target image is input into the artifact removal model, three artifact removal results corresponding to different preset window ranges are simultaneously output. For illustration, please refer to Figure 12 which shows the application process of the artifact removal model provided by one example embodiment of the present application, as shown in Figure 12 When the target image 1210 (in this embodiment, three different display modes of abdominal CT images in the same window range for the same abdominal tissue are provided, which are image 1211, image 1212 and image 1213), the three artifact removal results in different window ranges (including three artifact removal results 111 corresponding to image 1211, three artifact removal results 122 corresponding to image 1212, and three artifact removal results 133 corresponding to image 1213) are simultaneously output after the target image 1210 is input into the artifact removal model 1220. The artifact removal results are used to assist doctors in diagnosing the abdominal tissue.

[0234] In summary, the training method of the artifact removal model provided by the embodiments of the present application trains a plurality of sample removal models by using the reference image and the artifact image matched in image content, wherein the artifact image is input into the plurality of sample removal models during the training process, and a plurality of artifact removal results are output respectively, the prediction loss values between the plurality of artifact removal results and the reference image are determined, the prediction loss values are input into the sample weight model, and finally the weight parameters corresponding to each prediction loss value are obtained, the plurality of sample removal models are trained according to the prediction loss values and the weight parameters, and finally the artifact removal model containing a plurality of artifact removal sub-models is obtained. By training the plurality of sample removal models corresponding to different preset window ranges by using the weight parameters and the prediction loss values, the artifact removal model finally obtained can output the artifact removal images corresponding to different window ranges, meet the artifact removal requirements of different images, and improve the artifact removal accuracy of the artifact removal results.

[0235] The beneficial effects provided by the present application include the following points:

[0236] 1. The designed sample removal model (DICD-Net) has good interpretability, which can help users understand the function of each module in the model well;

[0237] 2. The sample weight model is introduced between different window ranges, so that the reconstruction and restoration learning process of different window ranges is more flexible, which has better potential for fully improving the fidelity of the organizational structure;

[0238] 3. The reconstructed and restored different contrast CT images (artifact removal results) are beneficial to more fine observation of different tissues and organs, thereby better facilitating subsequent diagnosis.

[0239] Figure 13 is a structural block diagram of the training device of the artifact removal model provided by an exemplary embodiment of the present application, as Figure 13 shown, the device includes the following parts:

[0240] The acquisition module 1310 is configured to acquire a reference image and an artifact image matched in image content, the reference image is an image generated after scanning a sample detection object not containing an implant, the artifact image is a reference image containing artifacts, and the artifact is a shadow generated by the implant in the scanning process of a sample detection object containing the implant;

[0241] The input module 1320 is configured to input the artifact image into a plurality of sample removal models, and output artifact removal results corresponding to the artifact image respectively, different sample removal models correspond to different preset window ranges, and the sample removal model is configured to remove artifacts in the artifact image by taking the corresponding preset window range as a reference;

[0242] The determination module 1330 is used to determine the predicted loss value corresponding to each of the multiple sample removal models based on the pixel differences between the artifact removal result and the reference image.

[0243] The input module 1320 is further configured to input the predicted loss values ​​corresponding to the multiple sample removal models into the sample weight model, and output the weight parameters corresponding to the multiple predicted loss values. The weight parameters are used to adjust the weights for updating the parameters of the sample removal model.

[0244] The training module 1340 is used to train multiple sample removal models based on the predicted loss value and the weight parameters to obtain a artifact removal model composed of multiple artifact removal sub-models. The artifact removal sub-models are used to remove artifacts from the target image based on a corresponding preset window range.

[0245] In an optional embodiment, such as Figure 14 As shown, the training module 1340 includes:

[0246] The determining unit 1341 is used to determine the weighted loss values ​​corresponding to multiple sample removal models based on multiple predicted loss values ​​and the weight parameters corresponding to the multiple loss values ​​respectively.

[0247] The adjustment unit 1342 is used to adjust the first model parameters of multiple sample removal models based on the weighted loss values ​​corresponding to the multiple sample removal models respectively, so as to obtain multiple artifact removal sub-models;

[0248] The determining unit 1341 is further configured to use the plurality of artifact removal sub-models as the artifact removal model.

[0249] In an optional embodiment, the determining unit 1341 is further configured to determine the weighted loss value corresponding to the s-th iteration training based on the predicted loss value obtained in the (s-1)-th iteration training and the weight parameters obtained in the s-th iteration training during the s-th iteration training process.

[0250] The adjustment unit 1342 is further configured to perform gradient adjustment on the first model parameters obtained by the (s-1)th iteration training of the sample removal model based on the weighted loss value corresponding to the s-th iteration training, to obtain the first model parameters corresponding to the s-th iteration training, and perform the (s+1)th cyclic adjustment until the artifact removal model training ends, where s≥1 and s is an integer.

[0251] In an optional embodiment, the input module 1320 is further configured to input the prediction loss obtained from the (s-1)th iteration training into the sample weight model obtained from the sth iteration training, and output the weight parameters corresponding to the sth iteration training.

[0252] In an optional embodiment, the obtaining module 1310 is further configured to obtain a verification reference image and a verification artifact image matched with the image content;

[0253] The input module 1320 is further configured to input the verification artifact image into a plurality of sample removal models, and respectively output a verification removal result corresponding to the verification artifact image.

[0254] The determination module 1330 is further configured to determine a verification loss value corresponding to each of the plurality of sample removal models based on a pixel point difference between the verification removal result and the verification reference image.

[0255] The training module 1340 is further configured to train the sample weight model based on the verification loss value.

[0256] In an optional embodiment, the training module 1340 is further configured to, in an s-th iteration training process, perform gradient adjustment on a second model parameter of the sample weight model based on a verification loss value obtained in an (s-1)-th iteration training, to obtain a sample weight model corresponding to the s-th iteration training.

[0257] In an optional embodiment, the determination module 1330 is further configured to determine a mapping relationship between the first model parameter and the second model parameter in the (s-1)-th iteration training based on the first model parameter obtained in the (s-1)-th iteration training, and determine the verification loss value obtained in the (s-1)-th iteration training based on the mapping relationship.

[0258] In an optional embodiment, the determination module 1330 is further configured to, in response to a number of times of cyclic iteration adjustment of the first model parameter reaching a number threshold, determine the first model parameter obtained in the last adjustment as the first parameter, or in response to an adjustment effect of the first model parameter meeting an adjustment effect condition, determine the first model parameter as the first parameter, the adjustment effect condition being used to represent a limitation requirement on the prediction loss value.

[0259] In an optional embodiment, the obtaining module 1310 is further configured to obtain a first learning decay rate, the first learning decay rate being a decay adjustment on a first learning rate according to a number of iterations, the first learning rate being a pre-set update step length for training the plurality of sample removal models.

[0260] The apparatus further includes:

[0261] The gradient descent module 1350 is configured to, in the process of training the plurality of sample removal models, perform gradient descent on the first learning rate based on the first learning decay rate, to obtain a target learning rate corresponding to the artifact removal model.

[0262] In an optional embodiment, the determining module 1330 is further configured to determine a window range corresponding to the i-th sample removal model, where i is a positive integer.

[0263] The apparatus further includes:

[0264] The conversion module 1360 is configured to perform window conversion on the artifact image and the i-1-th artifact removal result to obtain window conversion results corresponding to the artifact image and the i-1-th artifact removal result respectively as model inputs of the i-th sample removal model.

[0265] In summary, the training apparatus of the artifact removal model provided in the embodiments of the present application trains a plurality of sample removal models using the reference image and the artifact image matched in content, wherein, in the training process, the artifact image is input into the plurality of sample removal models to output a plurality of artifact removal results respectively, the prediction loss values between the plurality of artifact removal results and the reference image are determined, the weight parameters corresponding to each prediction loss value are finally obtained by inputting the prediction loss values into the sample weight model, the plurality of sample removal models are trained according to the prediction loss values and the weight parameters, and finally the artifact removal model containing a plurality of artifact removal sub-models is obtained. By training the plurality of sample removal models corresponding to different preset window ranges using the weight parameters and the prediction loss values, the artifact removal model finally obtained can output artifact removal images corresponding to different window ranges, meet the artifact removal requirements of different images, and improve the artifact removal accuracy of the artifact removal results.

[0266] It should be noted that: the training apparatus of the artifact removal model provided in the above embodiments is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the training apparatus of the artifact removal model and the training method of the artifact removal model provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0267] Figure 15 The structure of the server provided in an example embodiment of the present application is shown. Specifically:

[0268] The server 1500 includes a central processing unit (CPU) 1501, a system memory 1504, including a random access memory (RAM) 1502 and a read-only memory (ROM) 1503, and a system bus 1505 that couples the system memory 1504 to the central processing unit 1501. The server 1500 also includes a mass storage device 1506 for storing an operating system 1513, application programs 1514, and other program modules 1515.

[0269] The mass storage device 1506 connects to the central processing unit 1501 through a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1506 and its associated computer-readable media provide nonvolatile storage for the server 1500. That is, the mass storage device 1506 can include a computer-readable medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0270] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media would not, however, include communication media including wired or wireless signaling media that communicate program code in a modulated data signal. The system memory 1504 and mass storage device 1506 described above can be embodied as a memory component.

[0271] According to various embodiments of the present application, the server 1500 can further operate connected to a remote computer on a network through a network, such as the Internet. That is, the server 1500 can be connected to a network 1512 through a network interface unit 1511 connected to the system bus 1505, or can be connected to other types of networks or remote computer systems (not shown) using the network interface unit 1511.

[0272] The above-described memory further includes one or more programs stored in the memory configured to be executed by the CPU.

[0273] Embodiments of the present application also provide a computer device including a processor and a memory having at least one instruction, at least one program, a code set or an instruction set stored therein, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the training method of the artifact removal model provided by any of the above method embodiments.

[0274] Embodiments of the present application also provide a computer readable storage medium having at least one instruction, at least one program, a code set or an instruction set stored thereon, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the training method of the artifact removal model provided by any of the above method embodiments.

[0275] Embodiments of the present application also provide a computer program product or computer program including computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the training method of the artifact removal model described in any of the above embodiments.

[0276] Optionally, the computer readable storage medium can include a read-only memory (ROM), a random access memory (RAM), a solid state disk (SSD), an optical disk, etc. Among them, the random access memory can include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The above-mentioned application embodiment serial number is only for description, not representing the pros and cons of the embodiments.

[0277] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed to relevant hardware by program. The program can be stored in a computer readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0278] The above only describes optional embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of training an artifact removal model, the method comprising: The method comprises: obtaining a reference image matched with image content and an artifact image, the reference image being an image generated after scanning a sample detection object not containing an implant, and the artifact image being a reference image containing an artifact, the artifact being a shadow generated by the implant during scanning of a sample detection object containing the implant; inputting the artifact image into a plurality of sample removal models, and respectively outputting artifact removal results corresponding to the artifact image, different sample removal models corresponding to different preset window ranges, the sample removal model being used to remove the artifact in the artifact image based on the corresponding preset window range; determining prediction loss values respectively corresponding to the plurality of sample removal models based on pixel point differences between the artifact removal results and the reference image; inputting the prediction loss values respectively corresponding to the plurality of sample removal models into a sample weight model, and outputting weight parameters respectively corresponding to the plurality of prediction loss values, the weight parameters being used to weight adjust parameter updating of the sample removal model; determining weighted loss values respectively corresponding to the plurality of sample removal models based on the plurality of prediction loss values and the weight parameters respectively corresponding to the plurality of prediction loss values; adjusting first model parameters respectively corresponding to the plurality of sample removal models based on the weighted loss values respectively corresponding to the plurality of sample removal models, to obtain a plurality of artifact removal sub-models as artifact removal models, the artifact removal sub-model being used to remove artifacts from an image based on the corresponding preset window range, and the artifact removal model being used to output artifact removal images corresponding to different window ranges.

2. The method of claim 1, wherein, The method comprises: for each of the plurality of sample removal models, determining a weighted loss value corresponding to the s-th iteration training based on the prediction loss value obtained through the s-1-th iteration training and the weight parameter obtained through the s-th iteration training; The method comprises: for each of the plurality of sample removal models, gradient adjusting first model parameters obtained through the s-1-th iteration training of the sample removal model based on the weighted loss value corresponding to the s-th iteration training, to obtain first model parameters corresponding to the s-th iteration training, and performing s+1 cycle adjustment until the artifact removal model training is completed, s≥1 and s is an integer.

3. The method of claim 2, wherein, The method comprises: for each of the plurality of sample removal models, inputting the prediction loss value obtained through the s-1-th iteration training into the sample weight model obtained through the s-th iteration training, to output the weight parameter corresponding to the s-th iteration training.

4. The method according to any one of claims 1 to 3, characterized in that, Before the inputting the prediction loss value corresponding to each of the plurality of sample removal models into the sample weight model and outputting weight parameters corresponding to the plurality of prediction loss values, the method further comprises: obtaining a verification reference image and a verification artifact image matched with the image content; inputting the verification artifact image into the plurality of sample removal models to respectively output a verification removal result corresponding to the verification artifact image; determining a verification loss value corresponding to each of the plurality of sample removal models based on a pixel point difference between the verification removal result and the verification reference image; training the sample weight model based on the verification loss value corresponding to each of the plurality of sample removal models.

5. The method of claim 4, wherein, The training of the sample weight model based on the verification loss value corresponding to each of the plurality of sample removal models comprises: for each of the plurality of sample removal models, in an s-th iteration training process, performing gradient adjustment on a second model parameter of the sample weight model based on a verification loss value obtained in an (s-1)-th iteration training to obtain a sample weight model corresponding to the s-th iteration training.

6. The method of claim 5, wherein, Before the gradient adjustment on the second model parameter of the sample weight model based on the verification loss value obtained in the (s-1)-th iteration training to obtain the sample weight model corresponding to the s-th iteration training, the method further comprises: for each of the plurality of sample removal models, determining a mapping relationship between the first model parameter and the second model parameter in the (s-1)-th iteration training based on the first model parameter obtained in the (s-1)-th iteration training; determining the verification loss value obtained in the (s-1)-th iteration training based on the mapping relationship.

7. The method according to any one of claims 1 to 3, characterized in that, The model parameter in the artifact removal model is a first parameter. The method further comprises: in response to a number of times of cyclic iteration adjustment of the first model parameter reaching a number threshold, determining the first model parameter obtained in the last time of adjustment as the first parameter; or in response to an adjustment effect of the first model parameter meeting an adjustment effect condition, determining the first model parameter as the first parameter, the adjustment effect condition being used to represent a limitation requirement on the prediction loss value.

8. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: obtaining a first learning decay rate, the first learning decay rate being used to adjust a first learning rate in a decay form according to an iteration number, the first learning rate being a pre-set update step length for training the plurality of sample removal models; in the process of training the plurality of sample removal models, performing gradient descent on the first learning rate based on the first learning decay rate to obtain a target learning rate corresponding to the artifact removal model.

9. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: determining a window range corresponding to an i-th sample removal model, i being a positive integer; performing window conversion on the artifact image and an (i-1)-th artifact removal result to obtain window conversion results corresponding to the artifact image and the (i-1)-th artifact removal result respectively as model input of the i-th sample removal model. 10.A device for training an artifact removal model, comprising: The device comprises: The acquisition module is configured to acquire a reference image matched with image content and an artifact image, the reference image being an image generated after scanning a sample detection object without containing an implant, and the artifact image being a reference image containing an artifact, the artifact being a shadow generated by the implant during scanning of a sample detection object containing the implant; The input module is configured to input the artifact image into a plurality of sample removal models, and output artifact removal results corresponding to the artifact image respectively, different sample removal models corresponding to different preset window ranges, the sample removal models being configured to remove artifacts in the artifact image based on the corresponding preset window ranges; The determination module is configured to determine prediction loss values respectively corresponding to the plurality of sample removal models based on pixel point differences between the artifact removal results and the reference image; The input module is configured to input the prediction loss values respectively corresponding to the plurality of sample removal models into a sample weight model, and output weight parameters respectively corresponding to the plurality of prediction loss values, the weight parameters being configured to perform weight adjustment on parameter update of the sample removal models; The training module is configured to determine weighted loss values respectively corresponding to the plurality of sample removal models based on the plurality of prediction loss values and the weight parameters respectively corresponding to the plurality of prediction loss values, adjust first model parameters respectively corresponding to the plurality of sample removal models based on the weighted loss values respectively corresponding to the plurality of sample removal models, and obtain a plurality of artifact removal sub-models as artifact removal models, the artifact removal sub-models being configured to perform artifact removal on images based on the corresponding preset window ranges, and the artifact removal models being configured to output artifact removal images corresponding to different window ranges.

11. A computer device, comprising: The computer device includes a processor and a memory, the memory stores at least one program, the at least one program is loaded and executed by the processor to implement the training method of the artifact removal model as claimed in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, the at least one program is loaded and executed by the processor to implement the training method of the artifact removal model as claimed in any one of claims 1 to 9.

13. A computer program product, characterised in that, The computer instructions are executed by the processor to implement the training method of the artifact removal model as claimed in any one of claims 1 to 9.

Images