Image noise artifact correction method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to an image noise artifact correction method and device, electronic equipment and a storage medium. Obtaining a to-be-corrected artifact image; identifying the to-be-corrected artifact image, and determining a target category corresponding to the to-be-corrected artifact image; and inputting the to-be-corrected artifact image into a target artifact correction model corresponding to the target category, and outputting a target image corresponding to the to-be-corrected artifact image, the target image not including noise artifacts. Therefore, it is ensured that the output target image does not comprise noise artifacts, the definition and contrast of the target image are enhanced, the detail resolution capability is improved, the noise artifacts in the artifact image to be corrected are corrected through the method, errors can be reduced through correction, the accuracy of measurement and detection is improved, and the influence of noise and artifacts is reduced. And an automatic system can make a more accurate decision, and the production efficiency and the product quality are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image noise artifact correction method, device, electronic device and storage medium. Background Art

[0002] In the era of cloud computing, a large number of industrial CT images are uploaded to cloud platforms, and the industrial images are stored and operated through the cloud platforms.

[0003] However, during the CT image acquisition process, electronic noise in the sensor or circuit can cause random brightness or color changes in the image, or changes in environmental conditions such as lighting changes and temperature fluctuations, thereby affecting image quality and causing images to have serious noise artifacts. Noise artifacts may lead to misjudgments and affect the decision-making of automated systems, such as causing incorrect judgments in quality control and defect detection.

[0004] Therefore, how to correct the noise artifacts in CT images has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the present invention provides an image noise artifact correction method, device, electronic device and storage medium to solve the problem of how to correct noise artifacts in CT images.

[0006] In a first aspect, the present invention provides a method for correcting image noise artifacts, the method comprising:

[0007] Acquiring an image to be corrected for artifacts;

[0008] Identify the artifact image to be corrected and determine the target category corresponding to the artifact image to be corrected;

[0009] The artifact image to be corrected is input into a target artifact correction model corresponding to the target category, and a target image corresponding to the artifact image to be corrected is output, where the target image does not include noise artifacts.

[0010] The image noise artifact correction method provided in the embodiment of the present application obtains an artifact image to be corrected, identifies the artifact image to be corrected, determines the target category corresponding to the artifact image to be corrected, and ensures the accuracy of the target category corresponding to the determined artifact image to be corrected. The artifact image to be corrected is input into a target artifact correction model corresponding to the target category, and a target image corresponding to the artifact image to be corrected is output, wherein the target image does not include noise artifacts. This ensures that the output target image does not include noise artifacts, thereby enhancing the clarity and contrast of the target image and improving the detail resolution capability. By using the above method to correct the noise artifacts in the artifact image to be corrected, the correction can reduce errors, improve the accuracy of measurement and detection, reduce the impact of noise and artifacts, and help the automation system make more accurate decisions, improve production efficiency and product quality.

[0011] In an optional implementation, identifying the artifact image to be corrected and determining the target category corresponding to the artifact image to be corrected includes:

[0012] Identifying the artifact image to be corrected and determining the type of the target object included in the artifact image to be corrected;

[0013] According to the type of target object, the target category corresponding to the artifact image to be corrected is determined.

[0014] The image noise artifact correction method provided in the embodiments of the present application identifies the artifact image to be corrected and determines the type of target object included in the artifact image to be corrected, thereby ensuring the accuracy of the determined type of target object included in the artifact image to be corrected. Based on the type of target object, the target category corresponding to the artifact image to be corrected is determined, thereby ensuring the accuracy of the determined target category corresponding to the artifact image to be corrected. Furthermore, based on the target category corresponding to the artifact image to be corrected, a target artifact correction model corresponding to the artifact image to be corrected can be determined.

[0015] In an optional embodiment, inputting the artifact image to be corrected into a target artifact correction model corresponding to the target category, and outputting a target image corresponding to the artifact image to be corrected includes:

[0016] Determining a target artifact correction model corresponding to the target category based on a correspondence between the category and the artifact correction model;

[0017] The artifact image to be corrected is input into the target artifact correction model, and the target image corresponding to the artifact image to be corrected is output.

[0018] The image noise artifact correction method provided in the embodiments of the present application determines the target artifact correction model corresponding to the target category based on the correspondence between the category and the artifact correction model, thereby ensuring the accuracy of the target artifact correction model corresponding to the determined target category. The target artifact correction model then outputs a target image corresponding to the artifact image, thereby ensuring the accuracy of the output target image.

[0019] In an optional embodiment, the target artifact correction model includes an upper network branch, a middle network branch, and a lower network branch. The target artifact correction model inputs the artifact image to be corrected and outputs a target image corresponding to the artifact image to be corrected, including:

[0020] Input the artifact image to be corrected into the upper network branch, the middle network branch and the lower network branch respectively;

[0021] The upper network branch extracts features from the artifact image to be corrected and outputs the upper branch result;

[0022] The middle-layer network branch extracts features from the artifact image to be corrected and outputs the middle-layer branch results;

[0023] The lower network branch extracts features from the artifact image to be corrected and outputs the lower branch results;

[0024] According to the relationship between the upper-layer branch results, the middle-layer branch results and the lower-layer branch results, the target image corresponding to the artifact image to be corrected is output.

[0025] The image noise artifact correction method provided in the embodiment of the present application inputs the artifact image to be corrected into the upper network branch, the middle network branch and the lower network branch respectively; the upper network branch performs feature extraction on the artifact image to be corrected and outputs the upper branch result, thereby ensuring the accuracy of the output upper branch result. The middle network branch performs feature extraction on the artifact image to be corrected and outputs the middle branch result, thereby ensuring the accuracy of the output middle branch result. The lower network branch performs feature extraction on the artifact image to be corrected and outputs the lower branch result, thereby ensuring the accuracy of the output lower branch result. Based on the relationship between the upper branch result, the middle branch result and the lower branch result, the target image corresponding to the artifact image to be corrected is output, thereby ensuring the accuracy of the output target image.

[0026] In an optional embodiment, outputting a target image corresponding to the artifact image to be corrected according to the relationship among the upper-layer branch results, the middle-layer branch results, and the lower-layer branch results includes:

[0027] Splicing the upper branch results, the middle branch results, and the lower branch results to generate a splicing result graph;

[0028] Perform feature extraction on the splicing result image to generate noise estimation results;

[0029] The noise estimation result is subtracted from the artifact image to be corrected to output the target image.

[0030] The image noise artifact correction method provided in the embodiments of the present application splices the upper-layer branch results, the middle-layer branch results, and the lower-layer branch results to generate a spliced ​​result image, thereby ensuring the accuracy of the generated spliced ​​result image. Feature extraction is performed on the spliced ​​result image to generate a noise estimation result, thereby ensuring the accuracy of the generated noise estimation result. The noise estimation result is subtracted from the artifact image to be corrected to output a target image, thereby ensuring the accuracy of the output target image.

[0031] In an optional embodiment, the training process of the target artifact correction model includes:

[0032] Obtain a preset training data set, where the preset training data set includes a noisy image containing noise artifacts corresponding to the target category and an original image without noise artifacts corresponding to the noise image;

[0033] Inputting a preset training data set into an initial artifact correction network, the initial artifact correction network corrects the noisy image and outputs a corrected virtual image;

[0034] Based on the preset loss function, the target loss value between the virtual image and the original image corresponding to the noisy image is calculated;

[0035] Based on the target loss value, the initial artifact correction network is trained to generate the target artifact correction model.

[0036] The image noise artifact correction method provided in the embodiment of the present application obtains a preset training data set, inputs the preset training data set into an initial artifact correction network, and the initial artifact correction network corrects the noisy image and outputs a corrected virtual image, thereby ensuring the accuracy of the output corrected virtual image. Based on a preset loss function, a target loss value is calculated between the virtual image and the original image corresponding to the noisy image, thereby ensuring the accuracy of the obtained target loss value. Based on the target loss value, the initial artifact correction network is trained to generate a target artifact correction model, thereby ensuring the accuracy of the generated target artifact correction model. This can further ensure the accuracy of the target image output by performing noise artifact correction on the artifact image to be corrected based on the target artifact correction model.

[0037] In an optional embodiment, calculating a target loss value between the virtual image and the original image corresponding to the noisy image based on a preset loss function includes:

[0038] Calculating a first loss value between the virtual image and the original image corresponding to the noise image using a first loss function;

[0039] Calculating a second loss value between the virtual image and the original image corresponding to the noise image using a second loss function;

[0040] Calculate a target loss value based on the first loss value and the second loss value.

[0041] The image noise artifact correction method provided in the embodiments of the present application uses a first loss function to calculate a first loss value between a virtual image and an original image corresponding to the noisy image, thereby ensuring the accuracy of the calculated first loss value. A second loss function is used to calculate a second loss value between the virtual image and the original image corresponding to the noisy image, thereby ensuring the accuracy of the calculated second loss value. A target loss value is calculated based on the first and second loss values, thereby ensuring the accuracy of the calculated target loss value.

[0042] In a second aspect, the present invention provides an image noise artifact correction device, the device comprising:

[0043] An acquisition module, used for acquiring an image to be corrected for artifacts;

[0044] A determination module is used to identify the artifact image to be corrected and determine the target category corresponding to the artifact image to be corrected;

[0045] The output module is used to input the artifact image to be corrected into the target artifact correction model corresponding to the target category, and output the target image corresponding to the artifact image to be corrected, where the target image does not include noise artifacts.

[0046] The image noise artifact correction device provided in the embodiment of the present application obtains an artifact image to be corrected, identifies the artifact image to be corrected, determines the target category corresponding to the artifact image to be corrected, and ensures the accuracy of the target category corresponding to the determined artifact image to be corrected. The artifact image to be corrected is input into a target artifact correction model corresponding to the target category, and a target image corresponding to the artifact image to be corrected is output, wherein the target image does not include noise artifacts. This ensures that the output target image does not include noise artifacts, thereby enhancing the clarity and contrast of the target image and improving the detail resolution capability. The noise artifacts in the artifact image to be corrected are corrected by the above-mentioned device. Through correction, errors can be reduced, the accuracy of measurement and detection can be improved, the influence of noise and artifacts can be reduced, and the automation system can make more accurate decisions, thereby improving production efficiency and product quality.

[0047] In a third aspect, the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the image noise artifact correction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the image noise artifact correction method of the first aspect or any corresponding embodiment thereof.

[0049] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the image noise artifact correction method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 is a flow chart of a method for correcting image noise artifacts according to an embodiment of the present invention;

[0052] Figure 2 is a flow chart of another method for correcting image noise artifacts according to an embodiment of the present invention;

[0053] Figure 3 is a schematic structural diagram of a target artifact correction model according to an embodiment of the present invention;

[0054] Figure 4 is a flowchart of a method for training a target artifact correction model according to an embodiment of the present invention;

[0055] Figure 5 is a structural block diagram of an image noise artifact correction device according to an embodiment of the present invention;

[0056] Figure 6 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0058] In the era of cloud computing, a large number of industrial CT images are uploaded to cloud platforms, and the industrial images are stored and operated through the cloud platforms.

[0059] However, during the CT image acquisition process, electronic noise in the sensor or circuit can cause random brightness or color changes in the image, or changes in environmental conditions such as lighting changes and temperature fluctuations, thereby affecting image quality and causing images to have serious noise artifacts. Noise artifacts may lead to misjudgments and affect the decision-making of automated systems, such as causing incorrect judgments in quality control and defect detection.

[0060] Therefore, how to correct the noise artifacts in CT images has become an urgent problem to be solved.

[0061] Based on the above problems, an embodiment of the present application provides an image noise artifact correction method, which obtains an artifact image to be corrected, identifies the artifact image to be corrected, and determines the target category corresponding to the artifact image to be corrected, thereby ensuring the accuracy of the target category corresponding to the determined artifact image to be corrected. The artifact image to be corrected is input into a target artifact correction model corresponding to the target category, and a target image corresponding to the artifact image to be corrected is output, wherein the target image does not include noise artifacts. This ensures that the output target image does not include noise artifacts, thereby enhancing the clarity and contrast of the target image and improving the detail resolution capability. By using the above method to correct the noise artifacts in the artifact image to be corrected, the correction can reduce errors, improve the accuracy of measurement and detection, reduce the impact of noise and artifacts, and help the automation system make more accurate decisions, improve production efficiency and product quality.

[0062] It should be noted that the method for image noise artifact correction provided in the embodiments of the present application can be executed by an image noise artifact correction device, which can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. The electronic device can be a server or a terminal. The server in the embodiments of the present application can be a single server or a server cluster composed of multiple servers. The terminal in the embodiments of the present application can be a smart phone, personal computer, tablet computer, wearable device, smart robot or other smart hardware device. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0063] According to an embodiment of the present invention, an embodiment of a method for correcting image noise artifacts is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] In this embodiment, a method for correcting image noise artifacts is provided, which can be used in the above-mentioned electronic device. Figure 1 FIG. 1 is a flow chart of a method for correcting image noise artifacts according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0065] Step S101: Acquire an image to be corrected for artifacts.

[0066] Specifically, the electronic device may receive an artifact image to be corrected input by a user, or may receive an artifact image to be corrected sent by other devices. The electronic device may also search for the artifact image to be corrected from a storage space.

[0067] The embodiment of the present application does not specifically limit the manner in which the electronic device obtains the artifact image to be corrected.

[0068] Step S102 : identifying the image to be corrected for artifacts and determining the target category corresponding to the image to be corrected for artifacts.

[0069] Specifically, the electronic device may input the artifact image to be corrected into a category recognition model, recognize the artifact image to be corrected based on the category recognition model, and determine the target category corresponding to the artifact image to be corrected.

[0070] Among them, the category recognition model can be a model based on manual features, such as DPM (Deformable Parts Model), and the category recognition model can also be a model based on convolutional neural networks, such as YOLO (You Only Look Once) detector, R-CNN, (Region-based Convolutional Neural Networks, region-based convolutional neural network) model, SSD (Single Shot MultiBox, single shot multi-box) detector and Mask R-CNN (Mask Region-based Convolutional Neural Networks, masked region-based convolutional neural network) model, etc. The embodiment of the present application does not specifically limit the category recognition model.

[0071] This step will be described in detail below.

[0072] Step S103 : inputting the artifact image to be corrected into a target artifact correction model corresponding to the target category, and outputting a target image corresponding to the artifact image to be corrected.

[0073] The target image does not include noise artifacts.

[0074] Specifically, the electronic device may input the artifact image to be corrected into a target artifact correction model corresponding to the target category. The target artifact correction model extracts features from the artifact image to be corrected and outputs a target image corresponding to the artifact image to be corrected based on the extracted features.

[0075] This step will be described in detail below.

[0076] The image noise artifact correction method provided in the embodiment of the present application obtains an artifact image to be corrected, identifies the artifact image to be corrected, determines the target category corresponding to the artifact image to be corrected, and ensures the accuracy of the target category corresponding to the determined artifact image to be corrected. The artifact image to be corrected is input into a target artifact correction model corresponding to the target category, and a target image corresponding to the artifact image to be corrected is output, wherein the target image does not include noise artifacts. This ensures that the output target image does not include noise artifacts, thereby enhancing the clarity and contrast of the target image and improving the detail resolution capability. By using the above method to correct the noise artifacts in the artifact image to be corrected, the correction can reduce errors, improve the accuracy of measurement and detection, reduce the impact of noise and artifacts, and help the automation system make more accurate decisions, improve production efficiency and product quality.

[0077] In this embodiment, a method for correcting image noise artifacts is provided, which can be used in the above-mentioned electronic device. Figure 2 FIG. 1 is a flow chart of a method for correcting image noise artifacts according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0078] Step S201: Acquire an image to be corrected for artifacts.

[0079] For details about this step, please refer to the above description of step S101 and will not be repeated here.

[0080] Step S202 : Identify the image to be corrected for artifacts and determine the target category corresponding to the image to be corrected for artifacts.

[0081] Specifically, the above step S202 may include the following steps:

[0082] Step S2021 : Identify the artifact image to be corrected and determine the type of the target object included in the artifact image to be corrected.

[0083] Specifically, the electronic device may perform pre-processing operations on the input image to be corrected for artifacts, such as image enhancement, denoising, cropping, etc., to improve image quality and accuracy of subsequent processing.

[0084] The electronic device then inputs the preprocessed artifact-corrected image into the object detection model, which identifies the preprocessed artifact-corrected image and determines the location and range of the target object. The detected target object is then classified based on a pre-set classification model to determine its type.

[0085] Step S2022 : determining the target category corresponding to the artifact image to be corrected according to the type of the target object.

[0086] Specifically, the electronic device may determine the type of the target object as the target category corresponding to the artifact image to be corrected.

[0087] Step S203 : inputting the artifact image to be corrected into a target artifact correction model corresponding to the target category, and outputting a target image corresponding to the artifact image to be corrected.

[0088] The target image does not include noise artifacts.

[0089] Specifically, the above step S203 may include the following steps:

[0090] Step S2031 : determining a target artifact correction model corresponding to the target category according to the correspondence between the category and the artifact correction model.

[0091] Specifically, the electronic device can receive the correspondence between categories and artifact correction models input by the user and the artifact correction models corresponding to each category, and can also receive the correspondence between categories and artifact correction models and the artifact correction models corresponding to each category sent by other devices.

[0092] The electronic device may also train artifact correction models corresponding to various categories based on various preset training data sets, and generate corresponding relationships between categories and artifact correction models.

[0093] The embodiment of the present application does not specifically limit the manner in which the electronic device acquires the correspondence between categories and artifact correction models and the manner in which each category corresponds to each artifact correction model.

[0094] Then, the electronic device determines a target artifact correction model corresponding to the target category according to the correspondence between the category and the artifact correction model.

[0095] Step S2032 : input the artifact image to be corrected into a target artifact correction model, and output a target image corresponding to the artifact image to be corrected.

[0096] Specifically, if Figure 3 As shown, the target artifact correction model includes an upper network branch, a middle network branch, and a lower network branch. The above step S2032 may include the following steps:

[0097] In step a1, the artifact image to be corrected is input into the upper network branch, the middle network branch, and the lower network branch respectively.

[0098] Specifically, the electronic device may input the artifact image to be corrected into the upper network branch, the middle network branch, and the lower network branch in the target artifact correction model respectively.

[0099] In step a2, the upper network branch extracts features from the artifact image to be corrected and outputs the upper branch result.

[0100] Among them, the upper network branch can include multiple network structure layers, among which the number of network structure layers in the upper network branch can be 10, 8, or other numbers. The embodiment of the present application does not specifically limit the number of network structure layers included in the upper network branch.

[0101] Specifically, the upper network branch can extract features of the artifact image to be corrected based on multiple network structure layers, and output the upper branch result based on the extracted features.

[0102] For example, Figure 3 As shown in the figure, the upper network branch can be composed of 10 network structure layers, among which the 1st, 5th and 9th network structure layers are Conv+BRN+ReLU, 2-4, 6-8 are dilated convolutions, and the last layer is Conv.

[0103] Conv: Convolutional layer, used to extract features from the artifact-corrected image. BRN: Batch Normalization layer, used to accelerate training and improve model stability. ReLU: Activation function, used to increase the nonlinear expressiveness of the model. Dilated convolution: A special convolution operation that increases the receptive field by inserting holes in the convolution kernel, thereby capturing a wider range of contextual information.

[0104] In addition, the convolution kernel size of the upper network branch is 3, which means the size of the convolution kernel is 3x3. The number of channels of the upper network branch is 64, which means the number of channels of the feature map output by each convolution layer is 64.

[0105] In step a3, the middle-layer network branch extracts features from the artifact image to be corrected and outputs the middle-layer branch result.

[0106] Among them, the middle-level network branch can include multiple network structure layers, among which the number of network structure layers in the middle-level network branch can be 10, 8, or other numbers. The embodiment of the present application does not specifically limit the number of network structure layers included in the middle-level network branch.

[0107] Specifically, the middle-layer network branch can extract features of the artifact image to be corrected, and output the middle-layer branch results based on the extracted features.

[0108] For example, Figure 3 As shown, the middle network branch also consists of 10 layers. The first nine layers are Conv+BRN+ReLU, and the last layer is Conv. Conv: Convolutional layer, used to extract features from the artifact-corrected image. BRN: Batch Normalization layer, used to accelerate training and improve model stability. ReLU: Activation function, used to enhance the model's nonlinear expression capabilities.

[0109] In addition, the convolution kernel size of the middle network branch is 3, which means the size of the convolution kernel is 3x3. The number of channels of the middle network branch is 64, which means the number of channels of the feature map output by each convolution layer is 64.

[0110] In step a4, the lower network branch extracts features from the artifact image to be corrected and outputs the lower branch result.

[0111] Among them, the lower-level network branch can include multiple network structure layers, among which the number of network structure layers in the lower-level network branch can be 10, 8, or other numbers. The embodiment of the present application does not specifically limit the number of network structure layers included in the lower-level network branch.

[0112] Specifically, the lower network branch can extract features of the artifact image to be corrected based on multiple network structure layers, and output the lower branch result based on the extracted features.

[0113] For example, Figure 3 As shown in the figure, the lower network branch can be composed of 10 network structure layers, among which the 1st, 5th and 9th network structure layers are Conv+BRN+ReLU, 2-4, 6-8 are dilated convolution layers, and the last layer is Conv.

[0114] Conv: Convolutional layer, used to extract features from the artifact-corrected image. BRN: Batch Normalization layer, used to accelerate training and improve model stability. ReLU: Activation function, used to increase the model's nonlinear expressiveness. Dilated Convolutional layer: A special convolution operation that increases the receptive field by inserting holes in the convolution kernel, thereby capturing a wider range of contextual information.

[0115] In addition, the convolution kernel size of the lower network branch is 3, which means the size of the convolution kernel is 3x3. The number of channels of the lower network branch is 64, which means the number of channels of the feature map output by each convolution layer is 64.

[0116] Step a5: outputting a target image corresponding to the artifact image to be corrected according to the relationship among the upper-layer branch results, the middle-layer branch results, and the lower-layer branch results.

[0117] Specifically, the above step a5 may include the following steps:

[0118] Step a51 : splicing the upper-layer branch results, the middle-layer branch results, and the lower-layer branch results to generate a splicing result graph.

[0119] Specifically, the electronic device can connect the upper-level branch results, the middle-level branch results, and the lower-level branch results along the channel dimension to generate a splicing result graph of more channels.

[0120] For example, Figure 3 As shown in FIG, the target artifact correction model can use the concat layer to connect the upper branch results output by the upper network branch, the middle branch results output by the middle network branch, and the lower branch results output by the lower network branch along the channel dimension to generate a splicing result graph with more channels.

[0121] Step a52: extract features from the spliced ​​result image to generate a noise estimation result.

[0122] Specifically, electronic devices can select appropriate convolution kernel sizes and numbers based on specific needs and problems. The size of the convolution kernel determines the size of the receptive field, while the number of convolution kernels determines the number of channels in the output feature map.

[0123] Then, based on the convolution layer, the convolution kernel is convolved with the concatenated result image to obtain a new noise estimation result. The convolution operation can extract feature information at different locations and scales.

[0124] Step a53: Subtract the noise estimation result from the artifact image to be corrected to output a target image.

[0125] Specifically, the electronic device may subtract the noise estimation result from the artifact image to be corrected and output the target image.

[0126] The image noise artifact correction method provided in the embodiments of the present application identifies the artifact image to be corrected and determines the type of target object included in the artifact image to be corrected, thereby ensuring the accuracy of the determined type of target object included in the artifact image to be corrected. Based on the type of target object, the target category corresponding to the artifact image to be corrected is determined, thereby ensuring the accuracy of the determined target category corresponding to the artifact image to be corrected. Furthermore, based on the target category corresponding to the artifact image to be corrected, a target artifact correction model corresponding to the artifact image to be corrected can be determined.

[0127] Then, based on the correspondence between the categories and the artifact correction models, the target artifact correction model corresponding to the target category is determined, ensuring the accuracy of the target artifact correction model determined for the target category. The artifact image to be corrected is input into the upper, middle, and lower network branches, respectively. The upper network branch performs feature extraction on the artifact image to be corrected and outputs the upper branch results, ensuring the accuracy of the upper branch output results. The middle network branch performs feature extraction on the artifact image to be corrected and outputs the middle branch results, ensuring the accuracy of the middle branch output results. The lower network branch performs feature extraction on the artifact image to be corrected and outputs the lower branch results, ensuring the accuracy of the lower branch output results. The upper, middle, and lower branch results are spliced ​​together to generate a spliced ​​result image, ensuring the accuracy of the resulting spliced ​​result image. Feature extraction is performed on the spliced ​​result image to generate a noise estimation result, ensuring the accuracy of the generated noise estimation result. The noise estimation result is subtracted from the artifact image to be corrected to output the target image, ensuring the accuracy of the output target image.

[0128] In an optional embodiment of the present application, Figure 4 As shown, the training process of the target artifact correction model may include the following steps:

[0129] Step S301: Obtain a preset training data set.

[0130] The preset training data set includes a noise image containing noise artifacts corresponding to the target category and an original image without noise artifacts corresponding to the noise image.

[0131] Optionally, the electronic device may receive a preset training data set input by a user, or may receive a preset training data set sent by other devices.

[0132] Optionally, the electronic device may also generate a preset training data set. Specifically, the electronic device may add Poisson noise to the original image of different parts of the target category that do not contain noise, based on the noise characteristics of the artifact image to be corrected corresponding to the target category, to generate a corresponding noisy image with noise artifacts.

[0133] The embodiment of the present application does not specifically limit the manner in which the electronic device obtains the preset training data set.

[0134] In step S302 , a preset training data set is input into an initial artifact correction network, and the initial artifact correction network corrects the noisy image and outputs a corrected virtual image.

[0135] Specifically, the electronic device may input a preset training data set into an initial artifact correction network, and the initial artifact correction network extracts features from the noisy image and outputs a corrected virtual image based on the extracted features.

[0136] The process of the initial artifact correction network outputting a virtual image can refer to the process of the target artifact correction model outputting a target image, which will not be described in detail here.

[0137] Step S303: Calculate a target loss value between the virtual image and the original image corresponding to the noise image based on a preset loss function.

[0138] Specifically, the above step S303 may include the following steps:

[0139] Step S3031: Calculate a first loss value between the virtual image and the original image corresponding to the noise image using a first loss function.

[0140] Specifically, the electronic device may use a first loss function to calculate a first loss value between the virtual image and the original image corresponding to the noise image.

[0141] Among them, the first loss function can be an absolute loss function, or it can be any one of the loss functions such as mean square error loss function, mean absolute error loss function, cross entropy loss function, binary cross entropy loss function, Hinge loss function, logarithmic loss function, Kullback-Leibler divergence loss function, structural similarity index loss function, etc. The embodiment of this application does not specifically limit the first loss function.

[0142] Step S3032: Calculate a second loss value between the virtual image and the original image corresponding to the noise image using a second loss function.

[0143] Specifically, the electronic device may use the second loss function to calculate a first loss value between the virtual image and the original image corresponding to the noise image.

[0144] Among them, the second loss function can be an absolute loss function, or it can be any one of the loss functions such as mean square error loss function, mean absolute error loss function, cross entropy loss function, binary cross entropy loss function, Hinge loss function, logarithmic loss function, Kullback-Leibler divergence loss function, structural similarity index loss function, etc. The embodiment of this application does not specifically limit the second loss function.

[0145] It should be noted that the first loss function and the second loss function are different.

[0146] Step S3033: Calculate a target loss value based on the first loss value and the second loss value.

[0147] Optionally, the electronic device may calculate an average of the first loss value and the second loss value, and determine the calculated average as the target loss value.

[0148] Optionally, the electronic device may multiply the first loss value and the second loss value by corresponding weights respectively, and then add the first loss value and the second loss value multiplied by the corresponding weights to calculate the target loss value.

[0149] For example, the electronic device may calculate the target loss value based on the following formula:

[0150] L SL =1-SSIM(x,y)

[0151]

[0152] L mix =λ1L SL +(1-λ1)L1

[0153] Among them, L SL represents the first loss value, L1 represents the second loss value, L mix is the target loss value, x represents the virtual image, y represents the original image without noise, H and W represent the image height and image width.

[0154] Step S304: Based on the target loss value, the initial artifact correction network is trained to generate a target artifact correction model.

[0155] Specifically, the electronic device can compare the target loss value with a preset loss threshold. If the target loss value is greater than the preset loss threshold, the network parameters in the initial artifact correction network are updated. This cycle continues until the target loss value is less than the preset loss threshold, generating a target artifact correction model.

[0156] The image noise artifact correction method provided in the embodiment of the present application obtains a preset training data set, inputs the preset training data set into an initial artifact correction network, and the initial artifact correction network corrects the noisy image and outputs a corrected virtual image, thereby ensuring the accuracy of the output corrected virtual image. A first loss function is used to calculate a first loss value between the virtual image and the original image corresponding to the noisy image, thereby ensuring the accuracy of the calculated first loss value. A second loss function is used to calculate a second loss value between the virtual image and the original image corresponding to the noisy image, thereby ensuring the accuracy of the calculated second loss value. Based on the first loss value and the second loss value, a target loss value is calculated, thereby ensuring the accuracy of the calculated target loss value. Based on the target loss value, the initial artifact correction network is trained to generate a target artifact correction model, thereby ensuring the accuracy of the generated target artifact correction model. In addition, the accuracy of the target image output by performing noise artifact correction on the artifact image to be corrected based on the target artifact correction model can be ensured.

[0157] This embodiment also provides an image noise artifact correction device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0158] This embodiment provides an image noise artifact correction device, such as Figure 5 Shown, including:

[0159] An acquisition module 401 is used to acquire an image to be corrected for artifacts;

[0160] A determination module 402 is configured to identify the image to be corrected for artifacts and determine the target category corresponding to the image to be corrected for artifacts;

[0161] The output module 403 is configured to input the artifact image to be corrected into a target artifact correction model corresponding to the target category, and output a target image corresponding to the artifact image to be corrected, where the target image does not include noise artifacts.

[0162] In some optional implementations, the determination module 402 is specifically configured to identify the artifact image to be corrected and determine the type of target object included in the artifact image to be corrected;

[0163] According to the type of target object, the target category corresponding to the artifact image to be corrected is determined.

[0164] In some optional embodiments, the output module 403 is specifically used to determine the target artifact correction model corresponding to the target category based on the correspondence between the category and the artifact correction model; input the artifact image to be corrected into the target artifact correction model, and output the target image corresponding to the artifact image to be corrected.

[0165] In some optional embodiments, the target artifact correction model includes an upper network branch, a middle network branch, and a lower network branch, and an output module 403, which is specifically used to input the artifact image to be corrected into the upper network branch, the middle network branch, and the lower network branch respectively; the upper network branch performs feature extraction on the artifact image to be corrected and outputs the upper branch result; the middle network branch performs feature extraction on the artifact image to be corrected and outputs the middle branch result; the lower network branch performs feature extraction on the artifact image to be corrected and outputs the lower branch result; according to the relationship between the upper branch result, the middle branch result, and the lower branch result, the target image corresponding to the artifact image to be corrected is output.

[0166] In some optional embodiments, the output module 403 is specifically used to stitch the upper-level branch results, the middle-level branch results, and the lower-level branch results to generate a stitched result image; perform feature extraction on the stitched result image to generate a noise estimation result; and subtract the noise estimation result from the artifact image to be corrected to output a target image.

[0167] In some optional embodiments, the output module 403 is specifically used to obtain a preset training data set, which includes a noisy image containing noise artifacts corresponding to the target category and an original image without noise artifacts corresponding to the noise image; input the preset training data set into the initial artifact correction network, the initial artifact correction network corrects the noise image, and outputs a corrected virtual image; based on a preset loss function, calculates a target loss value between the virtual image and the original image corresponding to the noise image; based on the target loss value, trains the initial artifact correction network to generate a target artifact correction model.

[0168] In some optional embodiments, the output module 403 is specifically used to calculate a first loss value between the virtual image and the original image corresponding to the noise image using a first loss function; calculate a second loss value between the virtual image and the original image corresponding to the noise image using a second loss function; and calculate a target loss value based on the first loss value and the second loss value.

[0169] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0170] The image noise artifact correction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0171] An embodiment of the present invention further provides an electronic device having the above Figure 5 The image noise artifact correction device shown.

[0172] See also Figure 6 , Figure 6 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0173] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0174] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0175] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0177] The electronic device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0178] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0179] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0180] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0181] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for correcting image noise artifacts, characterized in that: The method comprises: Acquire an image to be corrected for artifacts; Identifying the artifact image to be corrected, and determining a target category corresponding to the artifact image to be corrected; The artifact image to be corrected is input into a target artifact correction model corresponding to the target category, and a target image corresponding to the artifact image to be corrected is output, wherein the target image does not include noise artifacts.

2. The method according to claim 1, characterized in that The identifying the artifact image to be corrected and determining the target category corresponding to the artifact image to be corrected includes: Identifying the artifact image to be corrected, and determining the type of the target object included in the artifact image to be corrected; The target category corresponding to the artifact image to be corrected is determined according to the type of the target object.

3. The method according to claim 1, characterized in that The step of inputting the artifact image to be corrected into a target artifact correction model corresponding to the target category and outputting a target image corresponding to the artifact image to be corrected comprises: Determining the target artifact correction model corresponding to the target category according to the corresponding relationship between the category and the artifact correction model; The artifact image to be corrected is input into the target artifact correction model, and the target image corresponding to the artifact image to be corrected is output.

4. The method according to claim 3, characterized in that The target artifact correction model includes an upper network branch, a middle network branch, and a lower network branch. The step of inputting the artifact image to be corrected into the target artifact correction model and outputting the target image corresponding to the artifact image to be corrected includes: Inputting the artifact image to be corrected into the upper network branch, the middle network branch and the lower network branch respectively; The upper network branch extracts features from the artifact image to be corrected and outputs an upper branch result; The middle-layer network branch extracts features from the artifact image to be corrected and outputs a middle-layer branch result; The lower network branch extracts features from the artifact image to be corrected and outputs a lower branch result; According to the relationship between the upper-layer branch result, the middle-layer branch result and the lower-layer branch result, the target image corresponding to the artifact image to be corrected is output.

5. The method according to claim 4, characterized in that Outputting the target image corresponding to the artifact image to be corrected according to the relationship between the upper-layer branch result, the middle-layer branch result, and the lower-layer branch result includes: Performing splicing processing on the upper-layer branch results, the middle-layer branch results and the lower-layer branch results to generate a splicing result graph; Performing feature extraction on the splicing result image to generate a noise estimation result; The target image is output by subtracting the noise estimation result from the artifact image to be corrected.

6. The method according to claim 1, characterized in that The training process of the target artifact correction model includes: Acquire a preset training data set, wherein the preset training data set includes a noise image containing noise artifacts corresponding to the target category and an original image without noise artifacts corresponding to the noise image; Inputting the preset training data set into an initial artifact correction network, the initial artifact correction network corrects the noisy image and outputs a corrected virtual image; Based on a preset loss function, calculating a target loss value between the virtual image and the original image corresponding to the noise image; Based on the target loss value, the initial artifact correction network is trained to generate the target artifact correction model.

7. The method according to claim 6, characterized in that The calculating, based on a preset loss function, a target loss value between the virtual image and the original image corresponding to the noise image comprises: Calculating a first loss value between the virtual image and the original image corresponding to the noise image using a first loss function; Calculating a second loss value between the virtual image and the original image corresponding to the noise image using a second loss function; The target loss value is calculated according to the first loss value and the second loss value.

8. An image noise artifact correction device, characterized in that: The device comprises: An acquisition module, used for acquiring an artifact image to be corrected; A determination module, used to identify the artifact image to be corrected and determine the target category corresponding to the artifact image to be corrected; The output module is used to input the artifact image to be corrected into the target artifact correction model corresponding to the target category, and output the target image corresponding to the artifact image to be corrected, wherein the target image does not include noise artifacts.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image noise artifact correction method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the image noise artifact correction method according to any one of claims 1 to 7.

Images