Training method of image processing model, and image processing method and device

Abstract

The invention provides an image processing model training method, an image processing method and an image processing device, and the method comprises the steps: constructing a to-be-trained candidate image processing model based on a visual effect; obtaining a first sample noise image, and obtaining M first noise reduction images after the first sample noise image is subjected to noise reduction processing based on the target image noise reduction model; obtaining respective first visual effect scores of the M first noise reduction images based on the target visual effect scoring model, and obtaining first training loss of the candidate image processing model; and adjusting model parameters of the candidate image processing model based on the first training loss, and returning to obtain a next first sample noise image to continue to train the candidate image processing model after parameter adjustment until training is finished, thereby obtaining a target image processing model. The visual experience of people on the noise-reduced image obtained after noise reduction processing is optimized, and the training method and the training effect of the image processing model are optimized.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a training method for an image processing model, an image processing method and a device. Background Art

[0002] In implementation, the noise on the image can be understood as the pixels and / or pixel blocks that appear on the image and cause a strong visual effect. In the scenario of information transmission based on images, the noise on the image has a certain degree of influence on the accuracy and effect of image information transmission.

[0003] In the related technology, a clear image corresponding to a noisy image can be obtained, and the clear image can be used as a label of the noisy image to train the image denoising model to be trained. After the trained image denoising model is obtained, denoising processing is performed on the noisy image based on the trained image denoising model in the scenario to obtain a processed denoised image.

[0004] However, the denoised image obtained by denoising the noisy image based on the image denoising model trained in this scenario may have reduced image clarity, which has a certain impact on people's visual experience and the denoising effect is poor. Summary of the invention

[0005] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0006] To this end, the first aspect of the present disclosure proposes a training method for an image processing model.

[0007] A second aspect of the present disclosure provides an image processing method.

[0008] A third aspect of the present disclosure provides a training device for an image processing model.

[0009] A fourth aspect of the present disclosure provides an image processing device.

[0010] A fifth aspect of the present disclosure provides an electronic device.

[0011] A sixth aspect of the present disclosure provides a computer-readable storage medium.

[0012] The first aspect of the present disclosure proposes a training method for an image processing model, the method comprising: constructing a candidate image processing model based on visual effects to be trained, wherein the candidate image processing model comprises a trained target image denoising model and a target visual effect scoring model; obtaining a first sample noise image, and obtaining M first denoised images after denoising the first sample noise image based on the target image denoising model, wherein the M first denoised images each have a different degree of denoising; obtaining a first visual effect score of each of the M first denoised images based on the target visual effect scoring model, and obtaining a first training loss of the candidate image processing model based on the first visual effect score and a reference second visual effect score; adjusting a model parameter of the candidate image processing model based on the first training loss, and returning to obtain a next first sample noise image to continue training the candidate image processing model after the parameter adjustment, until the training is completed, to obtain a trained target image processing model.

[0013] In addition, the training method of the image processing model proposed in the first aspect of the present disclosure may also have the following additional technical features:

[0014] According to an embodiment of the present disclosure, the step of acquiring a first sample noise image and obtaining M first denoised images after denoising the first sample noise image based on the target image denoising model includes: extracting a first image feature set of the first sample noise image based on the target image denoising model; acquiring M first image feature combinations from the first image feature set, wherein the first image feature combination includes at least part of the image features in the first image feature set, and the M first image feature combinations each include different image features; generating first reconstructed images of each of the M first image feature combinations through the target image denoising model to obtain the M first denoised images of the first sample noise image, wherein the M first denoised images each have a different degree of denoising.

[0015] According to an embodiment of the present disclosure, the construction of a candidate image processing model based on visual effects to be trained includes: obtaining a candidate image denoising model and a candidate visual effect scoring model to be trained; obtaining a second sample noise image, and training the candidate image denoising model based on the second sample noise image to obtain the trained target image denoising model; obtaining a third sample noise image, and obtaining N second denoised images after denoising the third sample noise image based on the target image denoising model, wherein the N second denoised images each have a different degree of denoising; training the candidate visual effect scoring model based on the N second denoised images to obtain the trained target visual effect scoring model; combining the target image denoising model and the target visual effect scoring model to obtain the candidate image processing model based on visual effects to be trained.

[0016] According to an embodiment of the present disclosure, the step of acquiring a second sample noise image and training the candidate image denoising model based on the second sample noise image to obtain the trained target image denoising model includes: inputting the second sample noise image into the candidate image denoising model to obtain a third denoised image after denoising the second sample noise image output by the candidate image denoising model; acquiring a label image of the second sample noise image, and acquiring a second training loss of the candidate image denoising model based on the third denoised image and the label image; adjusting model parameters of the candidate image denoising model based on the second training loss, and returning to acquire the next second sample noise image to continue training the candidate image denoising model with adjusted parameters until the training is completed to obtain the trained target image denoising model.

[0017] According to an embodiment of the present disclosure, the method of acquiring a third sample noise image and obtaining N second denoised images after denoising the third sample noise image based on the target image denoising model includes: extracting a second image feature set of the third sample noise image based on the target image denoising model; acquiring N second image feature combinations from the second image feature set, wherein the second image feature combination includes at least part of the image features in the second image feature set, and the N second image feature combinations each include different image features; generating second reconstructed images of each of the N second image feature combinations through the target image denoising model to obtain the N second denoised images of the third sample noise image, wherein the N second denoised images each have a different degree of denoising.

[0018] According to an embodiment of the present disclosure, the candidate visual effect scoring model is trained based on the N second denoised images to obtain the trained target visual effect scoring model, including: obtaining the third visual effect score of each of the N second denoised images; obtaining the fourth visual effect score of each of the N second denoised images output by the candidate visual effect scoring model; for any second denoised image, based on the third visual effect score and the fourth visual effect score of the second denoised image, obtaining a third training loss of the candidate visual effect scoring model; adjusting model parameters of the candidate visual effect scoring model based on the third training loss, and returning to obtain the next second denoised image to continue model training for the candidate visual effect scoring model with adjusted parameters until the training is completed, so as to obtain the trained target visual effect scoring model.

[0019] According to an embodiment of the present disclosure, the method of obtaining the fourth visual effect score of each of the N second denoised images output by the candidate visual effect scoring model includes: for any second denoised image, obtaining a third image feature combination for generating the second denoised image from the N second image feature combinations; obtaining a first image feature vector obtained by splicing each image feature in the third image feature combination, and a second image feature vector of the second denoised image; obtaining a third denoised image obtained by splicing the first image feature vector and the second image feature vector, and performing a visual effect score on the third denoised image based on the candidate visual effect scoring model to obtain the fourth visual effect score of the second denoised image corresponding to the third denoised image.

[0020] According to an embodiment of the present disclosure, the model parameters of the candidate image processing model are adjusted based on the first training loss, and the next first sample noise image is returned to continue training the candidate image processing model with the adjusted parameters until the training is completed to obtain a trained target image processing model, including: adjusting the parameters of the decoder of the image denoising model included in the candidate image processing model according to the first training loss to obtain the candidate image processing model with the adjusted parameters; and returning to obtain the next first sample noise image to continue model training on the candidate image processing model with the adjusted parameters until the training is completed to obtain the trained target image processing model.

[0021] The second aspect of the present disclosure proposes an image processing method, which includes: obtaining a trained target image processing model based on visual effects, wherein the target image processing model includes a trained target image denoising model and a target visual effect scoring model, and the target image processing model, the target image denoising model and the target visual effect scoring model are obtained based on the training method proposed in the first aspect above; obtaining a processed noisy image to be processed, extracting a candidate image feature set of the processed noisy image based on the target image denoising model, and generating S candidate denoised images obtained after denoising the processed noisy image based on the candidate image feature set, wherein the S candidate denoised images each have a different degree of denoising; obtaining a candidate visual effect score for each of the S candidate denoised images based on the target visual effect scoring model; obtaining a target visual effect score with the highest score among the candidate visual effect scores, and determining the denoised image corresponding to the target visual effect score in the S candidate denoised images as the target denoised image obtained by visual effect processing of the processed noisy image.

[0022] To achieve the above-mentioned purpose, the third aspect of the present disclosure proposes a training device for an image processing model, the device comprising: a construction module, used to construct a candidate image processing model based on visual effects to be trained, wherein the candidate image processing model comprises a trained target image denoising model and a target visual effect scoring model; a first acquisition module, used to acquire a first sample noise image, and obtain M first denoised images after denoising the first sample noise image based on the target image denoising model, wherein the denoising degree of each of the M first denoised images is different; a second acquisition module, used to acquire a first visual effect score of each of the M first denoised images based on the target visual effect scoring model, and acquire a first training loss of the candidate image processing model based on the first visual effect score and a reference second visual effect score; a training module, used to adjust the model parameters of the candidate image processing model based on the first training loss, and return to acquire the next first sample noise image to continue training the candidate image processing model after the parameter adjustment, until the training is completed, and a trained target image processing model is obtained.

[0023] In addition, the training device for the image processing model proposed in the third aspect of the present disclosure may also have the following additional technical features:

[0024] According to one embodiment of the present disclosure, the first acquisition module is further used to: extract a first image feature set of the first sample noise image based on the target image denoising model; obtain M first image feature combinations from the first image feature set, wherein the first image feature combination includes at least part of the image features in the first image feature set, and the M first image feature combinations each include different image features; generate a first reconstructed image of each of the M first image feature combinations through the target image denoising model to obtain the M first denoised images of the first sample noise image, wherein the M first denoised images each have a different degree of denoising.

[0025] According to one embodiment of the present disclosure, the construction module is also used to: obtain a candidate image denoising model and a candidate visual effect scoring model to be trained; obtain a second sample noise image, and train the candidate image denoising model based on the second sample noise image to obtain the trained target image denoising model; obtain a third sample noise image, and obtain N second denoised images after denoising the third sample noise image based on the target image denoising model, wherein the N second denoised images each have a different degree of denoising; train the candidate visual effect scoring model based on the N second denoised images to obtain the trained target visual effect scoring model; combine the target image denoising model and the target visual effect scoring model to obtain the candidate image processing model based on visual effects to be trained.

[0026] According to one embodiment of the present disclosure, the construction module is also used to: input the second sample noise image into the candidate image denoising model to obtain a third denoised image after the second sample noise image output by the candidate image denoising model is denoised; obtain a label image of the second sample noise image, and obtain a second training loss of the candidate image denoising model based on the third denoised image and the label image; adjust the model parameters of the candidate image denoising model based on the second training loss, and return to obtain the next second sample noise image to continue training the candidate image denoising model with adjusted parameters until the training is completed, so as to obtain the trained target image denoising model.

[0027] According to one embodiment of the present disclosure, the construction module is also used to: extract a second image feature set of the third sample noise image based on the target image denoising model; obtain N second image feature combinations from the second image feature set, wherein the second image feature combination includes at least part of the image features in the second image feature set, and the N second image feature combinations each include different image features; generate a second reconstructed image of each of the N second image feature combinations through the target image denoising model to obtain the N second denoised images of the third sample noise image, wherein the N second denoised images each have a different degree of denoising.

[0028] According to one embodiment of the present disclosure, the construction module is also used to: obtain the third visual effect score of each of the N second denoised images; obtain the fourth visual effect score of each of the N second denoised images output by the candidate visual effect scoring model; for any second denoised image, based on the third visual effect score and the fourth visual effect score of the second denoised image, obtain the third training loss of the candidate visual effect scoring model; adjust the model parameters of the candidate visual effect scoring model based on the third training loss, and return to obtain the next second denoised image to continue model training for the candidate visual effect scoring model with adjusted parameters until the training is completed, so as to obtain the trained target visual effect scoring model.

[0029] According to one embodiment of the present disclosure, the construction module is also used to: for any second denoised image, obtain a third image feature combination for generating the second denoised image from the N second image feature combinations; obtain a first image feature vector obtained by splicing each image feature in the third image feature combination, and a second image feature vector of the second denoised image; obtain a third denoised image obtained by splicing the first image feature vector and the second image feature vector, and perform a visual effect score on the third denoised image based on the candidate visual effect scoring model to obtain the fourth visual effect score of the second denoised image corresponding to the third denoised image.

[0030] According to one embodiment of the present disclosure, the training module is also used to: adjust the parameters of the decoder of the image denoising model included in the candidate image processing model according to the first training loss to obtain the candidate image processing model after the parameters are adjusted; return to obtain the next first sample noise image to continue model training on the candidate image processing model after the parameters are adjusted until the training is completed to obtain the trained target image processing model.

[0031] In a fourth aspect, the present disclosure proposes an image processing device, which includes: a third acquisition module, used to acquire a trained target image processing model based on visual effects, wherein the target image processing model includes a trained target image denoising model and a target visual effect scoring model, and the target image processing model, the target image denoising model and the target visual effect scoring model are obtained based on the training device proposed in the third aspect; a denoising module, used to acquire a processed noise image to be processed, extract a candidate image feature set of the processed noise image based on the target image denoising model, and generate S candidate denoised images obtained after denoising the processed noise image based on the candidate image feature set, wherein the S candidate denoised images have different denoising degrees; a scoring module, used to acquire a candidate visual effect score of each of the S candidate denoised images based on the target visual effect scoring model; and a determination module, used to acquire a target visual effect score with the highest score among the candidate visual effect scores, and determine the denoised image corresponding to the target visual effect score in the S candidate denoised images as the target denoised image obtained by the visual effect processing of the processed noise image.

[0032] The fifth aspect of the present disclosure proposes an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute instructions to implement the training method of the image processing model proposed in the first aspect above and / or the image processing method proposed in the second aspect above.

[0033] The sixth aspect of the present disclosure proposes a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of the image processing model proposed in the first aspect and / or the image processing method proposed in the second aspect.

[0034] The present invention discloses a training method and device for an image processing model, which constructs a candidate image processing model to be trained and obtains a first sample noise image, obtains M first denoised images of the first sample noise image based on a target image denoising model in the candidate image processing model, performs visual effect scoring on the M first denoised images based on a target visual effect scoring model, and obtains a first visual effect score for each of the M first denoised images, further obtains a first training loss for the candidate image processing model based on the first visual effect score for each of the M first denoised images and a reference second visual effect score, adjusts model parameters of the candidate image processing model based on the first training loss, returns to obtain the next first sample noise image, and continues to train the candidate image processing model after parameter adjustment until the training is completed, thereby obtaining a trained target image processing model. In the present disclosure, a first training loss is obtained based on a first visual effect score and a second visual effect score, and a candidate image processing model is trained based on the first training loss, so that the trained target image processing model can realize the processing of noisy images based on visual effects, thereby improving the visual effect of the denoised image obtained after the denoising process of the noisy image, improving the degree of adaptation between the visual effect of the denoised image and people's perception, optimizing people's visual experience of the denoised image obtained after the denoising process, and optimizing the training method and training effect of the image processing model.

[0035] It should be understood that the contents described in the present disclosure are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0037] Figure 1 A flowchart of a method for training an image processing model according to an embodiment of the present disclosure;

[0038] Figure 2 A flowchart of a method for training an image processing model according to another embodiment of the present disclosure;

[0039] Figure 3 A schematic diagram of an image denoising model according to an embodiment of the present disclosure;

[0040] Figure 4 A flowchart of a method for training an image processing model according to another embodiment of the present disclosure;

[0041] Figure 5 is a schematic diagram of an image processing model according to an embodiment of the present disclosure;

[0042] Figure 6 A flowchart of an image processing method according to an embodiment of the present disclosure;

[0043] Figure 7 A schematic diagram of the structure of a training device for an image processing model according to an embodiment of the present disclosure;

[0044] Figure 8 is a structural schematic diagram of an image processing device according to an embodiment of the present disclosure;

[0045] Fig. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0046] Fig.10 The present invention is a block diagram of an electronic device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0047] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0048] The following describes an image processing model training method, image processing method and device proposed in an embodiment of the present disclosure with reference to the accompanying drawings.

[0049] Figure 1 FIG. 1 is a flow chart of a training method for an image processing model according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0050] S101, constructing a candidate image processing model based on visual effects to be trained, wherein the candidate image processing model includes a trained target image denoising model and a target visual effect scoring model.

[0051] In the disclosed embodiments, a corresponding image processing model may be constructed based on visual effects, wherein the constructed model to be trained for performing image processing based on visual effects may be determined as a candidate image processing model.

[0052] Optionally, the candidate image processing models may include a model for denoising a noisy image and a model for performing visual effect scoring on the image obtained after the denoising process, wherein the trained model for denoising a noisy image can be determined as the target image denoising model, and the trained model for performing visual effect scoring on the image obtained after the denoising process can be determined as the target visual effect scoring model.

[0053] S102, acquiring a first sample noise image, and obtaining M first denoised images after denoising the first sample noise image based on a target image denoising model, wherein the M first denoised images have different denoising degrees.

[0054] In the disclosed embodiment, after the candidate image processing model is constructed, it is also necessary to perform model training on the candidate image processing model.

[0055] Optionally, a portion of the noisy image can be obtained from an open source image dataset and determined as the first sample noise image required for model training of the candidate image processing model, wherein the open source dataset can be the COCO (Common Objects in Context) 2017 dataset or other open source datasets, which are not specifically limited here.

[0056] Furthermore, the acquired first sample noisy image is input into a candidate image generation model to be trained, and the first sample noisy image is subjected to denoising processing by a target image denoising model in the candidate image generation model.

[0057] Optionally, noise image features and non-noise image features in the first sample noise image can be extracted based on the target image denoising model, and a denoised image can be generated based on the extracted noise image features and non-noise image features, thereby obtaining M denoised images with different degrees of denoising after the first sample noise image is denoised, which are marked as M first denoised images.

[0058] It should be noted that there are differences between the first image feature combinations used when the M first denoised images are generated. In this scenario, it can be determined that the denoising degrees of the M first denoised images are also different.

[0059] S103, obtaining a first visual effect score of each of the M first denoised images based on the target visual effect scoring model, and obtaining a first training loss of the candidate image processing model according to the first visual effect score and a reference second visual effect score.

[0060] In the disclosed embodiment, after obtaining M first denoised images of the first sample noisy image, the M first denoised images can be scored for visual effects respectively based on the target visual effect scoring model in the candidate image processing model, thereby obtaining a first visual effect score for each of the M first denoised images.

[0061] Optionally, the candidate image processing model has a reference visual effect score, marked as a second visual effect score. It can be understood that the visual effect score of the denoised image obtained by denoising based on the candidate image processing model needs to meet the scoring criteria corresponding to the second visual effect score.

[0062] In this scenario, the first visual effect scores of the M first denoised images can be obtained based on the loss values ​​of the second visual effect scores, and the loss values ​​can be determined as the first training loss of the candidate image processing model.

[0063] Among them, based on the loss value algorithm in the relevant technology, the M first visual effect scores and the second visual effect scores can be processed separately by algorithm, so as to obtain the loss values ​​of the M first visual effect scores based on the second visual effect scores, and then obtain the first training loss of the candidate image processing model.

[0064] It should be noted that after obtaining M loss values ​​obtained by obtaining M first visual effect scores based on the second visual effect scores, the M loss values ​​can all be used as the first training loss, or the M loss values ​​can be integrated to obtain the corresponding integrated loss value as the first training loss, which is not specifically limited here.

[0065] S104, adjusting the model parameters of the candidate image processing model based on the first training loss, and returning to obtain the next first sample noise image to continue training the candidate image processing model with adjusted parameters until the training is completed, thereby obtaining a trained target image processing model.

[0066] In the disclosed embodiment, the model parameters of the candidate image processing model may be adjusted based on the first training loss to obtain the candidate image processing model with adjusted parameters.

[0067] Furthermore, the next first sample noise image is obtained and the model training of the candidate image processing model after parameter adjustment is continued until the training is completed to obtain a trained target image processing model.

[0068] Optionally, corresponding training end conditions can be set based on the rounds of model training. If the training round of the candidate image processing model in the current round meets the preset training end conditions, the model training of the candidate image processing model can be stopped, and the model obtained at the end of the current round of training can be determined as the trained target image processing model.

[0069] Optionally, corresponding training end conditions can be set based on the training loss of model training. If the training loss of the candidate image processing model in the current round meets the preset training end conditions, the model training of the candidate image processing model can be stopped, and the model obtained at the end of the current round of training is determined as the trained target image processing model.

[0070] The present invention discloses a training method for an image processing model, which constructs a candidate image processing model to be trained and obtains a first sample noise image, obtains M first denoised images of the first sample noise image based on a target image denoising model in the candidate image processing model, performs visual effect scoring on the M first denoised images based on a target visual effect scoring model, and obtains a first visual effect score for each of the M first denoised images, further obtains a first training loss for the candidate image processing model based on the first visual effect score for each of the M first denoised images and a reference second visual effect score, adjusts model parameters of the candidate image processing model based on the first training loss, returns to obtain the next first sample noise image, and continues to train the candidate image processing model after parameter adjustment until the training is completed, thereby obtaining a trained target image processing model. In the present disclosure, a first training loss is obtained based on a first visual effect score and a second visual effect score, and a candidate image processing model is trained based on the first training loss, so that the trained target image processing model can realize the processing of noisy images based on visual effects, thereby improving the visual effect of the denoised image obtained after the denoising process of the noisy image, improving the degree of adaptation between the visual effect of the denoised image and people's perception, optimizing people's visual experience of the denoised image obtained after the denoising process, and optimizing the training method and training effect of the image processing model.

[0071] In the above embodiment, the training of the candidate image processing model can be combined with Figure 2 Further understanding, Figure 2 FIG. 1 is a flow chart of a training method for an image processing model according to another embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0072] S201, extracting a first image feature set of a first sample noise image based on a target image denoising model.

[0073] In the disclosed embodiment, the first sample noise image may be input into a target image denoising model in a candidate image processing model, and an encoder based on the target image denoising model may extract image features from the first sample noise image.

[0074] As an example, the first sample noise image can be input into Figure 3 In the encoder of the target image denoising model shown, image features of a first sample noisy image are extracted by the encoder.

[0075] like Figure 3 As shown, the encoder extracts features from the first sample noise image and obtains Figure 3 A feature set consisting of feature 1, feature 2, feature 3, ..., feature n is shown, and the set is determined as the first image feature set of the first sample noise image.

[0076] S202: Obtain M first image feature combinations from the first image feature set, wherein the first image feature combination includes at least part of the image features in the first image feature set, and the M first image feature combinations each include a different image feature.

[0077] In the embodiment of the present disclosure, at least part of the image features may be acquired from the first image feature set, and the acquired at least part of the image features may be determined as the first image feature combination acquired from the first image feature set.

[0078] In order to obtain M denoised images with different denoising degrees, M groups of partial image features can be extracted from the first image feature set, and each group of partial image features in the M groups of partial image features can be marked as a first image feature combination, thereby obtaining M first image feature combinations, wherein there are differences between the image features included in each of the M first image feature combinations.

[0079] As an example, Figure 3 As shown, if M is set to 2, the combination of feature 2 and feature 3 can be obtained from the first image feature set consisting of feature 1, feature 2, feature 3, ..., feature n as Figure 3 The first image feature combination obtained from the first image feature set shown.

[0080] Furthermore, a combination of feature 3 and feature n3 can be obtained from the first image feature set consisting of feature 1, feature 2, feature 3, ..., feature n as Figure 3 The first image feature combination obtained from the first image feature set shown.

[0081] Further, we get Figure 3 The M first image feature combinations with a value of 2 in the first image feature set are shown.

[0082] S203, generating first reconstructed images of respective M first image feature combinations through a target image denoising model, and obtaining M first denoised images of the first sample noisy image, wherein the denoising degrees of the M first denoised images are different.

[0083] like Figure 3 As shown, the M first image feature combinations can be respectively input into the decoder of the target image denoising model, and the decoder is used to reconstruct an image of each of the M first image feature combinations, thereby obtaining the first reconstructed image of each of the M first image feature combinations, and determining the first reconstructed image of each first image feature combination as the first denoised image of each of the M first image feature combinations.

[0084] For any first image feature combination, the decoder can be used to reconstruct an image of the first image feature combination, and the reconstructed first reconstructed image is determined as the first denoised image of the first image feature combination, thereby obtaining M first denoised images of the M first image feature combinations.

[0085] S204, obtaining a first visual effect score of each of the M first denoised images based on the target visual effect scoring model, and obtaining a first training loss of the candidate image processing model according to the first visual effect score and a reference second visual effect score.

[0086] For detailed information about S204, please refer to the relevant content in the above embodiment, which will not be repeated here.

[0087] S205, adjusting the model parameters of the candidate image processing model based on the first training loss, and returning to obtain the next first sample noise image to continue training the candidate image processing model with adjusted parameters until the training is completed, thereby obtaining a trained target image processing model.

[0088] Optionally, the decoder of the image denoising model included in the candidate image processing model can be parameter adjusted according to the first training loss to obtain the candidate image processing model after parameter adjustment, and the next first sample noise image can be returned to continue model training for the candidate image processing model after parameter adjustment until the training is completed to obtain a trained target image processing model.

[0089] In the disclosed embodiment, the model parameters of the decoder in the candidate image processing model can be adjusted based on the first training loss, thereby achieving parameter adjustment of the candidate image processing model and obtaining the candidate image processing model after parameter adjustment.

[0090] Furthermore, return to obtain the next first sample noise image, perform noise reduction processing and visual effect scoring on the next first sample noise image, and then continue to train the candidate image processing model until the training is completed to obtain a trained target image processing model.

[0091] The training method of the image processing model proposed in the present disclosure obtains a first image feature set of a first sample noise image, obtains M first image feature combinations from the first image feature set, and then obtains M first denoised images obtained after denoising the first sample noise image. Further, obtains a first visual effect score of each of the M first denoised images, obtains a first training loss of the candidate image processing model based on the first visual effect score of each of the M first denoised images and a reference second visual effect score, and adjusts the model parameters of the candidate image processing model based on the first training loss, returns to obtain the next first sample noise image to continue training the candidate image processing model after the parameter adjustment, until the training is completed, and obtains a trained target image processing model. In the present disclosure, a first training loss is obtained based on a first visual effect score and a second visual effect score, and a candidate image processing model is trained based on the first training loss, so that the trained target image processing model can realize the processing of noisy images based on visual effects, improve the visual effect of the denoised image obtained after the denoising process of the noisy image, improve the degree of adaptation between the visual effect of the denoised image and people's perception, optimize people's visual experience of the denoised image obtained after the denoising process, and optimize the training method and training effect of the image processing model.

[0092] In the above embodiment, the acquisition of the candidate image processing model to be trained can be combined with Figure 4 Further understanding, Figure 4 FIG. 1 is a flow chart of a training method for an image processing model according to another embodiment of the present disclosure. Figure 4 As shown, the method includes:

[0093] S401, obtaining a candidate image denoising model and a candidate visual effect scoring model to be trained.

[0094] Optionally, an image denoising model and a visual effect scoring model can be constructed based on the model building method in the relevant technology, and the constructed image denoising model can be determined as a candidate image denoising model to be trained, and the constructed visual effect scoring model can be determined as a candidate visual effect scoring model to be trained.

[0095] S402, obtaining a second sample noise image, and training a candidate image denoising model based on the second sample noise image to obtain a trained target image denoising model.

[0096] Optionally, the second sample noise image may be input into the candidate image noise reduction model to obtain a third noise reduction image after the second sample noise image output by the candidate image noise reduction model is subjected to noise reduction processing.

[0097] In the disclosed embodiment, the noise image used for training the candidate image denoising model can be determined as the second sample noise image, wherein the second sample noise image can be obtained from the COCO2017 open source dataset or from other datasets, which is not specifically limited here.

[0098] Among them, the second sample noisy image can be input into the candidate image denoising model, the image features of the second sample noisy image can be extracted by the encoder in the candidate image denoising model, and then the image features are reconstructed by the decoder in the candidate image denoising model to obtain the denoised image obtained by denoising the second sample noise image through the candidate image denoising model as the third denoised image.

[0099] Optionally, a label image of the second sample noisy image is obtained, and a second training loss of the candidate image denoising model is obtained based on the third denoised image and the label image.

[0100] In the embodiment of the present disclosure, a noise-free and / or low-noise clear image corresponding to the second sample noise image may be used as a label image of the second sample noise image.

[0101] In this scenario, the loss value of the third denoised image corresponding to the second sample noisy image based on the label image can be obtained, and the training loss of the candidate image denoising model can be obtained based on the loss value as the second training loss.

[0102] In some implementations, the third denoised image and the label image can be algorithmically processed pixel by pixel based on an algorithm for calculating the difference between pixels in the related technology to obtain the difference of the third denoised image based on the label image in the pixel dimension, and then obtain the loss value of the third denoised image based on the label image.

[0103] Optionally, the model parameters of the candidate image denoising model are adjusted according to the second training loss, and the next second sample noise image is returned to continue training the candidate image denoising model with adjusted parameters until the training is completed to obtain a trained target image denoising model.

[0104] In the disclosed embodiment, the model parameters of the candidate image denoising model can be adjusted based on the second training loss, and the next second sample noise image can be obtained to continue model training of the candidate image denoising model with adjusted parameters until the training is completed to obtain a trained target image denoising model.

[0105] Among them, the training end condition of the candidate image denoising model can be set based on the training rounds. When the training rounds of the candidate image denoising model meet the preset training end condition, the training can be ended to obtain the trained target image denoising model.

[0106] Furthermore, the training end condition of the candidate image denoising model can be set based on the training loss. When the training loss of the candidate image denoising model meets the preset training end condition, the training can be ended to obtain the trained target image denoising model.

[0107] S403, acquiring a third sample noise image, and obtaining N second denoised images after denoising the third sample noise image based on a target image denoising model, wherein the N second denoised images have different denoising degrees.

[0108] Optionally, a second image feature set of the third sample noise image is extracted based on the target image denoising model.

[0109] In the disclosed embodiment, the noise image used to train the candidate visual effect scoring model may be determined as the third sample noise image.

[0110] The third sample noise image may be input into a trained target image denoising model, and an image feature set of the third sample noise image may be extracted as the second image feature set through an encoder of the target image denoising model.

[0111] Optionally, N second image feature combinations are obtained from the second image feature set, wherein the second image feature combination includes at least part of the image features in the second image feature set, and the N second image feature combinations each include a different image feature.

[0112] In the disclosed embodiment, N groups of partial image feature combinations may be acquired from the second image feature set, and the acquired N groups of image feature combinations may be determined as N second image feature combinations.

[0113] It should be noted that, among the N second image feature combinations, each second image feature combination may include at least part of the image features in the second image feature set, and there are differences between the image features included in each second image feature combination.

[0114] Optionally, second reconstructed images of respective N second image feature combinations are generated by a target image denoising model to obtain N second denoised images of the third sample noisy image, wherein the denoising degrees of the N second denoised images are different.

[0115] In the disclosed embodiment, the decoder of the target image denoising model can be used to perform image reconstruction on each of the N second image feature combinations, and obtain the reconstructed image of each second image feature combination as the second reconstructed image.

[0116] Furthermore, the second reconstructed image of each second image feature combination is determined as the second denoised image of each second image feature combination, and then the second denoised images of the N second image feature combinations are obtained.

[0117] It should be noted that there are differences between the second image feature combinations used when the N second denoised images are generated. In this scenario, it can be determined that the denoising degrees of the N second denoised images are also different.

[0118] S404: Train the candidate visual effect scoring model based on the N second denoised images to obtain a trained target visual effect scoring model.

[0119] Optionally, a third visual effect score of each of the N second denoised images is obtained.

[0120] In the disclosed embodiment, the N second denoised images have visual scores that can be used as labels, wherein the visual scores that exist as labels for each of the N second denoised images can be determined as the third visual effect scores of each of the N second denoised images.

[0121] In some implementations, visual effect scores may be manually performed on the N second denoised images, and the visual effect scores may be determined as the third visual effect scores of the N second denoised images.

[0122] Optionally, a fourth visual effect score of each of the N second denoised images output by the candidate visual effect scoring model is obtained.

[0123] In the disclosed embodiment, the N second denoised images obtained by denoising the third sample noise image can be respectively input into the candidate visual effect models to be trained, and the visual effects of the N second denoised images can be scored respectively by the candidate visual effect scoring model.

[0124] Among them, for any second denoised image, a third image feature combination for generating the second denoised image is obtained from the N second image feature combinations, and a first image feature vector obtained by splicing each image feature in the third image feature combination and a second image feature vector of the second denoised image are obtained.

[0125] It can be understood that, for any second denoised image, the second image feature combination used by the decoder to generate the second denoised image can be obtained, and the second image feature combination can be determined as the third image feature combination for generating the second denoised image among the N second image feature combinations.

[0126] Furthermore, the image features included in the third image feature combination are feature spliced, and a feature vector obtained after the splicing is marked as a first image feature vector of the third image feature combination.

[0127] And, an image feature vector corresponding to the second denoised image is obtained as a second image feature vector.

[0128] Optionally, a third denoised image obtained by concatenating the first image feature vector and the second image feature vector is obtained, and a visual effect score is performed on the third denoised image based on a candidate visual effect scoring model to obtain a fourth visual effect score of the second denoised image corresponding to the third denoised image.

[0129] For any second denoised image, the first image feature vector of the third image feature combination corresponding to the second denoised image can be concatenated with the second image feature vector of the second denoised image, and the image corresponding to the concatenated feature vector can be determined as the third denoised image.

[0130] Furthermore, a candidate visual effect scoring model is obtained to perform visual effect scoring on the third denoised image, and the score output by the candidate visual effect scoring model is determined as the fourth visual effect score of the second denoised image corresponding to the third denoised image.

[0131] Optionally, for any second denoised image, based on the third visual effect score and the fourth visual effect score of the second denoised image, a third training loss of the candidate visual effect scoring model is obtained, and based on the third training loss, the model parameters of the candidate visual effect scoring model are adjusted, and the next second denoised image is returned to continue model training for the candidate visual effect scoring model with adjusted parameters until the training is completed, so as to obtain a trained target visual effect scoring model.

[0132] In the disclosed embodiment, a fourth visual effect score of the second denoised image output by the candidate visual effect scoring model can be obtained, a loss value of the third visual effect score based on the second denoised image as a label can be obtained, and a third training loss of the candidate visual effect score can be obtained based on the loss value.

[0133] Furthermore, the model parameters of the candidate visual effect scoring model are adjusted based on the third training loss, and the next second denoised image is returned to continue model training of the candidate visual effect scoring model with adjusted parameters until the training is completed, thereby obtaining a trained target visual effect scoring model.

[0134] S405, combining the target image denoising model and the target visual effect scoring model to obtain a candidate image processing model based on visual effects to be trained.

[0135] In the disclosed embodiment, the trained target image denoising model and the target visual effect scoring model may be combined, and the combined model may be determined as a candidate image processing model to be trained.

[0136] As an example, Figure 5 As shown, the target image denoising model and the target visual effect scoring model can be based on Figure 5 The combined candidate image processing model to be trained is obtained.

[0137] The training method of the image processing model proposed in the present disclosure obtains a candidate image denoising model and a candidate visual effect scoring model to be trained, and trains the candidate image denoising model and the candidate visual effect scoring model respectively to obtain a trained target image denoising model and a target visual effect scoring model. Optionally, the target image denoising model and the target visual effect scoring model can be combined to obtain a candidate image processing model to be trained. In the present disclosure, the candidate image processing model to be trained is obtained based on the trained target image denoising model and the target visual effect scoring model, so that the trained target image processing model can realize the processing of noisy images based on visual effects, improve the visual effect of the denoised image obtained after the denoising process of the noisy image, improve the degree of adaptation between the visual effect of the denoised image and people's perception, optimize people's visual experience of the denoised image obtained after the denoising process, and optimize the training method and training effect of the image processing model.

[0138] The present disclosure also proposes an image processing method that can be combined with Figure 6 understand, Figure 6 FIG. 1 is a flow chart of an image processing method according to an embodiment of the present disclosure. Figure 6 As shown, the method includes:

[0139] S601, obtaining a trained target image processing model based on visual effects, wherein the target image processing model includes a trained target image denoising model and a target visual effect scoring model.

[0140] Among them, the target image processing model, target image denoising model and target visual effect scoring model are based on Figures 1 to 4 The training method proposed in the embodiment is obtained.

[0141] In the disclosed embodiment, the noisy image can be denoised based on the visual effect by using a trained target image processing model.

[0142] Optionally, the target image processing model may include a trained target image processing model and a trained target visual effect scoring model.

[0143] It should be noted that the target image processing model, the target image denoising model and the target visual effect scoring model can be based on the above Figures 1 to 4 The training method of the image processing model proposed in the embodiment is obtained.

[0144] S602, obtaining a processed noisy image to be processed, extracting a candidate image feature set of the processed noisy image based on a target image denoising model, and generating S candidate denoised images after denoising the processed noisy image based on the candidate image feature set, wherein the S candidate denoised images have different denoising degrees.

[0145] In the disclosed embodiment, an image that needs to be subjected to noise reduction processing may be determined as a noise image to be processed, and the noise image to be processed may be input into a target image processing model.

[0146] Among them, the encoder based on the target image denoising model extracts the image features of the noisy image to be processed to form a candidate image feature set.

[0147] In this scenario, S candidate image feature combinations may be obtained from the candidate image feature set, wherein each candidate image feature combination may include at least part of the image features in the candidate image feature set.

[0148] Furthermore, each image feature combination in the S candidate image feature combinations is reconstructed through the decoder in the target image denoising model, and the reconstructed images of each of the S candidate image feature combinations are used as the denoised images of each of the S candidate image feature combinations, thereby obtaining S candidate denoised images after denoising the noisy image to be processed.

[0149] It should be noted that there are differences between the candidate image feature combinations used when the S candidate denoised images are reconstructed and generated. In this scenario, it can be determined that the denoising degrees of the S candidate denoised images are different.

[0150] S603, based on the target visual effect scoring model, obtaining candidate visual effect scores of each of the S candidate denoised images.

[0151] Optionally, the S candidate denoised images may be input into a trained target visual effect scoring model, and the visual effects of the S candidate denoised images may be scored and outputted respectively by the target visual effect scoring model.

[0152] Among them, the scores of the S candidate denoised images output by the target visual effect scoring model can be marked as the candidate visual effect scores of the S candidate denoised images.

[0153] S604, obtaining the target visual effect score with the highest score among the candidate visual effect scores, and determining the corresponding denoised image with the target visual effect score among the S candidate denoised images as the target denoised image obtained by visual effect processing of the noisy image to be processed.

[0154] In the disclosed embodiment, the specific scores of the candidate visual effect scores of the S candidate denoised images may be compared to obtain the highest score among all the candidate visual effect scores as the target visual effect score with the highest score among the candidate visual effect scores.

[0155] Furthermore, a denoised image corresponding to the target visual effect score in the S candidate denoised images is obtained, and the denoised image is determined as a target denoised image obtained by processing the noisy image to be processed based on the visual effect.

[0156] It can be understood that, compared with the remaining candidate denoised images except the target denoised image among the S candidate denoised images, the target denoised image corresponding to the target visual effect score has a better visual effect than the remaining candidate denoised images. In this scenario, the target denoised image can be used as the target image processing model to process the noisy image to be processed based on the visual effect to obtain the denoised image and output it.

[0157] The image processing method proposed in the present disclosure obtains a trained target image processing model, and extracts a candidate image feature set of the noise image to be processed based on the target image denoising model in the target image processing model, and then generates S candidate denoised images of the noise image to be processed. Furthermore, the candidate visual effect scores of the S candidate denoised images are obtained based on the target visual effect scoring model, and then the target visual effect score with the highest score is obtained from the candidate visual effect scores, and the corresponding denoised image with the target visual effect score in the S candidate denoised images is determined as the target denoised image. In the present disclosure, denoising processing of the noise image based on visual effects is realized based on the target image processing model, which improves the degree of adaptation between the visual effect of the denoised image obtained after the noise image is processed and people's perception, and optimizes the denoising processing method and denoising processing effect of the noise image.

[0158] Corresponding to the training methods of image processing models proposed in the above-mentioned embodiments, an embodiment of the present disclosure also proposes a training device for an image processing model. Since the training device for image processing models proposed in the embodiment of the present disclosure corresponds to the training methods for image processing models proposed in the above-mentioned embodiments, the implementation methods of the above-mentioned training methods for image processing models are also applicable to the training device for image processing models proposed in the embodiment of the present disclosure, and will not be described in detail in the following embodiments.

[0159] Figure 7 FIG. 1 is a schematic diagram of a structure of a training device for an image processing model according to an embodiment of the present disclosure. Figure 7 As shown, the training device 700 of the image processing model includes a construction module 71, a first acquisition module 72, a second acquisition module 73 and a training module 74, wherein:

[0160] A construction module 71 is used to construct a candidate image processing model based on visual effects to be trained, wherein the candidate image processing model includes a trained target image denoising model and a target visual effect scoring model;

[0161] A first acquisition module 72 is used to acquire a first sample noise image, and obtain M first denoised images after denoising the first sample noise image based on a target image denoising model, wherein the M first denoised images have different denoising degrees;

[0162] A second acquisition module 73 is used to acquire a first visual effect score of each of the M first denoised images based on the target visual effect scoring model, and acquire a first training loss of the candidate image processing model according to the first visual effect score and a reference second visual effect score;

[0163] The training module 74 is used to adjust the model parameters of the candidate image processing model based on the first training loss, and return to obtain the next first sample noise image to continue training the candidate image processing model with adjusted parameters until the training is completed to obtain a trained target image processing model.

[0164] In the disclosed embodiment, the first acquisition module 72 is further used to: extract a first image feature set of a first sample noise image based on a target image denoising model; obtain M first image feature combinations from the first image feature set, wherein the first image feature combination includes at least part of the image features in the first image feature set, and the M first image feature combinations each include different image features; generate a first reconstructed image of each of the M first image feature combinations through the target image denoising model, and obtain M first denoised images of the first sample noise image, wherein the M first denoised images each have a different degree of denoising.

[0165] In the disclosed embodiment, the construction module 71 is also used to: obtain a candidate image denoising model and a candidate visual effect scoring model to be trained; obtain a second sample noise image, and train the candidate image denoising model based on the second sample noise image to obtain a trained target image denoising model; obtain a third sample noise image, and obtain N second denoised images after the third sample noise image is denoised based on the target image denoising model, wherein the N second denoised images each have a different degree of denoising; train the candidate visual effect scoring model based on the N second denoised images to obtain a trained target visual effect scoring model; combine the target image denoising model and the target visual effect scoring model to obtain a candidate image processing model based on visual effects to be trained.

[0166] In the disclosed embodiment, the construction module 71 is also used to: input the second sample noise image into the candidate image denoising model to obtain a third denoised image after the second sample noise image output by the candidate image denoising model is denoised; obtain the label image of the second sample noise image, and obtain the second training loss of the candidate image denoising model based on the third denoised image and the label image; adjust the model parameters of the candidate image denoising model according to the second training loss, and return to obtain the next second sample noise image to continue training the candidate image denoising model with adjusted parameters until the training is completed, thereby obtaining a trained target image denoising model.

[0167] In the disclosed embodiment, the construction module 71 is also used to: extract a second image feature set of the third sample noise image based on the target image denoising model; obtain N second image feature combinations from the second image feature set, wherein the second image feature combination includes at least part of the image features in the second image feature set, and the N second image feature combinations each include different image features; generate a second reconstructed image of each of the N second image feature combinations through the target image denoising model, and obtain N second denoised images of the third sample noise image, wherein the N second denoised images each have a different degree of denoising.

[0168] In the disclosed embodiment, the construction module 71 is further used to: obtain the third visual effect score of each of the N second denoised images; obtain the fourth visual effect score of each of the N second denoised images output by the candidate visual effect scoring model; for any second denoised image, obtain the third training loss of the candidate visual effect scoring model based on the third visual effect score and the fourth visual effect score of the second denoised image; adjust the model parameters of the candidate visual effect scoring model based on the third training loss, and return to obtain the next second denoised image to continue model training for the candidate visual effect scoring model with adjusted parameters until the training is completed, so as to obtain a trained target visual effect scoring model.

[0169] In the disclosed embodiment, the construction module 71 is also used to: for any second denoised image, obtain a third image feature combination for generating the second denoised image from the N second image feature combinations; obtain a first image feature vector obtained by splicing the image features in the third image feature combination, and a second image feature vector of the second denoised image; obtain a third denoised image obtained by splicing the first image feature vector and the second image feature vector, and perform a visual effect score on the third denoised image based on the candidate visual effect scoring model to obtain a fourth visual effect score of the second denoised image corresponding to the third denoised image.

[0170] In the disclosed embodiment, the training module 74 is also used to: adjust the parameters of the decoder of the image denoising model included in the candidate image processing model according to the first training loss to obtain the candidate image processing model after the parameters are adjusted; return to obtain the next first sample noise image to continue model training on the candidate image processing model after the parameters are adjusted until the training is completed to obtain a trained target image processing model.

[0171] The training device of the image processing model proposed in the present disclosure constructs a candidate image processing model to be trained and obtains a first sample noise image, obtains M first denoised images of the first sample noise image based on a target image denoising model in the candidate image processing model, performs visual effect scoring on the M first denoised images based on a target visual effect scoring model, and obtains a first visual effect score for each of the M first denoised images, further obtains a first training loss of the candidate image processing model based on the first visual effect score of each of the M first denoised images and a reference second visual effect score, adjusts model parameters of the candidate image processing model based on the first training loss, returns to obtain the next first sample noise image, and continues to train the candidate image processing model with adjusted parameters until the training is completed, thereby obtaining a trained target image processing model. In the present disclosure, a first training loss is obtained based on a first visual effect score and a second visual effect score, and a candidate image processing model is trained based on the first training loss, so that the trained target image processing model can realize the processing of noisy images based on visual effects, improve the visual effect of the denoised image obtained after the denoising process of the noisy image, improve the degree of adaptation between the visual effect of the denoised image and people's perception, optimize people's visual experience of the denoised image obtained after the denoising process, and optimize the training method and training effect of the image processing model.

[0172] Corresponding to the image processing methods proposed in the above-mentioned embodiments, an embodiment of the present disclosure also proposes an image processing device. Since the image processing device proposed in the embodiment of the present disclosure corresponds to the image processing methods proposed in the above-mentioned embodiments, the implementation methods of the above-mentioned image processing methods are also applicable to the image processing device proposed in the embodiment of the present disclosure, and will not be described in detail in the following embodiments.

[0173] Figure 8 FIG. 1 is a schematic diagram of the structure of an image processing device according to an embodiment of the present disclosure. Figure 8 As shown, the image processing device 800 includes a third acquisition module 81, a noise reduction module 82, a scoring module 83 and a determination module 84, wherein:

[0174] The third acquisition module 81 is used to acquire a trained target image processing model based on visual effects, wherein the target image processing model includes a trained target image denoising model and a target visual effect scoring model, and the target image processing model, the target image denoising model and the target visual effect scoring model are based on Figure 7 The training device proposed in the embodiment is obtained;

[0175] The denoising module 82 is used to obtain a processed noisy image to be processed, extract a candidate image feature set of the processed noisy image based on a target image denoising model, and generate S candidate denoised images after denoising the processed noisy image based on the candidate image feature set, wherein the S candidate denoised images have different denoising degrees.

[0176] A scoring module 83 is used to obtain a candidate visual effect score of each of the S candidate denoised images based on a target visual effect scoring model;

[0177] The determination module 84 is used to obtain the target visual effect score with the highest score among the candidate visual effect scores, and determine the corresponding denoised image with the target visual effect score among the S candidate denoised images as the target denoised image obtained by visual effect processing of the noisy image to be processed.

[0178] The image processing device proposed in the present disclosure obtains a trained target image processing model, and extracts a candidate image feature set of the noise image to be processed based on the target image denoising model in the target image processing model, and then generates S candidate denoised images of the noise image to be processed. Furthermore, based on the target visual effect scoring model, the candidate visual effect scores of the S candidate denoised images are obtained, and then the target visual effect score with the highest score is obtained from the candidate visual effect scores, and the corresponding denoised image with the target visual effect score in the S candidate denoised images is determined as the target denoised image. In the present disclosure, denoising processing of noise images based on visual effects is implemented based on the target image processing model, which improves the degree of adaptation between the visual effect of the denoised image obtained after the noise image is processed and people's perception, and optimizes the denoising processing method and denoising processing effect of the noise image.

[0179] To achieve the above embodiments, the present disclosure also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0180] Fig. 9 1 shows a schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure. Fig. 9As shown, the device 900 includes a memory 91, a processor 92, and a computer program stored in the memory 91 and executable on the processor 92. When the processor 92 executes the program instructions, the training method of the image processing model provided in the above embodiment is implemented.

[0181] The present invention discloses a training method for an image processing model, which constructs a candidate image processing model to be trained and obtains a first sample noise image, obtains M first denoised images of the first sample noise image based on a target image denoising model in the candidate image processing model, performs visual effect scoring on the M first denoised images based on a target visual effect scoring model, and obtains a first visual effect score for each of the M first denoised images, further obtains a first training loss for the candidate image processing model based on the first visual effect score for each of the M first denoised images and a reference second visual effect score, adjusts model parameters of the candidate image processing model based on the first training loss, returns to obtain the next first sample noise image, and continues to train the candidate image processing model after parameter adjustment until the training is completed, thereby obtaining a trained target image processing model. In the present disclosure, a first training loss is obtained based on a first visual effect score and a second visual effect score, and a candidate image processing model is trained based on the first training loss, so that the trained target image processing model can realize the processing of noisy images based on visual effects, improve the visual effect of the denoised image obtained after the denoising process of the noisy image, improve the degree of adaptation between the visual effect of the denoised image and people's perception, optimize people's visual experience of the denoised image obtained after the denoising process, and optimize the training method and training effect of the image processing model.

[0182] Fig.10 1 shows a schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure. Fig.10 As shown, the device 1000 includes a memory 101, a processor 102, and a computer program stored in the memory 101 and executable on the processor 102. When the processor 102 executes the program instructions, the image processing method provided in the above embodiment is implemented.

[0183] The image processing method proposed in the present disclosure obtains a trained target image processing model, and extracts a candidate image feature set of the noise image to be processed based on the target image denoising model in the target image processing model, and then generates S candidate denoised images of the noise image to be processed. Furthermore, the candidate visual effect scores of the S candidate denoised images are obtained based on the target visual effect scoring model, and then the target visual effect score with the highest score is obtained from the candidate visual effect scores, and the corresponding denoised image with the target visual effect score in the S candidate denoised images is determined as the target denoised image. In the present disclosure, denoising processing of the noise image based on visual effects is realized based on the target image processing model, which improves the degree of adaptation between the visual effect of the denoised image obtained after the noise image is processed and people's perception, and optimizes the denoising processing method and denoising processing effect of the noise image.

[0184] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0186] The program code for implementing the method itself can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0187] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0188] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0189] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a grid browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication grid). Examples of communication grids include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain grid.

[0190] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication grid. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server for a distributed system, or a server combined with a blockchain.

[0191] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0192] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0193] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0194] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0195] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0196] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0197] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0198] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

[0199] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0200] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for an image processing model, characterized in that: The method comprises: Constructing a candidate image processing model based on visual effects to be trained, wherein the candidate image processing model includes a trained target image denoising model and a target visual effect scoring model; Acquire a first sample noise image, and obtain M first denoised images after denoising the first sample noise image based on the target image denoising model, wherein the M first denoised images have different denoising degrees; Obtaining a first visual effect score for each of the M first denoised images based on the target visual effect scoring model, and obtaining a first training loss for the candidate image processing model according to the first visual effect score and a reference second visual effect score; The model parameters of the candidate image processing model are adjusted based on the first training loss, and the next first sample noise image is returned to continue training the candidate image processing model with adjusted parameters until the training is completed, thereby obtaining a trained target image processing model.

2. The method according to claim 1, characterized in that The step of acquiring a first sample noise image and obtaining M first noise reduction images after noise reduction processing of the first sample noise image based on the target image noise reduction model includes: Extracting a first image feature set of the first sample noise image based on the target image denoising model; Acquire M first image feature combinations from the first image feature set, wherein the first image feature combination includes at least part of the image features in the first image feature set, and the M first image feature combinations each include a different image feature; The first reconstructed images of the M first image feature combinations are generated by the target image denoising model to obtain the M first denoised images of the first sample noisy image, wherein the denoising degrees of the M first denoised images are different.

3. The method according to claim 1, characterized in that The step of constructing a candidate image processing model to be trained based on visual effects includes: Obtaining a candidate image denoising model and a candidate visual effect scoring model to be trained; Acquire a second sample noise image, and train the candidate image denoising model based on the second sample noise image to obtain the trained target image denoising model; Acquire a third sample noise image, and obtain N second denoised images after denoising the third sample noise image based on the target image denoising model, wherein the N second denoised images have different denoising degrees; Training the candidate visual effect scoring model based on the N second denoised images to obtain the trained target visual effect scoring model; The target image denoising model and the target visual effect scoring model are combined to obtain the candidate image processing model based on visual effects to be trained.

4. The method according to claim 3, characterized in that The step of acquiring a second sample noise image and training the candidate image denoising model based on the second sample noise image to obtain the trained target image denoising model includes: Inputting the second sample noise image into the candidate image denoising model to obtain a third denoised image after denoising the second sample noise image output by the candidate image denoising model; Obtaining a label image of the second sample noisy image, and obtaining a second training loss of the candidate image denoising model according to the third denoised image and the label image; The model parameters of the candidate image denoising model are adjusted according to the second training loss, and the next second sample noise image is returned to continue training the candidate image denoising model with adjusted parameters until the training is completed, so as to obtain the trained target image denoising model.

5. The method according to claim 3, characterized in that: The step of acquiring the third sample noise image and obtaining N second noise reduction images after the third sample noise image is subjected to noise reduction processing based on the target image noise reduction model includes: Extracting a second image feature set of the third sample noise image based on the target image denoising model; Acquire N second image feature combinations from the second image feature set, wherein the second image feature combination includes at least part of the image features in the second image feature set, and the N second image feature combinations each include a different image feature; The second reconstructed images of the N second image feature combinations are generated by the target image denoising model to obtain the N second denoised images of the third sample noisy image, wherein the denoising degrees of the N second denoised images are different.

6. The method according to claim 3, characterized in that The step of training the candidate visual effect scoring model based on the N second denoised images to obtain the trained target visual effect scoring model comprises: Obtaining a third visual effect score for each of the N second denoised images; Obtaining a fourth visual effect score for each of the N second denoised images output by the candidate visual effect scoring model; For any second denoised image, obtaining a third training loss of the candidate visual effect scoring model based on a third visual effect score and a fourth visual effect score of the second denoised image; The model parameters of the candidate visual effect scoring model are adjusted based on the third training loss, and the next second denoised image is returned to continue model training of the candidate visual effect scoring model with adjusted parameters until the training is completed, so as to obtain the trained target visual effect scoring model.

7. The method according to claim 6, characterized in that The obtaining of the fourth visual effect scores of the N second denoised images output by the candidate visual effect scoring model comprises: For any second denoised image, obtaining a third image feature combination for generating the second denoised image from the N second image feature combinations; Acquire a first image feature vector obtained by concatenating the image features in the third image feature combination, and a second image feature vector of the second denoised image; Obtain a third denoised image obtained by concatenating the first image feature vector and the second image feature vector, and perform a visual effect score on the third denoised image based on the candidate visual effect scoring model to obtain the fourth visual effect score of the second denoised image corresponding to the third denoised image.

8. The method according to claim 1, characterized in that The method of adjusting the model parameters of the candidate image processing model based on the first training loss, and returning to obtain the next first sample noise image to continue training the candidate image processing model after the parameter adjustment until the training is completed, thereby obtaining a trained target image processing model, includes: According to the first training loss, adjusting parameters of a decoder of an image denoising model included in the candidate image processing model to obtain a candidate image processing model after parameter adjustment; Return to obtain the next first sample noise image and continue model training for the candidate image processing model after parameter adjustment until the training is completed, thereby obtaining the trained target image processing model.

9. An image processing method, characterized in that: The method comprises: Obtaining a trained target image processing model based on visual effects, wherein the target image processing model includes a trained target image denoising model and a target visual effect scoring model, and the target image processing model, the target image denoising model and the target visual effect scoring model are obtained based on the training method according to any one of claims 1 to 8; Acquire a processed noisy image to be processed, extract a candidate image feature set of the processed noisy image based on the target image denoising model, and generate S candidate denoised images after denoising the processed noisy image based on the candidate image feature set, wherein the S candidate denoised images have different denoising degrees; Based on the target visual effect scoring model, obtaining a candidate visual effect score for each of the S candidate denoised images; The target visual effect score with the highest score among the candidate visual effect scores is obtained, and the corresponding denoised image of the target visual effect score among the S candidate denoised images is determined as the target denoised image obtained by visual effect processing of the noisy image to be processed.

10. A training device for an image processing model, characterized in that: The device comprises: A construction module, used to construct a candidate image processing model to be trained based on visual effects, wherein the candidate image processing model includes a trained target image denoising model and a target visual effect scoring model; A first acquisition module is used to acquire a first sample noise image, and obtain M first denoised images after the first sample noise image is denoised based on the target image denoising model, wherein the M first denoised images have different denoising degrees; A second acquisition module is used to acquire a first visual effect score of each of the M first denoised images based on the target visual effect scoring model, and acquire a first training loss of the candidate image processing model according to the first visual effect score and a reference second visual effect score; A training module is used to adjust the model parameters of the candidate image processing model based on the first training loss, and return to obtain the next first sample noise image to continue training the candidate image processing model with adjusted parameters until the training is completed to obtain a trained target image processing model.

11. The device according to claim 10, characterized in that The first acquisition module is further used for: Extracting a first image feature set of the first sample noise image based on the target image denoising model; Acquire M first image feature combinations from the first image feature set, wherein the first image feature combination includes at least part of the image features in the first image feature set, and the M first image feature combinations each include a different image feature; The first reconstructed images of the M first image feature combinations are generated by the target image denoising model to obtain the M first denoised images of the first sample noisy image, wherein the denoising degrees of the M first denoised images are different.

12. The device according to claim 10, characterized in that The building blocks are also used to: Obtaining a candidate image denoising model and a candidate visual effect scoring model to be trained; Acquire a second sample noise image, and train the candidate image denoising model based on the second sample noise image to obtain the trained target image denoising model; Acquire a third sample noise image, and obtain N second denoised images after denoising the third sample noise image based on the target image denoising model, wherein the N second denoised images have different denoising degrees; Training the candidate visual effect scoring model based on the N second denoised images to obtain the trained target visual effect scoring model; The target image denoising model and the target visual effect scoring model are combined to obtain the candidate image processing model based on visual effects to be trained.

13. The device according to claim 12, characterized in that The building blocks are also used to: Inputting the second sample noise image into the candidate image denoising model to obtain a third denoised image after denoising the second sample noise image output by the candidate image denoising model; Obtaining a label image of the second sample noisy image, and obtaining a second training loss of the candidate image denoising model according to the third denoised image and the label image; The model parameters of the candidate image denoising model are adjusted according to the second training loss, and the next second sample noise image is returned to continue training the candidate image denoising model with adjusted parameters until the training is completed, so as to obtain the trained target image denoising model.

14. The device according to claim 12, characterized in that The building blocks are also used to: Extracting a second image feature set of the third sample noise image based on the target image denoising model; Acquire N second image feature combinations from the second image feature set, wherein the second image feature combination includes at least part of the image features in the second image feature set, and the N second image feature combinations each include a different image feature; The second reconstructed images of the N second image feature combinations are generated by the target image denoising model to obtain the N second denoised images of the third sample noisy image, wherein the denoising degrees of the N second denoised images are different.

15. The device according to claim 12, characterized in that The building blocks are also used to: Obtaining a third visual effect score for each of the N second denoised images; Obtaining a fourth visual effect score for each of the N second denoised images output by the candidate visual effect scoring model; For any second denoised image, obtaining a third training loss of the candidate visual effect scoring model based on a third visual effect score and a fourth visual effect score of the second denoised image; The model parameters of the candidate visual effect scoring model are adjusted based on the third training loss, and the next second denoised image is returned to continue model training of the candidate visual effect scoring model with adjusted parameters until the training is completed, so as to obtain the trained target visual effect scoring model.

16. The device according to claim 15, characterized in that The building blocks are also used to: For any second denoised image, obtaining a third image feature combination for generating the second denoised image from the N second image feature combinations; Acquire a first image feature vector obtained by concatenating the image features in the third image feature combination, and a second image feature vector of the second denoised image; Obtain a third denoised image obtained by concatenating the first image feature vector and the second image feature vector, and perform a visual effect score on the third denoised image based on the candidate visual effect scoring model to obtain the fourth visual effect score of the second denoised image corresponding to the third denoised image.

17. The device according to claim 10, characterized in that The training module is also used for: According to the first training loss, adjusting parameters of a decoder of an image denoising model included in the candidate image processing model to obtain a candidate image processing model after parameter adjustment; Return to obtain the next first sample noise image and continue model training for the candidate image processing model after parameter adjustment until the training is completed, thereby obtaining the trained target image processing model.

18. An image processing device, characterized in that: The device comprises: A third acquisition module is used to acquire a trained target image processing model based on visual effects, wherein the target image processing model includes a trained target image denoising model and a target visual effect scoring model, and the target image processing model, the target image denoising model and the target visual effect scoring model are obtained based on the training device according to any one of claims 1 to 8; a denoising module, configured to obtain a processed noisy image to be processed, extract a candidate image feature set of the processed noisy image based on the target image denoising model, and generate S candidate denoised images obtained after denoising the processed noisy image based on the candidate image feature set, wherein the S candidate denoised images have different denoising degrees; A scoring module, configured to obtain a candidate visual effect score for each of the S candidate denoised images based on the target visual effect scoring model; A determination module is used to obtain a target visual effect score with the highest score among the candidate visual effect scores, and determine a denoised image corresponding to the target visual effect score in the S candidate denoised images as a target denoised image obtained by visual effect processing of the noisy image to be processed.

19. An electronic device, characterized in that: include: processor; a memory for storing executable instructions for the processor; The processor is configured to execute instructions to implement the method according to any one of claims 1 to 8 and / or claim 9.

20. A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 8 and / or claim 9.

Images