Image enhancement method based on noise reduction processing

Abstract

The present disclosure relates to an image enhancement method based on noise reduction processing, and relates to the technical field of image processing; based on multiple to-be-processed images, corresponding image types are determined, based on the corresponding image types of the multiple to-be-processed images, first to-be-processed images and second to-be-processed images are divided, and different labels are configured for the first to-be-processed images and the second to-be-processed images, wherein the configured labels can represent the relationship between a noise reduction weight and an enhancement weight; then, based on the first to-be-processed images and the second to-be-processed images configured with labels, noise reduction and enhancement processing is respectively performed by using a pre-trained image processing model, and processed images are obtained. The technical solution effectively combines image noise reduction and image enhancement, realizes image enhancement based on noise reduction processing, and improves the noise reduction and enhancement effect of images.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image enhancement method based on noise reduction processing. Background Technology

[0002] With the development of image processing technology, various image enhancement techniques have also evolved. Image enhancement techniques are often accompanied by other image processing techniques, such as image noise reduction. Combining these techniques with others can better ensure the image enhancement effect.

[0003] In related technologies, image denoising and image enhancement are performed separately, without effectively combining them. Summary of the Invention

[0004] The purpose of this disclosure is to provide an image enhancement method based on noise reduction processing, which achieves image enhancement based on noise reduction processing and improves the noise reduction and enhancement effect of the image.

[0005] To achieve the above objectives, this disclosure provides an image enhancement method based on denoising processing, comprising: determining image types corresponding to multiple images to be processed, wherein the image types include noisy image types, non-noisy image types, and normal image types; determining a first image to be processed and a second image to be processed from the multiple images to be processed according to the image types corresponding to the multiple images to be processed; configuring a first identifier for the first image to be processed and configuring a second identifier for the second image to be processed, wherein the first identifier is used to indicate that the denoising weight is higher than the enhancement weight, and the second identifier is used to indicate that the denoising weight is lower than the enhancement weight; performing denoising and enhancement processing on the first image to be processed configured with the first identifier using a pre-trained image processing model to obtain a first processed image, and performing denoising and enhancement processing on the second image to be processed configured with the second identifier using a pre-trained image processing model to obtain a second processed image.

[0006] Optionally, determining the image type corresponding to each of the multiple images to be processed includes: determining the noise probability corresponding to each of the multiple images to be processed using a pre-trained image classification model, wherein the noise probability is used to characterize the probability that the image to be processed contains noise; determining the image type corresponding to the image to be processed as a noise image type for the image to be processed that has a noise probability higher than a first preset noise probability; determining the image type corresponding to the image to be processed as a normal image type for the image to be processed that has a noise probability lower than a second preset noise probability; and determining the image type corresponding to the image to be processed based on the image to be processed that belongs to the noise image type and the image to be processed that belongs to the normal image type for the image to be processed that has a noise probability between the first preset noise probability and the second preset noise probability.

[0007] Optionally, determining the image type corresponding to the image to be processed based on the image to be processed belonging to the noisy image type and the image to be processed belonging to the normal image type includes: determining a first target image feature corresponding to the image to be processed belonging to the noisy image type, determining a second target image feature corresponding to the image to be processed belonging to the normal image type, and determining a third target image feature corresponding to the image to be processed, wherein the first target image feature, the second target image feature, and the third target image feature belong to the same image feature type and are different from the type of noise feature learned by the pre-trained image classification model; determining the image type corresponding to the image to be processed based on the first target image feature, the second target image feature, and the third target image feature.

[0008] Optionally, determining the first image to be processed and the second image to be processed from the plurality of images to be processed according to their respective image types includes: determining the images to be processed belonging to the noisy image type as the first image to be processed; determining the images to be processed belonging to the normal image type as either the first image to be processed or the second image to be processed; determining a preset number of images from the images to be processed belonging to the non-noisy image type as the first image to be processed; and determining the remaining images from the images to be processed belonging to the non-noisy image type as the second image to be processed, wherein the preset number is determined according to the image enhancement requirements corresponding to the plurality of images to be processed.

[0009] Optionally, the pre-trained image processing model includes a preprocessing module, a denoising module, and an enhancement module. The step of performing denoising and enhancement processing on a first image to be processed, configured with the first identifier, using the pre-trained image processing model to obtain a first processed image includes: preprocessing the first image to be processed using the preprocessing module and outputting the obtained first preprocessed image to the denoising module according to the first identifier; performing denoising processing on the first preprocessed image using the denoising module according to the first identifier and outputting the obtained denoised image to the enhancement module; and enhancing the denoised image using the enhancement module according to the first identifier to obtain the first processed image.

[0010] Optionally, the step of performing noise reduction and enhancement processing on the second image to be processed, configured with the second identifier, using a pre-trained image processing model to obtain a second processed image includes: preprocessing the second image to be processed by the preprocessing module and outputting the obtained second preprocessed image to the enhancement module according to the second identifier; enhancing the second preprocessed image by the enhancement module according to the second identifier and outputting the obtained enhanced image to the noise reduction module; and performing noise reduction processing on the enhanced image by the noise reduction module according to the second identifier to obtain the second processed image.

[0011] Optionally, the image enhancement method further includes: acquiring a training dataset, the training dataset including multiple first training samples and multiple second training samples, each first training sample including: a first sample image, a first label image, a second label image, and a first sample identifier, the first label image being a preprocessed image of the first sample image, the second label image being a denoised and enhanced image of the first label image, and the first sample identifier being used to indicate that the denoising weight is higher than the enhancement weight; each second training sample including: a second sample image, a third label image, a fourth label image, and a second sample identifier, the third label image being a preprocessed image of the second sample image, the fourth label image being a denoised and enhanced image of the third label image, and the second sample identifier being used to indicate that the denoising weight is lower than the enhancement weight; and training the image processing model to be trained according to the training dataset to obtain the pretrained image processing model.

[0012] Optionally, training the image processing model to be trained based on the training dataset to obtain the pre-trained image processing model includes: sampling the plurality of first training samples to obtain sampled first sample images, first label images, and first sample identifiers; sampling the plurality of second training samples to obtain sampled second sample images, third label images, and second sample identifiers; training the pre-processing module to be trained based on the sampled first sample images, first label images, first sample identifiers, and sampled second sample images, third label images, and second sample identifiers to obtain the pre-trained pre-processing module; and training the pre-processing module based on the sampled first sample images, first label images, first sample identifiers, and sampled second sample images, third label images, and second sample identifiers. The first label image, second label image, and first sample identifier from the plurality of first training samples are used to train the denoising module and the enhancement module to be trained, resulting in the initially trained denoising module and the initially trained enhancement module. Based on the third label image, fourth label image, and second sample identifier from the plurality of second training samples, the initially trained denoising module and the initially trained enhancement module are trained again, resulting in the pre-trained denoising module and the pre-trained enhancement module. Based on the pre-trained preprocessing module, the pre-trained denoising module, and the pre-trained enhancement module, the pre-trained image processing model is obtained.

[0013] Optionally, the image enhancement method further includes: evaluating the image enhancement effects of the first processed image and the second processed image to obtain an image enhancement evaluation result; determining a first target processed image from the first processed image and the second processed image based on the image enhancement evaluation result; obtaining a target image to be processed corresponding to the first target processed image; adjusting the identifier configured on the target image to be processed to obtain a target image to be processed with adjusted identifier; and performing noise reduction and enhancement processing on the target image to be processed with adjusted identifier using the pre-trained image processing model to obtain an adjusted image corresponding to the target processed image.

[0014] Optionally, the step of evaluating the image enhancement effect of the first processed image and the second processed image to obtain an image enhancement evaluation result includes: determining a second target processed image from the first processed image and the second processed image whose image type is a non-noise image type; and evaluating at least the image enhancement effect of the second target processed image to obtain an image enhancement evaluation result.

[0015] The above technical solution involves determining the corresponding image types based on multiple images to be processed. Based on these image types, a first image and a second image to be processed are defined, and different identifiers are assigned to them. These identifiers characterize the relationship between denoising weights and enhancement weights. Then, a pre-trained image processing model is used to perform denoising and enhancement processing on the first and second images based on their respective identifiers, resulting in processed images. The identifier configuration based on image types ensures that the identifiers match the image features. Furthermore, the identifiers characterizing the relationship between denoising and enhancement weights allow the image processing model to perform denoising and enhancement processing according to the weight relationships, effectively combining denoising and enhancement. Therefore, this technical solution effectively combines image denoising and image enhancement, achieving image enhancement based on denoising processing and improving both denoising and enhancement effects.

[0016] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a schematic diagram illustrating an application scenario according to an exemplary embodiment.

[0019] Figure 2 This is a flowchart illustrating an image enhancement method based on noise reduction processing according to an exemplary embodiment.

[0020] Figure 3 This is a schematic diagram illustrating an image processing model according to an exemplary embodiment.

[0021] Figure 4 This is a schematic diagram of the internal processing logic of an image processing model according to an exemplary embodiment.

[0022] Figure 5 This is a schematic diagram illustrating the training process of an image processing model according to an exemplary embodiment.

[0023] Figure 6 This is a block diagram illustrating an image enhancement apparatus based on noise reduction processing according to an exemplary embodiment.

[0024] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0025] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0026] With the development of image processing technology, various image enhancement techniques have also evolved. Image enhancement techniques are often accompanied by other image processing techniques, such as image noise reduction. Combining these techniques with others can better ensure the image enhancement effect.

[0027] In related technologies, image denoising and image enhancement are performed separately, without effectively combining them.

[0028] Given the diversity of images, different image features may require different noise reduction and enhancement methods. Therefore, by configuring appropriate noise reduction and enhancement methods based on the actual situation of the image, the effect of image noise reduction and enhancement can be guaranteed.

[0029] Therefore, in this embodiment of the disclosure, based on multiple images to be processed, the corresponding image types are determined. Based on the image types corresponding to the multiple images to be processed, a first image to be processed and a second image to be processed are divided. Different identifiers are configured for the first image to be processed and the second image to be processed. The configured identifiers can characterize the relationship between denoising weights and enhancement weights. Then, using a pre-trained image processing model, denoising and enhancement processing are performed on the first image to be processed and the second image to be processed based on the configured identifiers to obtain the processed image.

[0030] By configuring identifiers based on image type, the configured identifiers can be matched with image features. Furthermore, by configuring identifiers that represent the relationship between denoising weights and enhancement weights, the image processing model can perform denoising and enhancement processing according to the weight relationship, effectively combining denoising and enhancement.

[0031] Thus, this technical solution effectively combines image denoising and image enhancement to achieve image enhancement based on denoising processing, thereby improving the denoising and enhancement effects of the image.

[0032] The technical solutions of this disclosure can be applied to various scenarios involving image processing, in which multiple images need to be noise-reducing and enhanced.

[0033] Figure 1 This is a schematic diagram illustrating an application scenario according to an exemplary embodiment, such as... Figure 1 As shown, this application scenario involves image acquisition equipment and image processing equipment.

[0034] Regarding image acquisition devices, these can be cameras or camera modules. Regarding image processing devices, these can be electronic devices including image processors, which have the capability for batch image processing.

[0035] Image acquisition equipment can acquire images periodically, and correspondingly, image processing equipment can process images periodically. Therefore, image processing equipment can process multiple images at once.

[0036] This application scenario can include: 3D image rendering, monitoring, and image projection. Correspondingly, the images to be processed can be 3D images, monitoring images, and images to be projected.

[0037] Understandable. Figure 1 The application scenarios shown are only examples; they can be applied to many more scenarios depending on different needs.

[0038] Figure 2 This is a flowchart illustrating an image enhancement method based on noise reduction processing according to an exemplary embodiment. This method can be applied to... Figure 1 The image processing apparatus shown includes the following steps:

[0039] Step S21: Determine the image type corresponding to each of the multiple images to be processed. The image types include noisy image type, non-noisy image type, and normal image type.

[0040] Step S22: Based on the image types corresponding to the multiple images to be processed, determine the first image to be processed and the second image to be processed from the multiple images to be processed.

[0041] Step S23: Configure a first identifier for the first image to be processed and configure a second identifier for the second image to be processed. The first identifier is used to indicate that the noise reduction weight is higher than the enhancement weight, and the second identifier is used to indicate that the noise reduction weight is lower than the enhancement weight.

[0042] Step S24: The first image to be processed with the first identifier configured is subjected to noise reduction and enhancement processing by a pre-trained image processing model to obtain the first processed image; and the second image to be processed with the second identifier configured is subjected to noise reduction and enhancement processing by a pre-trained image processing model to obtain the second processed image.

[0043] In step S21, with Figure 1 Taking the application scenario shown as an example, multiple images to be processed can be images acquired by the image acquisition device within the current image acquisition cycle.

[0044] Based on multiple images to be processed, they can first be classified into image types. Among them, the noisy image type can be characterized by the fact that the image to be processed contains more noise features than other image features that affect image quality. The non-noisy image type can be characterized by the fact that the image to be processed contains more non-noise features than noise features that affect image quality. The normal image type can be characterized by the fact that the image to be processed contains fewer image features that affect image quality, but not necessarily none, and may also affect the image enhancement effect.

[0045] The causes of noise characteristics can be found in mature technologies in this field, and will not be described in detail here.

[0046] As an optional implementation, step S21 includes: determining the noise probability corresponding to each of the multiple images to be processed using a pre-trained image classification model, wherein the noise probability is used to characterize the probability that the image to be processed contains noise; determining the image type corresponding to the image to be processed as a noise image type for the multiple images to be processed where the noise probability is higher than a first preset noise probability; determining the image type corresponding to the image to be processed as a normal image type for the multiple images to be processed where the noise probability is lower than a second preset noise probability; and determining the image type corresponding to the image to be processed based on the images to be processed that belong to the noise image type and the images to be processed that belong to the normal image type for the multiple images to be processed where the noise probability is between the first preset noise probability and the second preset noise probability.

[0047] In some embodiments, the image classification model can be an unsupervised learning model, which learns noise features during the initial training phase. After training, the model has the ability to recognize noise features and can then output noise probabilities based on the input image.

[0048] Alternatively, the image classification model can also be a supervised learning model. During the initial training, specific image types can be labeled. After training is complete, the model can output noise probability based on the image type classification ability and the input image.

[0049] Alternatively, other implementation methods may be used, which are not limited here.

[0050] The technology for identifying noise features is a mature technology in this field and will not be described in detail here.

[0051] Furthermore, image types can be specifically categorized based on noise probability.

[0052] In some embodiments, the first preset noise probability can range from 85% to 90%, and the second preset noise probability can range from 10% to 15%. Therefore, if the noise probability is higher than the first preset noise probability, it can be directly identified as a noisy image. If the noise probability is lower than the second preset noise probability, it can be identified as a normal image.

[0053] If the noise probability is between the first preset noise probability and the second preset noise probability, then the average similarity between the image to be processed and the image to be processed belonging to the noise image type, as well as the average similarity between the image to be processed and the image to be processed belonging to the normal image type, can be determined. If the average similarity with the image to be processed belonging to the noise image type is higher, then it is determined to be a noise image type. Otherwise, it is determined to be a normal image type.

[0054] The determination of average image similarity can be done by sampling a portion of the images or by determining the average similarity based on all images; no limitation is imposed here.

[0055] In some embodiments, the image type can also be determined by extracting image features.

[0056] Therefore, as an optional implementation, determining the image type corresponding to the image to be processed based on the image to be processed belonging to the noisy image type and the image to be processed belonging to the normal image type includes: determining a first target image feature corresponding to the image to be processed belonging to the noisy image type, determining a second target image feature corresponding to the image to be processed belonging to the normal image type, and determining a third target image feature corresponding to the image to be processed, wherein the first target image feature, the second target image feature, and the third target image feature belong to the same image feature type and are different from the type of noise feature learned by the pre-trained image classification model; and determining the image type corresponding to the image to be processed based on the first target image feature, the second target image feature, and the third target image feature.

[0057] It is understandable that the noise characteristics of an image can involve many aspects; here are some examples:

[0058] Statistical parameters of an image, such as mean, variance, and standard deviation. Noisy images typically have higher variance and standard deviation.

[0059] Frequency domain transformation: Fourier transform is used to convert an image from the spatial domain to the frequency domain. In the frequency domain, noise typically manifests as high-frequency components of the image. If these high-frequency components are unusually active, it may indicate the presence of noise in the image.

[0060] Apply a low-pass filter (such as a Gaussian filter) to the image, and then compare the images before and after filtering. If the filtered image differs significantly from the original image, it may indicate the presence of noise in the original image.

[0061] Noise model fitting: Fit the image to a known noise model (such as Gaussian noise, salt and pepper noise, etc.) to see if the model parameters are reasonable.

[0062] Image quality assessment metrics: Image quality assessment metrics, such as signal-to-noise ratio, peak signal-to-noise ratio, and structural similarity index, can quantify the noise level in an image.

[0063] Automatic noise detection algorithms: These algorithms use machine learning or deep learning methods to train models to automatically identify noise in images. These models can be trained on large datasets of both noisy and noise-free images.

[0064] Image local variance analysis: Analyzes the variance of local regions of an image. Noise usually increases the variance in local regions.

[0065] Image entropy analysis: Calculates the entropy of an image. Noise may increase the entropy value of an image because it introduces additional information.

[0066] Therefore, when training an image classification model, it can be trained based on one of the aforementioned dimensions. When extracting target image features, it can be extracted based on other dimensions, ensuring that the first, second, and third target image features belong to the same image feature type and are different from the noise features learned by the pre-trained image classification model.

[0067] Furthermore, the image type corresponding to the image to be processed can be determined based on the similarity between the features of the first target image, the features of the second target image, and the features of the third target image.

[0068] For example, images with high similarity are of the same type, while images with low similarity can be classified as non-noise images.

[0069] In step S22, the first image to be processed and the second image to be processed are determined from the multiple images to be processed according to the image types corresponding to them.

[0070] As an optional implementation, step S22 includes: determining the image to be processed that belongs to the noisy image type as the first image to be processed; determining the image to be processed that belongs to the normal image type as either the first image to be processed or the second image to be processed; determining a preset number of images from the images to be processed that belong to the non-noisy image type as the first image to be processed; and determining the remaining images from the images to be processed that belong to the non-noisy image type as the second image to be processed, wherein the preset number is determined according to the image enhancement requirements corresponding to the multiple images to be processed.

[0071] In this implementation, an image to be processed that belongs to the noisy image type can be identified as one type of image to be processed. An image to be processed that belongs to the normal image type can be randomly identified as either the first image to be processed or the second image to be processed. An image to be processed that belongs to the non-noisy image type can be partially identified as the first image to be processed and partially identified as the second image to be processed.

[0072] In some embodiments, image enhancement requirements can characterize the desired image enhancement effect. Accordingly, the higher the image enhancement effect represented by the image enhancement requirements for multiple images to be processed, the greater the preset number can be. For example, a higher image enhancement effect requirement corresponds to a higher preset number relative to the proportion of images to be processed that belong to the non-noise image type. It is understood that the preset number is not necessarily a specified value, but can be a proportion relative to the number of images to be processed that belong to the non-noise image type.

[0073] Furthermore, in step S23, for the first image to be processed, a first identifier can be configured to indicate that the denoising weight is higher than the enhancement weight. For the second image to be processed, a second identifier can be configured to indicate that the denoising weight is lower than the enhancement weight. These first and second identifiers can assist the image processing model in performing denoising and enhancement processing.

[0074] In some embodiments, the first identifier and the second identifier can determine not only the order of noise reduction and enhancement processes, but also the effectiveness of the noise reduction and enhancement processes.

[0075] Further, in step S24, the first image to be processed with the first identifier configured is subjected to noise reduction and enhancement processing by a pre-trained image processing model to obtain a first processed image, and the second image to be processed with the second identifier configured is subjected to noise reduction and enhancement processing by a pre-trained image processing model to obtain a second processed image.

[0076] Figure 3 This is a schematic diagram illustrating an image processing model according to an exemplary embodiment, such as... Figure 3 As shown, the image processing model includes a preprocessing module, a noise reduction module, and an enhancement module.

[0077] The preprocessing module can be a module used to implement image preprocessing. The algorithm it uses can be a variety of image preprocessing algorithms, such as image size adjustment, image region of interest identification, etc. For details, please refer to mature image preprocessing technologies in this field.

[0078] Regarding noise reduction processing, it can be a module used to implement image noise reduction, and the algorithm used can be various noise reduction processing algorithms, such as filtering algorithms, geometric correction algorithms, image format conversion algorithms, etc. For details, please refer to mature image noise reduction processing technologies in this field.

[0079] Regarding enhancement processing, it can be a module used to achieve image enhancement, and the algorithm used can be various enhancement algorithms, such as contrast enhancement algorithms, edge correction algorithms, and color correction algorithms. For specific details, please refer to mature image enhancement processing technologies in this field.

[0080] Figure 4 This is a schematic diagram of the internal processing logic of an image processing model according to an exemplary embodiment, such as... Figure 4 As shown, the preprocessing module can preprocess the input image and output the preprocessed image to either the noise reduction module or the enhancement module based on the image's configured identifier. If the image is output to the noise reduction module, it means noise reduction will be performed first, followed by enhancement. If the image is output to the enhancement module, it means enhancement will be performed first, followed by noise reduction.

[0081] Therefore, within this image processing model, there may be two processing logics: one is a preprocessing module-denoising module-enhancement module, and the other is a preprocessing module-enhancement module-denoising module.

[0082] In some embodiments, the configured identifier can determine which logic to employ. Specifically, if a first identifier is configured, it indicates that noise reduction is required before enhancement, thus employing the preprocessing module-noise reduction module-enhancement module processing logic. If a second identifier is configured, it indicates that enhancement is required before noise reduction, thus employing the preprocessing module-enhancement module-noise reduction module processing logic.

[0083] Then, the first image to be processed, configured with the first identifier, is subjected to noise reduction and enhancement processing by a pre-trained image processing model to obtain a first processed image, including: preprocessing the first image to be processed by a preprocessing module and outputting the obtained first preprocessed image to a noise reduction module according to the first identifier; performing noise reduction processing on the first preprocessed image by the noise reduction module according to the first identifier and outputting the obtained noise-reduced image to an enhancement module; and enhancing the noise-reduced image by the enhancement module according to the first identifier to obtain the first processed image.

[0084] In this implementation, the first image to be processed is processed via a preprocessing module, a noise reduction module, and an enhancement module because a first identifier is configured.

[0085] The second image to be processed, configured with a second identifier, is subjected to noise reduction and enhancement processing by a pre-trained image processing model to obtain a second processed image. This process may include: preprocessing the second image to be processed by a preprocessing module and outputting the obtained second preprocessed image to an enhancement module according to the second identifier; enhancing the second preprocessed image according to the second identifier by the enhancement module and outputting the obtained enhanced image to a noise reduction module; and denoising the enhanced image according to the second identifier by the noise reduction module to obtain a second processed image.

[0086] As can be seen, both the first and second images to be processed undergo denoising and enhancement, but the processing order differs. Furthermore, both denoising and enhancement processes are based on identifiers representing the relationship between denoising and enhancement weights.

[0087] As an optional implementation, the training process of the image processing model includes: acquiring a training dataset, which includes multiple first training samples and multiple second training samples. Each first training sample includes: a first sample image, a first label image, a second label image, and a first sample identifier. The first label image is a preprocessed image of the first sample image, and the second label image is a denoised and enhanced image of the first label image. The first sample identifier is used to indicate that the denoising weight is higher than the enhancement weight. Each second training sample includes: a second sample image, a third label image, a fourth label image, and a second sample identifier. The third label image is a preprocessed image of the second sample image, and the fourth label image is a denoised and enhanced image of the third label image. The second sample identifier is used to indicate that the denoising weight is lower than the enhancement weight. The image processing model to be trained is trained according to the training dataset to obtain a pretrained image processing model.

[0088] In this implementation, the training dataset includes two types of training samples: the first type is a training sample based on a first sample identifier, and the second type is a training sample based on a second sample identifier.

[0089] Furthermore, both training samples are configured with two types of label images. One type of label image can be used to train the preprocessing module, while the other type of label image can be used to train the denoising and enhancement modules.

[0090] In some embodiments, the configuration of the sample image, label image, and identifier can all be implemented manually.

[0091] Therefore, as an optional implementation, training the image processing model to be trained based on the training dataset to obtain the pre-trained image processing model includes: sampling multiple first training samples to obtain sampled first sample images, first label images, and first sample identifiers; sampling multiple second training samples to obtain sampled second sample images, third label images, and second sample identifiers; training the preprocessing module to be trained based on the sampled first sample images, first label images, first sample identifiers, and sampled second sample images, third label images, and second sample identifiers to obtain a pre-trained preprocessing module; training the denoising module to be trained and the enhancement module to be trained based on the first label images, second label images, and first sample identifiers from multiple first training samples to obtain an initially trained denoising module and an initially trained enhancement module; retraining the initially trained denoising module and the initially trained enhancement module based on the third label images, fourth label images, and second sample identifiers from multiple second training samples to obtain a pre-trained denoising module and a pre-trained enhancement module; and obtaining the pre-trained image processing model based on the pre-trained preprocessing module, the pre-trained denoising module, and the pre-trained enhancement module.

[0092] In some embodiments, since the preprocessing module needs to adapt to the processing of multiple images, two types of training samples can be sampled to obtain a first sample image, a first label image, a first sample identifier, and a sampled second sample image, a third label image, and a second sample identifier. Thus, the preprocessing module can be trained using the sampled first sample image, the first label image, and the first sample identifier, as well as using the sampled second sample image, the third label image, and the second sample identifier.

[0093] In some embodiments, when training the noise reduction module and the enhancement module to be trained to obtain the initially trained noise reduction module and the initially trained enhancement module, the training can be performed in two separate sessions. The noise reduction module is input first, and the enhancement module is input second, so as to achieve separate training effects.

[0094] Similarly, when retraining the initially trained noise reduction module and the initially trained enhancement module to obtain the pre-trained noise reduction module and the pre-trained enhancement module, the same training method can be used.

[0095] Furthermore, by integrating the pre-trained preprocessing module, pre-trained denoising module, and pre-trained enhancement module, a pre-trained image processing model can be obtained. For example, by connecting the trained modules according to their input-output relationships, the final image processing model can be obtained.

[0096] For easier understanding, please refer to Figure 5This is a schematic diagram illustrating the training process of an image processing model according to an exemplary embodiment, such as... Figure 5 As shown, the training process includes the following steps:

[0097] First, obtain the first training sample corresponding to the first sample identifier and the second training sample corresponding to the second sample identifier.

[0098] Then, the first training sample and the second training sample are sampled. Based on the sampling results, the preprocessing module is trained.

[0099] Next, based on the first training samples, the noise reduction module and the enhancement module are trained separately. And, based on the second training samples, the noise reduction module and the enhancement module are trained separately.

[0100] Finally, based on the three trained modules, the final trained image processing model is obtained.

[0101] This implementation method enables the image processing model to adopt image-adaptive noise reduction and enhancement strategies for various images, thereby improving the noise reduction and enhancement effect.

[0102] Furthermore, the resulting first and second processed images can be used as corresponding images, for example, for 3D rendering or anomaly detection.

[0103] In some embodiments, the method may further include: evaluating the image enhancement effect of the first processed image and the second processed image to obtain an image enhancement evaluation result; determining a first target processed image from the first processed image and the second processed image based on the image enhancement evaluation result; obtaining a target image to be processed corresponding to the first target processed image; adjusting the identifier configured on the target image to be processed to obtain a target image to be processed with adjusted identifier; and performing noise reduction and enhancement processing on the target image to be processed with adjusted identifier using a pre-trained image processing model to obtain an adjusted image corresponding to the target processed image.

[0104] In some embodiments, image enhancement effect evaluation may include contrast evaluation, color evaluation, and sharpening effect evaluation, etc. For details, please refer to the mature technologies in the field, which will not be described in detail here.

[0105] In some embodiments, image enhancement effects can be evaluated for each processed image, and then the processed image with poor image enhancement effects can be determined as the first target processed image.

[0106] In some embodiments, evaluating the image enhancement effect of the first processed image and the second processed image to obtain an image enhancement evaluation result may include: determining a second target processed image from the first processed image and the second processed image whose image type is a non-noise image type; and evaluating at least the image enhancement effect of the second target processed image to obtain an image enhancement evaluation result.

[0107] In this implementation, the image enhancement effect can be evaluated first on the designated second target image. If the image enhancement effect is good, there is no need to evaluate other processed images; if the image enhancement effect is poor, then it is necessary to further evaluate other processed images.

[0108] The judgment of the quality of image enhancement effect can be configured in different application scenarios, and is not limited here.

[0109] Furthermore, if this implementation method is adopted, when determining the first target image for processing, image enhancement optimization can be omitted if there is no need to evaluate other processed images. If further evaluation of other processed images is required, the image enhancement effects are compared, and the image with the poorer image enhancement effect is determined as the first target image for processing.

[0110] Furthermore, the target image to be processed corresponding to the first target image can be obtained, and the identifier configured on the target image to be processed can be adjusted. After adjustment, an image processing model is used for noise reduction and enhancement to optimize image enhancement. During adjustment, the identifier can be directly changed. For example, if the original identifier was the first identifier, it can be adjusted to the second identifier.

[0111] This implementation method allows for the optimization of images with poor enhancement effects, ensuring that the image enhancement effect is fully guaranteed.

[0112] Figure 6 This is a block diagram illustrating an image enhancement apparatus based on noise reduction processing according to an exemplary embodiment, such as... Figure 6 As shown, the device includes:

[0113] The determining module 601 is used to determine the image type corresponding to each of the multiple images to be processed, wherein the image type includes a noisy image type, a non-noisy image type, and a normal image type. Based on the image type corresponding to each of the multiple images to be processed, a first image to be processed and a second image to be processed are determined from the multiple images to be processed.

[0114] The configuration module 602 is used to configure a first identifier for the first image to be processed and a second identifier for the second image to be processed. The first identifier is used to indicate that the noise reduction weight is higher than the enhancement weight, and the second identifier is used to indicate that the noise reduction weight is lower than the enhancement weight.

[0115] The processing module 603 is used to perform noise reduction and enhancement processing on a first image to be processed with the first identifier configured using a pre-trained image processing model to obtain a first processed image, and to perform noise reduction and enhancement processing on a second image to be processed with the second identifier configured using a pre-trained image processing model to obtain a second processed image.

[0116] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0117] Figure 7 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0118] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the image enhancement method based on noise reduction processing described above. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 705 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0119] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the XXXX method described above.

[0120] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the image enhancement method based on noise reduction processing described above. For example, the computer-readable storage medium may be the memory 702 including the program instructions described above, which may be executed by the processor 701 of the electronic device 700 to complete the image enhancement method based on noise reduction processing described above.

[0121] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the image enhancement method based on noise reduction processing described above.

[0122] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0123] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0124] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. An image enhancement method based on noise reduction processing, characterized in that, include: The image types corresponding to the multiple images to be processed are determined, including noisy image types, non-noisy image types, and normal image types; Based on the image types corresponding to the multiple images to be processed, the first image to be processed and the second image to be processed are determined from the multiple images to be processed. A first identifier is configured for the first image to be processed, and a second identifier is configured for the second image to be processed. The first identifier is used to indicate that the noise reduction weight is higher than the enhancement weight, and the second identifier is used to indicate that the noise reduction weight is lower than the enhancement weight. A first processed image is obtained by performing noise reduction and enhancement processing on a first image to be processed with the first identifier configured using a pre-trained image processing model; and a second processed image is obtained by performing noise reduction and enhancement processing on a second image to be processed with the second identifier configured using a pre-trained image processing model. The pre-trained image processing model includes a preprocessing module, a noise reduction module, and an enhancement module; The step of performing noise reduction and enhancement processing on a first image to be processed, configured with the first identifier, using a pre-trained image processing model to obtain a first processed image includes: preprocessing the first image to be processed by the preprocessing module and outputting the obtained first preprocessed image to the noise reduction module according to the first identifier; performing noise reduction processing on the first preprocessed image by the noise reduction module according to the first identifier and outputting the obtained denoised image to the enhancement module; and enhancing the denoised image by the enhancement module according to the first identifier to obtain the first processed image. The step of performing noise reduction and enhancement processing on a second image to be processed, configured with the second identifier, using a pre-trained image processing model to obtain a second processed image includes: preprocessing the second image to be processed by the preprocessing module and outputting the obtained second preprocessed image to the enhancement module according to the second identifier; enhancing the second preprocessed image by the enhancement module according to the second identifier and outputting the obtained enhanced image to the noise reduction module; and performing noise reduction processing on the enhanced image by the noise reduction module according to the second identifier to obtain the second processed image.

2. The image enhancement method according to claim 1, characterized in that, Determining the image types corresponding to the multiple images to be processed includes: By using a pre-trained image classification model, the noise probability corresponding to each of the multiple images to be processed is determined. The noise probability is used to characterize the probability that the image to be processed contains noise. For the image to be processed among the multiple images to be processed where the noise probability is higher than the first preset noise probability, the image type corresponding to the image to be processed is determined to be a noise image type; For the image to be processed among the multiple images to be processed whose noise probability is lower than the second preset noise probability, the image type corresponding to the image to be processed is determined to be a normal image type. For the image to be processed in which the noise probability is between the first preset noise probability and the second preset noise probability, the image type corresponding to the image to be processed is determined according to the image to be processed that belongs to the noisy image type and the image to be processed that belongs to the normal image type.

3. The image enhancement method according to claim 2, characterized in that, The step of determining the image type corresponding to the image to be processed based on the image to be processed belonging to the noisy image type and the image to be processed belonging to the normal image type includes: A first target image feature corresponding to the image to be processed that belongs to the noisy image type is determined, a second target image feature corresponding to the image to be processed that belongs to the normal image type is determined, and a third target image feature corresponding to the image to be processed is determined. The first target image feature, the second target image feature, and the third target image feature belong to the same image feature type and are different from the type of noise feature learned by the pre-trained image classification model. Based on the first target image features, the second target image features, and the third target image features, the image type corresponding to the image to be processed is determined.

4. The image enhancement method according to claim 1, characterized in that, The step of determining the first image to be processed and the second image to be processed from the plurality of images to be processed according to their respective image types includes: The image to be processed that belongs to the noisy image type is identified as the first image to be processed; The image to be processed that belongs to the normal image type is identified as either the first image to be processed or the second image to be processed. A predetermined number of images belonging to the non-noise image type are identified as the first images to be processed, and the remaining images belonging to the non-noise image type are identified as the second images to be processed. The predetermined number is determined based on the image enhancement requirements corresponding to the multiple images to be processed.

5. The image enhancement method according to claim 1, characterized in that, The image enhancement method further includes: A training dataset is obtained, comprising multiple first training samples and multiple second training samples. Each first training sample includes: a first sample image, a first label image, a second label image, and a first sample identifier. The first label image is a preprocessed image of the first sample image, and the second label image is a denoised and enhanced image of the first label image. The first sample identifier is used to indicate that the denoising weight is higher than the enhancement weight. Each second training sample includes: a second sample image, a third label image, a fourth label image, and a second sample identifier. The third label image is a preprocessed image of the second sample image, and the fourth label image is a denoised and enhanced image of the third label image. The second sample identifier is used to indicate that the denoising weight is lower than the enhancement weight. The image processing model to be trained is trained based on the training dataset to obtain the pre-trained image processing model.

6. The image enhancement method according to claim 5, characterized in that, The step of training the image processing model to be trained based on the training dataset to obtain the pre-trained image processing model includes: The plurality of first training samples are sampled to obtain a sampled first sample image, a first label image, and a first sample identifier; The plurality of second training samples are sampled to obtain sampled second sample images, third label images, and second sample identifiers; The preprocessing module to be trained is trained based on the sampled first sample image, first label image, first sample identifier, sampled second sample image, third label image, and second sample identifier to obtain the pre-trained preprocessing module. Based on the first label image, second label image and first sample identifier in the plurality of first training samples, the denoising module to be trained and the enhancement module to be trained are trained to obtain the denoising module and the enhancement module trained for the first time. Based on the third label image, fourth label image, and second sample identifier in the plurality of second training samples, the noise reduction module and the enhancement module trained initially are retrained to obtain the pre-trained noise reduction module and the pre-trained enhancement module. The pre-trained image processing model is obtained based on the pre-trained preprocessing module, the pre-trained noise reduction module, and the pre-trained enhancement module.

7. The image enhancement method according to claim 1, characterized in that, The image enhancement method further includes: The image enhancement effects of the first processed image and the second processed image are evaluated to obtain image enhancement evaluation results; Based on the image enhancement evaluation results, a first target processing image is determined from the first processed image and the second processed image; Obtain the target image to be processed corresponding to the first target processing image; The identifier configured on the target image to be processed is adjusted to obtain the target image to be processed with the adjusted identifier; The pre-trained image processing model is used to perform noise reduction and enhancement processing on the target image to be processed after the labeling is adjusted, so as to obtain the adjusted image corresponding to the target image.

8. The image enhancement method according to claim 7, characterized in that, The evaluation of the image enhancement effects of the first processed image and the second processed image to obtain the image enhancement evaluation result includes: From the first processed image and the second processed image, a second target processed image is determined to be of the image type of a non-noise image. The image enhancement effect of at least the second target image is evaluated to obtain the image enhancement evaluation result.

Images