Image processing methods, image processing devices, storage media and electronic devices
By performing pixel point clustering and adding noise intensity determination methods on the image to be processed, low-quality images are generated, which solves the problem of lack of targeted image processing in the prior art and improves the flexibility and effectiveness of image processing.
Patent Information
- Application Number
- CN202110932196.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-13
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-08-13
AI Technical Summary
The existing image processing methods cannot perform targeted processing based on the actual situation of the image to be processed, resulting in large differences in image quality, affecting the accuracy of image repair and other processing.
By acquiring the image to be processed, clustering it based on the preset channel value of the pixel point, dividing it into multiple pixel point sets, and determining the noise intensity based on the preset channel value statistics of the pixel point sets, and the noise intensity is determined for the pixel point sets. Add noise to pixel points to generate corresponding low-quality images.
It realizes targeted noise addition for different pixel points and environmental scenarios, improves the flexibility and effectiveness of image processing, and has a wider range of applicable scenarios, ensuring the balance of image quality.
Smart Images

Figure CN113658070B_ABST
Abstract
Description
Background Art
[0002] In image processing such as image restoration or adding special effects to images, it is often necessary to process clear, high-quality images to obtain corresponding low-quality images. For example, noise is added to a clear image to generate a noisy image, and the image pairs consisting of the clear image and the noisy image are used as training data to train the image restoration model to perform image restoration on the noisy image to be restored.
[0003] The existing technology often uses a unified method to directly process high-quality images, causing them to degenerate into low-quality images. However, due to the different environments, times, and angles for shooting high-quality images, the quality of different high-quality images is inherently different. For example, the quality of images taken at night is much lower than that of images taken during the day. If the same method is used to add noise to images taken at night and during the day, the images taken at night will lose a lot of details and have worse quality. That is, the existing image processing methods cannot perform targeted processing according to the actual situation of the image to be processed, resulting in large differences in image quality after image processing, affecting the accuracy of image processing such as image restoration. Summary of the invention
[0004] The present disclosure provides an image processing method, an image processing device, a computer-readable storage medium and an electronic device, thereby at least to a certain extent improving the problem that the existing image processing methods cannot perform targeted processing according to the actual situation of the image to be processed.
[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0006] According to a first aspect of the present disclosure, there is provided an image processing method, comprising: acquiring an image to be processed; clustering the pixels in the image to be processed based on preset channel values of the pixels to divide the image to be processed into a plurality of pixel sets; determining the noise intensity of the pixel set according to statistical values of the preset channel values in the pixel set; and adding noise to the pixels in the pixel set using the noise intensity to obtain a low-quality image corresponding to the image to be processed.
[0007] According to a second aspect of the present disclosure, there is provided an image processing device, comprising: an image to be processed acquisition module, used to acquire an image to be processed; a pixel point set determination module, used to cluster the pixels in the image to be processed based on preset channel values of the pixels, so as to divide the image to be processed into multiple pixel point sets; a noise addition intensity determination module, used to determine the noise addition intensity of the pixel point set according to the statistical value of the preset channel value in the pixel point set; and a low-quality image acquisition module, used to add noise to the pixels in the pixel point set according to the noise addition intensity, so as to obtain a low-quality image corresponding to the image to be processed.
[0008] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method of the first aspect and possible implementation methods thereof are implemented.
[0009] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the image processing method of the first aspect and possible implementation thereof by executing the executable instructions.
[0010] The technical solution disclosed in this disclosure has the following beneficial effects:
[0011] Acquire an image to be processed; cluster pixels based on preset channel values of pixels in the image to be processed to divide the image to be processed into multiple pixel sets; determine the noise intensity of the pixel set according to the statistical value of the preset channel value in the pixel set; add noise to the pixels in the pixel set by adding noise intensity to obtain a low-quality image corresponding to the image to be processed. On the one hand, this exemplary embodiment proposes a new image processing method for adding noise to an image, which determines the noise intensity of different pixel sets based on the statistical value of the preset channel value of the pixel, and can add noise to different degrees according to the change of the preset channel value of different pixels in the same image. Compared with adding noise to different images or the same image in the same way, it has strong pertinence, and the noise addition can more effectively fit the actual pixel situation; on the other hand, in actual use, the pixels in the image to be processed under different light environments will have different preset channel values. Therefore, this exemplary embodiment can determine the noise intensity that better meets the actual needs based on the change of the preset channel value in different environmental scenes, and obtain low-quality images after adding noise to different degrees. The process of adding noise to the image to be processed is more flexible, more effective, and has a wider range of applicable scenarios.
[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0014] Figure 1 A schematic diagram showing a system architecture in this exemplary embodiment;
[0015] Figure 2 A structural diagram of an electronic device according to the exemplary embodiment is shown;
[0016] Figure 3 A flowchart showing an image processing method in this exemplary embodiment is shown;
[0017] Figure 4 A sub-flow chart showing an image processing method in this exemplary embodiment;
[0018] Figure 5 A schematic diagram showing the variation of the noise intensity with the statistical value of the grayscale value in this exemplary embodiment;
[0019] Figure 6 A schematic diagram showing another variation of the noise intensity with the statistical value of the grayscale value in this exemplary embodiment;
[0020] Figure 7 A flowchart showing another image processing method in this exemplary embodiment;
[0021] Figure 8 A structural diagram of an image processing apparatus in this exemplary embodiment is shown. DETAILED DESCRIPTION
[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0023] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0024] In view of one or more of the above problems, an exemplary embodiment of the present disclosure provides an image processing method. Figure 1 1 shows a system architecture diagram of the operating environment of this exemplary embodiment. Figure 1 As shown, the system architecture 100 may include a terminal device 110 and a server 120, which communicate with each other through a network. The terminal device 110 may include but is not limited to a smart phone, a tablet computer, a game console, a wearable device, etc. The server 120 refers to a backend server that provides Internet services.
[0025] It should be understood that Figure 1 The number of each device is only exemplary. According to the implementation requirements, any number of clients can be set, or the server can be a cluster formed by multiple servers.
[0026] The image processing method provided in the embodiments of the present disclosure may be executed by the terminal device 110, for example, after the terminal device 110 acquires the image to be processed, it directly processes it to generate a corresponding low-quality image; it may also be executed by the server 120, for example, after the terminal device 110 acquires the image to be processed, it uploads it to the server 120, so that the server 120 processes the image to be processed to obtain a corresponding low-quality image, etc. The present disclosure does not limit this.
[0027] An exemplary embodiment of the present disclosure provides an electronic device for implementing an image processing method, which may be Figure 1 The terminal device 110 or the server 120 in the embodiment of the present invention comprises at least a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the image processing method by executing the executable instructions.
[0028] Below Figure 2 Taking the mobile terminal 200 in the example, the structure of the above electronic device is exemplarily described. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Figure 2 The construction in can also be applied to fixed type equipment.
[0029] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a USB (Universal Serial Bus) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294 and a SIM (Subscriber Identification Module) card interface 295, etc.
[0030] The processor 210 may include one or more processing units, for example, the processor 210 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc. The encoder may encode (i.e., compress) the image or video data; the decoder may decode (i.e., decompress) the bitstream data of the image or video to restore the image or video data.
[0031] In some implementations, the processor 210 may include one or more interfaces, and may be connected to other components of the mobile terminal 200 via different interfaces.
[0032] The internal memory 221 may be used to store computer executable program codes, which may include instructions. The internal memory 221 may include volatile memory, non-volatile memory, etc. The processor 210 executes various functional applications and data processing of the mobile terminal 200 by running instructions stored in the internal memory 221 and / or instructions stored in a memory provided in the processor.
[0033] The external memory interface 222 can be used to connect an external memory, such as a Micro SD card, to expand the storage capacity of the mobile terminal 200. The external memory communicates with the processor 210 through the external memory interface 222 to implement a data storage function, such as storing music, video and other files.
[0034] The USB interface 230 is an interface that complies with USB standard specifications and can be used to connect a charger to charge the mobile terminal 200, or to connect headphones or other electronic devices.
[0035] The charging management module 240 is used to receive charging input from a charger. While the charging management module 240 is charging the battery 242, it can also power the device through the power management module 241; the power management module 241 can also monitor the status of the battery.
[0036] The wireless communication function of the mobile terminal 200 can be implemented by antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the mobile terminal 200. The wireless communication module 260 can provide wireless communication solutions including WLAN (Wireless Local Area Networks) (such as Wi-Fi (Wireless Fidelity) network), BT (Bluetooth), GNSS (Global Navigation Satellite System), FM (Frequency Modulation), NFC (Near Field Communication), IR (Infrared) and the like applied to the mobile terminal 200.
[0037] The mobile terminal 200 can realize the display function and display the user interface through the GPU, the display screen 290 and the AP. The mobile terminal 200 can realize the shooting function through the ISP, the camera module 291, the encoder, the decoder, the GPU, the display screen 290 and the AP, and can also realize the audio function through the audio module 270, the speaker 271, the receiver 272, the microphone 273, the headphone jack 274 and the AP.
[0038] The sensor module 280 may include a depth sensor 2801 , a pressure sensor 2802 , a gyroscope sensor 2803 , an air pressure sensor 2804 , etc., to implement different sensing detection functions.
[0039] Indicator 292 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc. Motor 293 may generate vibration prompts, and may also be used for touch vibration feedback, etc. Buttons 294 include a power button, a volume button, etc.
[0040] The mobile terminal 200 may support one or more SIM card interfaces 295 for connecting a SIM card to implement functions such as calls and data communications.
[0041] Figure 3 An exemplary process of the image processing method is shown, which can be performed by the terminal device 110 or the server 120, including the following steps S310 to S340:
[0042] Step S310, obtaining an image to be processed.
[0043] The image to be processed refers to an image with relatively good image quality, such as an image with a relatively high definition level or a relatively high resolution, such as an image with a resolution not less than 1024*1024; an image with relatively low noise, such as an image in which the difference between different pixels in the image is lower than a preset threshold; an image taken in a scene with relatively good lighting; and an image with good parameters such as picture brightness and contrast, etc. According to actual image processing requirements, the image to be processed can be a variety of images such as face images, building images, plant images, etc. In this exemplary embodiment, a large number of images can be collected by a camera at different times, scenes, and environments as the images to be processed, or a large number of images can be obtained from a specific image source as the images to be processed, etc., and the present disclosure does not make specific limitations on this.
[0044] Step S320: clustering the pixels in the image to be processed based on the preset channel values of the pixels to divide the image to be processed into a plurality of pixel sets.
[0045] Among them, the preset channel value of the pixel point refers to the channel value of a specific channel of the image to be processed in a specific color mode, such as a grayscale channel value or a brightness channel value, etc. This exemplary embodiment can pre-process the image to be processed according to the requirements of the preset channel. For example, when the image to be processed is a color image and the preset channel value is a grayscale channel value, the image to be processed can be first converted into a grayscale image, and then the grayscale channel value of each pixel therein, that is, the grayscale value, can be determined.
[0046] Then, based on the preset channel values of the pixels, the pixels can be clustered. In this exemplary embodiment, the pixels with the same preset channel value can be grouped into one category. In addition, considering that the pixels are not much different under similar preset channel values, the pixels with similar preset channel values can also be grouped into one category to reduce the amount of calculation. For example, the pixels with gray values in [98-100] in the image to be processed are grouped into one category, and the pixels with gray values in [101, 103] are grouped into one category, etc. The specific interval range standard of the same category can be set as needed. By clustering the pixels, the pixels in the image to be processed can be divided into multiple pixel sets, wherein each pixel set contains pixels belonging to the same preset channel value category.
[0047] In order to reflect the sensitivity of different preset channel values to noise, in an exemplary embodiment, targeted noise addition processing may also be performed on pixel points of different preset channel values to improve the accuracy of adding noise between pixel points of different preset channel values. Specifically, the above step S320 may include:
[0048] The pixels with the same preset channel value in the image to be processed are formed into a pixel point set.
[0049] That is, clustering can be performed according to each preset channel value, for example, pixels with a grayscale value of 100 in the image to be processed are classified into one category, pixels with a grayscale value of 150 are classified into one category, pixels with a grayscale value of 55 are classified into one category, and so on, to generate pixel sets under different preset channel value categories.
[0050] Step S330, determining the noise intensity of the pixel point set according to the statistical value of the preset channel value in the pixel point set.
[0051] Among them, the statistical value of the preset channel value refers to the statistical value determined according to the preset channel value of each pixel in the pixel set. Specifically, when the preset channel values of each pixel in the pixel set are the same, for example, a pixel set generated by clustering pixels with the same grayscale value in the image to be processed, the grayscale value of any pixel in the pixel set can be directly determined as the statistical value of the preset channel value of the pixel set; when the preset channel values of each pixel in the pixel set are different, for example, a pixel set generated by clustering pixels with similar grayscale values in the image to be processed, the maximum grayscale value, the minimum grayscale value or the average grayscale value in the pixel set can be determined as the statistical value of the preset channel value of the pixel set. Noise intensity refers to information that can reflect the degree of pixel noise. The greater the intensity, the greater the degree of added noise, the smaller the intensity, the smaller the degree of added noise, etc.
[0052] This exemplary embodiment can determine the noise intensity of each pixel point set according to the statistical value of the preset channel value in the pixel point set. Specifically, an equation or function for the noise intensity can be constructed according to the statistical value of the preset channel value, so that the noise intensity can be changed linearly or nonlinearly according to the statistical value of the preset channel value of the pixel point. For example, when the preset channel value is a grayscale value, a linear variation function of the grayscale value and the noise intensity is constructed, so that lower intensity noise can be added to the pixel point set with low statistical value of the grayscale value, and higher intensity noise can be added to the pixel point set with high statistical value of the grayscale value, thereby ensuring that images captured under different ambient light can be subjected to targeted noise addition to varying degrees, thereby ensuring the balance of image quality after adding noise.
[0053] Step S340, adding noise to the pixel points in the pixel point set by adding noise intensity, so as to obtain a low-quality image corresponding to the image to be processed.
[0054] Finally, by adding noise to the pixels in each pixel set by adding noise intensity, and then combining all the pixel sets, a low-quality image after adding noise to each pixel in the image to be processed can be obtained. This exemplary embodiment can form a large number of image pairs through the original large number of images to be processed and the low-quality images corresponding to each image to be processed. Through these image pairs, a machine learning model can be trained, and the images to be repaired can be processed by the trained model to restore the corresponding high-quality images. After obtaining the above-mentioned low-quality images, in order to simulate more situations where image information is lost, the address image can also be compressed, blurred, and other processing processes can be added to obtain images with lower resolution, less image details, more noise, or lower clarity.
[0055] In summary, in this exemplary embodiment, an image to be processed is obtained; pixels in the image to be processed are clustered based on preset channel values of the pixels to divide the image to be processed into multiple pixel sets; the noise intensity of the pixel set is determined according to the statistical values of the preset channel values in the pixel set; noise is added to the pixels in the pixel set by adding the noise intensity to obtain a low-quality image corresponding to the image to be processed. On the one hand, this exemplary embodiment proposes a new image processing method for adding noise to an image, which determines the noise intensity of different pixel sets based on the statistical values of the preset channel values of the pixel points, and can add noise to different degrees according to the changes in the preset channel values of different pixel points in the same image. Compared with adding noise to different images or the same image in the same way, this method is more targeted and the noise addition can more effectively fit the actual pixel situation. On the other hand, in actual use, the pixels in the image to be processed under different lighting environments will have different preset channel values. Therefore, this exemplary embodiment can determine the noise intensity that better meets the actual needs based on the changes in the preset channel values under different environmental scenarios, and obtain low-quality images after adding noise to different degrees. The process of adding noise to the image to be processed is more flexible, more effective, and has a wider range of applicable scenarios.
[0056] In an exemplary embodiment, determining the noise intensity of the pixel point set according to the statistical value of the preset channel value in the pixel point set includes:
[0057] According to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value, the initial noise addition intensity is adjusted to obtain the noise addition intensity of the pixel point set.
[0058] The initial noise intensity refers to an original value determined when adding noise to the processed image. The maximum preset channel value refers to the upper limit of the preset channel value. For example, when the preset channel is a grayscale channel and the preset channel value is a grayscale value, since the grayscale value is usually in the range of [0, 255], the maximum preset channel value is 255. This exemplary embodiment can be based on the ratio of the statistical value i of the preset channel value to the maximum preset channel value 255. The initial noise intensity is adjusted to obtain the noise intensity of the pixel point set. In this exemplary embodiment, the initial noise intensity can be determined based on experience, for example, the average value of the historical noise intensity is used as the initial noise intensity; or the maximum value, minimum value or median value of the historical noise intensity is extracted as the initial noise intensity; or multiple noise intensities that are gradually increased or decreased and change at equal intervals between the maximum value and the minimum value are determined, and each intensity is used as the initial noise intensity for adjustment to obtain the noise intensity at the intensity level.
[0059] In order to ensure that when adding noise to a large number of different images to be processed, noise with different noise levels is added to simulate an image with uniform noise distribution, in an exemplary embodiment, the image processing method may further include:
[0060] The initial noise intensity is randomly generated within a preset noise intensity range.
[0061] The noise intensity interval [scale0, scale1] is the preset lower and upper limits of the noise intensity. All images to be processed can randomly determine an initial noise intensity scale from this interval. init The noise intensity interval [scale0, scale1] can be obtained according to actual needs or experience. For example, the noise intensity interval can be set to [3, 15]. The initial noise intensity can be determined from this interval. init =3, the noise intensity is small. init When =15, the noise intensity is relatively large.
[0062] In an exemplary embodiment, if Figure 4 As shown, the above-mentioned adjustment of the initial noise intensity according to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value to obtain the noise intensity of the pixel point set may include the following steps:
[0063] Step S410, determining an adjustment amount according to a ratio of a statistical value of a preset channel value in a pixel point set to a maximum preset channel value, and a difference between an initial noise intensity and a basic noise intensity;
[0064] Step S420: adjusting the initial noise intensity based on the adjustment amount to obtain the noise intensity of the pixel point set.
[0065] In order to ensure that a certain amount of noise can be added to images of different image qualities to simulate an image with added noise, that is, a certain amount of noise can be added to images of good quality or poor quality. Therefore, when determining the noise intensity of a pixel point set, this exemplary embodiment introduces a basic noise intensity a, which indicates the minimum noise level to be added to pixels of different preset channel values. This exemplary embodiment can be calculated by the ratio of the statistical value i of the preset channel value to the maximum preset channel value. And the difference between the initial noise intensity and the basic noise intensity scale init -a, first determine an adjustment amount for the initial noise intensity, and further, based on the adjustment amount, adjust the initial noise intensity to obtain the noise intensity of the pixel point set. Specifically, the noise intensity can be expressed by the following formula:
[0066]
[0067] In which, taking the preset channel value as the grayscale value as an example, i represents the grayscale value or the statistical value of the grayscale value in the interval [0, 255], a represents the basic noise intensity, that is, the minimum noise level to be added to the pixels of different grayscale values, scale init Represents the initial noise intensity. According to the above formula, we can find the noise intensity scale of different pixel points i It can be linearly changed according to different gray values, such as Figure 5 It should be noted that this exemplary embodiment can set different basic noise intensity a according to different needs, for example, a=scale0=3 can be set, wherein the smaller the basic noise intensity a is, the higher the noise intensity scale of different pixel points is. i The trend of gray value change is more gradual.
[0068] In another embodiment, the noise intensity scale of different pixel points i It is also possible to make nonlinear changes or curve changes according to different gray values, such as Figure 6 The curve variation function shown can be, for example, obtained by presetting the statistical value of the channel value. Power Ratio to the maximum preset channel value And the difference between the initial noise intensity and the basic noise intensity scale init -a, first determine an adjustment amount for the initial noise intensity, and further, based on the adjustment amount, adjust the initial noise intensity to obtain the noise intensity of the pixel point set; or construct other functional relationships to determine the curve change relationship between the noise intensity and the grayscale value, etc., the present disclosure does not make specific limitations on this.
[0069] In an exemplary embodiment, adding noise to pixel points in a pixel point set by adding noise intensity includes:
[0070] Based on the noise intensity of the pixel set, Gaussian noise is added to the pixels in the pixel set.
[0071] In this exemplary embodiment, different pixel point sets can be firstly screened out from the image to be processed by the following formula:
[0072]
[0073] Wherein, HQgray represents the gray value or gray value statistics of the image HQ to be processed, and x and y represent the x-coordinate and y-coordinate of the pixel point, respectively. Based on the above formula, the pixels in the image to be processed can be divided by different gray values or gray value statistics to generate multiple pixel point sets.
[0074] Furthermore, the formula can be used:
[0075] HQnoise i =GaussianNoise(HQ, scale i )*mask i
[0076] The noise intensity of the pixel set is used as the benchmark, that is, the parameter scale is used i , add Gaussian noise to the image to be processed HQ, and obtain the corresponding Gaussian noise intensity when the gray value or the statistical value of the gray value is i.
[0077] Finally, the following formula can be used:
[0078]
[0079] Among them, HQnoise represents the noise intensity of the low-quality image. Based on the above formula, the pixel points after adding noise corresponding to each grayscale value or the statistical value of the grayscale value can be merged to generate a low-quality image after adding noise.
[0080] In an exemplary embodiment, the above image processing method may further include:
[0081] Perform at least one of the following degradation processing on the low-quality image: image blurring; downsampling and then upsampling to the original resolution of the low-quality image; image compression.
[0082] In order to ensure that the image can undergo a variety of degradation processing and simulate various real low-quality images to improve the accuracy in application scenarios such as model training and image restoration, this exemplary embodiment can also perform other degradation processing on the low-quality images obtained above.
[0083] Since the terminal device may have Gaussian blur due to inaccurate focus or motion blur due to instability when acquiring images, the low-quality image can be blurred to simulate the above-mentioned different blur situations. For example, a value can be randomly selected between the first blur interval [m0, m1] to add motion blur to the low-quality image, or a value can be randomly selected between the second blur interval [g0, g1] to add Gaussian blur to the low-quality image, etc., and a value can also be randomly selected in the first blur interval and the second blur interval to add motion blur and Gaussian blur respectively, etc. The range size of the specific interval can be customized according to actual needs, for example, the first blur interval [m0, m1] can be set to [0.5, 3], etc.
[0084] Typically, when processing an image, a machine learning model will perform a process of downsampling and upsampling. Although the image can still be restored to its original size after upsampling, some image information will be lost in the downsampling stage. Therefore, in order to simulate the above process, this exemplary embodiment can downsample the low-quality image and then upsample it to restore it to the original resolution size of the low-quality image.
[0085] In addition, considering that image information is often lost due to compressed images in actual applications, this exemplary embodiment can also simulate the image compression process for low-quality images. Specifically, a value can be randomly obtained in the interval [j0, j1] to compress the low-quality image in JPEG (Joint Photographic Experts Group) image file format.
[0086] It should be noted that the above-mentioned image blurring, downsampling and then upsampling to the original resolution of the low-quality image, and image compression processing can be to process the low-quality image in any way, or to process the low-quality image in sequence in a certain order, for example, first blur the low-quality image, then downsample and upsample the blurred image, and finally compress the upsampled restored image, etc.
[0087] Figure 7 A flowchart of another image processing method in this exemplary embodiment is shown, which may specifically include the following process:
[0088] Step S710, obtaining an image to be processed;
[0089] Step S720, adding noise to the image to be processed through the above steps S320 to S340 to obtain a low-quality image corresponding to the image to be processed;
[0090] Step S730 , performing one or more of image blurring, downsampling and then upsampling to the original resolution of the low-quality image, and image compression on the low-quality image to obtain a target image.
[0091] This exemplary embodiment can simulate the image degradation when the terminal device captures scene images as much as possible by adding noise to the image to be processed and performing one or more image degradation processing processes, and accurately and effectively construct a training set containing image pairs of high-quality images and low-quality images, thereby providing convenience for training image restoration models.
[0092] The exemplary embodiment of the present disclosure also provides an image processing device. Figure 8As shown, the image processing device 800 may include: a module 810 for acquiring an image to be processed, for acquiring an image to be processed; a module 820 for determining a pixel set, for clustering pixels based on preset channel values of the pixels in the image to be processed, so as to divide the image to be processed into a plurality of pixel sets; a module 830 for determining a noise intensity of the pixel set according to a statistical value of the preset channel value in the pixel set; and a low-quality image obtaining module 840, for adding noise to the pixels in the pixel set by adding noise intensity, so as to obtain a low-quality image corresponding to the image to be processed.
[0093] In an exemplary embodiment, the noise intensity determination module includes: an initial noise intensity adjustment unit, which is used to adjust the initial noise intensity according to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value to obtain the noise intensity of the pixel point set.
[0094] In an exemplary embodiment, the initial noise intensity adjustment unit includes: an adjustment amount determination subunit, which is used to determine the adjustment amount based on the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value, and the difference between the initial noise intensity and the basic noise intensity; and a noise intensity subunit, which is used to adjust the initial noise intensity based on the adjustment amount to obtain the noise intensity of the pixel point set.
[0095] In an exemplary embodiment, the image processing apparatus includes: an initial noise intensity generation module, configured to randomly generate an initial noise intensity within a preset noise intensity interval.
[0096] In an exemplary embodiment, the low-quality image acquisition module includes: a Gaussian noise adding unit, which is used to add Gaussian noise to the pixels in the pixel point set based on the noise added intensity of the pixel point set.
[0097] In an exemplary embodiment, the pixel point set determination module includes: a pixel point division unit, which is used to form pixel points with the same preset channel value in the image to be processed into a pixel point set.
[0098] In an exemplary embodiment, the image processing device includes: an image degradation module, which is used to perform at least one of the following degradation processing on the low-quality image: image blurring; downsampling and then upsampling to the original resolution of the low-quality image; image compression.
[0099] The specific details of each part of the above device have been described in detail in the implementation method part, so they will not be repeated here.
[0100] The exemplary embodiments of the present disclosure further provide a computer-readable storage medium, which can be implemented in the form of a program product, including program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification, for example, to execute Figure 3 , Figure 4 or Figure 7 The program product may be in the form of a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
[0101] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0102] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0103] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0104] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0105] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or implementations combining hardware and software aspects, which may be collectively referred to herein as "circuit", "module" or "system". Those skilled in the art will readily think of other implementations of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.
[0106] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An image processing method, characterized in that: include: Get the image to be processed; Clustering the pixel points in the image to be processed based on preset channel values of the pixel points, so as to divide the image to be processed into a plurality of pixel point sets; Each pixel point set contains pixels belonging to the same preset channel value category; There are multiple preset channel values, and the multiple pixel point sets are pixel point sets under different preset channel value categories; the preset channel value is a grayscale channel value; Determining the noise intensity of the pixel point set according to the statistical value of the preset channel value in the pixel point set; Adding noise to the pixel points in the pixel point set according to the noise adding intensity to obtain a low-quality image corresponding to the image to be processed; The step of determining the noise intensity of the pixel point set according to the statistical value of the preset channel value in the pixel point set includes: According to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value, the initial noise addition intensity is adjusted to obtain the noise addition intensity of the pixel point set.
2. The method according to claim 1, characterized in that The step of adjusting the initial noise intensity according to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value to obtain the noise intensity of the pixel point set includes: Determine the adjustment amount according to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value, and the difference between the initial noise intensity and the basic noise intensity; The initial noise intensity is adjusted based on the adjustment amount to obtain the noise intensity of the pixel point set.
3. The method according to claim 1, characterized in that The method further comprises: The initial noise addition intensity is randomly generated within a preset noise addition intensity interval.
4. The method according to claim 1, characterized in that The adding noise to the pixel points in the pixel point set by using the noise adding intensity includes: Based on the noise intensity of the pixel point set, Gaussian noise is added to the pixel points in the pixel point set.
5. The method according to claim 1, characterized in that The clustering of the pixels based on the preset channel values of the pixels in the image to be processed to divide the image to be processed into a plurality of pixel sets includes: The pixels with the same preset channel value in the image to be processed are formed into a pixel point set.
6. The method according to claim 1, characterized in that The method further comprises: The low-quality image is subjected to at least one of the following degradation processing: image blurring; downsampling and then upsampling to the original resolution of the low-quality image; image compression.
7. An image processing device, characterized in that: include: The module for acquiring the image to be processed is used to acquire the image to be processed; A pixel point set determination module, used for clustering the pixel points in the image to be processed based on preset channel values of the pixel points, so as to divide the image to be processed into a plurality of pixel point sets; Each pixel point set contains pixels belonging to the same preset channel value category; There are multiple preset channel values, and the multiple pixel point sets are pixel point sets under different preset channel value categories; the preset channel value is a grayscale channel value; A noise intensity determination module, used to determine the noise intensity of the pixel point set according to the statistical value of the preset channel value in the pixel point set; A low-quality image acquisition module, used for adding noise to the pixel points in the pixel point set according to the noise adding intensity, so as to obtain a low-quality image corresponding to the image to be processed; The noise intensity determination module is configured as follows: According to the ratio of the statistical value of the preset channel value in the pixel point set to the maximum preset channel value, the initial noise addition intensity is adjusted to obtain the noise addition intensity of the pixel point set.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
9. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.
Citation Information
Patent Citations
Model training and image processing method and device, medium and electronic equipment
CN110163237A
DenseNet-based image noise recognition method and device
CN111598184A
Medical image processing device, medical image processing method, and program
WO2020183799A1
