Image processing method, device and equipment

Abstract

The embodiment of the invention provides an image processing method, device and equipment. The method comprises the following steps: acquiring a RAW image; determining distribution data corresponding to a real pixel value according to the RAW image; receiving a clean RAW image; determining target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain a noisy pixel value corresponding to a real pixel value in the clean RAW image and a noisy RAW image corresponding to the noisy pixel value. According to the embodiment of the invention, the matching degree between the noisy RAW image and the real noisy image data can be improved, so that the processing effect of the image processing model can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular, to an image processing method, apparatus and device. Background Art

[0002] Images are very important information carriers, which contain rich information. However, images are easily contaminated by noise and lose details. The presence of noise will affect image quality, especially in practical application scenarios. Under low light conditions, the signal level is low, resulting in a low signal-to-noise ratio. In this case, noise has a greater impact on image quality. Therefore, removing noise to obtain high-quality images is very necessary and has always been an important task in the field of computer vision.

[0003] The image denoising model based on deep learning can be used to remove the noise contained in the input image. At present, the image denoising model is usually trained using training data. The training data specifically includes: noisy RAW (original) images and clean RAW images (clean RAW images can refer to RAW images without noise, and clean RAW images can be generated by data degradation).

[0004] In the related art, the process of acquiring a noisy RAW image is usually: constructing noise, and adding the constructed noise to a clean RAW image generated by a data degradation method to obtain a constructed noisy RAW image.

[0005] However, in practical applications, the input image corresponding to the trained image denoising model is usually a real noisy RAW image. When the constructed noisy RAW image is used to train the image denoising model, the processing effect of the image denoising model will be affected because the constructed noisy RAW image is inconsistent with the real noisy RAW image. Summary of the invention

[0006] The embodiment of the present application provides an image processing method, which can improve the matching degree between a noisy RAW image and actual noisy image data, thereby improving the processing effect of an image processing model.

[0007] Correspondingly, the embodiments of the present application also provide an image processing device and an electronic device to ensure the implementation and application of the above method.

[0008] In order to solve the above problems, an embodiment of the present application discloses an image processing method, which includes:

[0009] Collect RAW images;

[0010] Determine distribution data corresponding to real pixel values ​​according to the RAW image;

[0011] Receive clean RAW images;

[0012] Determine target distribution data corresponding to the real pixel values ​​in the clean RAW image from the distribution data corresponding to the real pixel values;

[0013] The target distribution data is randomly sampled to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

[0014] In order to solve the above problems, an embodiment of the present application discloses an image processing method, which includes:

[0015] Acquire training data; the training data includes: noisy RAW images;

[0016] Training the image processing model according to the training data;

[0017] Among them, the acquisition process of the noisy RAW image includes: collecting the RAW image; determining the distribution data corresponding to the real pixel value based on the RAW image; receiving the clean RAW image; determining the target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image and the noisy RAW image corresponding to the noisy pixel value.

[0018] In order to solve the above problems, an embodiment of the present application discloses an image processing device, the device comprising:

[0019] Acquisition module, used for acquiring RAW images;

[0020] A distribution data determination module, used to determine distribution data corresponding to real pixel values ​​according to the RAW image;

[0021] A receiving module, used for receiving a clean RAW image;

[0022] A target distribution data determination module, used to determine target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value;

[0023] The random sampling module is used to randomly sample the target distribution data to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

[0024] Optionally, the acquisition module includes:

[0025] The first acquisition module is used to acquire RAW images using an image acquisition module; the image acquisition module includes: an image sensor, or an image sensor and a filter.

[0026] Optionally, the acquisition module includes:

[0027] The second acquisition module is used to acquire multiple groups of RAW images under the conditions of multiple preset brightness values.

[0028] Optionally, the distribution data determination module includes:

[0029] A preset area determination module, configured to select a preset area including the center from the RAW image according to the center of the RAW image;

[0030] The statistical module is used to count the pixel points in the preset area, and the obtained pixel mean is the real pixel value, and the distribution data of the pixel points in the preset area is used as the distribution data corresponding to the real pixel value.

[0031] Optionally, the device further comprises:

[0032] The interpolation module is used to interpolate the distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value to obtain the distribution data corresponding to the third real pixel value.

[0033] Optionally, the random sampling module includes:

[0034] A mapping relationship determination module, used to determine the mapping relationship between the numerical range corresponding to the probability value of the pixel value and the pixel value according to the pixel value and the probability value contained in the target distribution data;

[0035] A generation module, used to generate random numbers;

[0036] A search module is used to search in the mapping relationship between the numerical range and the pixel value according to the random number, so as to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image.

[0037] In order to solve the above problems, an embodiment of the present application discloses an image processing device, the device comprising:

[0038] An acquisition module is used to acquire training data; the training data includes: a noisy RAW image;

[0039] A training module, used for training the image processing model according to the training data;

[0040] Among them, the acquisition process of the noisy RAW image includes: collecting the RAW image; determining the distribution data corresponding to the real pixel value based on the RAW image; receiving the clean RAW image; determining the target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image and the noisy RAW image corresponding to the noisy pixel value.

[0041] The embodiment of the present application further discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes the method described in the embodiment of the present application.

[0042] The embodiments of the present application include the following advantages:

[0043] In the technical solution of the embodiment of the present application, first, the distribution data corresponding to the real pixel value is determined based on the collected RAW image; then, the target distribution data corresponding to the real pixel value in the clean RAW image is determined from the distribution data corresponding to the real pixel value, and the target distribution data is randomly sampled, and then the noisy RAW image is determined based on the noisy pixel value corresponding to the real pixel value in the clean RAW image.

[0044] Since the collected RAW image is a real noisy image, the random sampling of the embodiment of the present application can be a process of sampling from the real noisy image data; in this way, the noisy pixel value as the random sampling result matches the real noisy image data; and the noisy RAW image is obtained based on the random sampling result. Therefore, the embodiment of the present application can improve the matching degree between the noisy RAW image and the real noisy image data. In other words, the embodiment of the present application can improve the authenticity of the noise data distribution in the noisy RAW image.

[0045] When a noisy RAW image is used as training data for an image processing model, since the embodiment of the present application can improve the matching degree between the noisy RAW image and the actual noisy image data, the embodiment of the present application can improve the processing effect of the image processing model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the steps of an image processing method according to an embodiment of the present application;

[0047] Figure 2 is an example of the interpolation process of the distribution data of one embodiment of the present application;

[0048] Figure 3 It is a flowchart of data degradation processing in one embodiment of the present application;

[0049] Figure 4 It is a schematic diagram of the steps of an image processing method according to an embodiment of the present application;

[0050] Figure 5 is a structural schematic diagram of an image processing device according to an embodiment of the present application;

[0051] Figure 6 is a structural schematic diagram of an image processing device according to an embodiment of the present application;

[0052] Figure 7 It is a schematic diagram of the structure of a device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The embodiments of the present application can be applied to image processing scenarios to provide image processing models with training data that better matches real data, thereby improving the processing effect of the image processing models.

[0055] The image processing model may be a machine learning model or a deep learning model with image denoising or target detection functions. For example, the image denoising model may be used to remove noise contained in an input image. The target detection model may be used to detect whether an input image contains targets such as pedestrians, and if so, the location information of the targets such as pedestrians may be provided. Targets may include pedestrians, vehicles, animals and other objects.

[0056] The embodiment of the present application can train the mathematical model to obtain the image processing model. The mathematical model can be trained by machine learning, deep learning methods, etc. The machine learning method may include: linear regression, decision tree, random forest, etc. The deep learning method may include: CNN (Convolutional Neural Networks), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), etc.

[0057] Image processing models such as target detection models or image denoising models can use noisy RAW images as training data. RAW images may refer to the original images output by the image sensor of the image acquisition module, and the RAW images have not been processed by the ISP (Image Signal Processing) of the image acquisition module.

[0058] In the related art, the process of acquiring a noisy RAW image is generally as follows: constructing noise, and adding the constructed noise to a clean RAW image to obtain a constructed noisy RAW image.

[0059] However, in practical applications, the input image corresponding to the trained image processing model is usually a real noisy RAW image. When the constructed noisy RAW image is used to train the image processing model, the processing effect of the image processing model will be affected because the constructed noisy RAW image is inconsistent with the real noisy RAW image.

[0060] In response to the technical problem that the noisy RAW image constructed in the related art is inconsistent with the real noisy RAW image and will affect the processing effect of the image processing model, an embodiment of the present application provides an image processing method, which may specifically include: acquiring a RAW image; determining distribution data corresponding to real pixel values ​​based on the RAW image; receiving a clean RAW image; determining target distribution data corresponding to the real pixel values ​​in the clean RAW image from the distribution data corresponding to the real pixel values; and randomly sampling the target distribution data to obtain noisy pixel values ​​corresponding to the real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

[0061] In an embodiment of the present application, first, distribution data corresponding to the real pixel values ​​are determined based on the collected RAW image; then, target distribution data corresponding to the real pixel values ​​in the clean RAW image are determined from the distribution data corresponding to the real pixel values, and the target distribution data are randomly sampled, and then the noisy RAW image is determined based on the noisy pixel values ​​corresponding to the real pixel values ​​in the clean RAW image.

[0062] Since the collected RAW image is a real noisy image, the random sampling of the embodiment of the present application can be a process of sampling from the real noisy image data; in this way, the noisy pixel value as the random sampling result matches the real noisy image data; and the noisy RAW image is obtained based on the random sampling result. Therefore, the embodiment of the present application can improve the matching degree between the noisy RAW image and the real noisy image data. In other words, the embodiment of the present application can improve the authenticity of the noise data distribution in the noisy RAW image.

[0063] When a noisy RAW image is used as training data for an image processing model, since the embodiment of the present application can improve the matching degree between the noisy RAW image and the actual noisy image data, the embodiment of the present application can improve the processing effect of the image processing model.

[0064] Method Example 1

[0065] Reference Figure 1, shows a schematic flow chart of the steps of an image processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0066] Step 101: acquiring a RAW image, where the RAW image may be a RAW image with noise;

[0067] Step 102: Determine distribution data corresponding to the real pixel value according to the RAW image;

[0068] Step 103, receiving a clean RAW image;

[0069] Step 104: determining target distribution data corresponding to the real pixel values ​​in the clean RAW image from the distribution data corresponding to the real pixel values;

[0070] Step 105 : randomly sampling the target distribution data to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

[0071] Figure 1 The method shown can be used to obtain a noisy RAW image corresponding to a clean RAW image based on real noisy image data.

[0072] In step 101, an image acquisition module may be used to acquire a RAW image.

[0073] In practical applications, the image acquisition module may include: image sensor, printed circuit board, filter, base, voice coil motor and lens. The image sensor is used to convert the light image on the photosensitive surface into an electrical signal proportional to the light image by using the photoelectric conversion function of the photoelectric device. The printed circuit board is responsible for power supply control and signal transmission. The filter is used to filter electromagnetic waves such as infrared light. The base is used to fix the lens. The voice coil motor is used to change the image distance to achieve automatic focus of the lens. The lens is used to receive light signals and converge the light signals on the photosensitive device.

[0074] In one implementation, an image acquisition module may be used to acquire RAW images; wherein the image acquisition module may include: an image sensor, a lens, a filter and other modules.

[0075] In another implementation, an image acquisition module may be used to acquire RAW images; wherein the image acquisition module may include: an image sensor, or the image acquisition module may include: an image sensor and a filter.

[0076] Among them, removing the lens in the image acquisition module can prevent the collected RAW image from being affected by the lens shadow, thereby improving the quality of the collected RAW image. Among them, the lens shadow can refer to the dark corners formed by insufficient incident light at the four corners of the picture. Affected by the optical characteristics of the lens, the edge area of ​​the image area of ​​the image sensor receives less light intensity than the center, which easily causes inconsistent brightness between the center and the four corners of the picture, which is the problem of lens shadow.

[0077] Removing the filter in the image acquisition module can avoid the influence of spectral shift caused by different incident angles of the filter, thereby improving the quality of the acquired RAW image.

[0078] Of course, the above-mentioned image acquisition module does not include a lens and / or a filter only as an optional embodiment. In fact, when the image acquisition module includes a lens and / or a filter, the captured RAW image can be corrected for lens shading and / or spectral shift.

[0079] In a specific implementation, multiple groups of RAW images can be collected respectively under the conditions of multiple preset brightness values. The preset brightness value can be determined by those skilled in the art according to actual application requirements. For example, the preset brightness value can correspond to a lower brightness limit value, and a group of RAW images can be collected under the condition of the lower brightness limit value. A group of RAW images may include: at least one RAW image.

[0080] Afterwards, based on the brightness lower limit value, the preset increment is added to the brightness lower limit value to obtain a set of brightness intermediate values, and a set of RAW images is collected under the condition of the brightness intermediate values. It can be understood that there can be multiple brightness intermediate values, and the brightness intermediate values ​​can be gradually adjusted upward according to the preset increments until the brightness intermediate values ​​reach the brightness upper limit value. An example of the brightness lower limit value can be 1 lux (lux), etc., and an example of the brightness upper limit value can be 10 lux, etc. It can be understood that the embodiments of the present application do not limit the specific preset brightness value, the specific brightness lower limit value, and the specific brightness upper limit value.

[0081] In practical applications, a light source can be used to evenly illuminate a collection object such as a flat surface, so that the image acquisition module obtains a RAW image corresponding to the collection object. The collection object can correspond to a color. The embodiment of the present application does not limit the color corresponding to the collection object. For example, the color corresponding to the collection object can be a preset color such as white.

[0082] In step 102, a RAW image may correspond to a real pixel value. The real pixel value may represent the pixel value represented by the RAW image. The distribution data corresponding to the real pixel value may represent the distribution of the pixel values ​​that may appear in the case of the real pixel value.

[0083] The color depth of a RAW image can be represented by n, and the pixel value range of a RAW image can be: 0 to 2 n -1. Taking n=12 as an example, the true pixel value range or pixel value range of the RAW image can be: [0,4095].

[0084] In a specific implementation, the above process of determining the distribution data corresponding to the real pixel value specifically includes: according to the center of the RAW image, selecting a preset area containing the center from the RAW image; counting the pixel points in the preset area, and the obtained pixel mean is the real pixel value, and the distribution data of the pixel points in the preset area is used as the distribution data corresponding to the real pixel value.

[0085] The embodiment of the present application obtains distribution data based on pixel points within a preset area including the center, which can reduce the row and column pixel value deviation caused by edge stress of the image sensor and peripheral circuit factors.

[0086] A RAW image may include W×H pixels, where W represents the width of the RAW image, H represents the height of the RAW image, and both W and H may be positive integers. In the embodiment of the present application, the center of the RAW image may be determined according to the coordinates corresponding to the W×H pixels.

[0087] The preset area may be a rectangular area. The preset area may include four boundaries: an upper boundary, a lower boundary, a left boundary, and a right boundary. The four boundaries may correspond to pixel average values, respectively, and the difference between the pixel average values ​​corresponding to any two boundaries may be less than the first difference threshold; in this way, the accuracy of the distribution data can be improved. The first difference threshold may be determined by those skilled in the art according to actual application requirements. For example, the first difference threshold may be a positive integer such as 1, or the first difference threshold may be a decimal less than 1.

[0088] Assuming that the preset area includes W1×H1 pixels, the pixel mean may be the average value of the pixel values ​​of W1×H1 pixels. In the embodiment of the present application, the pixel mean may be used as the real pixel value corresponding to the RAW image.

[0089] In the embodiment of the present application, the distribution data of the pixel points in the preset area can be used as the distribution data corresponding to the real pixel value. The distribution data may include: pixel value and number of occurrences, or pixel value and probability value, etc. Taking the real pixel value of 1915 as an example, in the corresponding distribution data, the range of pixel values ​​can be [1400, 2800]. The probability value corresponding to a pixel value can be the ratio of the number of occurrences of the pixel value to the sum of the number of occurrences of all pixel values ​​in the case of the real pixel value.

[0090] In the embodiment of the present application, N groups of RAW images can be acquired according to a plurality of preset brightness values. Assuming that a preset brightness value corresponds to a RAW image group, a RAW image group may include: at least one RAW image. The real pixel values ​​corresponding to the RAW images in the same RAW image group may be the same or similar, for example, the difference between the real pixel values ​​corresponding to different RAW images in the same RAW image group may be less than the second difference threshold.

[0091] The difference between the real pixel values ​​corresponding to the RAW images between different RAW image groups may be greater than the third difference threshold. For example, the third difference threshold may be a two-digit positive integer, or a three-digit positive integer, etc. In this way, the distribution data corresponding to the real pixel values ​​obtained based on the collected RAW images in the embodiment of the present application may be discrete. For example, the distribution data corresponding to the real pixel values ​​obtained based on the collected RAW images in the embodiment of the present application may include: distribution data corresponding to real pixel values ​​such as 0, 100, 200, 300, ... 4000, etc. Among them, the real pixel values ​​0 and 100, 100 and 200, etc. are discrete, for example, the distribution data corresponding to the real pixel values ​​between [101,199] are missing.

[0092] In order to increase the continuity and coverage of the distribution data of the real pixel value, the embodiment of the present application can interpolate the distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value to obtain the distribution data corresponding to the third real pixel value. For example, the embodiment of the present application can use 100 as the first real pixel value and 200 as the second real pixel value, and obtain the distribution data corresponding to the real pixel values ​​between [101,199] based on the interpolation.

[0093] Reference Figure 2 , showing an example of the interpolation process of distribution data according to an embodiment of the present application. Figure 2 The histogram distributions corresponding to three real pixel values ​​may be included: a histogram distribution of 1915, a histogram distribution of 2104, and a histogram distribution of 2010. The horizontal axis of the histogram distribution may represent the pixel value, and the vertical axis of the histogram distribution may represent the probability value of the pixel value.

[0094] The embodiment of the present application can interpolate the distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value. For example, Figure 2 In , the histogram distribution of 1915 and the histogram distribution of 2104 can be interpolated to obtain the histogram distribution of 2010.

[0095] The process of interpolating the distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value may include: translating the first distribution data corresponding to the first real pixel value to the third real pixel value, and translating the second distribution data corresponding to the second real pixel value to the third real pixel value; and performing weighted averaging on the translated first distribution data and the translated second distribution data to obtain the distribution data corresponding to the third real pixel value.

[0096] Assume that the smaller of the first true pixel value and the second true pixel value is p1, and the larger of the first true pixel value and the second true pixel value is p2, and further assume that the target true pixel value is p3, then p1 <p3<p2。

[0097] The distance between p2 and p1 can be written as:

[0098] diff=p2-p1 (1)

[0099] The first weight corresponding to p1 can be expressed as Weight_1, where the farther the distance between p2 and p3 is, the closer the distance between p1 and p3 is. The first weight can be expressed as:

[0100] Weight_1=(p2-p3) / diff (2)

[0101] The second weight corresponding to P2 can be expressed as:

[0102] Weight_2=1-weight_1 (3)

[0103] By shifting the distribution of p1 to the right by (p3-p1), we can obtain the first distribution Dist_1(x) after translation.

[0104] By shifting the distribution of p2 to the left by (p2-p3), we can obtain the second distribution Dist_2(x) after translation.

[0105] By taking a weighted average of the first distribution after translation and the second distribution after translation, we can get the distribution of p3, which can be expressed as:

[0106] Dist_3(x)=Dist_1(x)*Weight_1+Dist_2(x)*Weight_2 (4)

[0107] For example, the first distribution data corresponding to the first real pixel value is (X1, Y1), the second distribution data corresponding to the second real pixel value is (X2, Y2), and the second distribution data corresponding to the third real pixel value is (X3, Y3). The translation unit of the first distribution data can be (third real pixel value - first real pixel value), and the first distribution data after translation is (X'1, Y'1). Figure 2In the example, the translation unit of the histogram distribution of 1915 can be (2010-1915), and the translation direction can be: rightward. The translation unit of the second distribution data can be (second real pixel value-third real pixel value), and the second distribution data after translation is (X'2, Y'2), such as Figure 2 In , the translation unit of the histogram distribution of 2104 can be (2104-2010), and the translation direction can be: left shift. Figure 2 In the figure, since the distance between p1 and p3 is close to the distance between p2 and p3, the first weight is close to the second weight, X3 can be expressed as: (X'1+X'2) / 2, and Y3 can be expressed as (Y'1+Y'2) / 2.

[0108] By using the above interpolation method, the real pixel value that cannot be covered by the collected RAW image can be obtained. For example, the distribution data corresponding to the third pixel value located in [1916, 2103] can be interpolated based on the distribution data corresponding to the first pixel value 1915 and the second pixel value 2104.

[0109] In one example, assuming that the color depth of the RAW image is 12, the true pixel value range or pixel value range of the RAW image can be: [0,4095]. Then, the embodiment of the present application can collect RAW images corresponding to a plurality of preset brightness values ​​to obtain the distribution data corresponding to the first part of the true pixel values. The difference between the first part of the true pixel values ​​can be dozens or hundreds. Further, the embodiment of the present application can interpolate the distribution data corresponding to two adjacent true pixel values ​​in the first part of the true pixel values ​​to obtain the distribution data corresponding to the second part of the true pixel values. The union of the first part of the true pixel values ​​and the second part of the true pixel values ​​can be [0,4095].

[0110] The embodiment of the present application can save the distribution data corresponding to the true pixel values ​​such as the first part of the true pixel values ​​and the second part of the true pixel values. For example, the saved distribution data corresponding to the true pixel values ​​may include: pixel value and number of occurrences, or pixel value and probability, etc.

[0111] In step 103 , the clean RAW image may refer to a RAW image without noise.

[0112] In practical applications, data degradation processing can be performed on RGB (Red Green Blue) images in a preset image set to obtain clean RAW images.

[0113] Reference Figure 3, shows a schematic flow chart of data degradation processing of an embodiment of the present application, wherein the data degradation processing may be an inverse process of camera imaging, and may specifically include the following processes: performing global tone mapping inverse processing, gamma correction inverse processing, color correction inverse processing, white balance inverse processing, and Bayer arrangement inverse processing on the RGB image to obtain a RAW image (i.e., a clean RAW image).

[0114] In practical applications, the global tone mapping inverse processing, the gamma correction inverse processing, the color correction inverse processing, the white balance inverse processing and the Bayer arrangement inverse processing may correspond to independent modules, such as the global tone mapping inverse processing module, the gamma correction inverse processing module, the color correction inverse processing module, the white balance inverse processing module and the Bayer arrangement inverse processing module. Each module may have independent processing parameters. The above processing parameters may be controllable, thereby improving the robustness of converting the RGB image into the RAW image.

[0115] The global tone mapping inverse process is used to map the luminous intensity of the display device into the signal intensity output by the image sensor.

[0116] The inverse gamma correction process is used to simulate the nonlinear transformation of the camera front end to convert the optical signal into an electrical signal. This nonlinear transformation can be approximated as a power function:

[0117] Y=X γ (5)

[0118] Wherein, X is the RGB image to be converted, Y is the RGB image after the gamma correction inverse processing, and γ may be a first preset parameter.

[0119] Due to the difference between the spectrum responsivity of visible light of the human eye and the responsivity of the image sensor, the color values ​​of the obtained RGB image will be biased; usually a 3×3 color transformation matrix is ​​used for color correction. The inverse color correction process is used to multiply the image after the inverse gamma correction process by the inverse matrix of the transformation matrix.

[0120]

[0121] Where R, G, and B represent the pixel values ​​of red, green, and blue in the RGB image after inverse gamma correction. i ,b i ,c i , i∈(1,3), can be the second preset parameter. In practical applications, the ISP module can provide a i ,b i ,c i, i∈(1,3) is an initial value, and the embodiment of the present application can randomly select a value within a range of plus or minus 20% of the initial value to increase the diversity of the second preset parameter. The range of the second preset parameter after random selection can be: [0.8*initial value, 1.2*initial value].

[0122] White balance is used to restore white objects in different ambient color temperatures to true white. White balance inverse processing is used to convert the true white in the RGB image after color correction inverse processing to white objects in different ambient color temperatures.

[0123] An example of white balance inverse processing is shown in formula (7):

[0124] R"G"B"=R'G'B'*[1 / rgain, 1 / ggain, 1 / bgain] (7)

[0125] Among them, rgain, ggain, and bgain are the gains of the three channels of RGB, respectively, and the specific values ​​of rgain, ggain, and bgain can be determined by those skilled in the art according to actual application requirements. For example, the value range of rgain is [1.9, 2.4], ggain=1, and the value range of bgain is [1.5, 1.9].

[0126] Before light enters the image sensor, it passes through the Bayer filter to obtain a RAW image, and then the interpolation algorithm is used to convert the Bayer arrangement RAW image into an RGB image. The Bayer arrangement inverse processing is used to convert the RGB image after the white balance inverse processing into a Bayer arrangement RAW image.

[0127] An RGB image may contain three channels: R, G, and B. A RAW image may contain a single channel. In an application example of the present application, the process of inverse processing of the Bayer arrangement may include: selecting a target pixel from the RGB image after inverse white balance processing according to a preset pixel interval, and assigning the value of the target channel of the target pixel to the single channel of RAW. In other words, the value of the single channel of RAW may be the value of the target channel of the target pixel. The preset pixel interval may be determined by a person skilled in the art according to actual application requirements. For example, the preset pixel interval may be: taking a target pixel for every 2 pixels in the height direction and the width direction, so that 1 target pixel may be taken in a 2×2 pixel array. The embodiments of the present application do not limit the specific target channels. For example, in a 2×2 pixel array in a RAW image, the target channels of the first row are: G and R, and the target channels of the second row are: B and G.

[0128] Understandably, Figure 3The data degradation processing shown is only an example of obtaining a RAW image. In fact, the embodiment of the present application does not limit the specific method of obtaining a RAW image.

[0129] In step 104, the target distribution data corresponding to the real pixel value in the clean RAW image can be determined from the distribution data corresponding to the real pixel value. Specifically, the target distribution data corresponding to the real pixel value can be queried according to each pixel value in the clean RAW image to obtain the target distribution data corresponding to the real pixel value in the clean RAW image. For example, when the real pixel value in the clean RAW image is 1, the target distribution data corresponding to the real pixel value 1 can be obtained for the pixel value. For another example, when the real pixel value in the clean RAW image is 700, the target distribution data corresponding to the real pixel value 700 can be obtained for the pixel value.

[0130] In step 105, the target distribution data is randomly sampled to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values. The noisy pixel values ​​may be derived from the target distribution data.

[0131] The embodiment of the present application may utilize a random sampling method to perform random sampling on the target distribution data. It is to be understood that the embodiment of the present application does not limit the specific random sampling method.

[0132] In one implementation of the present application, the above-mentioned process of randomly sampling the target distribution data may specifically include: determining the mapping relationship between the numerical range corresponding to the probability value of the pixel value and the pixel value based on the pixel value and probability value contained in the target distribution data; generating a random number; and based on the random number, searching in the mapping relationship between the numerical range and the pixel value to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image.

[0133] The numerical range can be determined based on the probability value corresponding to the pixel value. Generally speaking, the larger the probability value corresponding to the pixel value, the larger the numerical range in the mapping relationship. For example, the target distribution data includes: pixel value 1 and probability value 0.1, pixel value 2 and probability value 0.2, pixel value 3 and probability value 0.3, pixel value 4 and probability value 0.4, then the numerical range corresponding to pixel value 1 may include: [0,1], the numerical range corresponding to pixel value 2 may include: (1,3], the numerical range corresponding to pixel value 3 may include: (3,6], the numerical range corresponding to pixel value 4 may include: (6,10]. In this example, the random number interval can be divided into M numerical ranges, and M can be equal to the number of pixel values ​​contained in the target distribution data; a pixel value contained in the target distribution data can correspond to a numerical range, and the length of a numerical range can be determined based on the probability value of the corresponding pixel value.

[0134] Applying the above example, when the generated random number is 0.5, the hit value range can be [0,1], so the noisy pixel value corresponding to the real pixel value in the clean RAW image can be the pixel value 1.

[0135] In another example of the present application, the target distribution data may be randomly sampled using a random sampling function provided by a scientific computing library.

[0136] An example of a random sampling function could be:

[0137] numpy.random.choice(a,size=None,replace=True,p=None).

[0138] Among them, numpy (Numerical Python) is a scientific computing library provided by Python language. The parameters of the above random sampling function are as follows:

[0139] a: represents the range of randomly sampled pixel values. Assuming that the pixel values ​​contained in the target distribution data are stored in a one-dimensional array, a can represent a one-dimensional array.

[0140] size: The number of sampling results. The default value is 1, and it can be greater than 1.

[0141] replace: Indicates whether the sampled sample is to be replaced. If the value of replace is True, the sampled elements will be repeated; if the value of replace is False, the sampling result will not be repeated.

[0142] p: represents the probability value of each element in a being sampled, which can be the probability value corresponding to the pixel value in the target distribution data, or p can represent a one-dimensional array.

[0143] Assuming that the clean RAW image contains W2×H2 pixels, the corresponding noisy pixel value can be obtained for the pixel value corresponding to each of the W2×H2 pixels. After completing the acquisition of the noisy pixel values ​​corresponding to the W2×H2 pixels, the corresponding noisy RAW image can be obtained. The embodiment of the present application can output the noisy RAW image.

[0144] In summary, in the image processing method of the embodiment of the present application, since the collected RAW image is a real noisy image, the random sampling of the embodiment of the present application can be a process of sampling from the real noisy image data; in this way, the noisy pixel value as the random sampling result matches the real noisy image data; and the noisy RAW image is obtained based on the random sampling result. Therefore, the embodiment of the present application can improve the matching degree between the noisy RAW image and the real noisy image data. In other words, the embodiment of the present application can improve the authenticity of the noise data distribution in the noisy RAW image.

[0145] Since the embodiment of the present application obtains the noisy pixel value from the real distribution data corresponding to the real pixel value for the real pixel value in the clean RAW image, the distribution corresponding to any real pixel value in the clean RAW image matches the real distribution data. For example, a clean RAW image includes: highlights, dark areas, and midtones. The highlight can refer to the brightest part near the highlight, and its corresponding real pixel value can be (2 n -1); the dark part may refer to the darker part of the boundary between light and dark, and the corresponding real pixel value may be around 0. The midtone may refer to the gray part close to the boundary between light and dark. The embodiment of the present application can not only match the distribution of the pixels in the dark part with the real distribution data, but also match the distribution of the pixels in the bright part (such as the overexposed area) with the real distribution data, which can reduce the truncation effect on the pixels in the dark and bright parts.

[0146] In the embodiment of the present application, since the noise data distribution in the noisy RAW image is derived from the real data distribution, and the real data distribution may include shot noise, read noise, quantization noise and all other non-spatially correlated noises, the noise data distribution in the noisy RAW image in the embodiment of the present application may also include shot noise, read noise, quantization noise and / or all other non-spatially correlated noises.

[0147] Moreover, when the noisy RAW image is used as training data for the image processing model, since the embodiment of the present application can improve the matching degree between the noisy RAW image and the actual noisy image data, the embodiment of the present application can improve the processing effect of the image processing model.

[0148] Method Example 2

[0149] Reference Figure 4 , shows a schematic flow chart of the steps of an image processing method according to an embodiment of the present application, and the method may specifically include the following steps:

[0150] Step 401: Acquire training data; the training data may specifically include: a noisy RAW image;

[0151] Step 402: training the image processing model according to the training data;

[0152] Among them, the acquisition process of the noisy RAW image can specifically include: collecting the RAW image; determining the distribution data corresponding to the real pixel value based on the RAW image; receiving the clean RAW image; determining the target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image and the noisy RAW image corresponding to the noisy pixel value.

[0153] Figure 4 The method shown can be used to use noisy RAW as training data, and train an image processing model based on the training data. The image processing model may include: an image denoising model or an object detection model, etc.

[0154] Taking the image denoising model as an example, its corresponding training data may include not only noisy RAW images but also clean RAW images, where the clean RAW images and the noisy RAW images appear in pairs.

[0155] The training process of the image processing model can include: forward propagation and back propagation.

[0156] Among them, forward propagation can be calculated in sequence from the input layer to the output layer according to the parameters of the image processing model to finally obtain the output information. Taking the image denoising model as an example, the output information may include: the image information after denoising. The embodiment of the present application can determine the loss information based on the difference between the image information after denoising and the clean RAW image.

[0157] Backward Propagation can calculate and update the parameters of the image processing model in sequence from the output layer to the input layer according to the loss information. The image processing model usually adopts the structure of a neural network, and the parameters of the image processing model can include: parameters such as the weight of the neural network.

[0158] In the back propagation process, the gradient information of the parameters of the image processing model can be determined, and the parameters of the image processing model can be updated using the gradient information. For example, the back propagation can calculate and store the gradient information of the parameters of the processing layers (including the input layer, the intermediate layer, and the output layer) of the image processing model in the order from the output layer to the input layer according to the chain rule in calculus.

[0159] Since the noisy RAW images included in the training data of the embodiment of the present application are obtained based on real distribution data, the embodiment of the present application can improve the authenticity of the noise data distribution in the noisy RAW images.

[0160] For example, the embodiments of the present application can improve the authenticity of the distribution of dark noise data; in this way, when a noisy RAW image is applied to an image processing model, the color cast problem caused by incorrect truncation of dark parts in the image processing model can be avoided to a certain extent, and the consistency between the training data and the real noisy image in the actual processing process can be improved, thereby improving the processing effect of the image processing model.

[0161] For another example, the embodiments of the present application can improve the authenticity of the noise data distribution in the bright part (overexposed area); in this way, when the noisy RAW image is applied to the image processing model, the mean shift problem caused by the erroneous addition of noise in the image processing model can be avoided to a certain extent, and the consistency between the training data and the real noisy image in the actual processing process can be improved, thereby improving the processing effect of the image processing model.

[0162] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0163] Based on the above embodiment, this embodiment further provides an image processing device, referring to Figure 5 The device may specifically include: a collection module 501, a distribution data determination module 502, a receiving module 503, a target distribution data determination module 504 and a random sampling module 505.

[0164] Wherein, the acquisition module 501 is used to acquire RAW images;

[0165] A distribution data determination module 502 is used to determine distribution data corresponding to real pixel values ​​according to the RAW image;

[0166] A receiving module 503, configured to receive a clean RAW image;

[0167] A target distribution data determination module 504 is used to determine target distribution data corresponding to the real pixel values ​​in the clean RAW image from the distribution data corresponding to the real pixel values;

[0168] The random sampling module 505 is used to randomly sample the target distribution data to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

[0169] Optionally, the acquisition module 501 may specifically include:

[0170] The first acquisition module is used to acquire RAW images using an image acquisition module; the image acquisition module includes: an image sensor, or an image sensor and a filter.

[0171] Optionally, the acquisition module 501 may specifically include:

[0172] The second acquisition module is used to respectively acquire RAW images under conditions of multiple preset brightness values.

[0173] Optionally, the distribution data determination module 502 may specifically include:

[0174] A preset area determination module, configured to select a preset area including the center from the RAW image according to the center of the RAW image;

[0175] The statistical module is used to count the pixel points in the preset area, and the obtained pixel mean is the real pixel value, and the distribution data of the pixel points in the preset area is used as the distribution data corresponding to the real pixel value.

[0176] Optionally, the device may further include:

[0177] The interpolation module is used to interpolate the distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value to obtain the distribution data corresponding to the third real pixel value.

[0178] In the embodiment of the present application, N groups of RAW images can be acquired according to a plurality of preset brightness values. The real pixel values ​​corresponding to the RAW images in the same RAW image group can be the same or similar, for example, the difference between the real pixel values ​​corresponding to different RAW images in the same RAW image group can be less than the second difference threshold.

[0179] The difference between the real pixel values ​​corresponding to the RAW images between different RAW image groups may be greater than the third difference threshold. For example, the third difference threshold may be a two-digit positive integer, or a three-digit positive integer, etc. In this way, the distribution data corresponding to the real pixel values ​​obtained based on the collected RAW images in the embodiment of the present application may be discrete. For example, the distribution data corresponding to the real pixel values ​​obtained based on the collected RAW images in the embodiment of the present application may include: distribution data corresponding to real pixel values ​​such as 0, 100, 200, 300, ... 4000, etc. Among them, the real pixel values ​​0 and 100, 100 and 200, etc. are discrete, for example, the distribution data corresponding to the real pixel values ​​between [101,199] are missing.

[0180] In order to increase the continuity and coverage of the distribution data of the real pixel value, the embodiment of the present application can interpolate the distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value to obtain the distribution data corresponding to the third real pixel value. For example, the embodiment of the present application can use 100 as the first real pixel value and 200 as the second real pixel value, and obtain the distribution data corresponding to the real pixel values ​​between [101,199] based on the interpolation.

[0181] Therefore, the first real pixel value and the second real pixel value may be the real pixel values ​​of the obtained distribution data. It can be understood that the embodiment of the present application does not limit the specific first real pixel value and the second real pixel value.

[0182] Optionally, the random sampling module 505 may specifically include:

[0183] A mapping relationship determination module, used to determine the mapping relationship between the numerical range corresponding to the probability value of the pixel value and the pixel value according to the pixel value and the probability value contained in the target distribution data; wherein the probability value corresponding to a pixel value may be the ratio of the number of occurrences of the pixel value to the sum of the number of occurrences of all pixel values ​​in the case of a real pixel value;

[0184] A generation module, used to generate random numbers;

[0185] A search module is used to search in the mapping relationship between the numerical range and the pixel value according to the random number, so as to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image.

[0186] In summary, in the image processing device of the embodiment of the present application, since the collected RAW image is a real noisy image, the random sampling of the embodiment of the present application can be a process of sampling from the real noisy image data; in this way, the noisy pixel value as the random sampling result matches the real noisy image data; and the noisy RAW image is obtained based on the random sampling result. Therefore, the embodiment of the present application can improve the matching degree between the noisy RAW image and the real noisy image data. In other words, the embodiment of the present application can improve the authenticity of the noise data distribution in the noisy RAW image.

[0187] Since the embodiment of the present application obtains the noisy pixel value from the real distribution data corresponding to the real pixel value for the real pixel value in the clean RAW image, the distribution corresponding to any real pixel value in the clean RAW image matches the real distribution data. The embodiment of the present application can not only match the distribution of the pixel points in the dark part with the real distribution data, but also match the distribution of the pixel points in the bright part (such as the overexposed area) with the real distribution data, which can reduce the truncation effect on the pixel points in the dark and bright parts.

[0188] In the embodiment of the present application, since the noise data distribution in the noisy RAW image is derived from the real data distribution, and the real data distribution may include shot noise, read noise, quantization noise and all other non-spatially correlated noises, the noise data distribution in the noisy RAW image in the embodiment of the present application may also include shot noise, read noise, quantization noise and all other non-spatially correlated noises.

[0189] Moreover, when the noisy RAW image is used as training data for the image processing model, since the embodiment of the present application can improve the matching degree between the noisy RAW image and the actual noisy image data, the embodiment of the present application can improve the processing effect of the image processing model.

[0190] Based on the above embodiment, this embodiment further provides an image processing device, referring to Figure 6 , the device may specifically include: an acquisition module 601 and a training module 602.

[0191] Wherein, the acquisition module 601 is used to acquire training data; the training data includes: a noisy RAW image;

[0192] A training module 602, used for training the image processing model according to the training data;

[0193] Among them, the acquisition process of the noisy RAW image specifically includes: collecting the RAW image; determining the distribution data corresponding to the real pixel value based on the RAW image; receiving the clean RAW image; determining the target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image and the noisy RAW image corresponding to the noisy pixel value.

[0194] The image processing model in the embodiments of the present application, for example: an image denoising model and a target detection model.

[0195] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0196] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0197] The embodiments of the present disclosure may be implemented as a device that uses any appropriate hardware, firmware, software, or any combination thereof to perform a desired configuration, and the device may include: electronic devices such as terminal devices and servers (clusters). Figure 7 An exemplary apparatus 1100 that can be used to implement various embodiments described in this application is schematically shown.

[0198] For one embodiment, Figure 7 An exemplary apparatus 1100 is shown having one or more processors 1102, a control module (chip set) 1104 coupled to at least one of the (one or more) processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory (NVM) / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0199] The processor 1102 may include one or more single-core or multi-core processors, and the processor 1102 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1100 can be used as a terminal device, server (cluster), etc. described in the embodiments of the present application.

[0200] In some embodiments, the apparatus 1100 may include one or more computer-readable media (e.g., memory 1106 or NVM / storage device 1108) having instructions 1114 and one or more processors 1102 combined with the one or more computer-readable media and configured to execute the instructions 1114 to implement a module to perform the actions described in the present disclosure.

[0201] For one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1102 and / or any suitable device or component in communication with the control module 1104 .

[0202] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0203] The memory 1106 may be used, for example, to load and store data and / or instructions 1114 for the device 1100. For one embodiment, the memory 1106 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1106 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0204] For one embodiment, control module 1104 may include one or more input / output controllers to provide an interface to NVM / storage device 1108 and input / output device(s) 1110 .

[0205] For example, NVM / storage 1108 may be used to store data and / or instructions 1114. NVM / storage 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0206] NVM / storage device 1108 may include storage resources that are physically part of the device on which apparatus 1100 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1108 may be accessed via input / output device(s) 1110 over a network.

[0207] (One or more) input / output devices 1110 may provide an interface for the apparatus 1100 to communicate with any other appropriate device, and the input / output device 1110 may include a communication component, an audio component, a sensor component, etc. The network interface 1112 may provide an interface for the apparatus 1100 to communicate through one or more networks, and the apparatus 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0208] For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers of the control module 1104 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104 to form a system-on-chip (SoC).

[0209] In various embodiments, the device 1100 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 1100 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1100 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0210] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information corresponding to the camera device, etc. are stored in the memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0211] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0212] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0213] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, terminal device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0214] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0215] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0216] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0217] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0218] The above is a detailed introduction to an image processing method and device, an electronic device and a machine-readable medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Collect RAW images; Determine distribution data corresponding to real pixel values ​​according to the RAW image; Receive clean RAW images; Determine target distribution data corresponding to the real pixel values ​​in the clean RAW image from the distribution data corresponding to the real pixel values; The target distribution data is randomly sampled to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

2. The method according to claim 1, characterized in that The collecting of RAW images comprises: The RAW image is collected by using an image collection module; the image collection module includes: an image sensor, or an image sensor and a filter.

3. The method according to claim 1, characterized in that The collecting of RAW images comprises: Under the conditions of various preset brightness values, multiple groups of RAW images are collected respectively.

4. The method according to claim 1, characterized in that: The determining of the distribution data corresponding to the real pixel value includes: According to the center of the RAW image, selecting a preset area including the center from the RAW image; The pixel points in the preset area are counted, and the average value of the obtained pixel points is the real pixel value. The distribution data of the pixel points in the preset area is used as the distribution data corresponding to the real pixel value.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The distribution data corresponding to the first real pixel value and the distribution data corresponding to the second real pixel value are interpolated to obtain the distribution data corresponding to the third real pixel value.

6. The method according to any one of claims 1 to 4, characterized in that: The randomly sampling the target distribution data comprises: Determine a mapping relationship between a numerical range corresponding to the probability value of the pixel value and the pixel value according to the pixel value and the probability value included in the target distribution data; Generate random numbers; According to the random number, a search is performed in the mapping relationship between the numerical range and the pixel value to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image.

7. An image processing method, characterized in that: The method comprises: Acquire training data; the training data includes: noisy RAW images; Training the image processing model according to the training data; Among them, the acquisition process of the noisy RAW image includes: collecting the RAW image; determining the distribution data corresponding to the real pixel value based on the RAW image; receiving the clean RAW image; determining the target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image and the noisy RAW image corresponding to the noisy pixel value.

8. An image processing device, characterized in that: The device comprises: Acquisition module, used for acquiring RAW images; A distribution data determination module, used to determine distribution data corresponding to real pixel values ​​according to the RAW image; A receiving module, used for receiving a clean RAW image; A target distribution data determination module, used to determine target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; The random sampling module is used to randomly sample the target distribution data to obtain noisy pixel values ​​corresponding to real pixel values ​​in the clean RAW image and a noisy RAW image corresponding to the noisy pixel values.

9. An image processing device, characterized in that: The device comprises: An acquisition module is used to acquire training data; the training data includes: a noisy RAW image; A training module, used for training the image processing model according to the training data; Among them, the acquisition process of the noisy RAW image includes: collecting the RAW image; determining the distribution data corresponding to the real pixel value based on the RAW image; receiving the clean RAW image; determining the target distribution data corresponding to the real pixel value in the clean RAW image from the distribution data corresponding to the real pixel value; and randomly sampling the target distribution data to obtain the noisy pixel value corresponding to the real pixel value in the clean RAW image and the noisy RAW image corresponding to the noisy pixel value.

10. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, causes the processor to execute the method according to any one of claims 1 to 7.

Images